Usenet: Everything You Need to Know About It

Saturday, 27 January 2018

Usenet: Everything You Need to Know About It

Posted by Rahul Gupta

Introduction to Usenet

The birth and subsequent evolution of the Internet is a fascinating study in ingenuity and a deep human desire to connect with others and exchange information. From the birth of ARPANET in 1969 to present day VR technology, use of the Internet has become not only a means of exchanging information, but a tool to enhance our own experiences. The bond between tech-savvy users both at the birth of the Internet and today is apparent through the use of forums. The Internet users of today are familiar with the exchange of ideas, files, and assets found on any modern day forum, but this link between others has a rich history, dating back to a time before Walkmans, DVDs, and the fall of the Berlin Wall. Usenet, developed in 1979 and still in use today, is one such pioneering service that brought forward-thinking people together to engage in dialogue and share what was important to them. 
Built on the “poor man's ARPANET”, Usenet was developed in 1980 at Duke University, and the University of North Carolina at Chapel Hill, well before the general public made use of the Internet in its current form, or anyone heard of the World Wide Web. Conceived as a bulletin board service (BBS) for text messages, Usenet originally only had the ability to exchange text discussions, although that changed with further technological advancement. Active to the present day, Usenet is considered one of the oldest network communications systems with active user participation. 

But how do users use Usenet? Back from its inception, Usenet utilized a transport protocol called Unix-to-Unix Copy (UUCP) to transfer files, mail, and announcements between users. Like forums today, posts and announcements are separated into hierarchical categories called newsgroups. When a user accesses a particular newsgroup, the news reader software provides a list of everything that has been uploaded, or posted, to that newsgroup.

Article responses in newsgroups typically branch from an original non-reply article posted, creating a branch of threads and subthreads that users can interact with. Initially, an article posted on a specific newsgroup is only available on that particular Usenet server. However, each server or “newsfeed” communicates with other feeds and eventually delivers the original article across the network for the rest of the user base to peruse. Unlike modern-day peer-to-peer networks such as torrent clients, Usenet depends on the sender to start the transfer of a file instead of the intended receiver initiating a request. Although the transmission of a message is similar to the sending of an e-mail, Usenet messages are visible to the entire newsgroup and anyone accessing Usenet instead of a single intended recipient.

Newsreaders, somewhat similar to aggregators, are used to access groups by acting as clients to connect to a news server using the Network News Transfer Protocol (NNTP). They then can download and upload articles to the group. Binary files are now commonly being distributed to groups dedicated to binaries instead of only text-based articles.  As a result, many users have steered away from the traditional discussion groups and use Usenet primarily as a tool to download media ranging from movies and music to pornography. A popular beginner's choice to access Usenet is the Newsbin and Newsrover readers and are frequently included by Usenet providers.

The now common binary content being distributed on Usenet was originally a groundbreaking development.  Content was originally encoded using uuencode, a method of binary-to-text encoding. Uuencode was later replaced by YEnc (literally for "why encode") encoding, which allowed e-mail to support non-ASCII text, media attachments, and complex message bodies. Today, binary files are uploaded by converting the file into RAR archives and then creating Parchive files. To ensure that a complete file reaches the server, parity files are used to recreate any missing data that was lost during transfer. 

The evolution of Usenet from an article sharing service to a method of quickly obtaining legally dubious files has caught the attention of Internet Service Providers (ISP's). As a result of the nature of Usenet, completely removing any copyrighted material is difficult if not downright impossible. The rapid sharing of files between newsgroups makes the material hard to remove completely. Similar to heads on a Hydra, the files are quickly duplicated across groups regardless of the take down of the original, incriminating file.  Although newsgroups can be dedicated to almost any subject imaginable, they are loosely grouped into nine hierarchies. Eight of these are regulated by subject and include comp.*, humanities.*, misc.*, news.*, rec.*, sci.*, soc.*, and talk.*. Alt.*, on the other hand, is not restricted by these regulations and acts as an “anything goes” set of newsgroups. The subset of newsgroups called alt binaries is dedicated to files, and there are thousands of categories that branch off to every subject possible.  As such, file distribution is most common on the alt binaries hierarchy which now accounts for many Terabytes of uploads per day.

Getting set up with Usenet is fairly straightforward and requires a service provider, optionally an NZB indexer, and a Usenet client. NewsHosting, Giganews, Astraweb, UseNeXT, UsenetServer, and Usenet Storm are considered high-quality choices as far as Usenet providers go and often offer a free trial to new users. After selecting a provider, the Usenet client software must be configured.

Clients like SABnzbd are lightweight and available across most operating systems. After running the installer, all options on the Choose Components screen should be selected, so SABnzbd runs at start up and all NZB files are associated with the app. The default web browser will then open a connection to localhost:8080 and begin the SABnzbd Quick-Start Wizard. A user will then be prompted to enter the server and login data for the provider they chose. Check the SSL box and ensure that port 563 will be used. 

Configuring access to SABnzbd is simple, and the default settings are usually sufficient if running on a home computer. If using two different indexing sites, configure the NZB indexer by providing the appropriate information. SABnzbd will then cycle and restart, prompting for the user's login information. The user will then be presented with their queue and download history. 

To get started, an NZB indexing site will be needed to discover content and files. Unlike open torrenting services, some indexing sites are not free. The cost, however, is nominal and averages only $10 a year. Raw indexes are essentially data dumps of all files and can be difficult to navigate if you are not sure what you are looking for. Hand-indexed databases are categorized and more user-friendly. 
NZBClub and are popular choices for raw indexes and are free with no registration required. Once you find a file, you wish to download, simply grab it and drop it into the Watch Folder. SABnzbd or any other client will then start the download, unpack the files and place them in the Finished Download directory. 

The repurposed and modern-day Usenet is a satisfactory alternative to public torrenting sites. 27 terabytes of new data are posted across all newsgroups every single day, with alt binaries newsgroups accounting for about 90% of that volume, and Usenet enjoys its status as the grandfather of all file-sharing networks. When a torrent explodes in popularity, the file was usually found on Usenet first. Torrents are also at the mercy of friendly seeders, with download speeds often bottoming out when no seeds are available. Files posted on Usenet are available for download at full speed immediately after being uploaded, allowing for a faster and more reliable download. User safety is also another Usenet asset. Unlike other P2P services, a downloader's identity is not visible on Usenet since the user directly connects to a server instead of another user. The network address cannot be traced once it moves past the original server. This guarantee of anonymity allows a freer flow of information and peace of mind when using the service. 

Although the technical jargon, interface and unfamiliar client names can initially be off-putting to new users, the three-step process above removes the confusion and mystique and allows you to dive right in. Once you become acclimated, Usenet will seem familiar and intuitive. Going back to torrents will seem clunky, out of date, and simply inefficient. One thing is certain, as technology changes and evolves, Usenet will change and evolve with it.


Post a Comment