Usenet binaries with Linux tutorial

Most of the techniques and software detailed in this FAQ can of course work with MacOSX, FreeBSD, and probably even with Windows.


How do I extract multiple-part Rar archives with Linux ? ( rar r00 r01 r02, etc )

With the unrar command line. Unrar is open source ( exotic licence). Notice that once unrar is installed, front-ends like File-roller, Ark, Xarchiver, or Gnome's right click's "Extract here" can extract Rar archives automatically.

$ unrar e file.rar

Do not install the "GNU unrar" it will not work on Rar3 files, use the unrar-nonfree ( Debian/Ubuntu ) , unrar from plf ( Mandriva ) packages, or the files from the author's website ( rarlab ).


How do I create Rar archives with Linux ? ( thanks to Ohmster for this part, see his comment bellow for reference )

The Rar command-line is the only way to do it. It costs about 30$ at rarlab.com. Like you probably already know, it is the de facto standard way for posting binaries.

example of use :

rar a -s -v15m ppv-erica_campbell-672 *.* ( LOL!!! )

This will create 15Mb volumes from the content of the current directory.
The file mask "*.*" will include all files ( including par2, nzb, nfo, screenshot, video sample, etc ) in the

To create a rar archive file that is broken into specified size parts, use this syntax:

rar a -v15000k hotstuff *

What this does:

• rar to run the rar program.
• "a" to command "Add to archive". If the archive does not exist, it will be created.
• -v15000k is the switch to tell rar to create archive parts no larger than 15,000Kb or 15Mb.
• hotstuff is the name of the archive you wish to create.
• * will tell the rar program to include all files in the current directory.

If you're not willing to pay 30$, consider splitting the file in chunks instead ( see question below ) since it's the second most common way to post usenet binaries these days.


How do I repair incomplete files with the par2 parity files ?

With the parchive package ( or par2 package if you use Debian/Ubuntu )and the par2 command :

$ par2 r file.par2

or :

$ par2repair file.par2

Or with Quickpar that works fine with Wine.



How do par2/repair by double-clicking from my file manager without opening a console ? Is there a par2 GUI ?

Those using KDE/Konqueror won't have any problem associating par2 files with par2repair and letting the console open after the operation, it's very straight forward.
With Gnome opening a gnome-terminal from Nautilus is tricky since it the window closes too quickly you can't see what happened, so let's use xterm instead :

In Nautilus, associate par2 extensions with custom command-line :

xterm -hold -e par2repair

About the Par2 GUIs there are several :

• Ben's Par2GUI KDE script, which judging by the comments is very handy.
• Gpar2
• PyPar2
  

How to join split files .001 .002 .003 .004... back together ?
Concatenation of multiple split files.

These split files are likely to be made by Quickpar users under windows, so the perfect way to assemble them is to use the par2 command line. ( from the parchive package )

par2 r myfile.par2 myfile.*

That way, all the split files will be considered as additionnal blocks, and it will check, repair, concatenate the files at the same time.

Note that it's also possible to join the split files back together and then use the new file to find additionnal blocks. It takes one step more but it can save some processing time.

Sometimes the posters mistakenly create parity files that will repair the split files instead of the actual file, after repairing the split files use one of these sweet shell wildcard to merge them back together :

$ cat *.[0-9][0-9][0-9] >output.avi

If the split files are not the only ones in the folder :

$ cat yourfile.avi.[0-9][0-9][0-9] >yourfile.avi

or :

$ cat yourfile.avi.??? >yourfile.avi


This is very rare but sometimes .001 .002 .003 files are not split files but actual Rar multiple-part archives, in that case, unrar should do the job :

$ unrar e yourfile.avi.001



Which binaries grabber for Linux ?

* Multi-server, multi-sources :

1. Pan 0.1** beta now uses less memory, handles multiple servers and nzb files. It only allows 4 connections per server though, which is kind of bad for those of us who pay for using more. I wish it could support SSL and nntps as well.
2. Klibido : GNU, handles Nzb files, OK but unstable
3. BNR2 : heavy, unstable, coded with Kylix, ( not free? )
4. Newsbin Pro is also reported to work fine with Wine and seem to use very few memory. ( not free )

* Nzb command-lines : nzb, hellanzb, NZBget, nzbperl, Knzb.
* Command-line binary grabbers : ubh, brag, aub, nget,


How do I split files, create parity files and post them ? ( Thanks a lot to "Nobody" @ a.b.m.a.d for this part! )

You don't need a file splitter do it, the split command-line will do the job easily.

Posting rules :

* The split files must be a multiple of the par2 block-size, if not some blocks can be lost between two split files.
* The par2 block-size should be equal to the size of the article, or a multiple of it.
* Posting with Newspost constrains the article size to be a multiple of 45, and thus, so must be the Par2 block-size and the split file size.


Here's an example of good settings :

# Par2 block-size : 225,000 bytes
# Article size : 225,000 bytes and 5000 lines
# Split files size : 9,000,000 bytes ( 40 articles )

#!/bin/sh
par2 create -s225000 $1
split --suffix-length=3 --numeric-suffixes --bytes=9000000 $1 $1'.'

Example of use :
$ sh parsplit.sh myfile.avi

To join the files back together and check them at the same time :
par2 r myfile.avi.par2 myfile.avi*

To post with Newspost, don't forget to specify the good number of lines :

$ newspost ........ -l 5000 ......

Here's another binary poster : yencee.


How do batch-repair all the files that I downloaded ? ( Thanks to Anthony @ a.b.m.a.d for this part! )

You can use this perl script.

$ mass-par2repair alt.binaries.*

Where arguments are directories, NOT files.


#!/usr/bin/perl -w
# Copyright (C) 2005 Anthony DeRobertis (netnews at derobert d0t net)
#
# This file is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# This file is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this file; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

sub run_par(@);

my %results = ();
foreach my $dir (@ARGV) {
opendir DH, $dir or die "Could not open directory $dir: $!";

my %par_files = ();
my %segment_files = ();
foreach my $file (readdir DH) {
my $base = undef;
if ($file =~ /.par2?$/i) {
if ($file =~ /^(.+)\.vol\d+\+\d+\.par2?$/i) {
$base = $1;
} elsif ($file =~ /^(.+)\.par2?/i) {
$base = $1;
} else {
die "Failed to find basename of $file.";
}
push @{$par_files{$base}}, $file;
} elsif ($file =~ /^(.+)\.\d\d\d$/) {
$base = $1;
push @{$segment_files{$base}}, $file;
}
}

foreach my $key (keys %par_files) {
print "$key:\n ";
print join("\n ", @{$par_files{$key}}), "\n";

my @args = ();
foreach my $file (@{$par_files{$key}}, @{$segment_files{$key}}) {
push @args, "$dir/$file";
}
my $retcode = run_par(@args);
if ($retcode == -1) {
die "Could not exec par2: $!";
} elsif ($retcode == 0) {
#$results{$dir}->{$key} = 'OK';
print " OK.\n\n"
} else {
$results{$dir}->{$key} = "FAILED ($retcode).";
print " FAILED ($retcode)\n\n";
}
}
closedir DH;
}

print "\n\n\n";
foreach my $dir (keys %results) {
print "SUMMARY FOR $dir:\n";
foreach my $file (sort keys %{$results{$dir}}) {
print "\t$file: " . $results{$dir}->{$file} . "\n";
}
print "\n";
}

sub run_par(@) {
my $pid = fork();

if ($pid == -1) {
return -1;
} elsif ($pid == 0) {
open STDOUT, '+>', '/dev/null' or die "> /dev/null failed: $!";
exec {'par2'} 'par2', 'r', '--', @_;
die "Exec of par2 failed: $!";
} else {
waitpid($pid, 0);
return $?;
}
}

########END OF PERL SCRIPT #############

As you noticed, I've messed up all the formatting but it still works. If you want the original file, contact the author, his address is in the headers.



Another simple solution is to do this : ( thanks to the anonymous poster for his comment )


for i in *.[pP][aA][rR]2; do par2 r $i; done


let's explain step by step :

echo *.par2
this will obviously print all the par2 files in the current directory

to get one filename per line :
for i in *.par2; do echo $i; done

So, to mass-repair everything in a directory simply type :
for i in *.par2; do par2repair $i; done

This will work 99% of the time, since usually the first par2 file of the a set ( the smallest one ) is not capitalized, whereas the rest if them are.

To make it work 100% of the time, you can do this :
for i in *.[pP][aA][rR]2; do par2 r $i; done
But beware, this is a brute strength method, you will run par2 several times for a same set of parity files.



Many thanks to the people on a.b.m.a.d for their help !


If you have other interesting tips to share about Usenet binaries with Linux, do not hesitate to drop a comment ( no registration required ). Stay tuned, this FAQ is regularly updated.

Last updated July 2007 : rar and Ben's Par2GUI