- How do I repair incomplete files with the par2 parity files ?
- How do par2/repair by double-clicking from my file manager without opening a console ? Is there a par2 GUI ?
- How to batch-repair the millions of files that I have downloaded legally
Friday, December 23, 2005
Usenet binaries with Linux tutorial
I've moved the FAQ to the new website
Subscribe to:
Post Comments
(
Atom
)
23 comments :
Hi Christian,
About the CPU usage to 50%, I don't know if it's possible. You could change the priority or renice the process to 19 though, it will still use 100% of the CPU, but the other processes wil not be affected.
Unfortunately I don't know any Perl, so you can contact the author, his email is in the headers of the script.
Each time the script runs "par2" or "par2repair" you need to add a couple of lines to the script tofind the PID of the new subprocess, and change its priority.
I think I know how to do it in python if it helps you, using the renice command line :
import subprocess
p = subprocess.Popen("par2 r blablabla")
subprocess.Popen( "renice 19 -p " + p.pid )
This is basically what you need to do but in Perl.
I don't understand what you're doing with par2 and your vdr, your vdr runs under linux and it also uses par2repair ? sorry I don't get it.
Good luck.
hmm the comment disappeared ?? wtf ??
here's a copy :
Hi,
the mass-par2repair perl script is what i was looking for, thx!
Is there a possibility to set the usage of the cpu to 50 for example?
I am running ur script as a chrown job every hour on a vdr (video disk recorder).
When recording via 2 cards and cutting some advertisment from recordings, its to much for the vdr using the script in the standart way.
Hope u understood my english...smile
So far greets from germany
Christian
Feel free to write me a email to
cPOINTmatthiessenATgmxPOINTde
Good article. Will take a closer look at the mass repair script. Two things for those that read this later:
1. it's good posting form to use par2 -l to limit the size of the par2 files.
2. do not, do not do not make the par2 file against original file instead of the posted files (the rars or the split files). This has two benefits for the downloader: you can check the files you have while the rest are downloading and automatic par2 checkers (nget and any other downloader worth considering) don't fail and download a bunch of unecessary par2 files.
Thank-you, thank-you thank-you!!
You've no idea how much googling I've done searching for how to use .par files in Linux until I stumbled across your blog!
Cat was failing me ~50% of the time, and your little par2 walkthrough works perfectly!
Many thanks once again!
I need a good page like this for reference and I do things pretty much the standard way for posting videos. I can do all of this in Windows but prefer not to tie up my Windows computer with these chores when I can offload them to the Linux machine on my LAN and even have the Linux machine post them for me. This is how I like to do things:
Create preview image with thumbnails of the video.
Create the rar archive set, broken into manageable parts like 10 or 15 megabytes.
Create the par2 recovery file set.
Use news post to post the entire directory of files in my "post" directory, that is where I put everything that I intend to post.
I often use the -E option in news post in order to create a text file at the beginning of the post, giving information or comments, etc.
Let 'er rip, that is to say, use news post to upload everything to my giganews server to the appropriate newsgroup.
I often use the screen command and work entirely from the CLI so that I can begin this job in a VT100 term window with putty from Windows and then detach the screen. I can then check up on it later by reattaching to the screen session from the term program or even the server itself, maybe even remotely with putty again.
Even though rar is not a free program, it is the de-facto method of posting large video files and I would appreciate you including the commands necessary to create multipart rar archive sets from a large video file, like this:
-----------------------------------------------------------------------------
rar Linux Examples
-----------------------------------------------------------------------------
rar a -s -v15m ppv-erica_campbell-672 *.*
Create 15Mb volumes from the contents of the current directory, the file mask,
"*.*" will include all files, in the new the archive and will be named
ppv-erica_campbell-672.partxx.rar.
-----------------------------------------------------------------------------
To create a rar archive file that is broken into specified size parts, use
this syntax:
rar a -v15000k hotstuff *
What this does:
rar to run the rar program.
"a" to command "Add to archive". If the archive does not exist, it will be
created.
-v15000k is the switch to tell rar to create archive parts no larger than
15,000Kb or 15Mb.
hotstuff is the name of the archive you wish to create.
* will tell the rar program to include all files in the current directory.
-----------------------------------------------------------------------------
Please include the rar commands in your page as it would make a great reference and I for one would like to reference it in my FAQ for video posting. I had a hell of a time getting rar Linux to do what I wanted as the instructions were pretty dry and I had to ask for help in the Linux newsgroups.
Other than the rar omission, this is a great page and I thank you for publishing it.
~Ohmster
Thank you Ohmster, I will include your comment.
To repair with par2s and then extract the rars in one go command stack with &&. A typical example would be: par2 r *par2 && rar x *rar. && ensures that if the first command fails it will not go on to the next command. Ideal in this case.
To get by the 4 server connection limit in Pan put the same server address in twice, or as many times as you like.
this is all awesome...thanks
I'm looking for a way to make directories from file names. Say I download 200 mp3's, I'd like some way to have directories made from a file name minus the extension. Either by right clicking on the file name and choosing make dir or a way to do it from the command line. im using debian and gnome.
I know there is a way to automate this whole process, i just have to find it :)
I installed Thunar File Manager and added a custom actions under edit>>configure custom actions.
mkdir $(ls %n|awk -F"." '{print $1}')
Add a new action, name it, put in the script, and then go to 'appearance and conditions' tab and select all the check boxes. Your done. Now just right click on the file you want and select the script.
I cant take credit for this, but it does what I wanted and removes one more step for processing binaries :)
I posted some useful scripts for working with usenet binaries. Its over http://bonzoli.com/?cat=4
Just about everything you could want to do with perl and usenet downloads.
Is there any other way of joining .001, .oo2 etc split files? I cant seem to get them to join. New to *nix and this is soo frustrating! Thanks!
jerry
For posting from the commandline, I've heard good things about newsmangler
It does yEnc encoding.
Requires Python.
Newsmangler is available here:
http://www.madcowdisease.org/mcd/newsmangler
Does work quite well.
1. STIC ('some tools for image collectors') is a set of commandline linux utilities for managing 'an unreasonable amount' of images, including downloading from and posting to usenet with yEnc encoding, and much more.
Includes an interesting-sounding utility called 'similar', which somehow is supposed to find images that are similar in content (including duplicates).
I haven't tried stic (looks much more than I need), but it might warrant exploration for anyone seriously into usenet images.
http://stic.sourceforge.net/
2. LUCY is a Perl commandline newsreader and poster.
http://www.geniegate.com/other/lucy/
Again, I haven't tried this yet (may do) and post this fyi, since I'm happy with newsmangler for posting.
I use usenext on linux and it works very well...
easy to use. for me it's the right thing!
http://www.usenext.com/UseNextDE/ShopInt/misc/support/miscSupport.cfm?SNUUID=36B7C32E%2D3048%2D8C24%2D4C3929519529F2AC&1203513255670&&CategoryID=8
Even though PAN and every other Linux nntp reader I know of do not support SSL directly, you can use the daemon `stunnel` to create a pipe. Your stunnel.conf file will look something like:
---
client = yes
[nntp.client]
accept = 127.0.0.1:119
connect = {your nntp server}:563
---
Then, you can point your client to localhost, and stunnel will handle the ssl connection to the actual server.
A few followup notes:
1. I thought rar 4.41 for linux was shareware. Needs checking, but I have a suspicion there isn't an obligation to buy.
2. Pan2 still is lacking in some ways, but it's all there really is as a half-decent native binary GUI newsreader for linux. It has got a lot right, so it's getting there. It's fine for relatively light use, or if you can get used to its quirks. It will improve.
3. Forte Agent runs well under the current wine release in linux. XNews also runs well under wine but requires the following dlls from a WinXP install to be copied into the XNews root directory: comctl32.dll, ole32.dll, oleaut32.dll, riched20.dll, riched32.dll. Run winecfg and set those libraries as 'native'.
Forte Agent should be installed in the ~/.wine/drive_c/Program Files directory. XNews should be installed in ~/.wine/drive_c (not sure how important that is).
I've heard QuickPar (free) also runs under wine but haven't tested this. In any event Peter Brian Clements' par2cmdline utilities for linux work very well:
http://parchive.sourceforge.net There are precompiled binaries in Debian repos and probably elsewhere.
4. LUCY sucks the sweat off dead donkey balls unfortunately.
5. Newsmangler (commandline, Python) is fine for posting binaries.
Also:
1. Concerning joining parts on the commandline:
I find if all the parts and par files are in their own directory, there's no need to put wildcards in the glob expression. The shell (bash in my case) seems to know I'm talking about numeric globbing only. So I just do:
$ cd thatdirectory
$ cat somefile.avi.* >somefile.avi
Never have any problems - the shell ignores the par2-ending files etc.
Be sure to put quotes around the filenames if they contain spaces, and use double quotes if these contain parentheses (Windows people will put all kinds of garbage in filenames!).
2. Concerning parts that are in fact split rar archive parts: there seems to be a typo in the article.
When rar itself makes split archives, the digits get put between the filestem and the "rar" suffix. So it's quite obvious:
somefile.001.rar
somefile.002.rar
somefile.003.rar
etc
and not as the author says. if it looks like somefile.rar.001 etc then the archive was made with rar and subsequently split with something else.
Combine and extract with:
$ rar x somefile.001.rar
I recommend using x not e because this will preserve any directory structures/paths inside the archive. Files may have been neatly organized into folders, but you'll lose all that with e.
Oh dear, I boo-booed.
Please *disregard* my comment (1) in the previous post.
You should indeed put wildcards in the globbing statement - I must have ended up repairing more files than I needed to!
So:
$ cat filestem.avi.[0-9][0-9][0-9] >filestem.avi
is quite correct.
Sorry!
For linux I recommend the URD package to download binaries. Check out http://www.urdland.com
hellanzb will do most of that automatically.
knewspost 1.1-1 version seems to work fine although i followed some advice from
http://ubuntuforums.org/showthread.php?t=691892 and found that posting from the command line is really sexy and fast. Same with archiving and many other things.
You can get knewspost from sourceforge via this http://fr2.rpmfind.net/linux/rpm2html/search.php?query=knewspost
Post a Comment