Saturday, November 29, 2003

Archiving to DVD

I've been playing around with QuickPar some more (previous post) as I finish up transferring all of my old CDR-based archives to DVD-R (packing 300 or 400 CD-Rs down to just 60-80 DVD-Rs). As a result, I've figured out a few tricks which I didn't realize last month.

1) Using the Options dialog, you should set the block size to be 1/4, 1/3, or 1/2 of your average file size. Files always get boundary-aligned, so if you have 512k files, but with block sizes of 2048k, you're going to waste 1536k for each file (efficiency value of 25%). OTOH, if you would have used 64k blocks, the average wasted space per file would only be 32k. In general, I use a block size that is 1/4 of the average file size.

2) If a file goes bad on a CD-R or DVD-R, Windows will not allow you to copy the file at all. So a single-sector ECC failure can render an entire 100Mb file unrecoverable (at least not, AFAIK, without hacking past the CD-ROM driver). Therefore, when protecting a file set, provide enough recovery data (or redundancy data) so that you can recover at least one file in the set. Making sure you can recover at least 2 files in the set would probably be safer.

2b) It's rumored that if you were to put all of the files into a single folder, and have a single PAR2 recovery set on that media, that you can rip damaged media to an ISO file, rename the ISO file to .PAR2, and QuickPar will recover the files. I've not tested it, but it's said that it's doable but since the current PAR2 spec doesn't support sub-folders that all files have to be in a single folder. (And I'm not sure if you can have multiple PAR2 sets on the same media and do the rip-to-ISO to PAR2 trick.)

3) QuickPar eats up memory when building the recovery files, so don't start 2 dozen large recovery set builds without checking how much memory is being used. I've mistakenly used up 50% more memory then I had installed in a particular machine which slowed the process to a complete crawl until I abandoned creating some of the recovery sets.

4) CFV is a good command line tool that you can use to verify PAR2 sets on a piece of media. It's main advantage is that you can easily have it recursively check all of the sub-folders on a CD/DVD. The specific command that I use to check is to CD to the root folder of the media, then issue the command "cfv -r --progress=no".

posted by Wuphon's at 12:08 PM