."\ Master source for slice manual : /usr/src/local/man/src/slice.rof .TH SLICE P "Manual V3.9 for Program V3.9" "Julian Stacey" "Julian Stacey, Vector Systems Ltd" .\" A4 the cludgy way. .pl 29.6c .UC 4 .SH NAME slice \- Slice blocks to & from sequentially numbered files. .SH SYNTAX .B slice [-b bytes_per_slice] [-d] [-r] [-n] [-s skip_blocks] [-v] [-w] [-0] [-1] [file or special_file] .SH SUMMARY .I Slice slices up large data blocks into lots of individually named .I Slice files. The large data blocks may come from a file, or a Unix device, such as a tape drive, CDROM, or DVD. .I Slice was written to recover data from large intermittently faulty tape software drivers (not really faulty tapes though), & to write new intact images back to tape. (Do not use .I Slice to recover or manipulate floppies as it's better to use my .I valid instead). .SH DESCRIPTION .I Slice slices binary files into equal sized blocks (of specifiable size), placing data unchanged in incrementally named .I Slice files. .I Slice can also work backwards, joining slices into one large file. .sp 0.5 .I Slice passes any odd trailing bytes at the end of a large contiguous input file without either truncating or padding. Ditto if .I Slice is running in reverse mode by use of the -w flag), & writes out just the number of bytes read, with no padding. .sp 0.5 .I Slice was written to help rescue data from tapes, as the SCS driver has a nasty habit of occasionaly returning 1 byte too few ! (Only happens with certain kernels in multi user mode). .SH OPTIONS .PP All flags must precede file name (if a filename is given). .PP All flags may be in any order relative to each other. .PP Flag default values are shown in {brackets}, Flags available include :- .TP .B \-0 {off} .br Warn if every bit in every byte in a block is Zero. .br (This is just a warning, not a error causing failure of .I slice as some zero or 0xFF blocks may be genuine, especially in truly raw data. However, if there are many in non raw data, such as gzip compressed, or .vob or .avi files etc, it almost certainly indicate serious data corruption.) .TP .B \-1 {off} .br Warn if every byte in a block is 0xFF. .TP .B \-v {off} .br Verbose. .TP .B \-r {on} .br Read mode. On by default, but automatically turned off if -w is invoked. .TP .B \-w {off} .br Write mode Off by default. This option re-assembles the slices into a contiguous file, (same effect as using cat/append from a shell script, but much faster). .TP .B \-d {off} Delete .I Slice files after appending slices to output file, and delete sub directories once each sub directory has been emptied of slices. .TP .B \-n {off} .br Suppress the initial rewind that .I Slice normally does before both write or read of an image or device. .br Useful if the image on tape is not the first file on the tape. (IE if there is a prior tape label or other content). .br Note this option does Not suppress any auto rewind after .I Slice finishes; that is of course outside the control of .I Slice, & is a funtion of the operating system. On a Symmetric 375 use /dev/nrcst2 for that, (if EG you were using .I Slice to rescue multiple sequential files from one tape). .TP .B \-b Bytes_Per_Slice {8 * 1024} .br Change block size. If the last character is a 'k' or 'K', block size in bytes is multiplied by 1024, if b or B by 512, if w or W by 2. (For Symmetric dump tapes a block size of 10K should be specified. For cdroms & DVDs: "-b 2k") .\" this is what restore failed with on one of Chuck's tapes: .\" partial block read: 10239 should be 10240 .TP .B \-s Skip_blocks .br Skip/search past specified number of blocks. ( .I Slice sized blocks, not Unix system blocks) If used with -r, all directories are still created; if used with -w & -d, skipped files & containing directories are deleted. .TP .B \-- {off} .br Pipe usage. .br This option can be useful in avoiding large temporary files. Example: .br First read a tape with some sort of faulty [media/drive/software driver combination] twice or more into the same tree containing sl_ files, until all .bad files have obtained equivalent .ok files; then use .ti +5 slice -w -- | tar xf - .sp 0.5 Note there is a limitation (you could call it a bug!): You can't use tar to merge a tree to a pipe, then slicing the input pipe into 720K chunks for floppy disc images. .ti +5 ( cd somewhere ; tar cf - . ) | slice -- -b 720K .br causes error reports, because Unix pipes only allows up to 4K per pipe. .ti +5 ( cd somewhere ; tar cf - . ) | slice -- -b 4k .br Causes error reports on the Author's Unix, with .I Slice complaining only 2K reads are achievable from the pipe. .br This works on the Author's Unix: .ti +5 ( cd somewhere ; tar cf - . ) | slice -- -b 2k .br However the safer usage would be .ti +5 ( cd somewhere ; tar cf - . ) | slice -- -b 512 .br or .ti +5 ( cd somewhere ; tar -c -f - -b 8 . ) | slice -- -b 2k .br later I will add an .in +5 if (pipe_f /* only for pipes, because if reading scs tape direct, dont want to change behaviour */ ) while(..) .in -5 around the read .SH EXAMPLE Assume a problem: .in +4 .I Restore fails to read the 22 Mbyte Usr file on your only copy of a distribution tape from defunct SCS computer manufacturer. The device driver appears to occasionally lose one byte per block. (The reason .I Slice's was written !) .in -4 Solution: .in +4 Read tape twice, slicing data into lots of files, delete duplicates, patch bad sectors, re-assemble & write to a new file (either normal file or EG tape device). .in -4 Syntax to use: .in +4 .nf cd /big_disc_with_lots_of_space mkdir 1 2 cd 1 make -f ~/public_html/symmetric/tape/makefile skip_to_usr slice -b 1k -n cd ../2 make -f ~/public_html/symmetric/tape/makefile skip_to_usr slice -b 1k -n find . -type f -name \\*.bad -exec rm {} \\; find . -type f -exec cmpd -d {} ../1 \\; echo cmpd deletes duplicates find . -type f -name \\*.ok -exec rm {} \\; find ../1 -type f -name \\*.bad -exec rm {} \\; echo "Hope sufficient .ok files to replace all .bad files." tar cf - . | ( cd ../1 ; tar xf - ) cd ../1 ; rm -rf ../2 & slice -w -b 1k -d new_disc_copy_of_tape_file cd .. ; rm -rf 1 2 .fi .in -4 If you are unlucky and both reads failed to read the same slice, .I Slice will stop writing the large disc file at the point where a slice is missing, and will cease deleting slices at that point, you will then need to reslice the disc file, slice the tape file, compare, merge, and run .I slice -w again. .SH LIMITATIONS .in +2 .ti -2 .ft B Error Handling .ft R .in +2 .ti -2 - .I Slice has no means of recognising the failure of the SCS (Symmetric Computer Systems Inc.) tape driver that returns 0 on eg the third 20K 'read()' of a tape device (special file) with tape containing eg 50K, However, when reading a disc file, the system read() call returns a 10k value which .I Slice recognises and handles properly. .ti -2 - Trailing half blocks & Symmetric Streamer Cassette Driver Bug. Although Slice tries to deal with trailing remainders, and does so successfully when reading normal files (for instance writing 193 * 20K + 1 * 10K file if given 387 * 10240 bytes of file data input), when reading the cassette, the read(20K) system call fails to return the last 10K, a value of 0 is returned, Slice has no way of detecting this isn't immediately the end of file, and so the last 10k is not processed. .ti -2 - .I Slice (like dd) is normally pointed at eg /dev/acd0 or /dev/cd0 fto read a DVD, a single read error causes this to fail. If the DVD is mounted as an ISO 9660 file system, .I FreeBSD tries harder at error recovery, re-reading odd bad blocks. So if you'r hoping to use .I slice to recover bad blocks, then compare & delete & assemble a new DVD image on hard disk to then feed to .I dvdrip you're going the wrong route. Do not use slice for that, just mount the DVD, & use eg ( cd /cdrom ; .I tar cvf - . ) | .I tar xf - .br or .br .I cp -R /cdrom . .br to copy all files to hard disk, Then run .I dvdrip on the copied tree. .in -2 .ti -2 .ft B Sub Directories: .ft R .in +2 .ti -2 - All files are written to files placed in sub directories created by .I Slice within the current directory. No option exists to instruct .I Slice to deposit files directly in the current directory, (though you could (if perverse) establish a series of symbolic links to the current working directory, with names being those of the directories .I Slice would later seek to create!). The reason .I Slice uses sub directories is to avoid overloading the system; (an early version caused its SCS Unix 4.2 BSD host to run progressively slower for a day, then crash, only half way toward creating (in a single directory) the 50,000 file entries necessary to recover a damaged 25 Mbyte /usr tape into 512 byte files). .ti -2 - Naturally .I Slice will need many inodes if you use a small block size, so you may wish to use .I pwd, mount, & df -i to discover how many inodes are available. .ti -2 - No check is made on the number of sub directories created, as modern FreeBSD systems slicing 4G DVDs seem to have no problem. .in -2 .ft B Untested .ft R .in +2 .ti -2 - .I Slice has not yet been written/ tested for stdio/ pipes. .ti -2 - .I Slice has not been tested on MSDOS (warning DOS runs pipes to completion, creating a temporary file in the root directory of the current drive). .in -2 .ti -2 - Writing slices to a raw device that has a different blocking size to the existing slices is not supported (ie moving slices between different architectures could be problematic, if tape blocking size varies). .ti -2 - When writing short slices .I Slice will complain that the slice is short, & then try to write it anyway, if you are writing to a raw device such as tape, your driver may not support writing short blocks. .I Slice will not normally have created slices shorter than the specified block size, (apart from the last slice of all possibly), however if you have a deficient SCS tape driver that occasionally when .I Slice calls read(8192), returns 8191 & then 1, in that case you will have something like .in +4 .nf 8192 sl_00200/20005.ok 8191 sl_00200/20006.bad 1 sl_00200/20007.bad 8191 sl_00200/20008.ok .fn .in -4 in this case, if the slices are to be written to a raw device, do manual correction of the short block, & creation of a skipping/continuation block: .in +4 .nf cat sl_00200/20006.bad sl_00200/20007.bad > sl_00200/20006.ok rm sl_00200/20006.bad sl_00200/20007.bad cp /dev/null sl_00200/20007.ok .fn .in -4 .ti -2 On FreeBSD-5.3, accesing an Atapi CDROM via the SCSI software interface adaptor (yes really, it Is possible!): Read failures from bad media cause an EOF so .I Slice gets no more data delivered to it from the device, so .I Slice can't currently easily be used for data recovery. However on 5.3 I have seen succesive read runs return different data to a slice, so somewhere the drive or kernel is failing to detect errors. .ti -2 .I Slice is not a standard Unix command, it is unique to VSL/JHS systems with the Ournix overlay. .in -2 .SH EXIT CODE 0 if no error has occurred, else 1; also 1 after a user interrupt occurs during -b looping. .SH BUGS .in +2 .ti -2 - Rewind might not work. .ti -2 - When using -w .I Slice complains the file after last can't be found. .ti -2 - When using -d -w .I Slice doesnt delete the last dir. .ti -2 - When using -w .I Slice should but doesnt check if the .ok files are bigger than the device block size. .ti -2 - When using -w .I Slice only reads the .ok files, & doesnt write the last .bad file which is created if the previous .I Slice -r did a short read at end. Ie if all slice ecept last are 8K & last slice is 4K you will have to manually do something like: cat sl_05744/tp_0574414.bad >> /usr1/tmp/OUT.iso .in -2 .SH HOST OP SYSTEM This utility runs on Msdos 3.2 & BSD Unix 4.2 & up to (& beyond) FreeBSD-4.10 & FreeBSD-5.3 (Dec. 2005). .SH COPYRIGHT Program and Document Copyright Julian H. Stacey, Munich, December 1990. .so author.jhs .SH ENHANCEMENTS LATER Read sliced files back to tape. .SH FILES sl_%05d/sl_%7d.[ok or bad] .SH SEE ALSO cmpd(vsl), split(vsl), valid(vsl). .\ End of file