Tar for OpenComputers

[mpmxyz 27]

Introduction

 

If you already know 'tar' and tar archives:

Yes, this is a tar implementation for OpenComputers.

It supports viewing, extracting and even creating archives on tape and file.

It supports most of the original tar format and the long name part of the ustar extension.

You can now skip the introduction.

 

Now for everyone else:

'tar' is a very old archive file format originating from early UNIX days.

It is an uncompressed archive format with the built in ability to store everything from files and directories to symbolic links or even device files together with their file permissions, last modification time/date and owner.

In easier words: You could backup and restore an entire UNIX file system without loosing anything. (not even modification times)

Because of this utility it's still widely used today. (most of the time compressed by another format, e.g. *.tar.gz files)

 

Why doesn't have OpenComputers have tar?

Well, there already is one program, but that only supports extracting.

I wanted to be able to create tar files, too.

There is a huge advantage in being able to:

You can now exchange whole filesystems with a single pastebin / wget operation.

Updating a bunch of files is also a lot easier: You can't forget a file if you extract an archive.

(compared to deleting and copying files manually)

 

Man Page

NAME
  tar - tar archiver for OpenComputers

SYNOPSIS
  tar <function letter> [other options] FILES...

DESCRIPTION
  allows working with tar archives in OpenComputers
  It supports the standard tar format + ustar extension.
  (not every feature but enough for most uses)
  
  That makes it a nice tool to exchange multiple files and directories between OpenComputers and "real computers".
  Tar a bunch of files, use pastebin / wget to transmit the archive and extract it at destination.
  Since this program is also able to create tar archives it is possible in both directions!
  
  When you want read from / write to a file you have to use the option -f / --file.
  It then uses the first file given as the archive.
  Else it uses the primary Computronics tape drive. (with the archive starting at the current position)
  
  A function flag is required to indicate what you want to do with the archive.
  Further options can be used to modify the behaviour of the program.
  
  By default the program does not print anything except errors.
  Use --verbose to make it 'talk'!
  
  By default it overwrites every existing file.
  Take care or use options like --keep-old-files!

FUNCTION LETTERS
  -c --create
    create a new tar archive
    
  -r --append
    append to the tar archive
    (tape only unless io.open is going to support rb+ mode)
    
  -t --list
    list contents of tar archive
    
  -x --extract --get
    extract the archive to the current directory
    
  --help
    show usage information
    
OTHER OPTIONS
  -f --file
    tell tar to use the first file parameter as a tar archive
    (When creating an archive, it will be the output file.)
    Without this option the primary tape drive is used instead.
    
  --address=ADDRESS
    tell tar to use the tape drive with the given address instead of the primary one
    
  -h --dereference
    follow symlinks (treating them as files / directories)
    
  --exclude=FILE1;FILE2
    exclude a semicolon separated list of files
    The archive file is automaticly excluded when writing to an archive.
    When using the --dereference option it might be necessary to exclude the archive manually.
    (i.e. when following a symlink directs the program to include the archive itself)
    The program will throw an error if an input file grew while being copied to the archive.
    (So it at least terminates when it tries to put the archive into itself.)
    
  -v --verbose
    print out what the program is doing
    If the library auto_progress is installed it also displays nice progress bars on large files.
    
  --keep-old-files
    throw an error when extracting and an output file already exists
  --skip-old-files
    do not overwrite existing files, but continues without errors
  --keep-newer-files
    overwrite existing files only if they are older than the one within the archive
  
  --dir=DIR
    change the reference directory i.e. to extract a tar file in a subdirectory
    (The original tar option is "-C DIR". But shell.parse() does not support an option+parameter combination.)
  
EXAMPLES
  tar --create --file all.tar / --exclude=/mnt
    creates an archive 'all.tar' containing the whole file system except "/mnt".
    
  tar -xf -v another.tar
    extracts the contents of 'another.tar' while showing the user what it's doing.
    
  tar -t
    displays the contents of the tape archive.
    The current position is read as the beginning of the archive.

Depencencies

 

Software

tar requires only standard libraries from OpenComputers.

It additionally uses but does not require the auto_progress library to display progress bars in verbose mode.

(-v --verbose)

It has been tested with OpenComputers 1.4.1.14.

 

Hardware

RAM: min. 256 KiB, recommended: 392 KiB

(rare, but possible out of memory errors are possible on minimum)

 

Did you know: The 'real' tar was originally created as a tape archiver.

Therefore I implemented support for Computronics tape drives.

Usage is quite similar except that you leave out the "-f" option.

 

Installation

 

Simply move the file to the 'bin' directory of your choice.

I recommend the directory "/home/bin".

 

Download (last update: 13.04.17)

 

github: program

github: man page

github: tar archive for easy installation

 

Ingame:

wget 'https://raw.githubusercontent.com/mpmxyz/ocprograms/master/home/bin/tar.lua'
wget 'https://raw.githubusercontent.com/mpmxyz/ocprograms/master/usr/man/tar.man'
wget 'https://raw.githubusercontent.com/mpmxyz/ocprograms/master/tars/tar.tar'

OR

oppm install mpm.tar

Known Issues

 

It does not support every version or extension of the tar file format.

Implementing them all would be too much work.

But if there is a feature you would like to see, feel free to tell me.

 

Another side note: I should write more in detail what is supported and what not.

 

Appending to tar files on a file system does not work because you would need to use the "rb+" mode to open the file.

That's currently not possible in OpenComputers and workarounds would have to include copying the file instead of simply working in place.

 

tar currently does not detect link cycles and would therefore follow them when creating an archive.

In that case it will terminate because it filled the whole filesystem. (You have been warned!)

 

You can only use the primary tape drive. (I simply didn't create an option. Maybe later...)

 

It currently doesn't check if you have enough storage space available. (Maybe later...)

[cinder1992 1]

This, plus Computronics tapes, is my new backup system. hands down.

 

they didn't call tar files Tape ARchive for no reason.

 

(BTW, there is a lua version of DEFLATE out there, maybe .tar.zip could become a thing for OC?)

[mpmxyz 27]

You mean this one?

https://github.com/OpenPrograms/Magik6k-Programs/blob/master/libdeflate/deflate.lua

That would at least enable extracting tar.gz files. (no compressing though...)

I could add this with an optional dependency.

If it isn't too complex I might even write another program + library to do both compressing and uncompressing.

(The file format doesn't look as complicated as *.tar; I just need to find a good reference for all the algorithms involved.)

In either case I'd let you know. ;-)

PS: If I decide to create such a library I'd also split the *.tar program into a program and a library. To make installation easier I'd have to create an installer program, too. (So much on my TODO list now. xD)

[Krutoy242 26]

Thank you for using this extractor for your projects!

Im used it for crunch and was very glad how simple this works. Much more userful then downloading many files from github.

Maybe i will use this for my own public projects, promoting your program. Dont forget to add links in source files to your pages (including e-mail and forum page) for feedback and bugreports.

 

 

Also, i found that if i just call tar without parameters if throw white error with stack traceback. I thought programs should just show help when called without parameters.

 

[mpmxyz 27]

Yeah, the traceback has been in there since I had to fix some nasty bugs.

 

Update (10.06.15): made traceback on error optional via --debug option

 

I also took the time to add a small header to all my source files.

I'm also researching more into Deflate since I introduced LZ77 compression to crunch.

It actually is the first step of Deflate. The huffman encoding would be the second one.

(I'd also like to include some more efficient algorithms for compression. -> It'll take some time.)

[dgelessus 26]

re. deflate: There is a new item in the works, the Data Card, which provides various hash functions as well as base64 and deflate codecs in component form. At the moment it is still an open pull request to the OC repo, but it seems to be mostly finished from the technical side. No idea when it will get merged, probably once the comment section is done arguing about how the corresponding Lua libs should be distributed

[mpmxyz 27]

Update (13.04.17)

-added --address option to specify used tape drive