Hoardr: Flickr as a cloud backup service

After years of being relegated to nothing more than a container for whacky cat antics, it turns out the humble Gif is capable of more than meets the eye. Specifically, it is able to carry more than just image data – you can actually append whatever you want to a Gif and still have it function as normal. This has a fairly scary implication, since it means any Gif on the internet could theoretically be hiding malicious code or other files that you want nothing to do with. A slightly less nefarious and more interesting use case lies in the fact that we can get files into or past a system that may not usually accept that filetype. And that’s what Hoardr does – it packs whatever files you want into gifs and turns your Flickr account into a cloud backup service.

Hoardr uses zip archives in particular due to the nature of both filetypes. Simply changing the file extension of a combined file from “.gif” to “.zip” allows it to be unzipped as usual.

If you’d like to play around with this idea but want nothing to do with breaking Flickr’s T.O.S ( keep reading for that ), you can use the following commands.

Create a gif/zip combination through concatenation:

cat your_gif_file.gif your_zip_file.zip > output_file.gif 

Change the gif into a readable zip:

mv output_file.gif output_file.zip

Unzipping the archive, to retrieve your files:

unzip output_file.zip

 Why?

You’re probably wondering why or if this is at all worthwhile, and the truth is it’s more novelty than anything else. However, I have been in Mexico for the past two months and just recently filled up my hard drive. I need to make some room and don’t feel like spending any money. I could pay for a bit of space on a cloud storage service, but that doesn’t sound like fun. Also, there may be rules here against spending your taco budget on computer junk.

 How

Here is a brief overview of why this trick works. Most of this is based off Wikipedia articles and brief pieces of the Gif and Zip specs. Let me know if something I’ve written here doesn’t seem right.

The Gif: Gif files are encoded with a few different data blocks that tell the decoder how the file should be handled. These include a header, a screen descriptor, the image data, some other stuff, and the trailer. This last one – the trailer – is the only block that really matters to our application. It delimits the end of the file, letting the decoder know the gif file is done and it can stop reading.

In ASCII the raw data for a mostly empty GIF looks like this:

GIF89a\x14\x00\x14\x00\x80\x00\x00\xFF\xFF\xFF\x00\x00\x00!\xF9\x04
\x01\x00\x00\x00\x00,\x00\x00\x00\x00\x14\x00\x14\x00\x00
\x02\x11\x84\x8F\xA9\xCB\xED\x0F\xA3\x9C\xB4\xDA\x8B\xB3
\xDE\xBC\xFB\xAF\x15\x00;

“GIF89a” is the header and “\x00;” is the trailer.

Concatenating a zip archive to this gif and again viewing the ASCII formatted data results in the following:

GIF89a\x14\x00\x14\x00\x80\x00\x00\xFF\xFF\xFF\x00\x00\x00!\xF9
\x04\x01\x00\x00\x00\x00,\x00\x00\x00\x00\x14\x00\x14\x00\x00\x02
\x11\x84\x8F\xA9\xCB\xED\x0F\xA3\x9C\xB4\xDA\x8B\xB3\xDE\xBC
\xFB\xAF\x15\x00;PK\x03\x04\x14\x00\b\x00\b\x00%\xB0\xF8D\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\b\x00\x10\x00test.txtUX\f
\x00\x884\xD5Su\xC8\xD1S\xF5\x01\x14\x00\v\xC9\xC8,V\x00\xA2D\x85
\x92\xD4\xE2\x1\x85\xB4\xCC\x9CT=\x00PK\a\bG\xFEj\xEC\x14\x00\x00
\x00\x14\x00\x0\x00PK\x01\x02\x15\x03\x14\x00\b\x00\b\x00%\xB0
\xF8DG\xFEj\xEC\x14\x00\x00\x00\x14\x00\x00\x00\b\x00\f\x00\x00\x00
\x00\x00\x00\x00\x00@\xA4\x81\x00\x00\x00\x00test.txtUX\b\x00\x884
\xD5Su\xC8\xD1SPK\x05\x06\x00\x00\x00\x00\x01\x00\x01\x00B\x00
\x00\x00Z\x00\x00\x00\x00\x00

The first chunk, from the header to the trailer is exactly the same (“GIF89a” to the first semicolon). From there on out we have the data belonging to the zip archive. As explained earlier, the Gif decoder will only read this first chunk and completely ignore the rest. Cool!

The Zip: Now for the zip file. We understand now that the concatenated data above can still be read as a Gif file, but you’re probably wondering how we can do the reverse – read it as a Zip file. And that’s actually the answer: Zips are read in reverse. How freaking cool. The first chunk of Zip data is stored at the end of the data string and is called the Central Directory. It stores the byte offsets for the files stored in the archive. Going backwards this is followed by the file signatures, which each end in 0x4b50, or “PK” in ASCII. Just as the Gif decoder ignores all data past the trailer, the Zip reader ignores anything after the last file signature.

After we do the concatenation of Zip to Gif we are left with an awesome Jekyll and Hyde / Frankenstein file. Because one is read forwards and the other backwards, there is no problem interpreting it as either filetype. Tell your filesystem it’s a Gif and it has no problem playing or opening it; tell it it’s a Zip and you can unzip as usual:
screencast

 Hoardr

Hoardr itself is a simple Ruby class that takes advantage of the above trick and pushes the resulting Gif to a Flickr account. I decided on Flickr because they recently gave all users one terabyte of free space for photo storage – perfect for backing up a hard drive!

Some notes on how it works and usage:

Next I plan on porting the functionality to OS X as a GUI app as a way of introducing myself to Swift.

If you want to use or contribute to Hoardr you can check out the source and documentation here.

 
19
Kudos
 
19
Kudos

Now read this

Shopify For WordPress

I just finished up and released a WordPress plugin for pulling and display products from Shopify web stores. If you’d like to check it out and jump into using it you can find it on Github right here. If you’d like to know a little more... Continue →