A few days ago I downloaded the 4.1.18 WIP. I noticed the packfile functions used vtables since 4.1.18. So immediately I thought finally zip support!

Anyway, I've been writing the attached zip file reader today, and I need some people to test it and I need some feedback, so it would be great if you could just make a little test with it and report back here whether you had any problems.

You need Allegro 4.1.18 and zlib installed.

documentation (also in zipfile.h):

EDIT: attached new version
EDIT: attached new version

Sweet, I'll try it later, but it sounds really cool.

I've thought for a while that this was the way to go. Then we won't need the dat utility any more; we can just use (Win)Zip.

Nice one Tobi!

Pete

I've seen a few games using zip for compression.. It is really great that we now have an addon to do it:). Thanks a lot!

Add a complete example, I'm too lazy to try it without one Sounds very useful in any case.

This sounds simply awesome. I'll grab a copy of it and see if I can check it out.

have i missed the point ?
why use compression ?

To save space?

Thanks!

I'll do some more testing by myself today and then I'll upload it to the depot. I'll also add an example.

And btw, the main reason IMO to use zip for game data is because of easy maintenance, not necessarily compression (you would zip the download anyway). Just make a dir with game data, right click->add to zip and voilà, there's your datafile.

It's still nice and polite to leave as small of a foot print on disk as reasonably possible.

hard drive space is cheap.
too cheap.. so cheap it is not worth the effort of compressing individual bits.
i no longer see it as useful, rather as a distraction and possible cause of problems. not part of the solution.

I would like to use uncompressed zips in my program.. mainly because that would make my game easier to modify.. And it is much better to have one big file instead of thousands of small ones

an "individual bit" can't be compressed any further:P
Sorry for the OT but this silly joke felt so burning under my nails.;D

It is nice to have all your files in one file. That way you don't flood peoples system with files, I know that I've noticed windows deals with lots of files much slower than a single large file.

I've updated the zip attached to my first post. Fixed some unicode handling with trace, added some assertions and an example.c and example.zip

btw, I'm thinking about a way to read the directory entries from the zip file. Now I would like to know, would you think it's useful to know more about a file than name+size (like crc, attributes, compressed size, comment, etc.)? And what API would you prefer?

Like this? Or something else?

int zip_get_num_dir_entries(PACKFILE *f);
const char *zip_get_name(PACKFILE *f, int i);
long zip_get_size(PACKFILE *f, int i);

It could be interesting to use an API similiar to allegro's directory API:

al_find_first, al_find_next, al_find_close, for_each_file_ex

But then, I'd actually prefer to get the number of entries and then use an index, myself

Hm, and this makes me wonder.. haven't some people worked on designs for a simple virtual file system? Seems like a good time to look at such designs now - they already must include an API to list files and get various information about them.

That's some nice work you did there! Great job! About the additional information (crc, attributes etc.) I'd say it wouldnt hurt to have them ready .

I whole heartedly disagree. Hard drive space may be cheaper, but not everyone is able to upgrade as needed. The more games that go by uncompressed, the more hard drive space will be needed, thus forcing the user to upgrade to bigger drives. Cheap or not, it's still $$$ not everyone has or is able to spend. "It's cheap" is a poor excuse because "it's cheap" is still too expensive for some. Do you think I'd be stuck with a Geforce 2 MX video card if I could get a Geforce 4 Ti because "it's cheap"?

Durring development is a different matter of course, but if HD space was such a non-issue, why use PNG, MP3/Ogg, and MPG/AVI/Ogg at all when TGA, WAV, and uncompressed BMPs have been around forever (and are directly supported by Allegro) and serve the same purpose? Now, if, and that's a big if, all your game data is precompressed PNG and Ogg (and even FLAC), then I can see compressed zip files as being unneeded (and potentially hindering), but your average game (especially average Allegro game) has plenty of uncompressed data.

The API is now like this:

int zip_get_num_dir_entries(PACKFILE *f);
char *zip_get_name(PACKFILE *f, int i, char *dest);
long zip_get_size(PACKFILE *f, int i);

I'm gonna send a mail to matthew to add it to the resource dir. I think the latest version (only on my HD at this moment) is fully unicode-aware now (even when debugging) and all others things seems to be fine. If you want to know the crc or other file props I'll just let you hack into my sources , I don't really feel the need to add it.

Lzip will prove you wrong!

With Quantum Mechanics it could..

Has anyone thought of adding tar.gz support? Because I generally have smaller files with it, like 1/5 smaller.

tar.gz would be more difficult, as the directory format (tar) is compressed globably, so it would have to be completely uncompressed and loaded into memory before anything can be read.

And if you want better compression, you should use LZMA.

Or if you want a truely fast decompressor, try LZO. On a p1 133 it does about 16MB/s, LZMA does about 8MB/s on a p3 1ghtz...

Heh. You know, I've read about that algorithm before, and I only remembered after seeing the pic on the main page. "Space Grade Technology".

But that shouldn't overshadow Zip support; it's awesome. I'm grateful for the people who made it possible. Thanks Tobi, vtable authors, et. al. I'll do some testing.

We should add this right into allegro, after it is more complete. Then that would make allegro dependent on zlib, but if the license allows it, we could include the needed zlib code right in allegro

Nah, the better route would be to include this into an extra distribution, which contains allegro + some useful addons, all packed up as a single download and easy to install.

As far as that goes, what about the licensing of Allegro? zlib isn't restrictive It's what I release under.

zlib is more restrictive than Allegro, but only in stuff that doesn't matter. (Ie, don't abuse our generosity vs please don't abuse our generosity.)

Allegro could take zlib and bundle it, but then Allegro's license would have be to be modified to be "Giftware, with some parts under the zlib license".

I think that may be a good idea myself.

As Elias said, I think this should be a supported or approved or bundled or gift wrapped or whatever addon, not part of the vanilla Allegro distribution.
Depending on zlib shouldn't be too much of a problem for the bigger package. We'd want PNG support in there anyway.

Yes.. maybe with 4.2.0, someone will create such a thing. A distribution which has zlib, libpng, loadpng, AllegroGL, glyphkeeper, jpgalleg, whatever else, all bundled together. The make process then also would build the examples and documentation of all the addons. But there still would be no need to move zlib into allegro's CVS repository.. everything would stay nicely modular (except the allegro core itself, which still is quite big ).

Maybe there should be a stripped down Allegro core then since this will get pretty big in terms of file size.

There will always be a vanila allegro distro.