Allegro.cc - Online Community

Allegro.cc Forums » Allegro Development » ZIP file reader encapsulated in PACKFILE functions

This thread is locked; no one can reply to it. rss feed Print
 1   2 
ZIP file reader encapsulated in PACKFILE functions
Tobi Vollebregt
Member #1,031
March 2001

A few days ago I downloaded the 4.1.18 WIP. I noticed the packfile functions used vtables since 4.1.18. So immediately I thought finally zip support!

Anyway, I've been writing the attached zip file reader today, and I need some people to test it and I need some feedback, so it would be great if you could just make a little test with it and report back here whether you had any problems.

You need Allegro 4.1.18 and zlib installed.

documentation (also in zipfile.h):

1/* Usage example:
2 * Suppose you have a file archive.zip containing the files
3 * foo and bar, you would read from them as follows:
4 *
5 * PACKFILE *zip, *foo, *bar;
6 *
7 * zip = pack_fopen_zip_dir("archive.zip");
8 *
9 * if (!zip) { show error message }
10 *
11 * foo = pack_fopen_zip(zip, "foo");
12 * bar = pack_fopen_zip(zip, "bar");
13 * pack_fclose(zip);
14 *
15 * if (!foo || !bar) { show error message }
16 *
17 * { read from packfiles foo and bar like
18 * you read from any other packfile }
19 *
20 * pack_fclose(foo);
21 * pack_fclose(bar);
22 */

EDIT: attached new version
EDIT: attached new version

________________________________________
website || Zipfile reader @ Allegro Wiki || Download zipfile reader

razor
Member #2,256
April 2002
avatar

Sweet, I'll try it later, but it sounds really cool.

Whoooo Oregon State University

Peter Hull
Member #1,136
March 2001

I've thought for a while that this was the way to go. Then we won't need the dat utility any more; we can just use (Win)Zip.

Nice one Tobi!

Pete

Avenger
Member #4,550
April 2004

I've seen a few games using zip for compression.. It is really great that we now have an addon to do it:). Thanks a lot!

Elias
Member #358
May 2000

Add a complete example, I'm too lazy to try it without one :P Sounds very useful in any case.

--
"Either help out or stop whining" - Evert

Evert
Member #794
November 2000
avatar

This sounds simply awesome. I'll grab a copy of it and see if I can check it out.

A J
Member #3,025
December 2002
avatar

have i missed the point ?
why use compression ?

___________________________
The more you talk, the more AJ is right. - ML

Richard Phipps
Member #1,632
November 2001
avatar

Tobi Vollebregt
Member #1,031
March 2001

Thanks!

I'll do some more testing by myself today and then I'll upload it to the depot. I'll also add an example.

And btw, the main reason IMO to use zip for game data is because of easy maintenance, not necessarily compression (you would zip the download anyway). Just make a dir with game data, right click->add to zip and voilà, there's your datafile.

________________________________________
website || Zipfile reader @ Allegro Wiki || Download zipfile reader

Kitty Cat
Member #2,815
October 2002
avatar

Quote:

not necessarily compression (you would zip the download anyway)

It's still nice and polite to leave as small of a foot print on disk as reasonably possible.

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

A J
Member #3,025
December 2002
avatar

hard drive space is cheap.
too cheap.. so cheap it is not worth the effort of compressing individual bits.
i no longer see it as useful, rather as a distraction and possible cause of problems. not part of the solution.

___________________________
The more you talk, the more AJ is right. - ML

Avenger
Member #4,550
April 2004

I would like to use uncompressed zips in my program.. mainly because that would make my game easier to modify.. And it is much better to have one big file instead of thousands of small ones

Dennis
Member #1,090
July 2003
avatar

Quote:

it is not worth the effort of compressing individual bits.

an "individual bit" can't be compressed any further:P
Sorry for the OT but this silly joke felt so burning under my nails.;D

razor
Member #2,256
April 2002
avatar

It is nice to have all your files in one file. That way you don't flood peoples system with files, I know that I've noticed windows deals with lots of files much slower than a single large file.

Whoooo Oregon State University

Tobi Vollebregt
Member #1,031
March 2001

I've updated the zip attached to my first post. Fixed some unicode handling with trace, added some assertions and an example.c and example.zip

btw, I'm thinking about a way to read the directory entries from the zip file. Now I would like to know, would you think it's useful to know more about a file than name+size (like crc, attributes, compressed size, comment, etc.)? And what API would you prefer?

Like this? Or something else?

int zip_get_num_dir_entries(PACKFILE *f);
const char *zip_get_name(PACKFILE *f, int i);
long zip_get_size(PACKFILE *f, int i);

________________________________________
website || Zipfile reader @ Allegro Wiki || Download zipfile reader

Elias
Member #358
May 2000

It could be interesting to use an API similiar to allegro's directory API:

al_find_first, al_find_next, al_find_close, for_each_file_ex

But then, I'd actually prefer to get the number of entries and then use an index, myself :)

Hm, and this makes me wonder.. haven't some people worked on designs for a simple virtual file system? Seems like a good time to look at such designs now - they already must include an API to list files and get various information about them.

--
"Either help out or stop whining" - Evert

Indeterminatus
Member #737
November 2000
avatar

That's some nice work you did there! Great job! About the additional information (crc, attributes etc.) I'd say it wouldnt hurt to have them ready ;).

_______________________________
Indeterminatus. [Atomic Butcher]
si tacuisses, philosophus mansisses

Kitty Cat
Member #2,815
October 2002
avatar

Quote:

hard drive space is cheap.
too cheap.. so cheap it is not worth the effort of compressing individual bits.
i no longer see it as useful, rather as a distraction and possible cause of problems. not part of the solution.

I whole heartedly disagree. Hard drive space may be cheaper, but not everyone is able to upgrade as needed. The more games that go by uncompressed, the more hard drive space will be needed, thus forcing the user to upgrade to bigger drives. Cheap or not, it's still $$$ not everyone has or is able to spend. "It's cheap" is a poor excuse because "it's cheap" is still too expensive for some. Do you think I'd be stuck with a Geforce 2 MX video card if I could get a Geforce 4 Ti because "it's cheap"?

Durring development is a different matter of course, but if HD space was such a non-issue, why use PNG, MP3/Ogg, and MPG/AVI/Ogg at all when TGA, WAV, and uncompressed BMPs have been around forever (and are directly supported by Allegro) and serve the same purpose? Now, if, and that's a big if, all your game data is precompressed PNG and Ogg (and even FLAC), then I can see compressed zip files as being unneeded (and potentially hindering), but your average game (especially average Allegro game) has plenty of uncompressed data.

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

Tobi Vollebregt
Member #1,031
March 2001

The API is now like this:

int zip_get_num_dir_entries(PACKFILE *f);
char *zip_get_name(PACKFILE *f, int i, char *dest);
long zip_get_size(PACKFILE *f, int i);

I'm gonna send a mail to matthew to add it to the resource dir. I think the latest version (only on my HD at this moment) is fully unicode-aware now (even when debugging) and all others things seems to be fine. If you want to know the crc or other file props I'll just let you hack into my sources :P, I don't really feel the need to add it.

________________________________________
website || Zipfile reader @ Allegro Wiki || Download zipfile reader

Steve Terry
Member #1,989
March 2002
avatar

Quote:

an "individual bit" can't be compressed any further

Lzip will prove you wrong!

___________________________________
[ Facebook ]
Microsoft is not the Borg collective. The Borg collective has got proper networking. - planetspace.de
Bill Gates is in fact Shawn Hargreaves' ßî+çh. - Gideon Weems

Richard Phipps
Member #1,632
November 2001
avatar

Jose Cuervo
Member #5,197
November 2004

Has anyone thought of adding tar.gz support? Because I generally have smaller files with it, like 1/5 smaller.

Thomas Fjellstrom
Member #476
June 2000
avatar

tar.gz would be more difficult, as the directory format (tar) is compressed globably, so it would have to be completely uncompressed and loaded into memory before anything can be read.

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Daniel Schlyder
Member #257
April 2000
avatar

And if you want better compression, you should use LZMA.

Thomas Fjellstrom
Member #476
June 2000
avatar

Or if you want a truely fast decompressor, try LZO. On a p1 133 it does about 16MB/s, LZMA does about 8MB/s on a p3 1ghtz...

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

 1   2 


Go to: