Allegro.cc - Online Community

Allegro.cc Forums » Programming Questions » Faster loading of pictures

This thread is locked; no one can reply to it. rss feed Print
Faster loading of pictures
A. van Patmos
Member #15,349
October 2013

Because my game loads many (300+ ATM, more to come) pictures of 1 to 4 KB each loading times have increased muchly.
If I search for PhysicsFS or physfs the latest threads are from Jan 2014. It makes me think I missed something like maybe store (under Files: https://www.allegro.cc/resource/Libraries/Files).
Is PhysicsFS the way to go? Does that speed up loading resources? is it a heavy weight library? (eg. I don't need its write access to archives).

Edgar Reynaldo
Member #8,592
May 2007
avatar

That sounds like a maintenance nightmare. It would be preferable to have a single (or a few) spritesheets with all the images on them already. It would make for much faster loading. Right now you're dealing with the overhead of opening hundreds of files when you could reduce that to a few. And that way you have less open points of possible failure as well.

EDIT
To answer your question, PhysFS might help in this instance, depending on how it works. If it loads a single file into memory, and then parses it, perfect. If however it has to decompress the archive every time it loads a file, it would be much worse for you.

A. van Patmos
Member #15,349
October 2013

One problem is that the depicted people cam stand still (20 by 46 pixels) but also crawl (46 by 12) (and run, walk, etc) so in a spritesheet all pics must be the maximum 46 by 46 and I need to calculate new offsets for every one.

( I was typing the above and realize I never thought about spritesheets containing differently sized pics. Like deviding a big bmp into 46 by 46 squares and outline every pic to the topleft. I'd still have to do this for the picture bmp and the transparency info bmp, then covert to png)

There's no way of loading from an archive like a tar? Retaining my pics/hero1 etc structure? The tar (like in WinRar choose 'store' for compression method, i.e. none) will be read fast if files a simply head-to-toe inside it.

Edgar Reynaldo
Member #8,592
May 2007
avatar

Well, you can test it easily enough.

Set the physfs_file_interface, put all your sprites in a zip in the root, and then access them the same way.

bamccaig
Member #7,536
July 2006
avatar

Technically you could have a spritesheet with variable sized/positioned images, but you'll need to store an "index" of them somewhere and you'll likely have waste in the image unless you get lucky, engineer it perfectly, or fragment your sprites within it. You can either store the index in the spritesheet itself as garbage data at a predictable position, or probably more easily in a separate custom file. Maintaining the file will likely be a pain though unless you can find a software tool that already meets you requirements. Or you take the time to develop one. Anything is possible. It's just a question of what's the better pay off.

Polybios
Member #12,293
October 2010

Besides loading times, it will also be faster to draw them if several pictures / sprites reside on one bitmap / texture. Minimizing texture switches is a real concern here.
Look for "texture atlas", "texture packing", there are solutions for that already available. You can stick to the ones not rotating the sprites for a start. Basically, you need some kind of table or xml or whatever to save some identifier for each sprite, its size and position on the large texture. For performance, it 'd be beneficial to group textures such that sprites needed at a time are on the same texture. If you use filtering, remember to check whether your texture packing tool has the ability to leave a 1px border around every sprite.

Chris Katko
Member #1,881
January 2002
avatar

I recently read an older book that said you could store sprite information... inside the bitmap itself. (That may be what Bam is suggesting.) It blew my mind. Basically, you use known colors (pure red, pure cyan, etc) to mark pixel points as "top left of sprite" / "bottom right of sprite", and other necessary features as needed. Then, your sprite parser checks for pixels instead of reading a metadata file. (I use an INI file with other object metadata. But it's quite laborious if you're talking hundreds or thousands of objects!) This method allows the artist to know exactly What-You-See-Is-What-You-Get. No reading position numbers and then writing them into an INI (or worse, source code!).

It's not the best solution for every case. I think if your sprites are ALWAYS a set size, like 32x32, then there's no point in hardcoding every size as if they could be different. And if an artist is copying or moving tiles around, they may forget to move the right "metadata pixels".

But the point is, there's another possible solution I hadn't ever considered.

-----sig:
“Programs should be written for people to read, and only incidentally for machines to execute.” - Structure and Interpretation of Computer Programs
"Political Correctness is fascism disguised as manners" --George Carlin

Edgar Reynaldo
Member #8,592
May 2007
avatar

I think that would be cool. You could encode names in the sprite border. And other fun stuff.(32 - 2)*4 - 1 = 119 bytes of char data in the top border alone!

Chris Katko
Member #1,881
January 2002
avatar

You could encode names in the sprite border.

The only problem there is, then you gotta figure out how to translate a user's actions like dropping pixels, into characters. Otherwise, I think some/many/most graphics formats already contain a metadata table for things like comments, ISO settings, etc.

My example simply had certain pixels mean concrete things. Like, start here, end here. Encoding actual variable data could be tricky.. but it might be possible to do elegantly..

-----sig:
“Programs should be written for people to read, and only incidentally for machines to execute.” - Structure and Interpretation of Computer Programs
"Political Correctness is fascism disguised as manners" --George Carlin

Edgar Reynaldo
Member #8,592
May 2007
avatar

Eric Johnson
Member #14,841
January 2013
avatar

We need a standardized atlas add on for allegro imo.

Aaannnnddd... We have a volunteer, everyone! Congratulations to Edgar for stepping up to the plate! :D

Chris Katko
Member #1,881
January 2002
avatar

I'd contribute. The issue is really, that I program in D. :P

But I could still contribute recommendations for API if someone wants to setup say, a Discord channel, and we could debate it.

-----sig:
“Programs should be written for people to read, and only incidentally for machines to execute.” - Structure and Interpretation of Computer Programs
"Political Correctness is fascism disguised as manners" --George Carlin

bamccaig
Member #7,536
July 2006
avatar

Edgar Reynaldo
Member #8,592
May 2007
avatar

Hold on guys. Let's wait til Monday to make any important decisions. ;)

Eric Johnson said:

Edgar Reynaldo said: said:

We need a standardized atlas add on for allegro imo.

Aaannnnddd... We have a volunteer, everyone! Congratulations to Edgar for stepping up to the plate! :D

I'd love to, but alas I've no experience in this matter. I don't know what a suitable packing algorithm might be. However, I have an idea, and it would be like a flow layout but better.

I'd contribute. The issue is really, that I program in D. :P

Well, it has to be in C for it to be a proper addon.

Chris Katko said:

But I could still contribute recommendations for API if someone wants to setup say, a Discord channel, and we could debate it.

I only have experience with Slack, but I assume it is similar. That would work. We would need threads for discussion at the least.

bamccaig said:

First we should elect a leadership committee and schedule a mandatory brainstorming session.

I elect committee by contribution. Contribute UML or API specs or rudimentary implementations with stub functions. If you want to be on the committee, you have to contribute something. Otherwise, it's all "let's do this great idea A", okay well why don't you?

Trezker
Member #1,739
December 2001
avatar

Start with making a good interface and crappy implementation.

Then someone will be irritated enough to make a better implementation.

Polybios
Member #12,293
October 2010

I have some old really crappy implementation of a texture packer I could probably supply. :P
I think there have been better ones referenced to on the forums, though.

First question would be what to do when sprites don't fit and how to decide a texture size (i.e. have it preset before attempting to pack and then just say: "sorry, doesn't fit", selecting an appropriate size (within some limit) automagically, produce several textures if sprites don't fit on a single one, etc.).

Also, I think, you'd usually want the packing to actually be done beforehand, e.g. at build time, so as an addon, it would probably be sufficient to just load a pre-packed spritesheet / atlas.

So maybe we could use an existing packer, just read the metadata it outputs and provide a subbitmap-creator or something like that?

bamccaig
Member #7,536
July 2006
avatar

I elect committee by contribution. Contribute UML or API specs or rudimentary implementations with stub functions. If you want to be on the committee, you have to contribute something. Otherwise, it's all "let's do this great idea A", okay well why don't you?

I was being sarcastic. ::) This is clearly a Monday project that will go nowhere. Software projects do not materialize out of forum threads. Software projects materialize out of some guy's basement because he needs to do something and couldn't find an existing solution.

Edgar is already committed to Eagle so it's unlikely he has time to spare on this unless it becomes a part of Eagle or he really needs this for a project he's already working on.

The same goes for everybody else. It would be neat to have it, but unless you have a real need to build it right now it's all just smoke and dreams. And it won't be collaborative project until somebody starts something, gets something working, and somebody else finds it easy or necessary to improve it.

Enough planning. Somebody GTFO and build something.

Edgar Reynaldo
Member #8,592
May 2007
avatar

bamccaig said:

Edgar is already committed to Eagle so it's unlikely he has time to spare on this unless it becomes a part of Eagle or he really needs this for a project he's already working on.

I already have rectangle union and subtraction and consolidation routines in eagle. Packing wouldn't be too big of a step I think.

Plus I need a good flow layout.

Go to: