al_load_bitmap_f and custom FILE_INTERFACE
kovarex

Hello,
I'm trying to save some allegro images directly in my save game data file.

I have Deserialiser/Serialiser class that works as stream and is basically wrapped FILE* f, so I thought this won't be hard.

I used the custom data pointer to store pointer to the the (De)serliaser and mapped all the methods of the FILE_INTERFACE.

 this->fileInterface = new ALLEGRO_FILE_INTERFACE;
  this->fileInterface->fi_fopen = &AllegroFileToDeserialiserConnection::fi_fopen;
  this->fileInterface->fi_fclose = &AllegroFileToDeserialiserConnection::fi_fclose;
  this->fileInterface->fi_fread = &AllegroFileToDeserialiserConnection::fi_fread;
  this->fileInterface->fi_fwrite = NULL;
  this->fileInterface->fi_fflush = &AllegroFileToDeserialiserConnection::fi_fflush;
  this->fileInterface->fi_ftell = &AllegroFileToDeserialiserConnection::fi_ftell;
  this->fileInterface->fi_fseek = &AllegroFileToDeserialiserConnection::fi_fseek;
  this->fileInterface->fi_feof = &AllegroFileToDeserialiserConnection::fi_feof;
  this->fileInterface->fi_ferror = &AllegroFileToDeserialiserConnection::fi_ferror;
  this->fileInterface->fi_fclearerr = &AllegroFileToDeserialiserConnection::fi_fclearerr;
  this->fileInterface->fi_fungetc = NULL;
  this->fileInterface->fi_fsize = &AllegroFileToDeserialiserConnection::fi_fsize;
  this->file = al_fopen_interface(this->fileInterface, (const char*)(&serialiser), "rb");

On the last line I use trick, I use the file name parameter to actually pass the pointer to the serialiser that will be used.

Saving of the bitmap worked fine, and it was saved in nice single call of fi_fwrite (with length of 50kb).

When I tried to load the bitmap, it just didn't work, I observed several calls of ftell (I just return 0, as it calls in the beginning), then reads and fseeks.

After that the result bitmap has 0 width 0 height (and it was saved as 2048X2048 bitmap), and not only that, the stream it reads from is left in some inconsistent state (It doesn't read exactly the same as it wrote to the stream).

Any ideas?
Can this work at all? Am I not using some obsolete method that is not working anymore?
Thanks for any pointers.

In the worst case, I can save those bitmaps as separate files, but I would really like to avoid that, it would be messy.

beoran

Allegro is a C library. I suggest you use plain C functions, structs and malloc to avoid the C++ insanity. That said, the "trick" with (const char*)(&serialiser) only eems to work by accident. I nreality it just can't work well like that.

I quote the docs:

The fi_open function must allocate memory for whatever userdata structure it needs. The pointer to that memory must be returned; it will then be associated with the file. The other functions can access that data by calling al_get_file_userdata on the file handle. If fi_open returns NULL then al_fopen will also return NULL.

What could probably work is to make fi_open call new Serializer(filename) and return that.

kovarex

Well I see it the other way around :)
I want to wrap the C insanity in C++ objects as soon as possible :)
Rest of the game is completely in C++ (several MB of codebase now) so I hardly want to rewrite it to C :)

Yes, I use the al_get_file_userdata and reinterpret it to the serialiser. This works fine, I checked that the object I get is the correct one.

I don't really need any other data then the pointer itself, so I don't need to allocate any memory.

I don't want to/can't use new serialiser, because I want to save the data of the picture(s) to the opened and half-filled file.

These are the (static) methods used by the interface:

#SelectExpand
1void* AllegroFileToDeserialiserConnection::fi_fopen(const char* path, const char* mode) 2{ 3 return (void*)path; 4} 5 6size_t AllegroFileToDeserialiserConnection::fi_fread(ALLEGRO_FILE* f, void* ptr, size_t size) 7{ 8 MapDeserialiser* deserialiser = (MapDeserialiser*)al_get_file_userdata(f); 9 deserialiser->freadOrThrow(ptr, size); 10 return size; 11} 12 13void AllegroFileToDeserialiserConnection::fi_fclose(ALLEGRO_FILE*) 14{ 15 // Nothing to clear, we just have pointer to Deserialiser 16} 17 18bool AllegroFileToDeserialiserConnection::fi_fflush(ALLEGRO_FILE*) 19{ 20 throw std::runtime_error("fflush called on deserialiser adapter"); 21 return false; 22} 23 24int64_t AllegroFileToDeserialiserConnection::fi_ftell(ALLEGRO_FILE*) 25{ 26 return 0; 27} 28 29bool AllegroFileToDeserialiserConnection::fi_ferror(ALLEGRO_FILE*) 30{ 31 throw std::runtime_error("ferror called on deserialiser adapter"); 32 return false; 33} 34 35void AllegroFileToDeserialiserConnection::fi_fclearerr(ALLEGRO_FILE*) 36{ 37 throw std::runtime_error("clearerror called on deserialiser adapter"); 38} 39 40off_t AllegroFileToDeserialiserConnection::fi_fsize(ALLEGRO_FILE*) 41{ 42 return -1; 43} 44 45bool AllegroFileToDeserialiserConnection::fi_fseek(ALLEGRO_FILE* f, int64_t offset, int whence) 46{ 47 return ((MapDeserialiser*)al_get_file_userdata(f))->seek(offset, whence); 48} 49 50bool AllegroFileToDeserialiserConnection::fi_feof(ALLEGRO_FILE *f) 51{ 52 return false; 53}

Elias

Are you sure there is no seek to the end before the ftell? Usually it's called to determine the size of a file so returning 0 won't work.

kovarex

This is the order of calls

fi_fopen
fi_fseek(offset = 0, whence = 1 (Current))
fi_fsize() (I return -1, as can't be determined)
fi_fread(size = 88)
fi_fseek(offset = 0, whence = 0 (Set?))
fi_fread(size = 22)
fi_fseek(offset = 0, whence = 0 (Set?))
fi_fread(size = 18)
fi_fseek(offset = 0, whence = 0 (Set?))
fi_fseek(offset = 0, whence = 0 (Set?))
fi_fread(size = 44)
fi_fseek(offset = -44, whence = 1)

The problem might be the fsize as well maybe.
I could wrap the file more.
While saving, I would just save it all into memory first instead to file directly, then I would save the count of bytes and the memory.

While loading, I would load the whole batch from memory, and those functions would just fiddle in the reserved memory for the file.

This way, I could return the value of fsize (unknown otherwise, when using it as stream) and I would also don't mind that it wouldn't leave the file marker at the right position. (Not even mentioning that the (De)Serialiser, will have also different sources than just file where seek will not be possible).

I'm afraid, that this might not be solvable at all, and I would have to save/load those pictures somewhere aside the data file, but I wouldn't be happy with it at all.

Thomas Fjellstrom

If it's your own serialized format, just stick the length in the data stream and slurp that up when "opening" the file. now you have something you can return as fsize.

Matthew Leverton

You can read/write multiple packed files in a single stream, but there can be problems due to the image loaders not necessarily being native to Allegro and not playing nicely with the concept.

An image loader may very well expect to be able to seek to the end of a file, back up, read a few bytes, seek to the beginning of the file, etc. If your custom I/O treats "seek to beginning" as "rewind to the start of my multi-file stream," it will fail with some loaders.

To get around it, you can use a file slice (a thin wrapper over any file):

ALLEGRO_FILE *fp_custom = open_custom_stream();

int32_t size = al_fread32le(fp_custom);

ALLEGRO_FILE *fp_image = al_fopen_slice(fp_custom, size, "r");
ALLEGRO_BITMAP *bmp = al_load_bitmap_entry(fp_image, ".png");
al_fclose(fp_image);

al_fclose(fp_custom);

When closing the slice, the outer stream will advance to the end of the slice. This requires that the outer supports random access.

Note that technically, this applies to writing files as well. A writer could seek to the beginning of the file and overwrite the wrong portion. You can use an expandable, writable file slice to avoid that.

Finally, while you could do this all in memory, there's no reason when writing that you couldn't store 0x00000000, write the file (to a slice), close the slice, go back fill in the size, and then seek to end of file.

beoran

I don't mean you should rewrite your game in C, just that you can use it where it makes sense, like, in interfacing with Allegro. No point in "wrapping" C, it seems a case of wrapping what you don't like.

Anyway, if I think a bit more about what you are trying to do then I think al_fopen_interface is the wrong solution for your problem. Off the top of my head, I can think of two ways that could be useful. Either save the image to a temporary file, read that back in, and write it out to your output stream. Or, use the memfile addon to write the file to a zeroed out block of memory, and then write that block to your output file.

Edit and on reading back the data , you can certainly use the memfile addon, if you did write the length of the image data in front of it. Then you can just allocade and read in the image data into a buffer and wrap that in a memfile.

kovarex

I can't express how much am I thankful for your advices. I had to do few changes, but the beast is now working.

The slicer works differently than I thought, it still calls fseeks on the custom stream, but yes, it sets the file position to the end after fclose is called, so it is working correctly.

There is one main problem I had to fix from the start (stupid me for not realizing it earlier):

When the whence was CUR (relative), it was working correctly.
But when the whence was SET (absolute), my call to fseek of the custom stream set the position relative to the start of the whole stream, not to the beggining of the part of the stream containing the image (the slice).

I solved it by creating custom variable called shift, I set it to 0 at the start, and I update it along any operation (read, seek).

Then, I used this shift variable to convert the seek with absolute set to the call of the relative seek, I can also use it to simulate the ftell function properly.

#SelectExpand
1int64_t AllegroFileToDeserialiserConnection::fi_ftell(ALLEGRO_FILE *f) 2{ 3 return ((AllegroFileToDeserialiserConnection*)al_get_file_userdata(f))->shift; 4} 5 6bool AllegroFileToDeserialiserConnection::fi_fseek(ALLEGRO_FILE* f, int64_t offset, int whence) 7{ 8 AllegroFileToDeserialiserConnection* instance = ((AllegroFileToDeserialiserConnection*)al_get_file_userdata(f)); 9 if (whence == ALLEGRO_SEEK_CUR) 10 { 11 instance->shift += offset; 12 if (instance->shift < 0) 13 throw std::runtime_error("Shift before beginning of slice part."); 14 return instance->deserialiser->seek(offset, SEEK_CUR); 15 } 16 else if (whence == ALLEGRO_SEEK_SET) 17 { 18 if (offset < 0) 19 throw std::runtime_error("Seek set before beginning of slice part."); 20 int64_t diff = offset - instance->shift; 21 instance->shift = offset; 22 return instance->deserialiser->seek(diff, SEEK_CUR); 23 } 24 else 25 throw std::runtime_error("wrong fseek type"); 26}

Thank you for the help, I got really frustrated and now I can move to the more creative tasks of the game making again :)

Matthew Leverton
kovarex said:

I solved it by creating custom variable called shift, I set it to 0 at the start, and I update it along any operation (read, seek).

Yeah, that's what the slice routine does too. It just remaps seeks, which is why you can only have one slice open at a time and cannot use the parent stream.

Remember that this can happen when writing too. I'm pretty sure I ran across at least one of the image saving routines on one of the platforms that seeks around.

kovarex

I would expect that, but it doesn't, I'm using the slice, and it still calls the SET version of seek.

Thread #612856. Printed from Allegro.cc