Making a grabber for Mugen's SFF file format
zesensei

Hi,
Does anyone have ever do a grabber to load a Mugen's sff file format or fnt file format for Allegro or can help making one, for KOF91 project ( http://koflinux.sourceforge.net )?
Thanks a lot for the help.

zebellah

why you want to write a new grabber.
You can make plugins for Allegro's current grabber.

It's also possible that you can load your file without a plugin.

zesensei

I don't specially want to write a new grabber, nor plugins, nor specific function. I just want to know if somebody have ever do it and would like to share his code with me to save me lot of time writing one of this solution. Especially because I'm not really familiar with file structure loading.
If nobody never do such a thing I will publish it when I wrote it (time, time, time) as part of the KOF91 source code.
Thanks a lot for any help.

spellcaster

I'd like to help, but I can't find a file spec for either sff or fnt.
Care to share a link?

zesensei

Thanks a lot spellcaster,
Here is the définition of the formats:

Quote:

/*--| SFF file structure
|--------------------------------------------------*\
Version 1.01
HEADER (512 bytes)
------
Bytes
00-11 "ElecbyteSpr\0" signature [12]
12-15 1 verhi, 1 verlo, 1 verlo2, 1 verlo3 [04]
16-19 Number of groups [04]
20-24 Number of images [04]
24-27 File offset where first subfile is located [04]
28-31 Size of subheader in bytes [04]
32 Palette type (1=SPRPALTYPE_SHARED or 0=SPRPALTYPE_INDIV) [01]
33-35 Blank; set to zero [03]
36-511 Blank; can be used for comments [476]

SUBFILEHEADER (32 bytes)
-------
Bytes
00-03 File offset where next subfile in the "linked list" is [04]
located. Null if last subfile

04-07 Subfile length (not including header) [04]
Length is 0 if it is a linked sprite
08-09 Image axis X coordinate [02]
10-11 Image axis Y coordinate [02]
12-13 Group number [02]
14-15 Image number (in the group) [02]
16-17 Index of previous copy of sprite (linked sprites only) [02]
This is the actual
18 True if palette is same as previous image [01]
19-31 Blank; can be used for comments [14]
32- PCX graphic data. If palette data is available, it is the last
768 bytes.
*--------------------------------------------------------------------------*/

/*--| SND file structure
|--------------------------------------------------*\
Version 1.01
HEADER
------
Bytes
00-11 "ElecbyteSnd\0" signature [12]
12-15 4 verhi, 4 verlo [04]
16-19 Number of sounds [04]
20-23 File offset where first subfile is located. [04]
24-511 Blank; can be used for comments. [488]

SUBFILEHEADER
-------
Bytes
00-03 File offset where next subfile in the linked list is [04]
located. Null if last subfile.
04-07 Subfile length (not including header.) [04]
08-11 Group number [04]
12-15 Sample number [04]
08- Sound data (WAV)

*--------------------------------------------------------------------------*/

/*--| FNT file structure
|--------------------------------------------------*\
/*

  • Very simple file format, formed by concatenating a pcx file and a text

  • file together and prepending a header.

  • May be optimized for size by stripping the text file of comments before

  • adding it to the .fnt file. Be sure text data comes last in the file.

*/

Version 1.0
HEADER
------
Bytes
00-11 "ElecbyteFnt\0" signature
[12]
12-15 2 verhi, 2 verlo
[04]
16-20 File offset where PCX data is located.
[04]
20-23 Length of PCX data in bytes.
[04]
24-27 File offset where TEXT data is located.
[04]
28-31 Length of TEXT data in bytes.
[04]
32-63 Blank; can be used for comments.
[40]
*--------------------------------------------------------------------------*/

I can't myself explain everything and hope this will look simplest for you than for me.
Thanks a lot if you can help me

spellcaster

it seems to be pretty straight forward.
I'll see what I can do :)

zesensei

It will realy help me, thanks a lot.

spellcaster

Just a couple of questions:

Do you have some files I could test my loader with?
What output do you expect? For the SFF file, you could get a list or an array of BITMAPs + some extra data (like the (x,y) offset; group, index, etc.) is that what you want?

zesensei

You could download Mugen on my website:
http://zesensei.free.fr/mugen/tools/mug10414.exe It has fnt, sff, snd files.
For me a good result would be:
for sff and snd a linked list with a structure of this type: 

struct sprites{
int group;
int number;
int x_axis;
int y_axis;
int index;
BITMAP * image;
RGB * palette;
struct sprites * next; 
};

and same for sound.
For fnt it will be great it Fonts became Allegro font.
Thanks for the help

spellcaster

Ok, here's the sff loader.
Right now it exports a linked list... but you can change that easily so it returns an array. Just check the source.

#SelectExpand
1/** 2 loader.h 3 4 Mugen SFF Loader (Header) 5 written by Lennart Steinke (http://www.steinke.net) 6 */ 7#ifndef SFF_LOADER 8#define SFF_LOADER 9 10#include <allegro.h> 11 12typedef struct { 13 char magic[12]; 14 char version[4]; 15 int countGroups; 16 int countImages; 17 int offsetFirstSubFile; 18 int sizeOfSubfileHeader; 19 char palType; 20 char padding[3]; 21 char comment[476]; 22} SFFHeader; 23 24typedef struct { 25 int nextSubFile; 26 int sizeOfData; 27 short x; 28 short y; 29 short group; 30 short curIndex; 31 short prevIndex; 32 char reusePalette; 33 char comment[13]; 34 char pcxData[1]; 35} SFFSubFileHeader; 36 37typedef struct SFFEntryStruct { 38 short x, y, group, curIndex, prevIndex; 39 char isLinkedImage; 40 RGB* pal; 41 BITMAP* image; 42 char comment[13]; 43 44 struct SFFEntryStruct *next; 45} SFFEntry; 46 47#define SFF_LOAD_RET_TYPE SFFEntry* 48 49SFF_LOAD_RET_TYPE sffLoad(const char* filename); 50SFF_LOAD_RET_TYPE sffLoadFromMemory(void *memory); 51#endif

Here's the implementation of the loader:

#SelectExpand
1/** 2 loader.c 3 Mugen SFF Loader (source) 4 written by Lennart Steinke (http://www.steinke.net) 5 */ 6 7#include <stdio.h> 8#include "loader.h" 9 10int sff_memloadInt(unsigned char *data, int *ofs) { 11 int a,b; 12 a = data[*ofs]; 13 (*ofs)++; 14 b = data[*ofs]; 15 (*ofs)++; 16 return a + (b <<8); 17} 18 19RGB* sffLoadPaletteFromMemory(unsigned char *data) { 20 RGB* pal = (RGB*) malloc(sizeof(RGB) * 256); 21 int a, ofs; 22 ofs = 0; 23 for (a=0; a < 256; ++a) { 24 pal[a].r = data[ofs++] / 4; 25 pal[a].g = data[ofs++] / 4; 26 pal[a].b = data[ofs++] / 4; 27 } 28 return pal; 29} 30 31BITMAP* sffLoadPcxFromMemory(char* data) { 32 BITMAP *bmp = NULL; 33 34 /* skip the first 3 bytes */ 35 int ofs = 3; 36 char planes = data[ofs++]; 37 int width = -sff_memloadInt(data, &ofs); 38 int height = -sff_memloadInt(data, &ofs); 39 int bpp = 0; 40 int bytesPerLine = 0; 41 int x,y; 42 int value; 43 int count; 44 45 width += sff_memloadInt(data, &ofs) +1; 46 height+= sff_memloadInt(data, &ofs) +1; 47 48 /* skip 4 bytes (dpi) */ 49 ofs += 4; 50 51 /* skip 16 color palette */ 52 ofs += 48; 53 54 /* padding */ 55 ofs++; 56 57 58 bpp = data[ofs++] *8; 59 if (bpp != 8) { /* || bpp != 24) {*/ 60 return NULL; 61 } 62 63 bytesPerLine = sff_memloadInt(data, &ofs); 64 65 /* image data starts at ofs 128*/ 66 ofs = 128; 67 bmp = create_bitmap_ex(bpp, width, height); 68 for (y=0; y < height; ++y) { 69 x = 0; 70 while (x < bytesPerLine) { 71 value = data[ofs++]; 72 73 /* check if upper 2 bit are set */ 74 if ((value & 0xc0) == 0xc0) { 75 /* bits are set, that means the 76 lower 6 bit contain the repeat count, 77 and the color is stored in the next byte 78 */ 79 count = value & 0x3f; 80 value = data[ofs++]; 81 } else { 82 /* value contains the color already */ 83 count = 1; 84 } 85 if (bpp == 8) { 86 while (count > 0) { 87 if (x < bmp->w) { 88 bmp->line[y][x] = value; 89 ++x; 90 } 91 count--; 92 } 93 } 94 } 95 } 96 97 98 return bmp; 99} 100 101SFF_LOAD_RET_TYPE sffLoadFromMemory(void *memory) { 102 char* data = (char*) memory; 103 SFFHeader* header = (SFFHeader*) memory; 104 SFFSubFileHeader *sub = NULL; 105 int index = 0; 106 SFFEntry* first = NULL; 107 SFFEntry* cur = NULL; 108 109 SFFEntry** sprArray = NULL; 110 111 sprArray = calloc(header->countImages, sizeof(SFFEntry*)); 112 113 sub = (SFFSubFileHeader*) &data[header->offsetFirstSubFile]; 114 while (sub != NULL) { 115 if (first == NULL) { 116 first = (SFFEntry*) calloc(1, sizeof(SFFEntry)); 117 cur = first; 118 } else { 119 cur->next = (SFFEntry*) calloc(1, sizeof(SFFEntry)); 120 cur = cur->next; 121 } 122 cur->x = sub->x; 123 cur->y = sub->y; 124 cur->group = sub->group; 125 cur->curIndex = sub->curIndex; 126 cur->prevIndex = sub->prevIndex; 127 128 if (sub->sizeOfData == 0) { 129 // This is just a link to another image 130 cur->isLinkedImage = TRUE; 131 cur->image = sprArray[index]->image; 132 } else { 133 cur->isLinkedImage = FALSE; 134 cur->image = sffLoadPcxFromMemory(sub->pcxData); 135 memcpy(cur->comment, sub->comment, 13); 136 if (!sub->reusePalette) { 137 /* Tha palette is stored in the last 768 bytes 138 of the pcx data 139 */ 140 cur->pal = sffLoadPaletteFromMemory((unsigned char*) &(sub->pcxData[sub->sizeOfData - 768]) ); 141 142 } 143 } 144 sprArray[index] = cur; 145 146 /* 147 According to the docs, the last subfile is indicated by a 0. 148 During tests it became obvious that this is not the case. 149 Thus I'm checking both for 0 and check if the index is 150 smaller than the number of sprites in the file. 151 */ 152 ++index; 153 if (sub->nextSubFile != 0 && index < header->countImages) { 154 sub = (SFFSubFileHeader*) &data[sub->nextSubFile]; 155 } else { 156 sub = NULL; 157 } 158 } 159 160 /* NOTE: You might want to return the sprArray instead; 161 if so, don't free it here :) 162 */ 163 free(sprArray); 164 165 return first; 166} 167 168SFF_LOAD_RET_TYPE sffLoad(const char* filename) { 169 long size = file_size(filename); 170 long toGo = size; 171 long start = 0; 172 long count = 0; 173 FILE *f = NULL; 174 char *data = NULL; 175 SFF_LOAD_RET_TYPE ret = NULL; 176 177 if (size > 0) { 178 f = fopen(filename, "rb"); 179 data = malloc(size); 180 while (toGo > 0 && !feof(f)) { 181 count = fread(&(data[start]),1,toGo, f); 182 if (count == 0) { 183 exit(0); 184 } 185 186 toGo -= count; 187 start += count; 188 } 189 fclose(f); 190 ret = sffLoadFromMemory(data); 191 free(data); 192 } 193 return ret; 194}

And here a small demo:

#SelectExpand
1/** 2 testloader.c 3 4 Mugen SFF Loader (test) 5 written by Lennart Steinke (http://www.steinke.net) 6 */ 7 8#include <allegro.h> 9#include "loader.h" 10 11int setGfxMode(int w, int h) { 12 int bpp[] = {16, 15, 32, 24, 8, 0 }; 13 int a; 14 for (a=0; bpp[a] > 0; ++a) { 15 set_color_depth(bpp[a]); 16 if (set_gfx_mode(GFX_AUTODETECT_WINDOWED, w, h, 0, 0) >= 0) { 17 return 1; 18 } 19 } 20 return 0; 21} 22 23 24 25int main(int argc, char** argv) { 26 27 SFFEntry *first, *cur; 28 int lastKey = 0; 29 allegro_init(); 30 31 first = sffLoad(argc > 1 ? argv[1] : "intro.sff"); 32 33 if(!setGfxMode(640, 480)) { 34 allegro_message("oups!"); 35 exit(0); 36 } 37 install_keyboard(); 38 cur = first; 39 40 if (cur->pal) { 41 set_palette(cur->pal); 42 } 43 44 while (!key[KEY_ESC]) { 45 blit(cur->image, screen, 0, 0, cur->x, cur->y, cur->image->w, cur->image->h); 46 47 if (key[KEY_SPACE]) { 48 if (lastKey != KEY_SPACE) { 49 lastKey = KEY_SPACE; 50 51 if (cur->next) { 52 cur = cur->next; 53 } else { 54 clear(screen); 55 cur = first; 56 } 57 if (cur->pal) { 58 set_palette(cur->pal); 59 } 60 } 61 } else { 62 lastKey = 0; 63 } 64 } 65 66 67 return 0; 68} 69END_OF_MAIN()

Please note that the file specs above seem to be incorrect.
Some fields seem to have different sizes than listed. And in some cases the logic doesn't seem to be correct (like the way to determine the last subfile).

zesensei

Thanks a lot for the code, it works well.
I think linked list will be fine, if there's any problem I could easely convert it into array.
Could I use your code in KOF 91 project (it's opend source, you'll be mentionned in the developement team if you want)?
Do you think you can do the same thing for other formats?
If not, the sff management is already a huge part, thanks again.

spellcaster

Sure, just use it - that's why I wrote it ;)

Regarding the array: It's already there, I'm using it internally ofr index sprites - so if you want it, you could just change the return type to the array. That way you'd have both array (since it was returned) and the linked list (since the array members are still linked).

I'm working on the other formats. Just a question to the FNT format - the txt file is pretty simple, but for some chars the it looks weird... mainly for the [ and ] I guess. Not sure why. Is there any documentation for the TXT part of the FNT?
I think I could guess how it's composed... but that would be just a guess. :)

(oh... and please answer to this post, so I can add the new loaders in my next reply :))
Might take some time though... the allegro font structure is quite complex IIRC - so I might have to play around with it a bit.

zesensei

Thanks a lot,
here's a litle explanation of how the elecbyte's fntmaker works, it could help about the format:

====================
III. Fntmaker
====================

The fonts generated by fntmaker work for 2000.06.27 and newer versions
of M.U.G.E.N.

pcx file - Supply a pcx file which contains all the characters in your font
alphabet, arranged from left to right in a single row. The pcx
file must be an 8-bit color sprite.
txt file - Copy a txt file from work/font in custom.zip, and edit it to fit
your font's pcx file. The first part is are the font settings.
It is followed by character and offset pairs. The character is a
letter or symbol in your pcx file you wish to represent. The
offset is the starting x location within the pcx file of the
character. For example,
B 55
means that character 'B' begins at an offset of 55 pixels from the
left. Semicolon is treated as a comment character. To put in
entries for special characters such as a semi-colon, you need to
use a hex sequence representing the ASCII number of the
character. Hex sequences start with 0x. For example, a semicolon
(ASCII value 5B) starting at pixel 100 would have an entry:
0x5B 100

Fonts with more than one color use "color banks" in their palette to store
multiple color arrangements. Each color bank in the palette has a number of
palette entries equal to the value of the "Color" parameter in the font's
txt file. The first color bank starts at color 255, and its entries are
laid out in reverse palette order. The next color bank's first palette entry
starts immediately following the first, and so on. For example, a font with
16 colors has its first bank's entries from 255 to 240. The second bank has
entries 239 to 224 and so on. The number of banks you can have is limited by
the palette space.

I hope it could help you.

spellcaster

Thanks for that info... makes sense now :)

Regarding the multiple color banks... allegro can't handle that.
I can copy the data into an allegro FONT and return a struct containing both FONT and palette .. but the actual color bank switch has to be made programmtically - if you're using hicolor display you could change the palette, set the changed pal and blit it. If you're in a 8 bit mode, things get a bit more messy.

You could either "clone" the fonts to get different versions (in different color sets) or create totally new font rendering functions.

Cloning would be faster, more easy to implement but does of course waste some memory.

What do you prefer?

Kitty Cat

I know you weren't directing this at me, but here's an idea. What if you load all the font data as true-color bitmap (according to their color banks), then do an optimized palette generation on it and make it an 8-bit bitmap to cut up for the font? Granted it won't look the same, but if you're gonna put it in FONT, you won't have much choice. :-/

spellcaster

The font is already an 8bit bmp. It has it's own palette (256 colors). The point is that you'd have to add an offset to each pixel to support the color bank feature.

Kitty Cat

Hmm.. I'm not sure I understand the color banks portion then. What does it do? Whats its purpose?

spellcaster

Assume you have a font with a gradient red->yellow using 8 colors.
The font is using the colors 247-255.
These 8 colors are color bank 0.
You could now use another 8 indices in the palette to define a blue->white gradient. Those colors would now be color bank 1.

I guess that mugens font drawing allows you to specify the color bank... so you could draw a text with font A, color bank 0 (red->yellow text) or using color bank 1 (blue->white text).

Allegro's glyph drawing functions don't work that way.

So, I could either generate several fonts (not much efford, not really memory wasting) or create functions mimicing the color bank switching.

I think, I'll go for the creation of multiple fonts :)

zesensei

I think the creation of clone fonts is a good idea, as it will become possible to have palettes that are not specify with the original font picture's pal.
Spellcaster is right, mugen call a font and then its color bank to show it with right palette. I think a call to an array of fonts (or linked list) with an index that is correspinding to the bank used to make the clone will be a good idea.
A little explaination too, Mugen use fixed fonts and proportional fonts. I don't understand really the differences, but it's one of the attribute you specify when you create a font. I don't know if it's really important, but I think I've seen something like that (fixed and proportionnal) in Allegro doc.
Thanks a lot for the help.

Evert
Quote:

Mugen use fixed fonts and proportional fonts. I don't understand really the differences

In a fixed width font all characters are the same width (for instance, W and I). In a proportional font, W is wider than I.

spellcaster: if you really wanted to, you could probably hack the FONT vtable so that it can draw the colourbanked fonts. That's probably going to be a lot of work for little obvious gain over what you're proposing.

spellcaster

Don't think that this is needed. Mugen fonts are pretty small - I doubt it'll be a big overhead if one would simply clone the fonts.

I'll add a 

FONT* fntGetColorBankFont( FntFont* fnt, int bank);

to my code. Or something similar. If somebody can come up with a good name for that function, I'd be grateful :)

Expect the font functions today (sorry... I was pretty busy yesterday).

Regarding the proportional fonts: What evert said... in theory, at least. It seems like a lot of mugen fonts are labeled as "proportional" but are in fact fixed size... but that's not a big problem, IMO.

I might need to bug Bruce about the wave formats, though :)

zesensei

Thanks again,
FONT* fntGetColorBankFont( FntFont* fnt, int bank); could become get_cfont( FntFont* fnt, int bank); for get colored font??
I'm not really good at function baptism...
Thanks for the help

spellcaster

Here's the loader.
I decided to create different palettes instead of different fonts - so this solution will only work in a high color mode.
If you want to support 8 bit, you need to clone the font and change the actual pixel data of the glyphs.

zesensei

Thanks again, and again,
It works good for some fonts. Just a detail, the code didn't compile on my Win98 with Mingw32.
It was because of a detail: declaration of variables MUST be before calling any function with my GCC. So for the testloader.c I got to put allegro_init() after declarations. Same problem in loader.c where I got to declare int count, and char * txt at the beginning of the function in order to make it compile.
I learn a lot with your code fully documented, really clean and structured.

For the bugs, here are some fonts that didn't work with the testloader prog. And if I pass the same name as the exemple font you give in your zip file (name1.fnt) as an argument color banks seems to disappear. Pretty weird...

I'll look later the how of this... It works with some fnt, a great beginning.

spellcaster

Get a new gcc - it should work with the later 3.0 versions... but to be honest, it was an mistake...
I normally try to declare my vars at the beginning. Regarding the testloader - I used only the font testing part and had the other stuff REMed. At the end I removed the comments and made a test run.

Since it worked, I was assuming it would be ok.

Quote:

Same problem in loader.c where I got to declare int count, and char * txt

Sorry. But, as I said, it'll work in any c99 compiler (say the later gccs). But I changed that - it wasn't intentional in the first place.
There's no need to put txt at the beginning, though. Just swap the char* txt; with the line above that.

Quote:

And if I pass the same name as the exemple font you give in your zip file (name1.fnt) as an argument color banks seems to disappear. Pretty weird...

Check the testloader code. There's a bug in name1.fnt. If you split it (I think I have the split.c code in that zip as well) and look into the txt.txt file, you'll see that it has colors=1 in it. And that's wrong. The numbers of colors should be 16 for that font.
That's why I added a small hack inside the loader - if it is using the default font, then it'll override the number of colors.

EDIT:
Just in case I didn't put the split stuff in the zip, here it is:

1#include <stdio.h>
2#include <stdlib.h>
3 
4int readIntFrom(FILE *f, int ofs) {
5 int i;
6 fseek(f, ofs, SEEK_SET);
7 fread(&i, 4, 1, f);
8 return i;
9}
10 
11int main(int argc, char** argv) {
12
13 FILE *f = fopen(argv[1], "rb");
14 FILE *o = NULL;
15 int pcxOfs = readIntFrom(f, 16);
16 int pcxLen = readIntFrom(f, 20);
17 int txtOfs = readIntFrom(f, 24);
18 int txtLen = readIntFrom(f, 28);
19
20 char *buffer = NULL;
21
22 printf("pcx: %i, %i\ntxt: %i, %i\n", pcxOfs, pcxLen, txtOfs, txtLen);
23
24 o = fopen("pcx.pcx", "wb");
25 buffer = malloc(pcxLen);
26 fseek(f, pcxOfs, SEEK_SET);
27 fread(buffer, 1, pcxLen, f);
28 fwrite(buffer, 1, pcxLen, o);
29 fclose(o);
30 free(buffer);
31
32 o = fopen("txt.txt", "wb");
33 buffer = malloc(txtLen);
34 fseek(f, txtOfs, SEEK_SET);
35 fread(buffer, 1, txtLen, f);
36 fwrite(buffer, 1, txtLen, o);
37 fclose(o);
38 free(buffer);
39
40 fclose(f);
41}

zesensei

Thanks for the reply.
For the name1.fnt that's my fault, I haven't read that this fount was malformed.
For GCC I'll do so later, it could explain some warning I did'nt have on others computers (or on Linux). Note: <mem.h> should not be include on my Mandrake 9.2, it didn't found it.
But what about fnts I put in my last post? Did you try them, did it work or crash (segmentation fault), like on my computer?
Do you have an idea fo the reason fo that crash?

spellcaster

Fixed.
There was a bug in my pcx loading code.

zesensei

Better and better...
All work good now. Thanks a lot.
Do you plan to do .snd as well?
I think I could release a new version of KOF 91 in some hours with your code and it will looks like far better than other versions...

spellcaster

I'll look into the snd format as well, but I'm not an expert with waveforms. On the other hand, I do have some spare time now...

Anyway, please reply to this post, so I can answer again as soon as I have something for you :)

zesensei

Well got to reply...
I've made a new version of KOF 91 with fnt support (a really limited one, as I haven't fix every call to text for now).
It'll be ready for dowload (binaries and sources) on my website : http://zesensei.free.fr
It's already nicer than ever, thanks to you.

Thread #342381. Printed from Allegro.cc