Making a grabber for Mugen's SFF file format

Making a grabber for Mugen's SFF file format

zesensei

Member #4,410

March 2004

Hi,
Does anyone have ever do a grabber to load a Mugen's sff file format or fnt file format for Allegro or can help making one, for KOF91 project ( http://koflinux.sourceforge.net )?
Thanks a lot for the help.

KOF 91 Project

zebellah

Member #4,275

January 2004

why you want to write a new grabber.
You can make plugins for Allegro's current grabber.

It's also possible that you can load your file without a plugin.

zesensei

Member #4,410

March 2004

I don't specially want to write a new grabber, nor plugins, nor specific function. I just want to know if somebody have ever do it and would like to share his code with me to save me lot of time writing one of this solution. Especially because I'm not really familiar with file structure loading.
If nobody never do such a thing I will publish it when I wrote it (time, time, time) as part of the KOF91 source code.
Thanks a lot for any help.

KOF 91 Project

spellcaster

Member #1,493

September 2001

I'd like to help, but I can't find a file spec for either sff or fnt.
Care to share a link?

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

Thanks a lot spellcaster,
Here is the définition of the formats:

Quote:

/*--| SFF file structure
|--------------------------------------------------*\
Version 1.01
HEADER (512 bytes)
------
Bytes
00-11 "ElecbyteSpr\0" signature [12]
12-15 1 verhi, 1 verlo, 1 verlo2, 1 verlo3 [04]
16-19 Number of groups [04]
20-24 Number of images [04]
24-27 File offset where first subfile is located [04]
28-31 Size of subheader in bytes [04]
32 Palette type (1=SPRPALTYPE_SHARED or 0=SPRPALTYPE_INDIV) [01]
33-35 Blank; set to zero [03]
36-511 Blank; can be used for comments [476]

SUBFILEHEADER (32 bytes)
-------
Bytes
00-03 File offset where next subfile in the "linked list" is [04]
located. Null if last subfile

04-07 Subfile length (not including header) [04]
Length is 0 if it is a linked sprite
08-09 Image axis X coordinate [02]
10-11 Image axis Y coordinate [02]
12-13 Group number [02]
14-15 Image number (in the group) [02]
16-17 Index of previous copy of sprite (linked sprites only) [02]
This is the actual
18 True if palette is same as previous image [01]
19-31 Blank; can be used for comments [14]
32- PCX graphic data. If palette data is available, it is the last
768 bytes.
*--------------------------------------------------------------------------*/

/*--| SND file structure
|--------------------------------------------------*\
Version 1.01
HEADER
------
Bytes
00-11 "ElecbyteSnd\0" signature [12]
12-15 4 verhi, 4 verlo [04]
16-19 Number of sounds [04]
20-23 File offset where first subfile is located. [04]
24-511 Blank; can be used for comments. [488]

SUBFILEHEADER
-------
Bytes
00-03 File offset where next subfile in the linked list is [04]
located. Null if last subfile.
04-07 Subfile length (not including header.) [04]
08-11 Group number [04]
12-15 Sample number [04]
08- Sound data (WAV)

*--------------------------------------------------------------------------*/

/*--| FNT file structure
|--------------------------------------------------*\
/*

Very simple file format, formed by concatenating a pcx file and a text
file together and prepending a header.
May be optimized for size by stripping the text file of comments before
adding it to the .fnt file. Be sure text data comes last in the file.

Version 1.0
HEADER
------
Bytes
00-11 "ElecbyteFnt\0" signature
[12]
12-15 2 verhi, 2 verlo
[04]
16-20 File offset where PCX data is located.
[04]
20-23 Length of PCX data in bytes.
[04]
24-27 File offset where TEXT data is located.
[04]
28-31 Length of TEXT data in bytes.
[04]
32-63 Blank; can be used for comments.
[40]
*--------------------------------------------------------------------------*/

I can't myself explain everything and hope this will look simplest for you than for me.
Thanks a lot if you can help me

KOF 91 Project

spellcaster

Member #1,493

September 2001

it seems to be pretty straight forward.
I'll see what I can do

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

It will realy help me, thanks a lot.

KOF 91 Project

spellcaster

Member #1,493

September 2001

Just a couple of questions:

Do you have some files I could test my loader with?
What output do you expect? For the SFF file, you could get a list or an array of BITMAPs + some extra data (like the (x,y) offset; group, index, etc.) is that what you want?

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

You could download Mugen on my website:
http://zesensei.free.fr/mugen/tools/mug10414.exe It has fnt, sff, snd files.
For me a good result would be:
for sff and snd a linked list with a structure of this type:

struct sprites{
int group;
int number;
int x_axis;
int y_axis;
int index;
BITMAP * image;
RGB * palette;
struct sprites * next; 
};

and same for sound.
For fnt it will be great it Fonts became Allegro font.
Thanks for the help

KOF 91 Project

spellcaster

Member #1,493

September 2001

Ok, here's the sff loader.
Right now it exports a linked list... but you can change that easily so it returns an array. Just check the source.

#SelectExpand
  1/**
  2 loader.h
  3 
  4 Mugen SFF Loader (Header)
  5 written by Lennart Steinke (http://www.steinke.net) 
  6 */
  7#ifndef SFF_LOADER
  8#define SFF_LOADER
  9
 10#include <allegro.h>
 11
 12typedef struct {
 13    char magic[12];
 14    char version[4];
 15    int  countGroups;
 16    int  countImages;
 17    int  offsetFirstSubFile;
 18    int  sizeOfSubfileHeader;
 19    char palType;
 20    char padding[3];
 21    char comment[476];
 22} SFFHeader;
 23
 24typedef struct {
 25    int   nextSubFile;
 26    int   sizeOfData;
 27    short x;
 28    short y;
 29    short group;
 30    short curIndex;
 31    short prevIndex;
 32    char  reusePalette;
 33    char  comment[13];   
 34    char  pcxData[1];
 35} SFFSubFileHeader;
 36
 37typedef struct SFFEntryStruct {
 38    short   x, y, group, curIndex, prevIndex;
 39    char    isLinkedImage;
 40    RGB*    pal;
 41    BITMAP* image;
 42    char    comment[13];
 43    
 44    struct SFFEntryStruct *next;
 45} SFFEntry;
 46
 47#define SFF_LOAD_RET_TYPE SFFEntry*
 48
 49SFF_LOAD_RET_TYPE sffLoad(const char* filename);
 50SFF_LOAD_RET_TYPE sffLoadFromMemory(void *memory);
 51#endif

Here's the implementation of the loader:

#SelectExpand
   1/**
loader.c
Mugen SFF Loader (source)
written by Lennart Steinke (http://www.steinke.net) 
*/
   6
   7#include <stdio.h>
   8#include "loader.h"
   9
  10int sff_memloadInt(unsigned char *data, int *ofs) {
  int a,b;
  a = data[*ofs];
  (*ofs)++;
  b = data[*ofs];
  (*ofs)++;
  return a + (b <<8);
  17}
  18
  19RGB* sffLoadPaletteFromMemory(unsigned char *data) {
  RGB* pal = (RGB*) malloc(sizeof(RGB) * 256);
  int  a, ofs;
  ofs = 0;
  for (a=0; a < 256; ++a) {
      pal[a].r = data[ofs++] / 4;
      pal[a].g = data[ofs++] / 4;
      pal[a].b = data[ofs++] / 4;
  }
  return pal;
  29}
  30
  31BITMAP* sffLoadPcxFromMemory(char* data) {
  BITMAP *bmp = NULL;
  
  /* skip the first 3 bytes */
  int     ofs = 3;
  char    planes = data[ofs++];
  int     width  = -sff_memloadInt(data, &ofs);
  int     height = -sff_memloadInt(data, &ofs);
  int     bpp  = 0;
  int     bytesPerLine = 0;
  int     x,y;
  int     value;
  int     count;
  
  width += sff_memloadInt(data, &ofs) +1;
  height+= sff_memloadInt(data, &ofs) +1;
  
  /* skip 4 bytes (dpi) */
  ofs += 4;
  
  /* skip 16 color palette */
  ofs += 48;
  
  /* padding */
  ofs++;
      
      
  bpp  = data[ofs++] *8;
  if (bpp != 8) { /* || bpp != 24) {*/
      return NULL;
  }
  
  bytesPerLine = sff_memloadInt(data, &ofs);
  
  /* image data starts at ofs 128*/
  ofs = 128;
  bmp = create_bitmap_ex(bpp, width, height);
  for (y=0; y < height; ++y) {
      x = 0;
      while (x < bytesPerLine) {
          value = data[ofs++];
          
          /* check if upper 2 bit are set */
          if ((value & 0xc0) == 0xc0) {
              /* bits are set, that means the
                 lower 6 bit contain the repeat count,
                 and the color is stored in the next byte
              */
              count = value & 0x3f;
              value = data[ofs++];
          } else {
              /* value contains the color already */
              count = 1;
          }
          if (bpp == 8) {
              while (count > 0) {
                  if (x < bmp->w) {
                      bmp->line[y][x] = value;
                      ++x;
                  }
                  count--;
              }
          }
      }
  }
  
  
  return bmp;
  99}
 100
 101SFF_LOAD_RET_TYPE sffLoadFromMemory(void *memory) {
  char*      data       = (char*) memory;
  SFFHeader* header     = (SFFHeader*) memory;
  SFFSubFileHeader *sub = NULL;
  int        index      = 0;
  SFFEntry*  first      = NULL;
  SFFEntry*  cur        = NULL;
  
  SFFEntry**  sprArray   = NULL;
  
  sprArray = calloc(header->countImages, sizeof(SFFEntry*));
  
  sub = (SFFSubFileHeader*) &data[header->offsetFirstSubFile];
  while (sub != NULL) {        
      if (first == NULL) {
          first = (SFFEntry*) calloc(1, sizeof(SFFEntry));
          cur   = first;
      } else {
          cur->next = (SFFEntry*) calloc(1, sizeof(SFFEntry));
          cur = cur->next;
      }
      cur->x = sub->x;
      cur->y = sub->y;
      cur->group = sub->group;
      cur->curIndex = sub->curIndex;
      cur->prevIndex = sub->prevIndex;        
      
      if (sub->sizeOfData == 0) {
          // This is just a link to another image
          cur->isLinkedImage = TRUE;            
          cur->image = sprArray[index]->image;
      } else {            
          cur->isLinkedImage = FALSE;
          cur->image = sffLoadPcxFromMemory(sub->pcxData);
          memcpy(cur->comment, sub->comment, 13);
          if (!sub->reusePalette) {
              /* Tha palette is stored in the last 768 bytes
                 of the pcx data
              */
              cur->pal = sffLoadPaletteFromMemory((unsigned char*) &(sub->pcxData[sub->sizeOfData - 768]) );
              
          }
      }
      sprArray[index] = cur;
      
      /*
      According to the docs, the last subfile is indicated by a 0.
      During tests it became obvious that this is not the case.
      Thus I'm checking both for 0 and check if the index is 
      smaller than the number of sprites in the file.
      */
      ++index;
      if (sub->nextSubFile != 0 && index < header->countImages) {
          sub = (SFFSubFileHeader*) &data[sub->nextSubFile];
      } else {
          sub = NULL;
      }                
  }
  
  /* NOTE: You might want to return the sprArray instead;
     if so, don't free it here :)
  */
  free(sprArray);
  
  return first;
 166}
 167
 168SFF_LOAD_RET_TYPE sffLoad(const char* filename) {
  long size  = file_size(filename);
  long toGo  = size;
  long start = 0;
  long count = 0;
  FILE *f    = NULL;
  char *data = NULL;
  SFF_LOAD_RET_TYPE ret = NULL;
  
  if (size > 0) {
      f = fopen(filename, "rb");
      data = malloc(size);
      while (toGo > 0 && !feof(f)) {
          count  = fread(&(data[start]),1,toGo, f);
          if (count == 0) {
              exit(0);
          }
              
          toGo  -= count;
          start += count;
      }
      fclose(f);        
      ret =  sffLoadFromMemory(data);
      free(data);
  }
  return ret;
 194}

And here a small demo:

#SelectExpand
  1/**
testloader.c

Mugen SFF Loader (test)
written by Lennart Steinke (http://www.steinke.net) 
*/
  7
  8#include <allegro.h>
  9#include "loader.h"
 10
 11int setGfxMode(int w, int h) {
  int bpp[] = {16, 15, 32, 24, 8, 0 };
  int a;
  for (a=0; bpp[a] > 0; ++a) {
      set_color_depth(bpp[a]);
      if (set_gfx_mode(GFX_AUTODETECT_WINDOWED, w, h, 0, 0) >= 0) {
          return 1;
      }
  }
  return 0;
 21}
 22
 23
 24
 25int main(int argc, char** argv) {
 26
  SFFEntry *first, *cur;
  int lastKey = 0;
  allegro_init();    
  
  first = sffLoad(argc > 1 ? argv[1] : "intro.sff");
  
  if(!setGfxMode(640, 480)) {
      allegro_message("oups!");
      exit(0);
  }
  install_keyboard();
  cur = first;
  
  if (cur->pal) {
      set_palette(cur->pal);
  }
  
  while (!key[KEY_ESC]) {
      blit(cur->image, screen, 0, 0, cur->x, cur->y, cur->image->w, cur->image->h);
      
      if (key[KEY_SPACE]) {
          if (lastKey != KEY_SPACE) {
              lastKey = KEY_SPACE;
                              
              if (cur->next) {
                  cur = cur->next;
              } else {
                  clear(screen);
                  cur = first;
              }
              if (cur->pal) {
                  set_palette(cur->pal);
              }
          }
      } else {
          lastKey = 0;
      }
  }
      
  
  return 0;
 68}
 69END_OF_MAIN()

Please note that the file specs above seem to be incorrect.
Some fields seem to have different sizes than listed. And in some cases the logic doesn't seem to be correct (like the way to determine the last subfile).

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

Thanks a lot for the code, it works well.
I think linked list will be fine, if there's any problem I could easely convert it into array.
Could I use your code in KOF 91 project (it's opend source, you'll be mentionned in the developement team if you want)?
Do you think you can do the same thing for other formats?
If not, the sff management is already a huge part, thanks again.

KOF 91 Project

spellcaster

Member #1,493

September 2001

Sure, just use it - that's why I wrote it

Regarding the array: It's already there, I'm using it internally ofr index sprites - so if you want it, you could just change the return type to the array. That way you'd have both array (since it was returned) and the linked list (since the array members are still linked).

I'm working on the other formats. Just a question to the FNT format - the txt file is pretty simple, but for some chars the it looks weird... mainly for the [ and ] I guess. Not sure why. Is there any documentation for the TXT part of the FNT?
I think I could guess how it's composed... but that would be just a guess.

(oh... and please answer to this post, so I can add the new loaders in my next reply )
Might take some time though... the allegro font structure is quite complex IIRC - so I might have to play around with it a bit.

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

Thanks a lot,
here's a litle explanation of how the elecbyte's fntmaker works, it could help about the format:

====================
III. Fntmaker
====================

The fonts generated by fntmaker work for 2000.06.27 and newer versions
of M.U.G.E.N.

pcx file - Supply a pcx file which contains all the characters in your font
alphabet, arranged from left to right in a single row. The pcx
file must be an 8-bit color sprite.
txt file - Copy a txt file from work/font in custom.zip, and edit it to fit
your font's pcx file. The first part is are the font settings.
It is followed by character and offset pairs. The character is a
letter or symbol in your pcx file you wish to represent. The
offset is the starting x location within the pcx file of the
character. For example,
B 55
means that character 'B' begins at an offset of 55 pixels from the
left. Semicolon is treated as a comment character. To put in
entries for special characters such as a semi-colon, you need to
use a hex sequence representing the ASCII number of the
character. Hex sequences start with 0x. For example, a semicolon
(ASCII value 5B) starting at pixel 100 would have an entry:
0x5B 100

Fonts with more than one color use "color banks" in their palette to store
multiple color arrangements. Each color bank in the palette has a number of
palette entries equal to the value of the "Color" parameter in the font's
txt file. The first color bank starts at color 255, and its entries are
laid out in reverse palette order. The next color bank's first palette entry
starts immediately following the first, and so on. For example, a font with
16 colors has its first bank's entries from 255 to 240. The second bank has
entries 239 to 224 and so on. The number of banks you can have is limited by
the palette space.

I hope it could help you.

KOF 91 Project

spellcaster

Member #1,493

September 2001

Thanks for that info... makes sense now

Regarding the multiple color banks... allegro can't handle that.
I can copy the data into an allegro FONT and return a struct containing both FONT and palette .. but the actual color bank switch has to be made programmtically - if you're using hicolor display you could change the palette, set the changed pal and blit it. If you're in a 8 bit mode, things get a bit more messy.

You could either "clone" the fonts to get different versions (in different color sets) or create totally new font rendering functions.

Cloning would be faster, more easy to implement but does of course waste some memory.

What do you prefer?

--
There are no stupid questions, but there are a lot of inquisitive idiots.

Kitty Cat

Member #2,815

October 2002

I know you weren't directing this at me, but here's an idea. What if you load all the font data as true-color bitmap (according to their color banks), then do an optimized palette generation on it and make it an 8-bit bitmap to cut up for the font? Granted it won't look the same, but if you're gonna put it in FONT, you won't have much choice. :-/

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

spellcaster

Member #1,493

September 2001

The font is already an 8bit bmp. It has it's own palette (256 colors). The point is that you'd have to add an offset to each pixel to support the color bank feature.

--
There are no stupid questions, but there are a lot of inquisitive idiots.

Kitty Cat

Member #2,815

October 2002

Hmm.. I'm not sure I understand the color banks portion then. What does it do? Whats its purpose?

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

spellcaster

Member #1,493

September 2001

Assume you have a font with a gradient red->yellow using 8 colors.
The font is using the colors 247-255.
These 8 colors are color bank 0.
You could now use another 8 indices in the palette to define a blue->white gradient. Those colors would now be color bank 1.

I guess that mugens font drawing allows you to specify the color bank... so you could draw a text with font A, color bank 0 (red->yellow text) or using color bank 1 (blue->white text).

Allegro's glyph drawing functions don't work that way.

So, I could either generate several fonts (not much efford, not really memory wasting) or create functions mimicing the color bank switching.

I think, I'll go for the creation of multiple fonts

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

I think the creation of clone fonts is a good idea, as it will become possible to have palettes that are not specify with the original font picture's pal.
Spellcaster is right, mugen call a font and then its color bank to show it with right palette. I think a call to an array of fonts (or linked list) with an index that is correspinding to the bank used to make the clone will be a good idea.
A little explaination too, Mugen use fixed fonts and proportional fonts. I don't understand really the differences, but it's one of the attribute you specify when you create a font. I don't know if it's really important, but I think I've seen something like that (fixed and proportionnal) in Allegro doc.
Thanks a lot for the help.

KOF 91 Project

Evert

Member #794

November 2000

Quote:

Mugen use fixed fonts and proportional fonts. I don't understand really the differences

In a fixed width font all characters are the same width (for instance, W and I). In a proportional font, W is wider than I.

spellcaster: if you really wanted to, you could probably hack the FONT vtable so that it can draw the colourbanked fonts. That's probably going to be a lot of work for little obvious gain over what you're proposing.

spellcaster

Member #1,493

September 2001

Don't think that this is needed. Mugen fonts are pretty small - I doubt it'll be a big overhead if one would simply clone the fonts.

I'll add a

FONT* fntGetColorBankFont( FntFont* fnt, int bank);

to my code. Or something similar. If somebody can come up with a good name for that function, I'd be grateful

Expect the font functions today (sorry... I was pretty busy yesterday).

Regarding the proportional fonts: What evert said... in theory, at least. It seems like a lot of mugen fonts are labeled as "proportional" but are in fact fixed size... but that's not a big problem, IMO.

I might need to bug Bruce about the wave formats, though

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

Thanks again,
FONT* fntGetColorBankFont( FntFont* fnt, int bank); could become get_cfont( FntFont* fnt, int bank); for get colored font??
I'm not really good at function baptism...
Thanks for the help

KOF 91 Project

spellcaster

Member #1,493

September 2001

Here's the loader.
I decided to create different palettes instead of different fonts - so this solution will only work in a high color mode.
If you want to support 8 bit, you need to clone the font and change the actual pixel data of the glyphs.

--
There are no stupid questions, but there are a lot of inquisitive idiots.

zesensei

Member #4,410

March 2004

Thanks again, and again,
It works good for some fonts. Just a detail, the code didn't compile on my Win98 with Mingw32.
It was because of a detail: declaration of variables MUST be before calling any function with my GCC. So for the testloader.c I got to put allegro_init() after declarations. Same problem in loader.c where I got to declare int count, and char * txt at the beginning of the function in order to make it compile.
I learn a lot with your code fully documented, really clean and structured.

For the bugs, here are some fonts that didn't work with the testloader prog. And if I pass the same name as the exemple font you give in your zip file (name1.fnt) as an argument color banks seems to disappear. Pretty weird...

I'll look later the how of this... It works with some fnt, a great beginning.

KOF 91 Project

spellcaster

Member #1,493

September 2001

Get a new gcc - it should work with the later 3.0 versions... but to be honest, it was an mistake...
I normally try to declare my vars at the beginning. Regarding the testloader - I used only the font testing part and had the other stuff REMed. At the end I removed the comments and made a test run.

Since it worked, I was assuming it would be ok.

Quote:

Same problem in loader.c where I got to declare int count, and char * txt

Sorry. But, as I said, it'll work in any c99 compiler (say the later gccs). But I changed that - it wasn't intentional in the first place.
There's no need to put txt at the beginning, though. Just swap the char* txt; with the line above that.

Quote:

And if I pass the same name as the exemple font you give in your zip file (name1.fnt) as an argument color banks seems to disappear. Pretty weird...

Check the testloader code. There's a bug in name1.fnt. If you split it (I think I have the split.c code in that zip as well) and look into the txt.txt file, you'll see that it has colors=1 in it. And that's wrong. The numbers of colors should be 16 for that font.
That's why I added a small hack inside the loader - if it is using the default font, then it'll override the number of colors.

EDIT:
Just in case I didn't put the split stuff in the zip, here it is:

1#include <stdio.h>
2#include <stdlib.h>
3 
4int readIntFrom(FILE *f, int ofs) {
  int i;
  fseek(f, ofs, SEEK_SET);
  fread(&i, 4, 1, f);
  return i;
9}
10 
11int main(int argc, char** argv) {
  
  FILE *f = fopen(argv[1], "rb");
  FILE *o = NULL;
  int pcxOfs = readIntFrom(f, 16);
  int pcxLen = readIntFrom(f, 20);
  int txtOfs = readIntFrom(f, 24);
  int txtLen = readIntFrom(f, 28);
  
  char *buffer = NULL;
  
  printf("pcx: %i, %i\ntxt: %i, %i\n", pcxOfs, pcxLen, txtOfs, txtLen);
  
  o = fopen("pcx.pcx", "wb");
  buffer = malloc(pcxLen);
  fseek(f, pcxOfs, SEEK_SET);
  fread(buffer, 1, pcxLen, f);
  fwrite(buffer, 1, pcxLen, o);
  fclose(o);
  free(buffer);
  
  o = fopen("txt.txt", "wb");
  buffer = malloc(txtLen);
  fseek(f, txtOfs, SEEK_SET);
  fread(buffer, 1, txtLen, f);
  fwrite(buffer, 1, txtLen, o);
  fclose(o);
  free(buffer);
  
  fclose(f);
41}

--
There are no stupid questions, but there are a lot of inquisitive idiots.