I've been trying at this over the past days and the solution evades me.
I'm trying to mod Sonic Generations(released on Steam 2 months ago), to import the geometry stages from an old Xbox 360 game, Sonic Unleashed, which uses pretty much the same engine. The evidence is clearly there since I just copied and modified some of the stage files, and the geometry loads.
The problem is the texture mapping is all wrong. My question for the 3D techies here is, what could the problem be? Perhaps manually modifying the textures to be flipped or something could solve it? Or would I have to fix the UV mapping? It's all in binary format so it would require heavy hex editting to figure it out, so perhaps I can just do a batch conversion of the textures and fix it.
Here's an example of the issue, and the source texture to spot the problem:
This is a screencap of the original game(youtube video, bad quality):
{"name":"605421","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/4\/d\/4d8617376de68568576f47d660865865.png","w":635,"h":462,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/4\/d\/4d8617376de68568576f47d660865865"}
And here it's ported over to the PC game:
{"name":"605422","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/7\/a74c7d94549817a72bb736b53d1913b4.png","w":1600,"h":900,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/7\/a74c7d94549817a72bb736b53d1913b4"}
I've been focusing in those small colored strips to spot the texture problem. Here's the original texture:
{"name":"605423","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/c\/4\/c456e387a103f0d76745efd70e3eedaf.png","w":1024,"h":512,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/c\/4\/c456e387a103f0d76745efd70e3eedaf"}
As you can see, the original game just renders the "Green" part I marked below, but the PC game seems to render the "Red" section I marked.
{"name":"605420","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/6\/8\/68eb799fed3a5bc0ede21cf3868600a8.png","w":1024,"h":512,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/6\/8\/68eb799fed3a5bc0ede21cf3868600a8"}
Is there any texturing differences on the Xbox 360 I should be aware off?
EDIT: I can't say how accurate that "red" section I marked is. After all, it's not really different from any other tall rectangle you could mark in the texture. 
EDIT2: It seems rotating the image and keeping the same size, plus stretching and flipping fixed it.
I hope allegro 5 has dds support so I could write a batch utility.
I'm not aware of any difference in PC and XBox uv coordinates, but if you're modding a game then it's most likely a difference in the way the game is on PC and XBox...
Honestly though, it looks like they are simply rotated as you said. Have you tried rotating the textures or is that just too complex?
Also, the XBox 360 uses big-endian byte ordering and on the PC you have little-endian byte ordering. Could be an endianness issue...
Curiously enough, a 90 degrees CW rotation and horizontal flip solves the problem for JUST a few textures. Luckily, it's the majority, but stuff such as skies for example don't need to be flipped. I think it's just weird handling by their engine or something...
Luckily, if I hadn't created this thread I wouldn't have noticed it tho.
Some weird magic is present on this site for problem solving...
Also, the XBox 360 uses big-endian byte ordering and on the PC you have little-endian byte ordering. Could be an endianness issue...
That reminds me, I've been also trying to port over the Havok binary files from the game. Apparently, they're saved specifically for the XBox 360(I've seen the option on the SDK), and it's not compatible with the PC version... from looking over it, the only difference is the endianness in fact. I suppose I don't really have a way of converting the file without knowing the format, do I? It's closed source.
(And the file is pretty darned big, nearly 1 MB)
Oh no! 1MB.
Post it... I kind of like living in binary and I have the Havok SDK... Don't know why I have it, but I can take a look. Can't be too crazy... Is there a PC and Xbox version you can post from the different games?
Having worked on AATranslator I have helped work out a few file formats here and there. I working on a new library for AAT right now but need a distraction for the moment...
This is the Xbox 360 hkx binary file from Sonic Unleashed. It handles the level collision data. I've downloaded the Havok SDK PC tools and they can't just open it, claiming it's not for the specified platform. According to their page, I would need a fully licensed Havok SDK for opening it(SAVING to the 360 version is possible, but not opening), but my guess is that it can probably be converted...
Since of course Sonic Unleashed was never released on PC, there's no PC version of those binary files to compare to. Generations uses a similar engine ported over to PC tho, so I took the chance to convert and compare the same binary hkx file. The (free) Havok content tool can open them since they were saved for the PC after all.
Green Hill Zone PC HKX
Grenn Hill Zone PC HKX resaved to XBOX360 format
Well, so far the Green Hill Zone PC HKX resaved to XBOX360 format looks to be in little-endian as well, not sure why. It is different, just no change in endianness...
[edit] You also seem to have an older version of the SDK then I do. And some of my SDK programs are not working for some reason...
That's because I'm a retard and uploaded the wrong binary file.
Here's the real converted 360 file. Double checked this time, I can't open it on the havok tools.
As for the older version, that's intentional because that's the version the game on the PC is using.
Ahh... So I can confirm then, the byte at 0x11 determines endianness of the file. That's the only thing I'm really sure of here...
Trying to figure out where to go from there... Have to do some other stuff right now though...
If it's confusing rows with columns in the image, I'd suspect much more than endianness.
Edited the structure hvk_section in my post above. It actually gets a lot easier the more you dig into the format. It's even easier if you know what data is already there. Right now my Havok SDK doesn't seem to be working. It could be because the computer I'm trying to run it on is about 10 years old now...
As long as you double check the endianness between the PC and 360 version you should be able to create a nice conversion utility... Just can't work on it anymore myself right now... Need to sleep. Lots to do tomorrow...
Wow, thanks for that.
Do you think switching the byte at 0x11 could probably make it load up if you modified the header to be compatible with the PC version?
Well perhaps their reader will only accept little endian on PC, but I wouldn't have problem writing a conversion utility. I'm just a little dumb at trying to figure out file formats.
The Havok reader won't load up on your PC if you haven't installed it. If it's a folder from an old installation, it won't work. It needs some system dlls, so a proper reinstall should do the trick. AFAIK, any new version will read the old versions without any trouble. These collision files don't really have much visual data to preview on their tool tho, it's just loads and loads of vertexes(someone who was researching them told me it was a set ammount of objects, and depending on that the vertex data would be interpreted).
Given that it's the same game engine underneath, switching the endianness at the right spots where it's affect, and a compatible header, should make it work. 
EDIT: I IS DERP.
Turns out the headers do hold some relevant information which would probably save you a stupid ammount of research... 
Here's a folder of the headers. Some relevant files: hkPackfileHeader.h, hkPackfileSectionHeader.h. I guess there might be a way after all. 
I haven't got much time to crack on the format yet, but I got the header to be recognized by the Havok content tools as compatible at least. It won't say "wrong platform for packfile" now, but it will fail to load. Here's the little application I wrote for converting the havok header and the section headers. Now I need to figure out what all these offsets mean. 
main.cpp
I forced the layout rules to be 4101 to be recognized as a PC header. It takes a *source.hkx*(the xbox 360 havok file) and writes to out.hkx.
I'm gonna try to read the classnames now, but the order seems different than the PC one. And so do some of the int32s before it. Some were affected by endiannes, but others seem totally different values.
Not sure if they're even offsets anyway, some of them seem to have negative values. Or perhaps they're just unsigned...
Arg. I don't seem to have those header files... well I can't seem to find them. A quick look in that shows header files in "Packfile/Binary" which does look interesting though... Haven't had any time to work on it myself though. Still working on this other library.
Yes, those values before the class names are probably some kind of ID and not offsets. The differences would probably be due to different versions of the SDK creating the files... That's my guess at least...
I'm guessing that if you can find every int16, int32 and int64 that needs swapping and just swap them then it should be ok as long as you change the endianness flag in the header. No need to really find out what anything does until another problem comes up...
Not sure if it was mentioned already, but another thing I noticed was that each section was aligned on a 16 byte boundary. The character 0xff was used to fill in any gaps...
Anyway, it looks like you've made some progress at least. Have to go again.
Ha... That's awesome. I liked that quote at the end of the video. 
[edit]
The blue lighting is probably a difference in color depth... I think that maybe the console version is in 16-bit mode and the PC is in either 24 or 32 bit. The color values that are stored in the file are probably 32-bit integers in both, but using only the needed bits. This would make things look more blue/green in a higher bit depth. Just a theory though...
I think the bit depth color theory is actually right for the Global Illumination textures.
After struggling with them for a while, I figured out that they need to be uv-flipped as well. But they are big textures which hold several sub-textures most of the time.
So I looked at a little binary file next to them called "atlasinfo". Thanks to your previous binary guessing efforts, I learned from that and REd the whole structure. It contained the UV coords of each sub-texture, so I just swapped the U with V in there and walla!
Now the global illumination AND the textures are correctly mapped. The only thing left to figure out so far is the Shadow casting but, the stage shouldn't look this dark. How could I convert the texture to the right color values if it perhaps was orignally 16-bit, and now must be switched to 32-bit? The final color should look as if it was lit by the same intensity as you can see in the rest of the objects.
If it's something doable in Photoshop 7 with automated batch, even better.
Otherwise I'll have to rely on Allegro.
Nice job. 
How could I convert the texture to the right color values if it perhaps was originally 16-bit, and now must be switched to 32-bit?
Well, if each texture is it's own individual file then it could be easy. You might change the format to 32-bit and multiply each color value by 8 if they are still too dark. The value for the green component might need to be multiplied by 4 if it was actually 565 16-bit color...
If each texture is inside a larger file then it might be more difficult. You'd need to figure out the structures of that file and create a new one with the new textures.
It could just be some color values near the UV coordinates that need shifting too, but all this is just theory without knowing more about the game files.
[edit]It actually looks more blue then red or green, but the green seems to be brighter then the other colors when it's on screen...
It could also be a color table or color ramp stored somewhere. The values for that would probably be 1024 bytes, words or dwords. Just something to look out for...
[edit2]
Strike all that... I turned the brightness up on my monitor and then it looks perfect. The images might be stored as 32-bit but with only 5-bits for each component being used... It might have been done that way to keep alpha maps there... Or they are 16-bit and just need to be converted to 32-bit....
Ok. I'm done.