Allegro.cc - Online Community

Allegro.cc Forums » Programming Questions » Texture downscaling - highest quality possible, and texture packing

Credits go to Billybob, gnolam, Kitty Cat, and Thomas Harte for helping out!
This thread is locked; no one can reply to it. rss feed Print
Texture downscaling - highest quality possible, and texture packing
james_lohr
Member #1,947
February 2002

I want to downscale my textures before loading them into OpenGL. This need only be done every time the user changes texture quality, so computation time is not an issue.

My first thought was bicubic interpolation, but, on second thought, is it actualling going to make any difference whatsoever with downscaling compared to bilinear interpolation (since no additional information is actually needed).

The second issue is texture packing:

One technique I've seen in the past (in fact I think AllegroGL used to do it, and might still) is to scale up textures that are not POTs. This gives really bad results in my experience (perhaps because it used bilinear for the upscaling?). In fact better results can be obtained by only using a small part of the texture (for example, for an 800x600 texture, using: 0 to 0.78125 and 0 to 0.5859375 texture coords for a 1024x1024 texture). Of course this wastes loads of memory (55% in this case) and, more importantly, is not a general solution because some textures are required to wrap and must therefore occupy the full 1x1 area.

Another issue, which is linked in with the above, is that of texture resolution relative to the viewport pixel dimensions. From what I've seen, textures become noticeably better quality when there's a 1:1 mapping from texels to pixels. So, for example, if you're in a 1024x768 pixel graphics mode, and you draw a 256x256 texture covering exactly 256x256 pixels, it's going to look good (assuming no rotation).

Linking in with the first issue, now suppose that you've got a 256x200 texture. If you scale it up to 256x256 and then draw it to cover 256x200 pixels, you no longer have this 1:1 mapping since your GPU is squashing your slightly stretched texture back to its original ratio (and potentially you're compounding the problem by bilinearly scaling the texture up in the first place!). However, if you had instead simply used just the first 256x200 texels and scaled your texel coordinates appropriately, you could be maintaining the 1:1 mapping, and therefore be getting better results.

So anyway, my currently planned solution is as follows:

The general intention is to always maintain a 1:1 pixel-texel mapping, while minimizing wastage. This should be the case regardless of what resolution the user chooses.

So, all textures will be stored on disk at a very high resolution (2048x1536 if a texture were to cover the whole screen). Textures will then be downscaled in software to match the screen resolution chosen by the user, however; textures requiring wrapping will be downscaled to the closest POT dimension that maintains at least a 1:1 mapping of texels to pixels (note: downscaled and never upscaled!).

Finally, textures will be group by pixel format (i.e. 24 bit and 32 bit), and then packed into larger textures so as not to waste video memory. Texture packing will allow for a generic packing algorithm implementation to be used, which will be required to support a few configurables such as maximum texture size. The output will be a list of texture objects (which may actually share the same OpenGL texture but with disjoint texel areas) that hide texel coords from the user.

Does this sound sensible? - It's something that I'll probably use across many of my projects in the future, so I want to be sure that I get it right which is why I'd like to run it past you guys before I start writing it.

Billybob
Member #3,136
January 2003

What video cards are you targeting? Most modern and old video cards support non-POT textures. The card has to be pretty old to only support POT textures.

Mark Oates
Member #1,146
March 2001
avatar

You could always use the POTs and your textures would look real nice.

:D ?

wah-wah-wah :(

Kitty Cat
Member #2,815
October 2002
avatar

I think games will typically make use of mipmaps for texture quality. For instance, .DDS images can store multiple mipmap levels. For high-quality textures, you would load the 1st texture level as the 1st, 2nd level as the 2nd, etc. For medium, you could load the 2nd mipmap level as the 1st, 3rd level as the 2nd, etc. And for low, load the 3rd level as the 1st, 4th level as the 2nd, etc.

As a bonus, this will load the images faster since you don't have to generate mipmaps for them, and low-quality texture modes will use up significantly less texture memory.

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

james_lohr
Member #1,947
February 2002

Kitty Cat: I appreciate that most games do it that way. However, it would give very poor results in my case. The reason for this is that most games will transform textures so much (think any 3d game) that it makes no sense whatsoever to think about the mapping of texels to pixels. In my case I'm using an orthog view with textures usually maintaining the same size - almost as if they were bitmaps. Simply committing to mipmap levels would give horrendous results.

I've already gone into this in detail in my OP, but perhaps you'll see what I mean with another example.

Suppose I have a large texture that is going to cover the whole screen. It is stored on disc at 2048x1536. Let's say my screen resolution is 1280x1024 (for the moment ignore the inconsistent aspect ratio). Taking your approach, the texture would be loading onto the card at 2048x1536, and then OpenGL itself would do the downsampling while rendering it to the screen.

Now, doing it the way I have proposed: I downsample in software (using the most accurate technique available) to 1280x1024. This is converted to an OpenGL texture as a padded 2048x1024: 1280x1024 texels containing the texture while the remaining 768x1024 is padding (but this will hopefully be filled by the texture packer).

Now when I render the texture to cover the screen, I have 1:1 mapping from texels to pixels, and I get far better results.

In the more general case you can see that for lower resolutions (say 1024x768), generating OpenGL textures at their highest quality would be totally redundant because everything is going to be smaller than their first mipmap.

Do you see what I'm getting at?

Kitty Cat
Member #2,815
October 2002
avatar

Suppose I have a large texture that is going to cover the whole screen. It is stored on disc at 2048x1536. Let's say my screen resolution is 1280x1024 (for the moment ignore the inconsistent aspect ratio). Taking your approach, the texture would be loading onto the card at 2048x1536, and then OpenGL itself would do the downsampling while rendering it to the screen.

Now, doing it the way I have proposed: I downsample in software (using the most accurate technique available) to 1280x1024. This is converted to an OpenGL texture as a padded 2048x1024: 1280x1024 texels containing the texture while the remaining 768x1024 is padding (but this will hopefully be filled by the texture packer).

Now when I render the texture to cover the screen, I have 1:1 mapping from texels to pixels, and I get far better results.

I think it depends on the image. In my GL games, bilinear with mipmapping (GL_LINEAR_MIPMAP_NEAREST) gives quite good results for things like text and HUD graphics. Though it likely won't be the best possible, it won't be horrible. Pre-scaling in software would be fine though, as long as the extra computational time isn't an issue.

FWIW, I also second Billybob. Any semi-recent video card should have GL_ARB_texture_non_power_of_two, which lets you use non-power-of-two texture sizes. For older cards, there's also GL_ARB_texture_rectangle (though there's some caveats with those).

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

james_lohr
Member #1,947
February 2002

Kitty Cat said:

FWIW, I also second Billybob

Cool, in that case I'll dispense with the texture packing, although it's really a separate issue to that of texture pre-scaling.

[edit]

Just tested, and on my computer there's less than a 1% difference in render speed between POTs and NPOTs. :)

Thomas Harte
Member #33
April 2000
avatar

Re: prescaling, as I recall the comp.graphics.algorithms FAQ recommends a Gaussian blur with a radius equal to the scaling factor (or, I guess, you'd use an ellipse if you aren't preserving the output ratio), following by a simple point sampling.

That said, I can't immediately find that advice in that text, so possibly I imagined it and/or read it elsewhere. Please take with a pinch of salt.

gnolam
Member #2,030
March 2002
avatar

To be specific, non-power-of-two texture support is mandatory from OpenGL 2.0 and onwards.
To check if they are supported in the general case, though, you have to check both the ARB extension and the OpenGL version - even though ARB_texture_non_power_of_two was AFAIK merged into 2.0 without any changes. OpenGL is pretty darned neat to have around, but it has a frighteningly high WTF/(design decision) ratio. :P

--
Move to the Democratic People's Republic of Vivendi Universal (formerly known as Sweden) - officially democracy- and privacy-free since 2008-06-18!

Kitty Cat
Member #2,815
October 2002
avatar

gnolam said:

To check if they are supported in the general case, though, you have to check both the ARB extension and the OpenGL version

I can't think of a halfway decent GL implementation that doesn't expose GL_ARB_texture_non_power_of_two (or indeed, any extension that was merged into core) even if it's of a sufficient version that requires it. There were/are some drivers (IIRC, the Linux FGLRX drivers) that purposely did not expose that extension despite having a sufficient GL version, because it caused the driver to drop to software rendering.

--
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer." -- Bruce Graham

gnolam
Member #2,030
March 2002
avatar

Kitty Cat said:

I can't think of a halfway decent GL implementation that doesn't expose GL_ARB_texture_non_power_of_two

Neither can I, but that's just because ATI's drivers will probably never qualify as "halfway decent". :P

It's the same situation with FBOs, BTW.

--
Move to the Democratic People's Republic of Vivendi Universal (formerly known as Sweden) - officially democracy- and privacy-free since 2008-06-18!

Billybob
Member #3,136
January 2003

It doesn't look like there's a definitive answer for the downscaling question. My suggestion would be to just open Photoshop, Gimp, Paint.NET and any other application you can get your hands on. Downscale a few sample textures using every method they have. Downscale to various levels, 50%, 25%, 77%, 42%, etc, etc. See what gives the best results. Really, though, it's hard to go wrong with downscaling. No information has to be created.*

  • Unless you consider the high spatial frequencies that were lost in the initial sampling.

james_lohr
Member #1,947
February 2002

I've gone with weighted averages (weights are calculated by exact pixel area coverage), which seems to give perfect results. I've compared the results with what I considered to be the best photoshop downscaler, and the difference is infinitesimally small. It actually up-scales surprisingly well too as long as it's not more than a factor of around 3.

So now I can store my master textures at ridiculously high resolution and not worry about memory usage on older machines because of software downscaling.

Downscaling is indeed quite slow (mainly because I've gone for really massive masters). However, I've already implemented a "DownSizeSaver" which will only generate downsized textures when either the screen resolution has been changed, or the "lastWriteTime" has changed on the master file. So I don't even notice it while developing.

In the past I've always stuck to 1024x768 or 1280x1024 equivalent sized masters. With my new 2048x1536 masters, I tried running my game fullscreen on my 1900x1200 laptop monitor; the results were quite stunning. ;D

So overall, very happy, especially since I'm just using NPOTs instead of wasting my time with packing into POTs.

Billybob
Member #3,136
January 2003

Screenies? I love eye candy!

Thomas Harte
Member #33
April 2000
avatar

I've gone with weighted averages (weights are calculated by exact pixel area coverage), which seems to give perfect results.

The likely issues will revolve around downscaling of high-frequency signals, have you been careful to include a bunch of those in your testing?

Logically, I think the best approach would be a 2d discrete cosine transform, chop out anything above half the output sampling frequency, transform back to image space from there.

james_lohr
Member #1,947
February 2002

Billybob said:

Screenies? I love eye candy!

{"name":"599447","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/3\/a3bdda15b553b4b0ae7ae9cd3620f6e2.png","w":1024,"h":820,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/3\/a3bdda15b553b4b0ae7ae9cd3620f6e2"}599447

That's at a fairly low res, but uploading a 1900x1200 png would be silly.

Logically, I think the best approach would be a 2d discrete cosine transform, chop out anything above half the output sampling frequency, transform back to image space from there.

Having studied DCTs and Wavelets at Uni, it's sad that they didn't spring to mind more readily. :( Anyway, it sounds good, and I suppose I could even go on to select a set of downscaling parameters based upon the distribution of cosine coefficients. It would need testing though, and is probably a lot more work than it's worth considering that I'm perfectly happy with the results I'm getting now, even for high frequencies.

Billybob
Member #3,136
January 2003

Shiny! 8-)

[quote]Logically, I think the best approach would be a 2d discrete cosine transform, chop out anything above half the output sampling frequency, transform back to image space from there.[/quote]
That would give the cleanest results, but would be the most inaccurate. Yes, the high frequencies are "lost" when you sample at a lower frequency, but they still perturb the signal (aliasing). Removing them out-right would remove too much information, unless you want to completely avoid aliasing.

Go to: