HTML5 Massively-Multiplayer Real-time Strategy. Play for Free. Carve your empire, Change the world!
Time for some code... How to use Pixel Shaders to do decompression in WebGL
Posted by BenAdams on Dec 15th, 2011
Time for some code, Pixel Shader decompression in WebGL
This is a follow up HTML5 WebGL Experiments: 3d Town with some technical developer details...
Texture compression was a bit of journey - as no one at Illyriad had ever implemented anything in 3d before; to us texture compression was mostly a tick box on a graphics card.
It started when we found out our 90MB of jpegs expanded to 2GB of on-board memory and we were worried we'd made a terrible mistake, as this was certainly beyond the possibilities of low-end hardware! Half of it this was due to Three.js keeping a reference to the image and the image also being copied to the GPU process - so essentially the required texture memory doubled.
Dropping the reference Three.js held after the texture was bound to WebGL resolved this. I'm not sure how this will play out with context lost events - as I assume we will have lost the texture at that point - but local caching in the file system and reloading may help with recreating them at speed.
With 1GB of memory remaining we were faced with three choices - either deciding what were were trying to do wasn't possible; reducing the texture sizes and losing fidelity or trying to implement texture de-compression in the shader. Naturally we opted for the last.
We were originally planning to use 24bit S3TC/DX1; however this proved overly complex in the time we had available as the pixel shaders have no integer masking or bitshifts and everything needs to be worked in floats. The wonders we could unleash with binary operators and type casting (not conversion) - but I digress...
In the end we compromised on 256 colour pallettized textures (using AMD's The Compressonator to generate P8 .DDS textures). This reduced the texture to one byte per pixel - not as small or high colour as DX1 - but already 4 times smaller than our original uncompressed RGBA textures.
It took a while to divine the file format; which we load via XMLHttpRequest into an arraybuffer. The files have 128 bytes of header which we ignore, followed by the 256x4 byte palette which we load into a lookup table texture RGBA. The rest we load into a Luminance texture. Both textures need to use NearestFilter sampling and not use mipmapping to be interpreted sensibly.
We have created our own compressed texture loaders - the colour texture loader looks a little like this:
When we first did the decompression in the pixel shader, it was very blocky as we had turned off filtering to read the correct values from the texture. To get around this we had to add our own bilinearSample function to do the blending for us. In this function it uses the diffuse texture with the colour look up table and using the texture size and texture pixel interval samples the surrounding pixels. The other gotcha is that the lookup texture is in BGRA format so the colours need to be swizzeled. This makes that portion of the shader look like this:
This performs fairly well; and certainly better than when your computer feels some virtual memory is required because you are using too much! However, I'm sure on-board graphics card decompression should be swifter and hopefully open up the more complex S3TC/DX1-5 compression formats.
There is a major downside however with decompressing this way in the pixel shader. You have to turn off mipmapping! Not only does turning off mipmapping cause a performance hit as you always have to read the full-size textures - but more importantly it doesn't look good. In fact in the demo - we had to use full-size textures for the grass so we could apply mipmapping as otherwise in the distance it was a wall of static!
Unfortunately, as far as I'm aware, WebGL while you can create mipmaps with generateMipmap - you can't supply your own. Again, real compressed textures should help here.
EDIT: Benoit Jacob has pointed out this is possible by passing a non-zero ‘level’ parameter to texImage2D - one to look into.
Some caveats on the demo:
So the actual game requirements will be much much lower.