Post feature RSS Late research and White World

Technical article that explains the rest of the research and the development of the White World.

Posted by on

The Unleashed Renderer

Discovery

Korama discovered the functionality of an alternate renderer left over from some point in development. His work into finding it involved a lot of debugging what was in the Sonic Generations .exe itself, and eventually led to this hidden renderer tucked away in the executable. This alternate renderer apparently still had some of the visual features from Sonic Unleashed that didn't transition into Sonic Generations, like the inclusion of dynamic lightshafts, directional shadows, and different motion blur/bloom. However, the first results were less than ideal:

Swapping the vtables of both Rendering Managers allowed us to enable this alternate renderer. There wasn't much more research done on it at the time other than thinking it was a cool find, but nowhere near viable to use yet.

Compatible shaders found

Eventually, Lobotomy found a bunch of unused packfiles in the game named shader_s and shader_s_add (the real ones used by the game are shader_r and shader_r_add). Swapping these files actually gave the alternate renderer a completely different look. This probably hinted that at one point in development, they were using these shader files before they decided to overhaul the rendering pipeline completely.

However, this still wasn't enough, as everything was white for some reason. In the regular game, a variable called "mrgPlayableParam" handles whether or not to color an object. This is used in White World in the vanilla game to "restore" color when stages are completed. The alternate renderer was defaulting to "not restored" for the actual stages, so everything was white. Overriding this variable in the .exe led to getting back the color, but there were still more issues to work out:

While the directional shadows were a very cool feature to regain, the fact that they didn't blend properly with the enviroment really defeated the purpose of having them in the first place. This didn't seem fixable by just finding something else in the executable, so more work from a different angle was needed.

Another approach
I decided to look into why the old shaders worked fine with the alternate renderer while the modern ones didn't. There were some alternate materials that just didn't work correctly or look as they should, so this could have led to some bad results in the long run if not solved.

Instead, I decided to learn how to work with some basic disassembling and recompilation of HLSL vertex and fragment shaders. With some small scale testing I eventually figured out the differences, and I could get a regular shader from the vanilla game (in shader_r and shader_r_add) to look correct on the alternate renderer. The issues were:

  • The cause of the red was a parameter common to all shaders named "g_ForceAlphaColor". In the alternate renderer, this value defaulted to (255, 0, 0, 0) in RGBA, and all the shaders in the regular game use this to force the transparency of some objects. In the regular game, this value defaulted to (255, 255, 255, 255). Forcing the shaders to correct the value for the buggy default did the trick, and finally the regular shaders didn't look red anymore.
  • I never really understood why the shadow blending didn't work, so instead I decided to cheat and blend the dynamic directional shadowmap data with the lightmap/shadowmap used by the terrain itself on real time. This achieved what I was looking for.

These changes were implemented automatically to a set of shaders as necessary. The batch program works like this:

  • Use D3DDisassemble to translate the binary shader file to HLSL ASM instructions, along with commentary on the name of the registers.
  • Read the comments automatically to figure out the context of what registers to look for.
  • Look for lines handling certain registers, such as calls for retrieving texture information, or any place where that register was used for multiplication.
  • Add some constant registers as necessary at the start of the file.
  • Insert new lines after the correct instructions to automatically fix the two prominent issues of the alternate renderer.
  • Use D3DCompileFromFile on the new modified HLSL ASM file.

The result is compatible with the regular game.

The new White World development

Music
The music system used by the hub is actually poorly implemented in the final game, as the correct way the songs should fade between each other only works when the framerate is in unplayable levels of slowdown (1-2 FPS). The system they used relies on 13 mini-tracks of 1.739s length each, and the songs are supposed to transition between each other by fading into the next piece based on the level Sonic's currently standing at. The appropiate cuts of each song go like this:

  • 0.000 - 1.739 : A
  • 1.739 - 3.478 : B
  • 3.478 - 5.217 : C
  • 5.217 - 6.956 : D
  • 6.956 - 8.695 : E
  • 8.695 - 10.434 : F
  • 10.434 - 12.173 : G
  • 12.173 - 13.912 : H
  • 13.912 - 15.651 : i
  • 15.651 - 17.39 : J
  • 17.39 - 19.129 : K
  • 19.129 - 20.868 : L
  • 20.868 - 22.607 : M

On the actual final game, the tracks just switch on/off without any kind of fading.

Some words from our White World tune composer, Falk:

"There were a couple of issues with the adaptation of Unleashed music to the White World - the first was that all the source music was at a much higher BPM than the demands of White World implementation (1.739s per 'bar' meant you had to use a very specific BPM). The second was due to the nature of the clips switching on the downbeat of each 'bar', the adaptations needed to be able to mask this change which limited what you could do with rhythms. This meant being a little creative with instrumentation in some cases."

Concept and design

With the new music tunes being finalized, it was also time to start with the final White World for the level selection. The key concepts behind its current design are:

  • Emulate the style of the regular Sonic Generations White World, but add a twist: make the enviroments blend between each other. This would also play well in highlighting the contrast from one level to the other, while also filling the gaps with something more fun to look at than just a white background.
  • Only use existing terrain models for decoration and reference. All assets except for the ground are picked straight out of the original stages, modified to fit as needed. Even if we don't have professional modelers that could work full time on this, we have to make it look good to keep a nice overall presentation.
  • Keep it compatible with savefiles from the vanilla game. For this, the shape of the ground and its collision needed to be the exact same. For the remaining space that was unused, add a teleport to the center of the White World in case the player saved outside of the edges. This prevented the White World from being 3D, since that's the only way it would remain compatible with regular savefiles correctly.
  • Get rid of boss stages, mission gates, and the Collection Room, as they serve no purpose and are largely incompatible with the mod.

Modeling and polishing

The modeling of the White World was a group effort done in 3DS Max. Since the concept art detailed exactly what assets needed to be picked out, it was simply a matter of exporting all these stages into 3D models, and cutting anything necessary out from them.

The art itself went by a similar philosophy to what Generations does in the White World: There's multiple levels of "depth", the scale of big objects is reduced by a lot and the scale of small objects near the first plane of depth are in scale with Sonic. The objects are artificially scaled instead of relying on the camera to show how far away they are. The organization in these elements was tweaked a lot to get the depth correct, as shown in comparisons of some early renders to the final product:


The downside to this approach was that this led to performance issues, as these objects were originally designed for much larger areas and contained a lot of unnecessary detail. I did two things to address this. First, I commissioned Dude from Sonic Retro to work on polishing the level in general and reducing the poly count as much as possible. Second, support for generating triangle strips in models was implemented into SonicGLvl. Once I figured out a good algorithm to do that, there were huge performance improvements in custom geometry importing.

All the progress and changes the hub went through can be seen in this progress video, which shows multiple revisions done in the span of a month. The skyboxes shown in the video were placeholders, which is why they change in between clips. The idea behind it right from the start was to use a skybox that simulated being in space, similar to how the Sonic Unleashed level selection looks.

Stage Signs

The stage signs were done by ripping 3D models of the letters that were currently available in Sonic Generations, and making the new names by positioning them manually. You can download a complete rip of them here to make your own signs if you need them.

Global Illumination Rendering

Rendering Global Illumination at the right resolution for an entire map is a very complicated task for hobbyists, but it's nowhere near impossible. One common mistake is assuming that you can just do it all in one go and let the system grind it out for several hours or days, and that just won't work properly if something goes wrong and you have to redo it all over again. There's a much better approach to it which requires some careful set-up.

  • Optimize the stage in chunks so they can share the same GI map easily. Huge models that cover entire areas of the stage in a single node are not a good approach, but the other extreme of lots of small nodes isn't good either.
  • Don't go overboard with the resolution. Start with a low resolution render for most of the terrain and see how much you can improve it before the game's limits are hit. Create rendering groups at different resolution levels to prioritize the quality of some nodes over others. There's no need to give background elements the same level of quality as the nodes close to the player.
  • And most importantly, DONT RENDER THE ACTUAL GLOBAL ILLUMINATION LIGHTMAPS FOR TESTING. The process of rendering GI can take hours and hours. The lightmaps used in White World took an entire day to render on a fairly modern CPU. This is not something you want to include in your iteration time, and it is best left for last. What you should do instead is ONLY render the shadowmaps as they should be, and render out some temporary, incredibly low quality (very simple render settings) lightmaps that take barely any time to render. This is the method I used for White World, and once the quality of the maps in the stage was good enough overall, I proceeded to do the final render over the course of a day. With the method used, I only needed one attempt at the lightmaps to get it working correctly.

Getting the GI rendered and finalized for White World took a couple of days, right around the time we were preparing for the final release.

Playthrough adaptation

Another problem to solve was making this White World work without needing a complete savefile, since with our changes the player wouldn't be able to start a new game. The game handles most of the stuff related to the game's level progression with LUA scripts, but they're compiled in a binary format. However, the Xbox 360 version of the game features the exact same LUA scripts in plain text, and the PC version of Generations can read these perfectly fine. With this, it was possible to completely modify the way the game's progression system worked. There were some key problems that needed to be solved, which were all fixed with a lot of LUA coding:

  • Prevent Classic Sonic from being playable at the start. The ability to play as him is changed to unlock only after the last level is beaten.
  • Beating Jungle Joyride should be the end of the game. Super Sonic is the reward for beating the game, as he can't be unlocked by beating Time Eater.
  • The regular game progression needed to be changed to a linear style that only used the Modern Acts. Boss keys unlocking via missions and boss gates were removed completely. The game's unlock system was instead changed to simply require beating a level to unlock the next one.
  • The player needed to complete the GHZ100 level slot (Green Hill Zone Classic) for the game to enable saving. If this level isn't completed, the game refuses to save your progress. Given we had to prevent Classic Sonic from appearing until the game was beaten, we forced Modern Sonic to start a new game in GHZ100. This was a cloned copy of Windmill Isle Act 1.
  • Level slots can't be added without changing the game's coding somehow, so we had to make use of the already available slots as much as possible.
  • The coding of the stage gates only boots into the Classic version if you're using Classic Sonic, or Modern version if you're playing as Modern. There's no way around this without changing the game's coding, and we wanted to make Windmill Isle Act 1 accessible from White World as well. To solve this, we implemented some trickery with unlocking "Planet Wisp" early, which was actually just Windmill Isle Act 1. Planet Wisp's gate was placed right at the start of White World, which is why its state is marked as "Not Beaten" when you start a new game, even though you just went through that stage. To be compatible with Classic Sonic, he got an even stranger swapping job with PLA100 (Planet Wisp Classic) being Windmill Isle Act 2, since GHZ100 was already Windmill Isle Act 1. That's why the gates swap around in the final mod - just to work around a savefile limitation.
  • Why not just move Windmill Isle Act 2 to Planet Wisp internally and leave GHZ200 as the "Modern" version of Windmill Isle Act 1? This is where another limitation comes in, which is the Red Rings being unlocked only once you've beaten the GHZ200 level slot. Since the proper level progression was Windmill Isle Act 1 and then Act 2, we couldn't really justify making the player play the same stage two times just for the sake of hardcoded limitations. That's why we went with the weird swapping trick between these acts.
  • Get rid of all cutscenes between stages. They would've been pointless to the mod.

Rankings

The time limits for achieving S-ranks was changed to a much harsher system, involving pushing the time required to its limit, while only requiring use of techniques that fit with the level design (no glitches). However, this was met with very mixed reception, so I think it's important I explain my opinion on this after many months of discussion.

I've pretty much agreed to disagree on this issue at this point. It's not really a bad design decision objectively, since it's optional and it doesn't unlock anything extra. However, I'll admit that it's a bad decision subjectively in the situation this mod falls into, since you're dealing with the expectations of players that already had experiences with Unleashed and with other Sonic games. You can't force them to re-learn what certain things mean that easily, especially in this case of porting content that's already been played by a majority of users.

Even the brand new Sonic Lost World has been met with some disagreement due to the harsher rank requirements in Time Attack mode when compared to the rest of the series, even though it's a completely optional goal. However, Lost World has a much worse problem than that: the degree of difficulty to even complete the game. This is something that I would never agree to do in a mod or in a future game. I think of difficulty that prevents you from completing a level at all, and difficulty for optional challenges that don't even unlock anything, to be very different things from a game design standpoint. You can see examples of the latter on the level development article, where certain layout changes were made just to accommodate for lower-skilled players.

Just for the record, keep in mind none of these time rankings need any sort of glitches, skips, level design exploits, or additional skills equipped. They're designed around getting a good time with the maximum ring bonus at the end of the stage, which is 100 rings. Unlike in Unleashed, getting more than 100 rings or doing other things like defeating enemies and doing tricks doesn't contribute to your score at all; that's what the Generations scoring system factors in, and there was no way to change it to match Unleashed more closely. There's plenty of footage on YouTube showing this. WI2, SC, RR, DR, CE, AS, EC, JJ. (Notice the score offsets on the videos from 50,000 points to see what the actual time limit would be and how much room for error there is)

My ideal system would be something that doesn't "rank" you, but rather encourages you to try better and still compliments you for getting less than the best. I've actually seen this used in the daily challenges of Rayman Legends, in which apparently some players don't even know that Diamond Cups exist at all and are perfectly content with Gold Cups! Any cup leads to an eventual Lum reward daily and the baseline for getting a Bronze Cup is pretty low, while getting a Diamond Cup requires some practice. As a result, during my time with it, I was always satisfied with getting either a Silver or Gold Cup, but I was very happy the time I went the extra mile to get a Diamond. They all gave me rewards daily, so I was never focused on the idea that I just HAD to get a Diamond.

That's the feeling I wanted to emulate with these time rankings: getting a B or an A is perfectly fine and you should be happy to get one, but going the extra mile and practicing for the S-Rank should be even more highly rewarded. That's why the top times are designed around the depth of exploring and optimization your route much more than Unleashed, and it's the other extreme of what the Generations scoring system can do. The ideal system I mentioned would require an overhaul of the entire concept of the ranking system in Sonic games, but I'm definitely considering it for future games of my own.

At this moment there's no point in patching these times, because there's already another mod available for those who wish to have easier ranking requirements. The choice is left to users, then, since there's supporters on both sides of the argument. There's been both positive and negative feedback. In some cases people were glad to have an extra challenge to push their skills, and they were surprised to find new ways to play the same level. In other cases people just thought the ridiculous time gap shown was unachievable because the game never explains the scoring system properly. Some people just didn't enjoy the particular playstyle required or thought that it was simply too different from the original game's. It's all stuff that a new system needs to consider more carefully.

Post a comment
Sign in or join with:

Only registered members can share their thoughts. So come on! Join the community today (totally free - or sign in with your social account on the right) and join in the conversation.