The Darkmod 2.10 is HERE!!! news

Dedicated to Grayman

This release is dedicated to our departed friend and lead developer Rand Phares “Grayman” who passed away in May 2021.

Before joining the TDM team and rising to become the lead developer, Grayman was making mods for Doom, Quake, and Return to Castle Wolfenstein (including the excellent “Stolen Artifact” mod).

Grayman used his deep knowledge of id Tech AI and scripting to repair and improve TDM AI from versions 1.02 to 2.08.

Grayman’s “William Steele” series missions were always filled with complicated sequences and had a cinematic quality that few authors achieved. Grayman often recorded his own voice-acting and his stories often were tinged with a little of his unmistakable humor.

It is impossible to fathom all the things Grayman has done for TDM over the years and he will missed as both a developer and mentor for team members, mappers, and players alike.

If you would like to pay your respects, please visit the dedicated forum thread here:

Rest in Peace Grayman

NOTICE: tdm_updater is obsolete and will NOT work to upgrade to TDM 2.10. You must use tdm_installer to upgrade (or install) instead !

The Dark Mod 2.10 is HERE!!!

TDM 2.10 FULL Download:

LOADING TIMES ARE FAST!!!

Ever since Doom 3 released back in 2004, people have noted that it's approach to level loading was a little rough. At the time, most games would only load part of a level and would have a complex paging system to move items in and out different tiers of memory. The downside of this conventional approach is that it requires additional CPU overhead and memory bandwidth to constantly manage and move the data.

In a strange twist of fate, both "id Software" and "Valve" decided to rebel against this conventional approach at roughly the same time.

Doom 3 would see a design where the entire mission would be moved into memory right at the start with only a single option ( image_preload ) to load some texture data "just in time" or at level load. Valve, on the other hand, would survey the available RAM and storage and would define a chunk of a level that would load all at once ( similar to an entire Doom 3 level ) then when players reached the end of the chunk, a new estimate would be made and a new chunk would load.

Since all loading was done "up front" there would be more CPU and RAM resources to use for advanced AI, Physics, and Rendering.

Sounds great doesn't it?

Well, in theory these approaches are a nice way to skirt the painful task of building a tiered management system but in practice they are absolutely brutal for player experience.

Load times for games were going down up until this point and then suddenly two of the most anticipated games ( Doom 3 and Half Life 2 ) had the longest load times ever. Doom 3 was the worse of the two since the "chunk" system in HL2 split the long load times into little breaks between mission sections.

Players began to complain about Doom 3 load times right from the start and modders spent considerable time trying to figure out ways to improve things. People would unzip all the pk4 packages in the Doom 3 directory, disable texture compression, defrag their hard drive, reinstall Doom 3, and scramble for any hint of improvement.

When Thief players who never played Doom 3 came to play The Dark Mod, the loading times were quite shocking to them. Nearly every new player would grumble about it until defeated by the fact that we were helpless to improve things much.

At one point very shortly after TDM 1.0 was out, a modder "MoroseTroll" showed up with modified Doom 3 executables that could load Normal Maps that were compressed with ATI's "3Dc" compression. Since we weren't pre-compressing normal maps at the time, this might have saved some loading time and improved the compressed texture quality and memory footprint of TDM. In a move that was very prescient, MoroseTroll also made vendor neutral variants that used RGTC and LATC textures ( vendor neutral 3Dc \ BC5 ). Sadly, we could not use these hacked executables since they would break compatibility with other Doom 3 mods and Linux. The dream of using 3Dc ( or RGTC ) would have to wait until Doom 3 was open source.

Thus began the long term goal of adding 3Dc ( or something similar ) compression to TDM.

As early as TDM 1.07, ( around when Doom 3 source was released ) the TDM team began to investigate ways to improve loading times. On profiling the code, one of the biggest time consumers for mission loading was found to be mipmap generation.

This meant ( in theory ) that the dream of quicker loading due to 3Dc might not even be needed. You could just pack an uncompressed DDS texture with pre-created mipmaps and the majority of the loading time benefits would be there already. TDM coder "Serpentine" kept tinkering with the loading code and actually had a working prototype of LATC loading but then found that Intel wouldn't support it so he moved to working on RGTC loading. That project was put on the back-burner in favor of working to make TDM "Standalone" and sometime after TDM 2.0 was released Serpentine no longer had time to work on TDM.

In 2.05 Duzenko tried to increase loading time performance by migrating mipmap generation to the OpenGL drivers. Presumably Nvidia and AMD would now have better native performance for this in their drivers than the mipmap generator in Doom 3? It improved things a little for some people.

When TDM 2.06 introduced the GLSL backend, there was a renewed interest in moving to RGTC texture loading. One of the complications of RGTC is that it has a different data structure ( colors ) than standard tangent space normal maps so all shaders that do normal map lookups would need to be rewritten to handle both the raw mode and the compressed mode. Fortunately, there were already a few Doom 3 projects like fhDOOM that had already implemented LATC texture compression so Duzenko was able to review the needed changes and begin work on our own implementation. Initial RGTC loading and shading were buggy but working in 2.08. In 2.09 everything was converted to RGTC on load and finally in 2.10 both pre-compressed RGTC loading and rendering are fully functional ( Orbweaver fixed the loading format. )

Did this work finally achieve faster loading times???

Nope.

If we pre-compressed a number of our existing normal maps, load times would definitely improve. The problem with that "was" that none of the material blend modes worked with compressed textures so anyone who wanted to use these modes would have to either decompress the existing asset (and gain artifacts) or find the original raw data ( defeating the point of saving hard-drive space and hosting space ). So this last step was left as an open question. Which textures would mappers never use custom blend modes on? Wood? AI textures? We have a pretty good idea about the good candidates but all these decisions were made obsolete by the following developments.

In 2.09 Cabalistic and Duzenko began looking at modern OpenGL texture loading design and began to find ways to load things in parallel or in the background. 2.09 saw some nice improvements with these changes and 2.10 saw even greater improvements.

Perhaps due to some sort of rivalry, Stgatilov was motivated to profile the loading code again. Undeterred by the fact that most people would presume that AMD, Nvidia, Intel, and id Software would already have tried to solve these issues ( and the current versions should already be as optimized as possible ), he planned to write a very fast 'on-the-fly" RGTC compressor and a very fast "on-the-fly" mipmap generator!

Since he had already seen great success with optimizing Doom 3 code with modern SIMD approaches, he tried this same thing with compression and mipmap code. ( Surprisingly, there was no literature of anyone ever applying SIMD to vectorize mipmap generation even though it was trivial to implement. ) The results were ASTONISHING!!! Maps loading times dropped to as low as a quarter of the previous results. Missions were loading so quickly that they had to be edited because the "player hints" would go by too quickly.

So this is where we are now, we have all the tools to start compressing normal maps to RGTC but the original need to do it has been solved. It would still save some loading time but there is no need to rush into it now that we have loading time nirvana in 2.10.

Oh yeah, remember that problem where you cannot use pre-compressed images with material blend modes ? Stgatilov solved that too.

NEW MISSION MANAGEMENT GUI

The way that TDM installs and starts missions was designed to avoid scenarios where players would encounter Doom 3's main menu and mod loading process. It was an attempt to preserve the mood and reduce confusion and apprehension for Thief players seeing unfamiliar Doom 3 menus. Unfortunately, the design was hampered by the limitations of how mod installs work in Doom 3 and was still pretty kludgy for new players. During the 2.04 planning phase, Obsttorte watched a Let's Play where the player was really frustrated that they had to go to the New Mission page just to "start" the currently installed mission. He made a prototype of a new rearranged GUI where "Start This Mission" was prominently displayed right on the main menu screen when a mission is installed. Now the only reason to go to the New Mission page is to select or download new missions. This is so much more logical that once you get used to it, you will have a hard time playing older TDM versions.

Also, Obsttorte added the long requested feature of tool-tip text to describe different settings in the GUI:

Other GUI improvements:

* A new Reverb Volume slider to reduce the EFX reverb effect

(A big THANK YOU to Daniel Gibson for adding this feature to Dhewm3 which we ported to TDM)

* The aspect ratio of some screen resolutions have been corrected

* Vsync now offers an "Adaptive" mode where vsync is only enforced at or above your monitor refresh rate. ( r_swapInterval -1 )

* The in-game downloader can update the currently installed mission

GUI Code Stability

Stgatilov set out an agenda to attempt to cleanup the GUI code. To do this, he first had to enforce restrictions about what things are allowed to change when mission authors customize the menu GUI for their missions. With the old GUI design the only way to customize was to completely override the default configuration. If the TDM team changed anything in the default GUI, the mission author might have to fix whatever got broken on update. Also, this potentially made trouble-shooting issues with TDM a nightmare since you might always be stuck in a damaged GUI created by some mapper. To avoid this bad design, dedicated GUI keywords and GUI def pages have been created to allow mission authors to customize the screen without unintentionally breaking other things. With approval from the mission authors, all the non-compliant missions were fixed by Stgatilov to use the new System.

Subtitle Support!!!

Non-Native speakers and Hearing Impaired players have long requested that we offer sub-titles. We even had one modder NDK offer an implementation a long time ago. Tels was planning on testing that implementation but felt that text Translation should take precedence so the subtitle support project went dormant. Duzenko then expressed the desire to revive it but NDK's code was no longer available. As such, he built his own implementation and then Stgatilov improved on it. Now we have all the pieces necessary to offer sub-titles for videos, narration, cut-scenes, player voice, and even AI. We currently have story narration working in Mission 2:Tears of St Lucia:

Improved dragging and object manipulation!

• Afraid to grab a candle near a sleeping guard? Hopefully, you will make less noise while moving objects in 2.10. And if you hold the creep button, the moved item will not make any noise at all.
• Also, dragging unconscious bodies should feel better now. Rats slighty spoiled the joy of this improvement (rats often do), but hopefully it is fixed now
• Finally, now you can control how far a rock flies by holding the 'throw' button for a different amount of time.

Better Performance!!

Did you think we would take a break from optimizing performance after the major new backen in 2.09? If so, you were gladly mistaken!

A performance regression for how complex meshes are handled in vertex cache was identified in 2.09 and tracked to changes made after 2.07. This has been fixed and now performance for these operations are back to 2.07 levels.

We found that stencil shadows were wasting bandwidth on Anti-aliasing ( AA ) where no AA was needed. This has been substantially improved ( more improvements planned for 2.11 )

Many more entities and lights have been given LOD and hide_distance functions. Now mappers can use LOD to optimize more things.

Linux SIMD and AVX support has been improved

Fire Elementals no longer perform expensive leg collision calculations on complex surfaces when "flying"

A new Hash Table has been added.

Entity Limit Increase!!

While there is a single parameter for changing the entity limit, increasing this value has carry on effects to other limits in the engine. Stgatilov has done the hard work to evaluate all this additional architecture and make it work with higher limits. In testing, he generated over 100k entities and was able to have a functional mission load. The limit has currently been set to 65K which is practically unlimited for any sane mission design and even a few insane ones. Mappers will no longer need to worry about merging entities or trying to use SEED to save on entity count ( although SEED is still a more convenient way to automatically generate large amounts of entities such as vegetation ).

Volumetric Lights ( God Rays )

In many of the best looking TDM missions, mappers fake rays of light coming through windows via particles or patches with transparent additive textures. Some times this can look quite handsome but there are clear artifacts with this approach and it doesn't come close to realistic behavior. In 2.10 if you enable shadow maps ( MAPS mode in the advanced menu ) missions that use volumetric lights will have this new more beautify lighting mode.

Under normal circumstances, you would need to wait for a mission author to use this feature after 2.10 released. For 2.10, I arranged a "Connections Contest" where there is a risk \ reward bonus for using features that were in the 2.10 beta release. This means that there are many missions that have this effect now already available in the mission database!

I humbly request that you visit the contest thread:

Forums.thedarkmod.com

where you will find links to each of the participating missions:

"Down and Out on Newford Road" by thebigh

"Hazard Pay" by Kingsal

"Written in Stone" by Bikerdude, Amadeus and Dragofer

"Iris" by Wellingtoncrab

"Paying the Bills: Moving Day" by Jedi_Wannabe

Please play as many as possible to meet the voting deadline roughly two weeks from this post!

( Disclaimer: While "Iris" does use some volumetric lights, the above screen uses the traditional particle approach. Likewise, "Paying the Bills: Moving Day" is shown above using the old patch based approach which is still quite handsome. )

One mission did not meet the 2,10 Connect Contest deadline but nonetheless should be mentioned with regard to showing off 2.10 features:

A Noble Affairs by Goldwell

Full female vocal set!

Why am I singling out a mission during a TDM update article? Simply put, several of the contest entries raise the bar of excellence and are probably each worthy of their own articles but "Noble Affairs" completes another long sought after feature for The Dark Mod, a "fully realized female protagonist".

Every grunt and vocalization of the default male thief character in TDM now has a corresponding female vocal. To give context to the importance of this development to the TDM community, Greebo had once made stub code for a menu to switch player gender in the GUI back before the 1.0 release. While this release is not on the scale of "T2X: Shadows of the Metal Age" ( which also had a female protagonist ) I feel that it should at least have a prominent place in the history of TDM and deserves to be highlighted here.

And if you think the female protagonist is the only thing of value that this mission brings to the table, think again! This mission not only takes excellent advantage of 2.10 features but also has tons of custom models, animations, scripted events, textures, and sounds. The game play design is top notch with tons of options to either barge through and evade guards or ghost the mission.

Also, did I mention how pretty this thing is?

It goes toe-to-toe with some of the best looking missions ever produced ( and some of those are also TDM 2.10 contest entries ) .

Security Camera Improvements!!!

Dragofer with a little assistance from Duzenko, Epifire, and Kingsal went absolutely crazy with improvements to both the appearance and functionality of Security Cameras in TDM 2.10! The camera is now practically an AI and can follow the player to instigate targeted attacks from trap arrows, gas traps, etc.

X-Ray Render

Duzenko and Dragofer worked hard to complete the resurrection of a Doom 3 stub feature "X-Ray render". Now mission authors can create special glasses or viewers that will show hidden innards or phantom objects and AI that are not normally visible:

New Assets:

Wellingtoncrab created an expanded furniture set based on the "Cabinet 1" model:

Sotha's "Automaton AI" has been made part of TDM:

The buggy "reflective tea tray" has been made "less buggy" along with adding blur to the reflection while still retaining light reactivity. It works best when positioned vertically such as in the mission "Crucible of Omens: Behind Closed Doors":

The "cubic light" texture from Rich_is_Bored has been adjusted so that the top most shadow isn't pitch black and thus it should now be a fully suitable replacement for original desk lamp with the 2D projection:

Other Assets improvements

* AluminumHaste has improved the alpha gradient for the lock-pick success \ fail image

* Kingsal has (likewise) improved the alpha gradient for player damage and added more blood effects

* The broken splash puddle animation has been restored

* Most paintings now have torn skins

Improved Window Management

TDM has had support for Alt-Tab to Desktop since 2.05 but it has worked with varying success since then, especially for Linux users.

In 2.10 cabalistic moved (Linux) Mouse, Keyboard, and Window handling to the GLFW framework.

Now Linux players can safely Alt-Tab back and forth to TDM without losing camera control or encountering control issues.

On the Windows side, Windowed and Borderless Fullscreen modes have been improved to be more compliant with modern standards.

Secrets System

The tradition of hiding secret objects in missions goes all the way back to Thief 1 fan missions. As such, even TDM missions prior to 2.10 sometimes included such items but until now this either required custom scripts or abusing the Objectives system. Now there are dedicated secrets spawnargs and entities so that mission authors can go crazy and easily add all sorts of secret things to find along with rewarding messages to the player.

A little detour: Frob Outline

When we removed the obsolete ambient material stages after TDM 2.07 so that mappers would not be forced to make their materials needlessly complicated, we noticed that the Frob highlight effect was also a good candidate to remove from materials and hard-code into the engine.

Duzenko started by creating a new frob cvar "r_newFrob" that would disable the frob stages in the material and render it's own frob highlight. The initial design was broken and did not take diffuse colors or alpha into account. During many experiments to improve it, one approach was to add a strong fresnel highlight which sort of looked like an outline. Cabalistic took this change to it's obvious conclusion and created an object outline feature. The initial implementation was image based and had a soft glow but Stgatilov added a version with geometric outlines. Finally, r_newFrob was also fixed to include diffuse and alpha processing.

We offered these options to 2.10 testers and found that players had mixed feelings about this change. Also, some transparent objects that use cubemap reflections or other GL Blends would not highlight and would only have the outline. Thus the feature was turned off by default in 2.10 ( r_newFrob and r_frobOutline cvars set to 0 ).

Still, if you wish to play with it, the console variables are available to enable it and there are even easy "frob presets" to choose from.

r_newFrob 1

r_frobOutlinePreset

AMD Driver Woes

TDM 2.09 saw two hotfix releases due to AMD driver bugs. By the end of the 2.10 development cycle we had cured most of these by a combination of changing order-of-operations and turning off some new performance optimizations by default. In particular, "Bindless Textures" would cause the renderer to swap diffuse and normal map textures sometimes resulting in the whole game taking on a strange green look. At the end, the only AMD hardware we were not able to get working correctly was TeraScale generation 1 ( AMD HD 2xxx, 3xxx, 4xxx ). This generation would not render 3D unless we broke OpenGL specification and thus also breaking rendering for a large number of Intel graphics options or other Open Source drivers. ( We might offer a special TeraScale 1 build to address this. ). Just before 2.10 was released, we discovered that "Multi-Draw Indirect" would result in NAN data ( Not A Number. Eg. error data or divide by zero ) to be passed to shaders and cause huge square artifacts. We disabled this feature too. The performance hit for Nvidia users ( where these features work as expected ) was negligible since the driver already does it's own parallel optimizations and is pretty good with texture residency and CPU overhead. A big reason for trying to use these features was to help AMD performance but in the end they did more harm than good. Oddly, given that these were the main performance features of the new 2.09 backend, the new backend still performs better than the old backend on AMD hardware. I would hazard to guess that the new GL standards in the new backend align better with AMD's modern drivers.

If you have nvidia hardware or "really new" AMD hardware, you can try re-enabling these features via:

r_useBindlessTextures 1

r_useMultiDrawIndirect 1

for a small bump in performance. r_useMultiDrawIndirect is hard-coded to revert to off on restart so to keep it enabled you can add the arg to an autoexec.cfg file.

Mapper Improvements

Since TDM 2.09, 3 DarkRadiant releases have arrived:

2.12.0

* Customizable GUI Layout

* Material Editor GUI

* 3D Grid background

* Duplicate finder

* Improvements to managing attached lights and light radius

2.13.0

* 3 way map merge and map diff operation

* BC5 \ RGTC texture support

* Git version control integration for shared maps

2.14.0

* Significant Texture manipulation tool improvements

* FBX import support

* Texture search allows logical operators

* Game Connection GUI for syncing to TDM is greatly improved

Greebo and Orbweaver have been busy!

In addition, stgatilov improved mapping tools in the TDM side.

* Dmap has been made much much faster and has been given more precision to avoid floating point bugs

* A new diagnostic to identify broken or overlapping "Locations" (areas) has been added

* Hot-Reload supports more spawnargs

* The mission management database has been moved to SVN so that uploads happen to a single destination and missions can be reverted to previous versions via SVN revert

Tracy Profiler

Cabalistic replaced the ad-hoc profiler tool from 2.08 with the more sophisticated "Tracy Profiler" tool. Now the TDM team can do very detailed performance analysis and find even more areas for improvement!

Grayman's Legacy

Before Grayman passed, he was working on William Steele 6, 7, and 8. Both 6 and 7 were in beta testing and were nearly complete. Grayman expressed the wish that these missions would not be put into the mission database or completed by other authors. If the download links are still working in the beta test forums, players and team members may acquire the beta versions but it is to be made very clear that these are not the intended "final product" and that only William Steele 1 through 5 are to represent the final releases.

Grayman donated two other "Work in Progress" missions to Jack Farmer and Bikerdude.

JackFarmer completed the mission and it has been released as "The Black Mage" ( pictured above ). Bikerdude is currently hard at work with Grayman's other donated map now that his Connections Contest submission is completed.

Unofficial Add-Ons

I have created a new 2.10 compliant version of my Fresnel Mod.

It has the following features:

* Ambient lighting has enhanced ( increased ) specular reaction

* Ambient lighting has Fresnel brightening on glancing angles ( modulated by specular )

* Ambient lighting gets darker further from the player when SSAO is enabled

Comparisons:

Default:

Fresnel:

Fresnel and SSAO with Distance Darkening

Wesps Unofficial Patch:

The "Unofficial Patch" by Wesps has been update to be 2.10 compliant. It now features GUI options for enabling the experimental Frob Outline feature.

---

Mission Releases since 2.09

No Honor Among Thieves v4 : EFX Edition

Bikerdude, with a little assistance from Dragofer, Amadeus, and nbohr1more has upgraded the 3 Mission "No Honor Among Thieves" campaign to include EFX reverb. Along with this change, the missions have been both beautified and optimized. New areas have been opened up and more readables have been added to expand the world of this story.

---

Blackgrove Manor

New mapper "ddaazzaa" has made an excellent horror themed mission with exquisite lighting and excellent puzzles and scares. This is a new classic!

---

Hare in the Snare: Part 1

Frost_Salamander and Kerry000 ( also new mappers ) make excellent use to TDM modular content along with adding custom assets and textures to make a very handsome mission! The video briefing raises the bar for visual appeal with a tasteful combination of illustration shaded 3D models and hand drawn images.

---

Hidden Hands: The Anomaly

The fourth installment in JackFarmer's "Hidden Hands series"! An excellent mission about infiltrating an heavily guarded compound!

---

Away 0 - Stolen Heart

A prequel to Geep's "Away 1 - Air Pocket" which was created in collaboration with _Atti_, this mission has a very clever design and intricate scripted sequences.

---

Chronicles of Skullduggery 0: To Catch A Thief

Biene just keeps expanding the excellent COS series which make excellent use of modular buildings! This time, you are tasked to stop an abduction from happening.

---

Cole Hurst 1: Eaton

New author bwyan has made quite a splash with this handsome new mansion heist mission.

---

Deadeye

Sotha, the author of the excellent "Thomas Porter Series" missions, returns with a fantastic small mission about infiltrating a gang compound and assassinating a target.

---

The Black Mage

Just a little additional taste of Grayman and JackFarmer's "The Black Mage" mentioned above since it was also released after 2.09

Upcoming development: Bats!

STRUNK worked with Dragofer to implement flying AI improvements for his custom bat model! Jedi_Wannabe has already incorporated this AI into his contest mission "Paying the Bills: Moving Day". Once this is fully completed, we expect to integrate it into the core TDM project!