Thief: Deadly Shadows ESRGAN Pack mod

Prerequisites

You NEED to install Sneaky Upgrade: Unofficial patch for Thief 3 with at list one mod enabled (I suggest briefing videos and Collective Texture Pack since we will replace completely the second one). And I don't know why would you want to play Thief 3 without this community patch.

Full Mod vs Light Mod considerations

Since loading times in Thief 3 became irrationally slow in full ESRGAN textures pack, I created a Light version of the mod with all the textures being downscaled twice and some menu upscaled reverted. With this game still gains overall improved quality with restored/fixed textures and especially normals and all the profits of improved MatLib (material library) with restored normal maps and bugfixed textures. And loading times just 3x times longer as in the original game. If you have PC from the future or if anyone would be able to debug extremely long loadings in Unreal Engine 2 or if you want to create good screenshots - use Full pack. If you want to actually play the game with bugfixed textures and improved details - use Light pack.

Performance

I tested loading times performance on my computer, and here what I got. This is the duration of loading a save game. Basically light mod takes 3x time, full mod takes 6x time comparing to vanilla game.
I optimized performance in the latest version (5.3) of the mod by creating mip-maps for most of the textures (except some HUD elements, fonts, loading screens).

Vanilla game: 00:10
Full mod v5.2: 01:40
Full mod v5.3: 01:08
Light mod v5.2: 00:40
Light mod v5.3: 00:34

Also I tested performance for the latest version with and without materials improvements (deleting T3Fix.mlb)
Full mod v5.3: 01:08
Full mod v5.3 without materials: 01:05
Light mod v5.3: 00:34
Light mod v5.3 without materials: 00:29

There is very little difference so I suggest to play with improved materials, they add a lot

Concept for textures

For some reasons a lot of vanilla textures in Thief 3 were already upscaled version of the XBOX textures so most of the textures were bad quality. My assumption is this was done to reduce DXT BC1 compression artifacts in DDS files. So I took several steps:
-run all textures through BC1 smooth 2.0 ESRGAN model (BC1 compression artifacts removal model)
-created a two copy of de-artifacted textures (one original size one downscaled twice)
-run two copies through 6-7 several upscaling models using ImageEnhancingUtility and its tiling option if necessary (if original texture was seamless)
-chose one good looking final result from 12-14 textures. Most of the time preliminarily downscaled texures looked better after upscaling
-manual edit if necessary by combining parts from several textures from another upscale options. Sometime one texture had some ornament in good size which I can use in another texture with the same ornament etc

A lot of textures in Thief 3 were missing normal maps or it is broken in some way. I upscaled all the normal maps using Normal Maps by Alsa or any other model with splitting RGB channels in ImageEnhancingUtility. For diffuse textures without normal maps I used normal generator by LyonHrt or Awesome Bump program to recreate them. And finally user snobel from TTLG specially created a tool for editing material files (MatLib) in Thief 3. So I fixed a lot of bugs in materials (material in game terms is a combination of diffuse texture (the texture itself) with normal map and any other maps and parameters - reflection, specular and so on)

Improved materials

Unfortunately I didn't created a separate versions of the materials editings to distinguish fixing vanilla game bugs (like there is normal map present in game files, but it is not added to the diffuse texture material) from creating/improving other texture's normal maps and other entries. So it's hard to extract only bugfixes to use in some other graphical mod.

• But you can edit materials in Thief 3 any way you want. All you need is:
  My google sheets table with all the Thief 3 materials extracted with changes made by me marked with different colors
• mlb-tool. You need to create a copy of my table, edit anything you want, export csv file with changes and do magic with mlb-tool (read documentaion or ask snobel) to create mlb file. You could also decompile any UE2 game which use material libraries (Deus Ex 2 for example) into csv and edit entries. Or decompile Thief 3 materials yourself without the need of my table.

Additional fixes

SkeletalMesh was edited for Caduca model to use her own texture (existing in vanilla game) instead of generic Keeper texture. Vanilla game bug. Ported from some another mod.
There is also existing bug with Orland glasses which uses inappropriate texture, but I don't know how to fix this mesh. If anyone could do it - you are welcome.

Concept for videos

All the BIK cinematics from game were converted to separate frames. Frames were processed by JPG (40-60%) or JPG (60-80%) artifacts removal models. Some videos (based of quality) were converted to lossless x264 files and deinterlaced in XviD4PSP 5 program to remove some lines artifacts and converted back to frames. After that all frames were upscaled using Manga109Attempt model. Frames were downscaled to 1920xYYYY size and converted to videos, slightly filtered in XviD4PSP 5 program and compressed to final result in MKV container with x264 codec. One video was edited frame by frame in graphics editor to remove artifacts.

Installation

0. Install Sneaky Upgrade: Unofficial patch for Thief 3

1. Cinematics pack. Move all videos to this folder
c:\ProgramData\Thief 3 Sneaky Upgrade\Installed Mods\Briefing Videos\Content\T3\VideoTextures\

2. Textures and material labs
Unpack archive into folder, replacing all files
c:\ProgramData\Thief 3 Sneaky Upgrade\Installed Mods\Collective Texture Pack\

P.S. You could use the mod path from any of the Sneaky Upgrade mods you installed. I just used Briefing Videos and Collective Texture Pack as an example

Special thanks

To users Glypher and snobel from TTLG forums for providing feedback, technical details and coding material editing utility

Credits

ESRGAN models that were used (mostly from ESRGAN wiki Upscale.wiki):
Manga109Attempt by Kingdomakrillic
mymanga109_250000 by patrikspacek
Skyrim Misc by Deorder
Skyrim Armory by Alsa
Normal Maps by Alsa
Skyrim Alpha by Deoder
JPG (40-60%) A by Alsa
JPG (60-80%) A by Alsa
BC1 smooth 2.0 by BlueAmulet
normal generator by LyonHrt
rocks_100000 by I don't know who (It is not a nickname)
FSBox by Pandalism
Trixie_Star Wars by LyonHrt
DigiPaint35000
ThiefGold by me (based on EP2 textures)
ThiefGoldMod by me (based on EP2 textures)