===============================================================================
Crash To Desktop Due To Out Of Virtual Memory Issue Solution
===============================================================================
Author: JakFrost
Created: 2009-02-09
Updated: 2009-02-10
Game: STALKER Shadow of Chernobyl v1.0006
Mod: Supermod Pack 2.2.5 for OL2.2 by Wolfhunter
-------------------------------------------------------------------------------
Cause
-------------------------------------------------------------------------------
The cause of some of the Crash To Desktop (CTD)s problems for the game
Stalker are caused by a out-of-memory error when the game tries to
allocate more than 2 GB of memory when running this mod while having
the Texture Detail setting at a high value such as at 3/4 or Maximum
and when entering certain complex maps such as the Bar. These crashes
happen when you enter a new map, when you try to save your game on the
current map, when you try to load a previously saved game, when moving
around a map and you hit one of the prefetch load zones such as the one
where you walk to the Bar across the ditch in the road where there is a
hanging corpse and there are always dogs, or at other times where the
game might try to load more information into memory.
The reason for this crash is not related to how much physical RAM you
have in your computer but to the design of the Windows NT/2000/XP
32-bit kernel, which divides the 4 GB of memory address space into 2 GB
for user mode processes such as applications and games, and the
remaining 2 GB for the kernel and system cache. Basically no
application or game can allocate more than 2 GB of memory all at the
same time because of this design decision. There is a special /3GB
switch that you can put into the &#xSy;stemDrive%\boot.ini file that will
change the memory split so that the user mode processes get up to 3 GB
of the memory address space and the kernel will only get the remaining
1 GB. This will allow the game to allocate more than the previous 2 GB
limit, up to 3 GB of memory, for itself. The memory values that I am
referring to here are for the virtual memory address space and not
actually physical RAM that is in your system since the limits mentioned
above affect all computers no matter if they have 1 GB, 2 GB, or 4 GB
or RAM. Also the limits here are for each process individually since
every process gets a new private virtual memory address space that is
independent of any other processes and not shared between them.
As explained earlier the Available Virtual Memory limit is individual
to each process and this is the one that the solutions mentioned below
try to address. Be aware that there is also a memory Commit Charge
Limit that is composed of the amount of available physical memory and
all the page files capacities combined. This amount is a total amount
of memory that can be allocated to all processes running on the system.
If you are running your system without a page file or a page file that
is too small and the game tries to allocate more memory than the
available physical memory and the total combined size of all page files
the system will fail during the allocation and also cause a Crash To
Desktop. This can still happen even if you implement the Available
Virtual Memory solution below to increase the Available Virtual Memory
limit for the game. This second issue can be caused by running other
programs at the same time as the game that try to allocate a lot of
memory or if your page file size is too small or you do not have one
configured.
I came across this issue when I was diagnosing of the cause of the
Crash To Desktop issues that I was having with my system. It has 4 GB
of RAM and couldn't understand how I could run out of so much memory. I
found that Stalker v1.0005/1.0006 with the Wolfhunter's Supermod Pack
2.2.5 for OL2.2 mod installed and Texture Detail setting at 3/4 was
allocating 1.96 GB of virtual memory for the game process when I was in
the Bar area. This value turned out to be very close to the 1.98 GB
limit for Available Virtual Bytes that my Microsoft Windows XP
Professional SP3 had configured by default. When I tried to make a new
saved game or enter another area my game would Crash To Desktop each
and every type unless I lowered the Texture Detail counter to 1/2 and
muddied up the game textures.
My hypothesis was that the game was trying to allocate additional
memory above the 1.98 GB Available Virtual Memory limit for my system
causing the crash and this turned out to be true after further
investigation and after implementing the solution below. After
performing more tests running through various maps in the game I found
that the when the Texture Detail value is set to Maximum the game will
would allocate up to 2.46 GB of virtual memory on my system and that is
way past the 2.00 GB Total Virtual Memory limit.
-------------------------------------------------------------------------------
Diagnosis
-------------------------------------------------------------------------------
Determine The Available Virtual Bytes Value For The System
----------------------------------------------------------
First find out the value for the Total Virtual Memory and Available
Virtual Memory counters for your system. On a standard Microsoft
Windows XP/2000/XP 32-bit system the value for Total Virtual Bytes will
be 2.00 GB and Available Virtual Bytes will be ~1.98 GB. The Available
Virtual Bytes value shows the limit that any user mode process such as
a game or an application can allocate for itself.
1. Go to the Start menu, Run command and type in "msinfo32.exe" and hit Enter to start the System Information utility .
2. Check the Total Virtual Bytes value and the Available Virtual Bytes values.
3. Under File select the Export option and save the output to a text
file such as "Before-3GB.txt" to have a record of your system
configuration.
Determine The Virtual Bytes Peak For The Process
------------------------------------------------
Before you try to implement the solution you should first follow the
instructions below to confirm my diagnosis of this problem and ensure
that the CTDs that you are experiencing are in fact related to running
out of Available Virtual Memory for the process. There are other
reasons for getting CTDs and they might be related to a problem with
the mod scripting in the game having an error, merged mods that are
conflicting with each other, or other issues with the game itself so it
is important to verify the actual cause of the CTD problem before
implementing the solution.
1. Start Notepad and create a new file.
2. Cut-and-paste the line below into the file.
@ typeperf.exe -y -f CSV "Process(XR_3DA)\Virtual Bytes Peak"
3. Save this file as "StalkerVirtualBytesPeak.cmd".
4. Execute this script by double-clicking on it and it will open up a
command line window while it is running and recording the value for the
Virtual Bytes Peak counter every second into a file called
"output.csv". You can Minimize this window while it is running.
5. Start up Stalker and play your game trying to do something that
causes a CTD to happen such as moving around the maps, and entering the
Bar area, saving games, opening large inventories, etc.
6. If you cannot get the CTD to happen when you are trying don't
despair and just try to move around the game world as much as possible
to get the game to allocate as much memory as possible so that you at
least have a relative value for the counter that we are trying to
monitor.
7. When finished exit the game and open up the "output.csv" file in
Notepad that was created in the same folder as the script that was
created and see the highest value at the bottom of in this file that
was recorded while the game process was running.
8. Open up windows Calculator by doing Start, Run command "calc.exe" and use the View menu to enter Scientific mode.
9. Convert the bytes into gigabytes by dividing the value by 2^30 so
cut-and-paste the value from the output file into calculator then hit
divide (/), type "2", hit the (x^y) button, type "30", then hit Enter.
10. Write down this value as your Virtual Gigabytes Peak value.
The Available Virtual Bytes value is the limit for your system and if
the value that you just received from the Virtual Bytes Peak is very
close or near this limit then the game process is likely to CTD when it
tries to allocate memory above this limit. If this is the case for you
then you can use the solution below to apply the /3GB switch and
increase the Total Virtual Bytes limit on your system up to 3 GB.
If these values are very far apart, hundreds of megabytes then it is
likely that your CTDs are not caused by running out of Virtual Memory
address space but maybe by something else, an overclocked video card,
old video drivers, corrupted game files, overclocked CPU or RAM, RAM
going faulty, scripting errors in the mod, problems with merging
multiple mods, issues with the scripts in the game itself, game
characters performing unusual actions or getting stuck in strange
places on the map, etc. Look for a solution elsewhere in this case.
Determine If Your Memory Commit Charge Limit Is Large Enough
------------------------------------------------------------
Since the memory Commit Charge Limit is composed of the Total Physical
Memory and all the sizes of your Page Files this limit can also be
reached if the game tries to allocate a lot of memory and your system
has not enough physical memory while also having no page file or one
that is too small. Use the procedure below to get the value for this
limit.
1. Go to the Start menu, Run command and type in "msinfo32.exe" and hit Enter to start the System Information utility .
2. Check the Page File Space value, this is actually the memory Commit
Charge Limit value composed of the Total Physical Memory and Page File
Sizes counters.
You can also see this counter in Task Manager or by using the " typeperf.exe "Memory\Commit Limit" "command on the command line.
If this counter is smaller than the Available Virtual Bytes counter or
close to it in size you should increase the size of your Page File or
select the option to let the operating system manage the size so that
it can increase the size as required when processes try to allocate new
memory. You can refer to the article below from Microsoft about
increasing the Page File size.
One recommendation about choosing the size of the Page File. You can
follow the amount of physical memory x 1.5 to set your page file, or
you can select the System Managed option to let the operating system
grow and shrink the file as required. However, the correct way of
choosing the size of the page file is to actually load the games and
processes that your system would normally be running under the heaviest
foreseen load and look at the Commit Charge Peak value to determine the
maximum amount of memory that was allocated to all the running
processes at the same time, then subtract the Total Physical Memory
amount from the peak value, and set your page file to the remaining
value plus a bit more space for overhead. This will ensure that the
size of your page file is actually large enough to accommodate the
heaviest memory usage you expect but that the file is not so large that
it wastes space.
Also, be sure to defragment your page file if it is manually set or
system managed to ensure that it is not heavily fragmented and so that
it does not cause unnecessary hard disk seek delays. You can use
Microsoft SysInternal's Page Defrag (pagedfrg.exe) from Technet.microsoft.com to run the defrag at the next reboot.
Microsoft Help and Support
RAM, Virtual Memory, Pagefile and all that stuff
Article ID: 555223 - Last Review: December 12, 2004 - Revision: 1.0
Microsoft Help and Support
How to set performance options in Windows XP
Article ID: 308417 - Last Review: November 3, 2008 - Revision: 4.1
-------------------------------------------------------------------------------
Solution
-------------------------------------------------------------------------------
Upgrade Your Operating System to 64-bit Version
-----------------------------------------------
The most effective but most difficult solution to implement is to
actually upgrade your 32-bit operating system to a 64-bit operating
system. On a 64-bit Windows operating system any user mode processes
receive the full 4 GB of virtual memory address space for Total Virtual
Memory and this practically doubles the 2 GB limit that is the cause of
the problem for 32-bit operating system. If you are planning on
building or upgrading your computer in the future I would strongly
suggest and recommend a 64-bit operating system since games and mods
that are going to be released in the future will likely pass the 2, 3,
and 4 GB barriers and they are also likely to be compiled with 64-bit
executables pushing the Total Virtual Memory limit to 8 TB making it
unlikely in the next few years from being reached.
Applying The /3GB boot.ini Switch on Microsoft Windows NT/2000/XP 32-bit
------------------------------------------------------------------------
Disclaimer: The procedure below will modify your boot.ini file and this
file is required to properly start the operating system so any errors
made in this process might make your system temporarily unbootable. You
can perform a repair if you make a mistake by booting your operating
system's installation disc and selecting [R] for Repair mode, or using
a bootable Linux or Windows XP CD to edit the boot.ini file to fix it.
Recovery: Have a bootable CD available that can mount NTFS volumes
before you start the procedure below so that if your computer crashes
or does not start up after this change you can use the CD to edit the
boot.ini file to remove the /3GB switch. You can make your own Windows
XP PE Live bootable CD with BartPE software from Nu2.nu or download the Knoppix Linux live CD or DVD from Knoppix.net
and one of the mirrors. Either of these will let you boot from and then
mount your hard drive volume to change the boot.ini file back.
Warning: Be aware that the /3GB switch might cause problems with older
video card or other drivers that were not designed to use this switch
so upgrade to the latest video card and other drivers before attempting
this procedure. Since the kernel will now be located within the last
1GB of space that is shared with the PCI Memory Mapped I/O address
range there is a chance of conflict due to certain drivers not being
properly written to share the address range with kernel mode processes.
This change can have a negative impact on your system causing random
crashes, video or audio corruption, or blue screens if you have a
driver that was written improperly.
Performance: By limiting the kernel address space from 2 GB to 1 GB and
forcing the kernel to share this space with the PCI Memory Mapped I/O
area for devices such as network, video, and sound cards there is a
chance for a performance degradation for the system as the kernel tries
to swap some of it's data to the page file while managing less memory.
Video cards that try to allocate a lot of PCI MMIO memory will further
shrink that last 1 GB of space available to the kernel forcing it to
have a lot less memory resources available to service the operating
system and this can cause performance degradation and at worst
instability due to out-of-memory or out-of-resource issues in the
kernel itself.
Turning: Following the procedure below there is a section describing
how to the /UserVA= boot.ini switch to give back some of the memory
back to the kernel once you find out your game's actually Virtual Bytes
Peak value after playing the game for a while.
Permissions: The procedure below requires editing the boot.ini file and
this can only be accomplished by using an account with Administrator
privileges.
1. Go to the Start menu, Run command and type in "msinfo32.exe" and hit Enter to start the System Information utility .
2. Check the System Type field and if the value is displayed as "x86"
you have a 32-bit operating system and you can proceed with the change,
if it is "x64" "amd64" or "ia64" then it is 64-bit and you should stop
this procedure now since you can't use this switch because it is
already in effect.
3. Check the Total Virtual Memory and if the value is 2.00 GB then the
/3GB switch can be applied to your system. If the value is 3.00 GB or
4.00 GB then the switch is already active.
4. Go to the Start menu, Run command and type this in "attrib.exe -r +h
+s &#xSy;stemDrive%\boot.ini" to remove the Read Only attribute on this
file allowing you to change this file and save the changes.
5. Start the Notepad application and under File, Open menu type in "&#xSy;stemDrive%\boot.ini" then press Enter.
6. A text file should open and it should contain a line similar to
"multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /noexecute=optin /fastdetect" but the switches, the
folder, and the (0) (1) values might be different and that is okay. Do
not edit any of the numbers or remove any of the switches.
7. Go to the end of the line and add the " /3gb" switch to the end with
a space to separate this switch from the previous one. Do not
cut-and-paste the line from this example since yours might be different
and you could make your system unbootable by pointing it to the wrong
disk, partition, or installation folder.
8. The line should look similar to this one after adding the switch
"multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /noexecute=optin /fastdetect /3gb" but once again the (0)
(1) values and the switches might be different.
9. Reboot your computer to make the switch take effect.
10. After rebooting go to the Start menu, Run command and type in "msinfo32.exe" and hit Enter.
12. Check the Total Virtual Memory and if the value is 3.00 GB then the
/3GB switch is active and user mode processes can now allocate 3 GB or
memory space instead of the previous 2 GB.
13. Under File select the Export option and save the output to a text
file such as "After-3GB.txt" to have a record of your system
configuration.
14. Start Stalker enjoy the game without Crashes To Desktop* caused by
out-of-memory errors. You can also increase your Texture Detail setting
if you wish.
* - Stalker might still Crash To Desktop if there are errors in the
coding for the mod or by errors in the game itself and they might
happen due to overloaded inventory, people or monsters in the game
doing unforeseen actions causing crashes, etc.
Below is an example of the System Information before and after the /3GB switch.
Before the /3GB Switch
----------------------
OS Name: Microsoft Windows XP Professional
Version: 5.1.2600 Service Pack 3 Build 2600
OS Manufacturer: Microsoft Corporation
System Type: X86-based PC
Total Physical Memory: 4,096.00 MB
Available Physical Memory: 2.45 GB
Total Virtual Memory: 2.00 GB
Available Virtual Memory: 1.96 GB
Page File Space: 7.99 GB
After the /3GB Switch
---------------------
OS Name: Microsoft Windows XP Professional
Version: 5.1.2600 Service Pack 3 Build 2600
OS Manufacturer: Microsoft Corporation
System Type: X86-based PC
Total Physical Memory: 4,096.00 MB
Available Physical Memory: 2.56 GB
Total Virtual Memory: 3.00 GB
Available Virtual Memory: 2.96 GB
Page File Space: 7.99 GB
Calculating The Increase To The New User Mode Virtual Memory Value
------------------------------------------------------------------
If the Virtual Bytes Peak and Available Virtual Bytes values are very
close in your system and you suspect that this is the cause of your CTD
issues then you can apply the solution with the /3GB switch to set the
Total Virtual Bytes value to 3.00 GB and end up with Available Virtual
Bytes at ~2.96 GB. However, you can also choose a different value
anywhere between 2 and 3 GB and use it instead allowing the remaining
portion to be used by the kernel for system processes and data.
The first step in this process is to enable the /3GB switch to allocate
the full 3 GB for user mode processes using the process below and then
to play the game again while executing the StalkerVirtualBytesPeak.cmd
script to determine what the new peak value will be after you try to
play without Crashing To Desktop. Once you get the new Virtual Bytes
Peak value you can then use the procedure below to use the "/UserVA="
switch to assign enough memory for the game and to leave the rest for
the kernel, freeing up a few hundred megabytes of memory and lessening
the effect of the full /3GB switch on the kernel.
The process of estimating is a bit difficult since it is hard to
predict how much memory exactly the game will allocate in the future
and if any future updates to the mod will cause more memory to be
allocated to the game. This could lead to CTDs in the future and force
you to increase the value for the "/UserVA=" switch.
In my experimentation I found that at the time of writing these
instructions the versions of this game with the mod, written at the top
of the document, would allocate 2.46 GB of virtual memory with Texture
Detail set to Maximum while I was running between the Garbage, Bar, and
Wild Territory area. It is possible that if I entered new areas such as
Yantar and Army Warehouses the game might try to allocate more memory.
I ran a test to determine how much impact the Texture Memory setting
has on how much memory the game tries to allocate at the start after
loading a saved game in the Bar area while standing on the Watertower
on top of the Arena building. Below are the results and you can use them
Virtual Bytes Peak Virtual GB Peak Texture Detail
2,497,961,984 2.33 100%
2,314,375,168 2.16 75%
1,919,328,256 1.79 50%
1,905,266,688 1.77 25%
1,898,594,304 1.77 0%
If you followed the instructions above after setting the /3GB switch to
run the script to determine the new Virtual Bytes Peak value you should
now have a number that you can use to predict a little how much the
game will try to allocate. You can use this number along with a little
headroom to give back the remaining memory to the kernel for system
processes and data. You will have to convert the bytes value to
megabytes and then add a little extra for space.
1. Start the Windows Calculator by and then using the View, Scientific option.
2. Type in or paste the new Virtual Bytes Peak value that you received after playing the game after the /3GB switch.
3. Hit the Divide [/] button, type in 2, then hit the [x^y], type in 20, hit Enter.
4. The result should contain a number such as digits such as
"2382.2421875" and you can round of anything after the decimal point.
5. Add approximately 250 MB to this value to give yourself some head
room. You can change this number up or down depending on how much more
you expect the game to load or if you are planning on lowering your
Texture Detail level.
6. You should now have the number of megabytes to use for the Total Virtual Memory value for the "/UserVA=" boot.ini switch.
Adding the /UserVA= Boot.ini Switch to Return Memory Back To The Kernel ----------------------------------------------------------------------
1. Go to the Start menu, Run command and type this in "attrib.exe -r +h
+s &#xSy;stemDrive%\boot.ini" to remove the Read Only attribute on this
file allowing you to change this file and save the changes.
2. Start the Notepad application and under File, Open menu type in "&#xSy;stemDrive%\boot.ini" then press Enter.
3. A text file should open and it should contain a line similar to
"multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /noexecute=optin /fastdetect" but the switches, the
folder, and the (0) (1) values might be different and that is okay. Do
not edit any of the numbers or remove any of the switches.
4. Go to the end of the line and add the " /userva=" switch with the
value that you calculated to the end with a space to separate this
switch from the previous one. Do not cut-and-paste the line from this
example since yours might be different and you could make your system
unbootable by pointing it to the wrong disk, partition, or installation
folder.
5. The line should look similar to this one after adding the switch
"multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /noexecute=optin /fastdetect /3gb /userva=2816" but once
again the (0) (1) values and the switches might be different.
6. Reboot your computer to make the switch take effect.
7. After rebooting go to the Start menu, Run command and type in "msinfo32.exe" and hit Enter.
8. Check the Total Virtual Memory and if the value is now less than
3.00 GB and matches the value in gigabytes that you calculated in
megabytes then you are finished. (Convert megabytes to gigabytes by
dividing by 1024.)
3. Under File select the Export option and save the output to a text
file such as "After-3GB and UserVA.txt" to have a record of your system
configuration.
If you experience CTD issues in the future run the
StalkerVirtualBytesPeak.cmd script again in case the game is now trying
to allocate more memory if you entered a new map that requires more
textures or if you updated the mod or the game to a new version and
larger texture files are now being loaded into memory. The value for
the /UserVA= switch is flexible so you will have to accommodate some
changes to it if you experience issues in the future. Now that you know
how to calculate the values you can repeat this process to get a new
value.
Remember that the long term solution is switching to a 64-bit operating
system to let the 32-bit processes access 4 GB or memory and later to
use 64-bit executables that will be able to address 8 TB of memory.
After the /3GB Switch and /UserVA=2816
--------------------------------------
OS Name: Microsoft Windows XP Professional
Version: 5.1.2600 Service Pack 3 Build 2600
OS Manufacturer: Microsoft Corporation
System Type: X86-based PC
Total Physical Memory: 4,096.00 MB
Available Physical Memory: 2.56 GB
Total Virtual Memory: 2.75 GB
Available Virtual Memory: 2.71 GB
Page File Space: 7.99 GB
-------------------------------------------------------------------------------
Other Information
-------------------------------------------------------------------------------
Alternative Procedure for checking Available Virtual Memory amount
------------------------------------------------------------------
1. Download the TestLimit.exe utility from Microsoft's SysInternal web page Download.sysinternals.com and uncompress the archive to a folder.
2. Start the Command Prompt and navigate to that folder using the "cd c:\folder" commands.
3. Execute the utility as "testlimit.exe -r" and see if the memory
allocation value is greater than 2032 MB, and it should be a little
above 3000 MB.
4. If it is greater than 3000 MB then the /3GB switch is now active and
your memory allocation space is split as 3GB user mode / 1GB kernel
mode.
Large Address Aware Executable Flag
-----------------------------------
The /LARGEADDRESSAWARE switch is a compile-time switch applied by the
developer when they are compiling the executable file, in the case of
Stalker it is the XR_3DA.exe and the presence of this switch will be
shown as having the flag "Application can handle large (>2GB)
addresses" when doing "dumpbin.exe /headers XR_3DA.exe" for example.
The Stalker executable was compiled with this switch already so to make
it take advantage of more than 2GB of memory is either the adding the
/3GB to the boot.ini file or running the game through a 64-bit OS that
would allow the game to allocate 3 GB of memory without any special
switches. This switch is not something that you should try to change,
unless you are a developer and want to experiment with the Microsoft
Visual C 9.0 "editbin.exe" utility that let's you change the
"/LARGEADDRESSAWARE" switch on executable files. You can also use
NTCore's CFF Explorer utility to open up any executable file and
navigate into the NT Headers, File Header, Characteristics section and
toggle the "App can handle >2gb address space" flag.
-------------------------------------------------------------------------------
Utilities
-------------------------------------------------------------------------------
Microsoft SysInternals - Process Explorer (ProcExp.exe)
Replacement for Task Manager that shows much more detailed System and Process information including Handle and DLL per process.
NTCore - Explorer Suite (Multi-Platform Version, Recommended) - CFF Explorer
CFF Explorer shows information about executable files and allows
changing execution flags, including the /LARGEADDRESSAWARE flag.
Microsoft SysInternals - Page Defrag (pagedfrg.exe)
-------------------------------------------------------------------------------
References
-------------------------------------------------------------------------------
Microsoft Help and Support
Available switch options for the Windows XP and the Windows Server 2003 Boot.ini files
Article ID: 833721 - Last Review: November 28, 2007 - Revision: 6.2
Microsoft Help and Support
RAM, Virtual Memory, Pagefile and all that stuff
Article ID: 555223 - Last Review: December 12, 2004 - Revision: 1.0
Microsoft Help and Support
How to set performance options in Windows XP
Article ID: 308417 - Last Review: November 3, 2008 - Revision: 4.1
Ask Dan: What's with the 3Gb memory barrier? - Date: 4 June 2007, Last modified 03-Aug-2008.
Mark Russinovich's technical blog covering topics such as Windows troubleshooting, technologies and security.
Mark's Blog - Pushing the Limits of Windows: Physical Memory
Techno Portal - The best Technology Portal
TechNet - Pushing the Limits of Windows: Virtual Memory
I'm running Vista but still getting crashes. Should I try this fix?
Yeah My Too Looks like you guys make a mod only for OS x64 System & XP!
i hope you make 2.5 for vista too !!
Oh ok sry about my dumb comment i have fixed for vista now : I
Sry again : )
This is brilliant and works great! Just add the line /3gb to the Boot.ini and you can play High rez with winxp 32bit OS. Plus it will help your other games.