Description

A Lua based solution to change Stalker Anomaly LTX Files on the fly once the game starts. This Library Collects changes from several 3rd party scriptfiles and applies them once on game start. **Note this is a Pre-Release Version, so things are subject to change until 1.0.0 is done and as such Semantic Versioning (important for Mod Developers only) does not apply yet - this Library is a proof of concept for now and as such I would recommend to not use it for "productive" mods yet**

Preview
Author

Update 2017-03-25:

Some smaller Changes, nothing big yet. Disclaimer further below still applies:

- Ability to define Changeset and Changes as optional (see Readme)
- Allow multiple Changesets in one scriptfile, using ChangesetCollection (see Readme)
- Refactored the way systemIni is Reloaded (only done once now)

----

I would not recommend it for production usage (aka other mods) other than to test and give me feedback yet.

Currently for JSGME or Manual Installation (so non MO2) Users the Process for Updating Mods is convoluted - I have an issue open ( Github.com ) where I'll try to remedy this somewhat (this won't be completely alleviated just yet though - also not sure if it will ever be - perhaps once I get around to Github.com )

----

There are no mods that use this Library yet, I do work on something small that serves as an example once I got this Library to an acceptable user-experience for End-Users.

Author

A quick note - this will probably take a bit longer since I'm currently working on ravenascendants (see comment below) request.

Author

The New Version which can modify Trader Files is up.

Info for Mod Developers: When switching from 0.1.0 to 0.2.0 - you need to

Rename scriptfiles that end on _ltx.script to _system_mod.script

Change the method in those scriptfiles from registerLtxModifications to registerSystemLtxModifications

-----

I'd recommend reading the Readme on Github for now, it now also contains a Table of Content Github.com

interesting approach, i had always considered using utils_data.scripts cfg_file class to edit individual LTX files.

What would it take to point this at LTX that are not part of ini_sys, most notably trade profiles?

what do you have against else statements?

Author

> What would it take to point this at LTX that are not part of ini_sys, most notably trade profiles?

Ah I see, those are not loaded in the system ini - dangit.

I did write some of the lua files inside the scripts/config/ folder to be a bit configurable but I haven't tested that part yet.

What I don't know yet is how the game behaves when these files are changed on the fly - the system.ltx has a reload function that the CoC Dev Alundaio introduced a few years back - but I don't know if changes to the Trader files would also be refreshed.

I'm gonna try a few things over the coming days.

> what do you have against else statements?

I assume you mean the part that use luas "goto" statements?

Definitely some horrible parts there. Originally I had goto in some parts to simulate what other languages have as a "continue" statement for loops.

Probably something I'll refactor out in the coming days.

Regarding the question - I generall try to avoid else unless neccessary, I used to work in a bigger Team (PHP Backend Dev) where we had tons of linters running to ensure we had a "codestyle" that was the same for everyone working there (so we could get people onboard of different projects efficiently by having some familiarity)

Author

I've made an Issue for this feature. As I assumed in my previous comment my code can already be used to create "copies" of trader files. It's a bit finicky because I made no proper API for this yet, but theoretically it works.

As predicted however, an issue arises with updating - the call to update the system ltx does not update the trader ltx files (as expected) and as such the modified trader files aren't read (probably because they get read at startup, before the changes are made via my code)

I'll investigate in the following days if one workaround I wrote in the Issue is feasible, or perhaps I'll find another one.

Author

You may be happy to hear that this seems indeed possible. I finally managed to force the game to read the modified trader ltx.

Well at least for my hardcoded testcase with sidor.

I don't yet fully understand why I have to do a weird workaround though.

It's still a lot of debugging trash I did, so I need isolate which steps are enough to clean that up in the following days so it becomes actually usable. Not really sure how long that takes, since I have to plan some sort of API for this that isn't to hard to use.

There is a high chance I have to overwrite a vanilla scriptfile (trade_manager.script) though, but again not sure what final solution I will arrive at yet - I just basically got this to work mere minutes ago.

I'll write up a more complete description of what I did in the Github ticket later.

Edit: Ops replied to the wrong comment, oh well...

With the capability to modify trader files, this looks extremely promising. Nice job.

Author

Well hopefully it works - currently it's a case of "Works for Me" since I haven't heard otherwise.

Technically (I haven't tried) when using the "*_trader_mod.script" to create Changesets you can modify any LTX already - not to say the game will load them ofcourse (depends how and when the ltx is loaded in the engine)

However that is more of a side-effect (since I don't limit the Changesets to the trade folder directly) rather than deliberate - I'll probably rework that at some point to be more generic (because there is no point in running some trader specific code on non trader files)

Well... I tried to change trader files using your readme example literally copy-pasted, and it didn't work. Not sure what's going on.

I loaded it in MO2, loaded the *_trader_mod.script file after it, but nothing changes in-game. No crashes, though, so I guess that's good?

EDIT: MO2 is run with admin privileges, so it should work. I do notice that it hasn't created any of the backup ltx files in the overwrite.

Author

Did you start a new game or did you load a game?

Can you check the logfile and look for messages starting with "LTX-LIBRARY" (or just post a link to the logfile on pastebin or something)?

New game. Also, my fault for not looking at the log earlier - I should have done that before going to sleep.

Here's the relevant stuff:

! [LUA] SCRIPT RUNTIME ERROR

and later...

! [LUA] SCRIPT RUNTIME ERROR
no file '.\gamedata\scripts\config\Ini.lua'
no file 'C:\Anomaly\bin\lua\gamedata\scripts\config\Ini.lua'
no file 'C:\Anomaly\bin\lua\gamedata\scripts\config\Ini\init.lua'
no file '.\gamedata\scripts\config\Ini.dll'
no file 'C:\Anomaly\bin\gamedata\scripts\config\Ini.dll'
no file '.\gamedata\scripts\config\Ini.lua'
no file 'C:\Anomaly\bin\lua\gamedata\scripts\config\Ini.lua'
no file 'C:\Anomaly\bin\lua\gamedata\scripts\config\Ini\init.lua'
no file '.\gamedata\scripts\config\Ini.dll'
no file 'C:\Anomaly\bin\gamedata\scripts\config\Ini.dll'

And other similar errors for the other LTX Loader scripts.

Which is weird, since the config folder and the necessary contents are all there in Anomaly\gamedata\scripts\config\. But I'm also curious - why is the directory pointing to Anomaly\bin\lua\gamedata instead of Anomaly\gamedata?

Author

Thanks. That's odd to say the least. Which Operating System are you on?

The reason it looks in Anomaly\bin\lua\gamedata is because when requiring a module it looks in several different folders (this is controlled by the package.path setting in lua)

At first I though your package.path is different but I just intentionally named a FileLoader in the autoloader.script require wrong and we seem to have the virtually the same package.path (well except the Disk and Anomaly Foldername)

In fact the very first "no file" message is the same and it should find the file there (if you remove the extra "r" at the end of my logs), but doesn't

My Logs (extra "r" just to see the package.path errors)

Yet for me it works... that's so odd.

Can you try and replace

with

in the autoloader.script and tell me if it the error message of autoloader.script disappears?

In the meantime I'll create an Installation on my C: Drive aswell - I wonder if this is perhaps some sort of file protection issue on Windows (assuming you are on Windows) even if you do run the game in Admin mode.

Edit: Its not - at least on Windows 7 - I moved my installation there and tried with "manual" installation aswell as MO2. Though I am on a non-admin account on my system.

In the worst case I can just move all files back into the script folder itself I think.

I'm on Windows 10 x64

The change you suggested gives the same error, more or less:

! [LUA] SCRIPT RUNTIME ERROR
no file '.\gamedata.dll'
no file 'C:\Anomaly\bin\gamedata.dll'
no file '.\gamedata.dll'
no file 'C:\Anomaly\bin\gamedata.dll'

Yeah, it very well could be some kind of hidden Windows protection.

Also, thanks for the prompt replies. This LTX loader does really seem like the next big step for Anomaly modding if it can work across most systems

Author

I got it I think - I always launch the game via the Launcher (from MO2) if I use one of the EXEs directly I get the same issue. Which I guess is to be expected because the Launcher is one Folder "up" in the file hirarchy.

I'll see what I can do.

Perhaps you can try to verify if this works for you aswell (launching from the launcher, that is)

Edit: Documented this as Issue on Github: Github.com

Hey, it works! And the trader example works perfectly, too. Thanks for finding the issue - interesting that that was the cause. Yeah, I usually start from the exe since it's faster. Hopefully something can be done to make it functional in either scenario, since any given end-user could use either the launcher or an exe to load.

I do have some comments about the LTX files created by the loader, if you don't mind:

- The backup and temp files created in the overwrite directory still have .ltx extensions. Some config files read wildcard files in their directory, like #include "items_*.ltx" which will read any file starting with 'items_', which could potentially lead to duplicate sections? Perhaps putting the 'backup' or 'temp' designations at the start of the file might solve any potential future issues that might pop up? Or temporarily changing the file extension to .bak or .backup, etc. I think this would be necessary if you want to expand this loader to be able to modify config xmls dealing with squad defines and quests, since those read entirely wildcard filenames. But you know far more about these details than I do, so maybe I'm completely wrong.

- I can foresee issues where an end-user with MO2 installed this loader as a requirement for an addon, and then deletes the addon and loader, but doesn't manually clear the accumulated LTX files in the Overwrite. Is it possible for this library to instead create the backup/temp files in the library's MO2 folder instead of the overwrite? That way, deleting the loader also deletes the extra LTX files. The user would have to place the LTX loader last in their load order, but that seems easier for end-users to do versus remembering to delete stuff in the Overwrite.

Anyway, thanks again for this loader and your help - looking forward to using it for some addons I'm making!

Author

> The backup and temp files created in the overwrite directory still have .ltx

Good point - I'll made an Issue for that Github.com.

Not entirely sure if it works with the ini_file_ex() function (and therefore the temp file) - but I'll take a look

> I can foresee issues where an end-user with MO2 installed this loader as a requirement for an addon, and then deletes the addon and loader, but doesn't manually clear the accumulated LTX files in the Overwrite

Yeah that is my gripe with my Library aswell - it's why there is an entire Uninstall Section dedicated to this.

I did try, infact, to delete files when quitting the game (still leaves the Problem when the game Crashes though), but some simply cannot be deleted (perhaps due to pointers not getting cleared or the game simply writing the files again). I'm not entirely sure why.

Currently as it stands, when the game quits it at least installs the "vanilla" file from the backup that exists. But well again, if the game crashes that doesn't happen.

I don't think I can save to the MO2 Mod Directory because the LUA files have no knowledge of the Folderstructure once the game launches (because MO2 uses some sort of Virtual Filesystem). The fact that the files end up in Overwrite is probably because MO2 cannot ascertain which mod they came from.

However played with the thought of at least moving the backup file to another folder. However since the temp file is loaded with ini_file_ex() it errors on ini files that include other ini files - probably because ini_file_ex also parses the files and cannot find the included inis.

Another Idea I have written down in an issue for this: use luas IO functions to write my own "ini handler" which does not parse the files -> Github.com

Then I could move the temp file to a custom folder aswell. Still leaves the issue with the "original" file though. This does make editing the system.ltx more complex though. That is because currently you can just say "change section X" and due to the fact that the "system.ltx" gets parsed (and thus includes all sections of weapons, items, ... aswell) you don't need to specify the ltx you want to edit, unlike for the trader files where you need to specify them.

I wish we had a way to get ahold of the "in-memory" ini instances the game uses, that way one could just do the modifications in memory directly - but I don't think that's possible currently - at least I haven't found anything

Ah, thanks for the detailed answers.

I think if you fix the issue about the backup/temp LTX file names, my latter comment isn't a huge deal. It would be extra (small) file bloat, but harmless in the end. It's probably not worth trying to address, especially with crashes (something I completely forgot).

If you can get this working with either .exe or launcher startups, I already have an addon that I want to use with this loader. Definitely looking forward to future updates - again, nice work.

Author

The issue with it only working from the Launcher should be fixed

However I would not recommend it for production usage (aka other mods) other than to test and give me feedback yet.

I've edited my first comment to include some detail about that

Edit: Ah and I also removed the .ltx from both the backup and temp file - works for me.

Amazing! And so quickly, too. Thanks for the update - I'm gonna test this for an addon in the works, and I'll send what feedback I can to you.

What kind of feedback would be most useful to you?

Author

I'd say if you encounter any properties that won't work since I haven't tested all possible properties that LTX files can have. Some might not even be fixable (e.g. if some of the properties get "cached" in some obscure way in the engine) but documentation about that would be helpful.

Otherwise general User Experience (from an End-User / Modders perspective, for example if anything in the Readme is off), Bugs, etc.

Currently I'm working on making the detection of changed vanilla / modded LTX files a bit better (so it requires less End-User interaction) but I don't foresee that being done quickly since I have to explore different solutions. Github.com

"If you did the archaic "I installed everything manually" way to install mods, you need to at least remove the following files (or better yet, start fresh - I mean that's what you get by doing it manually when there are better ways like JSGME or even better MO2)" .. less bias please.

Author

I'll change it.

Author

Done - on the rchive Readme / Moddb Description aswell as the Github Readme.

tyvm. :D

Alright, cool - It works perfectly for both system ltx and trader ltx changes.

Question: is there a way to bundle multiple changesets into a single *_mod.script file? For example, if I want to change two different trader's inventories with the same Changes, is the only way to make two separate *_trader_mod.script files?

I'm thinking about a situation where a mod creator (I guess me, in this case) wants to add new items to most of the traders in the Zone (~12 different LTX files). Is there a way to condense those into less than 12 separate files?

Author

Not right now - I have something similar in mind (only noted in the Roadmap for now but not yet planned out - something like Autoloading LTX files from mod specific folders and generating the Changesets automatically) Github.com

But that is probably still some time in the future.

I think I could implement it so you can nest Changesets one level, for example

local someChangeA = Change("supplies_1", "device_pda_1", nil)
local someChangeB = Change("supplies_1", "device_pda_1", nil)

local a = Changeset({someChangeA}, "Sid", "Sid.ltx")
local b = Changeset({someChangeB}, "Barman", "Barman.ltx")

return Changeset({a, b}, "Barman & Sid")

Not sure if I like the code but if I'm not mistaken that would be relatively simple to tack on.

I'll take a look at it this weekend - I'm kind of exhausted after 8 hours of trying to wrestle with the *.db files Github.com (been trying to find a solution to make updating / removing other Mods easier on End-Users)

No worries, take your time, you deserve the break. Thanks again for the info and support

Author

Just as a heads-up I've implemented the above solution a little bit differently - there will be a new class (ChangesetCollection) that you can return instead of a Changeset (so you can return either a Changeset or a ChangesetCollection)

local someChangeA = Change("supplies_1", "device_pda_1", nil)
local someChangeB = Change("supplies_1", "device_pda_1", nil)

local a = Changeset({someChangeA}, "Sid", "Sid.ltx")
local b = Changeset({someChangeB}, "Barman", "Barman.ltx")

return ChangesetCollection({a, b})

I like this a bit more than the nesting approach I was going to take originally.

It isn't out yet though, I addressed a Performance Issue that would've reared it's head eventually and the last thing I need to finish before I'll release 0.3.0 is a way for Modders to specify if they want a changeset to still apply, even if a change inside the changeset is faulty (right now a single faulty change out of hundreds could cause the entire changeset to not apply). Probably takes a few days still.

After that I'll probably have to spend some weeks on the bigger issue with making updating / removing / etc easier on the End-User aswell as some badly needed architectural changes (e.g. tests, actual error handling instead of this weird error bubbling approach I implemented)

The progress you've made on this is really amazing, dude. I especially like the changeSetCollection addition.

As I was testing the older version, I tried a test where there were two changesets altering the same property/line. Example: changing Sid's [supplies_1] to have 3 PDAv1, and then having a separate file change it to 10 PDAv1.

From my limited testing, it seems like the priority of the changes is alphabetical based on the first letter of the filename, right?

This could conceivably become an issue if the LTX Loader becomes the de-facto way to make config file changes. In such a future, load order would be ignored in favor of the alphabetical sorting. Is this simply a limitation of the LUA library and MO2? The ideal way to handle this would be to grab the MO2 priority data and load the changeset scripts in the correct order based on that priority data. But is that even possible? Since, as you said before, MO2 doesn't distinguish between individual mods once the game is running, and this loader doesn't run until the game starts.

In any case, what you've done here should be really helpful for future addons that want to make less-intrusive, more compatible changes.

Author

Ah yes, for now I've designed it so that it's based on the alphabetical order(or at least it should work that way) - that way mod authors already have a way to somewhat control the load order until I come up with something better.

"Very rudimentary load order system (not dependency management!) so modders can specify in which order their mods should be applied given that other mods exist"

This would be the best way I can come up with for now, since this works with any way to install Mods and not just MO2. I suppose the Mod Author(s) should also know best in which way the mods should load (compared to the End-User).

Further down the line, perhaps there could be some sort of config file where an End-User can overwrite the Load Order (incase a mod author goes missing), if there happens to be a need for it. Not sure how / if MO2 has some sort of Plugin Capabilities, but if so perhaps it could be used to write a Plugin that exports the config which specifies the order aswell.

----

I'm not sure if any updates will come out in the coming weeks. Right now I'm hitting my head against a concrete wall, since I'm trying to find out if I can somehow "update" the gamedata Path (to e.g. gamedata_temp) or add a new path and make the engine read loose files from there.

There are methods for this, but update_path doesn't seem to do what it says it does (it just seems to return a "parsed" path and not update the path the engine uses, the few uses I found in vanilla scriptfiles seem to support this, regrettably) and append_path just crashes on the spot with some generic windows file error and no actual detail as to where it happens.

Doesn't help that I'm not experienced with C++ or Debugging it either (tried to use WinDbg on the mdmp generated by Anomaly, but that too did not yield anything someone like me could work with, pushing aside the fact that I miss the neccessary Debug Symbols aswell. I suppose I could google around and see if I can somehow "generate" debug symbols myself If I download the Anomaly engine and build it or something along those lines)

No worries, this was something I noticed during testing that would only become a problem in a far-off hypothetical future.

As the loader stands now, I think it's probably ready to be used as a dependency for mods going forward.

Again, very nice work

Author

> As the loader stands now, I think it's probably ready to be used as a dependency for mods going forward.

You can if you want, but in general I would feel better about this, once I made some improvements to the end-user experience with regards to updating their mods.

Since due to the nature how this Library currently works, if you for example have a mod that edits an LTX directly in addition to a Mod that uses this Library, the End-User has to manually remove the .backup and .temp files each time the Mod that edits the LTX directly is updated.

If this is not done, the End-User won't get the updates of the Mod that edits the LTX directly, since the Library uses the .backup file (that remains unchanged once it has been initially created) as a "source" to write the Modifications to.

For MO2 users that keep their install clean that's easy since they can just nuke their "Overwrite" entry but for everyone else that uses JSGME or Manual install this is a hassle.

Once I've solved that I'd feel better about encouraging the usage of this Library.

Especially because this might become a de-facto standard I want to get this mostly right before it is widely adopted.

Would this be viable for ltx files loaded in via scripts? Such as:

Said file has a number of npc_loadout_xxx.ltx #includes located within that same items\\settings\\ folder.

I'm going to assume no, this ltx loader wouldn't be viable for this - as those npc_loadout.ltx files are loaded in via a specific init_settings() within a different script, loaded in during an on_game_start() - but I wanted to check just in case, as I'd rather not modify those scripts directly to account for the behaviour I desire

Author

Sorry for not seeing your comment for a day, I'm currently in the process of moving from windows to linux so I cannot check as I'm in the middle of getting MO2 + Anomaly to work properly - once I got it running I'll take a look if what you want to do works.

----

It might work if the file in question doesn't get cached somehow when starting the game from desktop.

As long as the function ini_file(...) is not called directly in the script (so outside any function like on_game_start) it should in theory work.

My script runs before on_game_start is executed and makes changes to the files.

Anomalies "on_game_start", that most scripts use to register callbacks or run other scripts, runs when the "loading" screen appears - so only once you started a new game or loaded an existing savegame.

A note on the #include - if you edit "items\\settings\\npc_loadouts\\npc_loadouts.ltx", the #includes inside that file get parsed and the file then contains the contents of all files that are normally included via #include

(this is expected for now and working "properly" - this is a sideeffect of me using the vanilla anomaly functions, until I get around to write my own methods to handle the ltx files Github.com ).

This means if you want to change any properties of the included files you can either use the main file "items\\settings\\npc_loadouts\\npc_loadouts.ltx" or you can edit included files like for example "items\\settings\\npc_loadouts\\npc_loadouts_stalker.ltx".

But you should not try to do both at once, as this can have side-effects like changes from included files not applying.

In this case I would suggest modifying the included files, given that you do not need to change any properties like "scope_chance" in "items\\settings\\npc_loadouts\\npc_loadouts.ltx"

Did you get MO2 Working on Linux!
Bruh, Please god tell me, then I can finally stop dual booting

Only registered members can share their thoughts. So come on! Join the community today (totally free - or sign in with your social account on the right) and join in the conversation.

Tags