Perfect ESF<->XML converter and revolution in ESF modding [Released!]

[taw]

So now that every single DB table except one is moddable, the next step is ESF.

Current ESF architecture makes even the simplest changes really difficult, and you're only allowed one change - there's no esf merging ever (hybrid startpos.esf tricks aside).

I didn't like any of it, so I wrote a pair of scripts that convert ESF to XML, and then XML back to ESF. ESF->XML->ESF round trip is perfect to the very last byte on startpos.esf, regions.esf, and every other esf I tested.

Once data is in XML format, all sort of mass modifications like removing every single fort (db-based no forts mod horribly confuse AI, esf-based no forts mod don't have such problems), flipping factions from minor to major, changing character skills and abilities, giving every minister a mistress, changing towns minor to major, and many more change from long painful effort to trivial find-and-replace-via-xquery, and a brave new world of far more interesting modding becomes wide open - including all CAI stuff that hides inside ESF for no apparent reason, and maybe even new campaign map because why the hell not.

This is good kind of XML - there are binary offsets, or other nonsense in it - conversion scripts handle all that, you can just go ahead and edit.

Right now the most serious practical problems are:

• These XMLs are really huge - startspos.xml is 473MB (75% of that is CAI-related stuff, not even binaries). I should be able to cut it at least in half without losing anything of value, but it will still need a really good computer.
• Conversion takes about 4 minutes each way. 4 minutes is nothing compared to an evening of trying to get things done with ESFEditor, but it should be faster.
• It's all Ruby code that might be hard to get running on Windows. I'll write some detailed instructions at least.

Once these three are done I'll publish the scripts, hopefully in a few days and together with collection of easy to use recipes for making amazing things with ESFs.

If you want early alpha access, ask me on PM or email, but it's still quite messy, and stress testing has been limited.