Belovèse tool for extracting data from EDU
Welcome everyone,
Here is the link to the new thread for this tool I've posted the new version, that can also export EDA and EDCT
Please post here any bug report, feedback and suggestions.
How to use:
- Download and unzip.
- Keep the folder as it is: for the program to work you must have EDU_to_CSV.exe, EDU_to_CSV.mxf and subfolder "resources" in the same folder
- Run EDU_to_CSV.exe (not .mxf)
- Select a file, eventualy select what parameters you want to see, then click save and save in .csv format.
- Finally open the .csv file with a spreadsheet editor (Excel, LibreCalc, what you like)
- If asked, use TAB as separator for the import to a spreadsheet.
- enjoy!
Changelog
Here's a vague roadmap
Here is also a tool to parse EDU to export_unit.txt (to add more info on unit cards), provided by HaruHalas and based on a script by Cryoshakespear.
Instructions:
Original post:
Re: SSHIP - General Discussion
I don't use it, frankly speaking. You may have a look at the script.txt for 097, I think I've made some order and put a few hints for the next modders.
What I need the most is a program that would transform EDU (export_descr_units.txt) into a spreadsheet file so that I can get something like this (this one is for 092):
https://i.imgur.com/OA5xuFt.jpg
Re: SSHIP - General Discussion
Jurand, I tried a few things and could make something like this: https://we.tl/t-rwled9oEKW (it's a excel file)
It's a first draft, and I did not went through the whole file to search for mistakes: I would like to know first if it's what you need, before putting more work on it.
A few limitations: I can't use the stats that are in two words (like "banner holy") or are on two lines (like "officer")
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Belovèse
Jurand, I tried a few things and could make something like this:
https://we.tl/t-rwled9oEKW (it's a excel file)
It's a first draft, and I did not went through the whole file to search for mistakes: I would like to know first if it's what you need, before putting more work on it.
A few limitations: I can't use the stats that are in two words (like "banner holy") or are on two lines (like "officer")
Hi Beleovese, this is what I was looking for. It will make my life easier, for sure.
Is it possible to have in a separate table:
- the text that is after ; in the entry: dictionary (there're in-game names and types of the units, so it's an important entry)
- the numbers in the "soldier" entry (so that there're two columns: one for soldier, the other for numbers (it's important as the first of the numbers signifies the size of the unit)
- in "stat_pri" the first two numbers in one column, and the rest in another column
- in "stat_sec" similarly, but if the numbers are "0 0" then the it'd be left empty
- "stat_pri_armor" - also numbers and text separated
- "stat_ground" - the best would be if each number would be in the separate column
- "stat_mental" - each should be in the separate column (morale, discipline, training)
- "stat_cost" - first three numbers should be in the separate columns,
Re: SSHIP - General Discussion
I'll give it a try tomorow, to see what I can do. If it work I'll post the program. In the dictionnay entry I think I'll have trouble separating data, so it'll likely be a single column for that.
Since last post I got curious and managed to integrate the banner and era entries in the program.
What I'm writing is not very efficient, as I only know one programming software that is designed for live music! But hey, if it work's, it ain't stupid, or so they say.
Re: SSHIP - General Discussion
I would also be interested in hearing how this ends up working out. I myself am not super gifted with crafting such programs, but a friend of mine has fun making crawlers and bots to glean information from certain websites (nothing sinister; he uses it to help him figure out what kind of fishing rod to buy, and :wub: like that, by comparing prices and models across multiple platforms). But he is good to making rather simple things that are good at collecting and crunching info from tabular environments (the structure of the EDB makes it likely that he could do something for that as well). So yeah, I'll keep an eye on this.
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Belovèse
In the dictionnay entry I think I'll have trouble separating data, so it'll likely be a single column for that.
In this entry there's always ; used, I mean: each and every unit has it (or should have it). Given this "no exception" rule, I think that it could be not so difficult.
Re: SSHIP - General Discussion
Here is what you asked, Jurand, I think: https://we.tl/t-ZLVFyGVjfU
If it's what you need, I just have to make the interface more understandable and a few tweak, and I could post it with a mega or dropbox link
I'll make it so you can choose the stats you want to export. If you point me to the right documentation I'll use better names for the columns also. If most of the mods use the same EDU structure, I'll post it in the worshop I think
Quote:
In this entry there's always ; used, I mean: each and every unit has it (or should have it). Given this "no exception" rule, I think that it could be not so difficult.
indeed you were right!
Quote:
I would also be interested in hearing how this ends up working out. I myself am not super gifted with crafting such programs, but a friend of mine has fun making crawlers and bots to glean information from certain websites (nothing sinister; he uses it to help him figure out what kind of fishing rod to buy, and :wub: like that, by comparing prices and models across multiple platforms). But he is good to making rather simple things that are good at collecting and crunching info from tabular environments (the structure of the EDB makes it likely that he could do something for that as well). So yeah, I'll keep an eye on this.
I also would like to learn how to program like your friend, I think he could make something lighter and more powerfull! Someday...
Re: SSHIP - General Discussion
That's great, @Belovese!
I think it's very difficult to split the "dictionary" so that the "type" (feudal knight, urban militia...) are in a different column? What I'd like to have is possibility to group the units by type - to compare them to see discrepancies etc.
It's very close, would it be possible to have:
- stat_pri numbers (and stat_sec as well) in two different columns?
- stat_pri armour in three columns (this is very important: they describe quite different characteristics of the unit),
Re: SSHIP - General Discussion
My bad, I used the same link twice and can't edit my posts yet, here is the program: https://www.dropbox.com/s/bx1pf4ez01...o_CSV.zip?dl=0
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Belovèse
@Jurand and @Kilo, here is the program:
https://www.dropbox.com/s/c1rusr30hp...space.txt?dl=0
I could do what you asked, except for the "dictionary" entry. The names and types of units are not always the same number of words, and I could not find a way around that. Best solution I can think of is to use the filter options in Excel (like I did in the file included), sorry!
There is a small bug, caused by empty lines inside og a unit stats. In the file you sent me Jurand, it was in Druhzina, Mutatawwi and Sipahi. Here is a file without these lines:
https://www.dropbox.com/s/c1rusr30hp...space.txt?dl=0
Would it be possible for you to make this separate entry if we'd put in the dictionary line for each of the units an additional " ; " (or any other character) between the name and the type?
The empty lines will be removed.
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Jurand of Cracow
Would it be possible for you to make this separate entry if we'd put in the dictionary line for each of the units an additional " ; " (or any other character) between the name and the type?
The empty lines will be removed.
I'll give it a try first, I'm really not sure so I have to test. Howerver, It fells like reworking the entire EDU is maybe more work than just using the "filter" option in Excel. But if it'll help you I will try of course!
Asides that, does it work as expected? I forgot to replace 0 0 by --- in stat_sec, and I will also search for what is exactly is each number to give the columns better names, more descriptive.
Re: SSHIP - General Discussion
yes, it does, it's already a great tool for me to compare units.
on the column titles you may have a look at the SS6.4 units data file
on the dictionary - I think I can make very easily (search ", feudal levy", replace with " ; feudal levy" will do for 107 units, and other types as well, then only looking for exceptions)
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Jurand of Cracow
Would it be possible for you to make this separate entry if we'd put in the dictionary line for each of the units an additional " ; " (or any other character) between the name and the type?
The empty lines will be removed.
I'll give it a try first, I'm really not sure so I have to test. Howerver, It fells like reworking the entire EDU is maybe more work than just using the "filter" option in Excel. But if it'll help you I will try of course!
Asides that, does it work as expected? I forgot to replace 0 0 by --- in stat_sec, and I will also search for what is exactly is each number to give the columns better names, more descriptive.
Re: SSHIP - General Discussion
Hi Belovese,
your program does work as expected. Here is the EDU without empty lines (those 3 cases), and also with " ; " in dictionary line for each unit separating a very important info for us: our type of unit.
If you could add the column for that additional info, and separate two numbers in the stat_sec, this would be great!cheers
JoC
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Jurand of Cracow
Hi Belovese,
your program does work as expected.
Here is the EDU without empty lines (those 3 cases), and also with " ; " in dictionary line for each unit separating a very important info for us: our type of unit.
If you could add the column for that additional info, and separate two numbers in the stat_sec, this would be great!cheers
JoC
Happy to hear it's working well! I followed your idea and can indeed make separate entries for "dictionary", but can't use punctuation (because of how the program I use works). So I will used a word instead, and since "type" is already an entry is the EDU, I will use "class" instead if you agree. I will update the program and the EDU you sent me, sometime today I think.
If you have other improvement ideas I'll be happy to hear them!
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Belovèse
Happy to hear it's working well! I followed your idea and can indeed make separate entries for "dictionary", but can't use punctuation (because of how the program I use works). So I will used a word instead, and since "type" is already an entry is the EDU, I will use "class" instead if you agree. I will update the program and the EDU you sent me, sometime today I think.
If you have other improvement ideas I'll be happy to hear them!
Hey Jurand, I tried to implement the dictionary entry but it kind of messed the program (or I did something, don't know what!). I'll continue to work on it but can I open another thread to easily post updates and stop spamming the general discussion thread?
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Belovèse
Happy to hear it's working well! I followed your idea and can indeed make separate entries for "dictionary", but can't use punctuation (because of how the program I use works). So I will used a word instead, and since "type" is already an entry is the EDU, I will use "class" instead if you agree. I will update the program and the EDU you sent me, sometime today I think.
If you have other improvement ideas I'll be happy to hear them!
Class is also an entry in the EDU, and actually it's pertinent (light, heavy, missile or spearmen).
On the other hand, "type" is an entry, but it has not relation to the concept of "type" (many similar units), it simply an internal code name of the unit (one unique name).
So I'd stick to the "type" name of they types, but I think I'd tinker a bit with the types (discussion on this in another thread).
I'm looking forward to that programm from you, already the current enabled me to spot many things in the EDU.
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Jurand of Cracow
Class is also an entry in the EDU, and actually it's pertinent (light, heavy, missile or spearmen).
On the other hand, "type" is an entry, but it has not relation to the concept of "type" (many similar units), it simply an internal code name of the unit (one unique name).
So I'd stick to the "type" name of they types, but I think I'd tinker a bit with the types (discussion on this in another thread).
I'm looking forward to that programm from you, already the current enabled me to spot many things in the EDU.
I figured that halfway through changing the EDU you sent me... :whistling so I went for "SSHIP_class" as keyword, and here is the EDU based on the last you sent me. I hope you haven't yet made any changes!
About the program I went for a full rework, but it should more flexible and easier for me to add new functionalities, I think it'll take me a few days.
Thank you for opening the thread, as soon as I can edit my own posts I'll clean the first one and will put everything in the same place.
Re: SSHIP - General Discussion
Quote:
Originally Posted by
Belovèse
I figured that halfway through changing the EDU you sent me... :whistling so I went for "SSHIP_class" as keyword, and here is the
EDU based on the last you sent me. I hope you haven't yet made any changes!
Well, as a matter of fact, we did. I didn't know you'd be making changes in the EDU. There're some accents changed by JLMP; I've also run your program, had a look at some units, and adjusted many plances (eg. recruitment times of urban militia, or some heat stats, and also added type "artillery" to the relevant units). Your program is very helpful in finding odd things.
EDU improved version is here.
However, I think it's easy for you to make your changes: just Ctrl-F " ; " and replace with your text, and you'll get 484 cases (so 10 units missing, to be found otherwise).
But I would be inclined to use in the EDU something more simple, why to make long < dictionary Javelinmen ; Levy Javelinmen SSHIP_class feudal levy >, and not simply < dictionary Javelinmen ; Levy Javelinmen ; feudal levy > ?
And just drop a line before if I shouldn't meddle with EDU ;-)