Atlas of the Imperium Data?

[Nathan Brazil]

Did anyone ever actually compile the Atlas of Imperium into some data format? I mean specifically, take the Atlas, record the information for each hex, stand alone without cross-referencing it with other and perhaps more accurate sources?

 

[aramis]

The source data for it was released.... on GEnie, many years ago.

 

[BytePro]

Did anyone ever actually compile the Atlas of Imperium into some data format? ...

Yep - Marc!

Now I never had the AotI, but in researching data files over the decades, I stumbled upon the Genie/Sunbane data and later sets and here is what I came up with (at least from comments in my source code - can't vouch for any of it - maybe Marc wasn't actually the programmer?)...

Marc wrote a program a BASIC program that ran on the Apple II+ to generate the data (though, also did manual editing) for the Atlas of the Imperium's 35 sectors, but did not include UWPs and TLs.

DGP released 30 sectors of (poorly generated) data based on AotI (assume data files from Marc) to GENII (and with copies - possibly data corrupted - hosted on Sunbane) that included UWPs, but no Stellar data and only names from AotI (IIRC). The 'missing' sectors were from published sources (Old Expanses, Reaver's Deep, Reft, Solomani Rim, Spinward Marches).​

If you wanted the 'originals' - missing 5 having come after - grab the Genie data (not sure where that is anymore - try The Missouri Archive at http://traveller.mu.org/) and strip the DGP data (see http://www.downport.com/amv/software/Genie.html).

Travellermap.com, IIRC, matches pretty well, with the credits stating Robert Eaglestone and Jim Fetters (of this board) had 'scrubbed' the data to match the AotI.

 

[RainOfSteel]

Did anyone ever actually compile the Atlas of Imperium into some data format? I mean specifically, take the Atlas, record the information for each hex, stand alone without cross-referencing it with other and perhaps more accurate sources?

There was once an Access 95 database of the UWP information available on Doug's World Software Page, however that website no longer exists. The URL was:

http://traveller.geekoids.com/software/mainpage.html

I tried the Wayback Machine, but while it has a copy of the page, it does not have a copy of the file to download.

The database contained a single table that resembled a standard flat file. I had originally imported into Access 2000 and did a variety of things with it like:

How Do *You* Calculate YTU's Budgets?

I still have a copy of the unaltered original download (well, it went through import into Access 2000, but that is it). It is 5 megabytes in size. If you want a copy, PM me your e-mail address.

I can also export to a variety of other formats, but MS Excel would probably be the most common.

I make no claims about the quality of the data, but a great deal of it looked correct.

 

[Gray Lensman]

http://www.travellerrpg.com/CotI/Discuss/showthread.php?p=369899#post369899

I have Forgotten more websites than I remember so I do not remember where I got this from.

If I step on someone's toes I am sure the Moderators can pull it quick enough. o:

 

[aramis]

That's a pre-Excell-97 file... o:

 

[RainOfSteel]

The file name of data.xls looks very similar to the data.mdb file I downloaded from Doug's web site many years ago.

Except the MS Excel file splits each sectors into its own worksheet page. (Since I think in terms of importing data into MS Access or some other database where I can use SQL on it, that's a disadvantage.)

EDIT------

I just spent some time counting the number of rows, and they're not consistent, so maybe they're not the same.

Another EDIT------

Further digging reveals I also have a copy of data.xls from back in 2003. There may be some differences between it and data.mdb. I never really checked before as the data.xls file was not important to me at the time.

 

[RainOfSteel]

I happened to be glancing at:

http://www.travellermap.com/

I was skimming through Core sector Core subsector, glancing at Capital, and was looking for Ion/Core 2016 per my data, the source of which is the files we've just been discussing.

On the website above, hex 2016 contain Zuarkri/Core. I'm not sure how the difference came into being.

I've already done some work on Ion/Core, writing it into various bits of MTU Capital and Imperium lore and frankly have no desire to change it.

 

[RainOfSteel]

I think I've located it. Zuarkri is a Milieu T0 name.

 

[inexorabletash]

FWIW, the T5 data cleanup thread is over yonder:

http://www.travellerrpg.com/CotI/Discuss/showthread.php?t=5832&page=14

That part of it actually mentions Core sector world names. Robert (robject) may remember the details more completely than I do, but I seem to recall the guideline of using Vilani/Sylean names from T4 in preference to names introduced by HIWG if the worlds aren't named in AOTI, unless there were other canon references to the world.

The map site uses preliminary T5 data where possible, but that's still being worked on and I haven't updated the site's data recently. The data also doesn't live anywhere I can point at (in the details at the bottom of the map), so it's a bit messy - sorry!

 

[RainOfSteel]

FWIW, the T5 data cleanup thread is over yonder:

http://www.travellerrpg.com/CotI/Discuss/showthread.php?t=5832&page=14

I recall it.

 

[Nathan Brazil]

Thank you for the replies, I think that I might not have asked the correct question. I was looking for data in a format where all that is entered is the data in the format of the actual Atlas of the Imperium product. Specifically:

Hex Number
World type (Desert, Asteroid Belt, Other)
Gas Giant Presence
Naval Base type (if present)
Starport Type
Secondary Bases
Name (if pop 9+)

and that is it. Otherwise the actual precursor data which was used to generate the actual maps for the Atlas of the Imperium product. Did any of the previous data match that? I get the DGP part not being original, just wanted to make sure about the other data sources.

 

[inexorabletash]

If you look at the thread I referenced, it was basically an attempt to take the GEnie data and ensure it matched what was published in the AOTI (exactly the parameters you list), for worlds inside the Imperium. That project's output is probably the closest thing to what you're looking for that is known to exist. I'm not sure if robject is still hosting the data anywhere public, though.

(There is also this tool: http://www.downport.com/amv/software/Genie.html - but it's not going to correct mistakes in the transcription.)

 

[Vargas]

Keep in mind that, if you didn't already know, the UWPs in the original files are incredibly buggy. There was a portion of an article in Challenge which addressed this fact in the early 90s.

 

[RainOfSteel]

Thank you for the replies, I think that I might not have asked the correct question. I was looking for data in a format where all that is entered is the data in the format of the actual Atlas of the Imperium product. Specifically:

Hex Number
World type (Desert, Asteroid Belt, Other)
Gas Giant Presence
Naval Base type (if present)
Starport Type
Secondary Bases
Name (if pop 9+)

and that is it. Otherwise the actual precursor data which was used to generate the actual maps for the Atlas of the Imperium product. Did any of the previous data match that? I get the DGP part not being original, just wanted to make sure about the other data sources.

There was no "one" original source of data. Supplements 3 and 10 were the only fully official GDW sectors published that I recall. Everything else was produced by <insert question marks here>, sometimes by procedures that did not appear to match those in Book 3, 6, or anywhere.

I have no clear idea how the "GEnie/Sunbane" data was originally produced.

GEnie was a pre-Internet BBS (Bulletin Board Service) where lots of interesting things happened, but it adamantly refused to convert to Internet Usenet Newsgroups or a Forum-style system. It ultimately died. The point is, the data was originally assembled there (again, as far as I know).

I have seen other sources of data. They conflict and, IMO, are inconsistent with each other and the AotI map.

The main source of data I had ("main" because it was in Microsoft Access database format, making it easy to work with), had a ridiculous mixture of allegiance information from different eras (Deneb was all Vargr controlled). Back in the early 00s I spent days working on trying to fix it, and I never did get it completely right. I was only just able to go through it and correct the rimward border world allegiance codes (thanks to http://www.travellermap.com/).

The reason for that was that though I had AotI in the 1990s, I could no longer locate it and had to just guess based on the large scale poster map.

However, if the map at (http://www.travellermap.com/) is the same as the AotI, then if you look over the Antares sector, you will locate three planets with the name "E". Yes, three planets with a one-letter name, and two of them are in the same subsector (hexes 0309, 0607, 1031). The rimward borders being displayed on it do not match the border on the original "space map" of the Imperium showing a large bulge from Lishun into Meshan. So, the information being displayed here is not perfect.

On the other hand, the GEnie data file I have doesn't even list planet names for Antares, although the UWPs appear to match.

Here's a factoid, the files show fewer than 11,000 worlds with Imperial allegiance codes.

8792 in my most recently corrected GEnie data, 8425 in the data I picked up from the T5 playtest, and 9073 in a file I picked up from a website I can no longer remember that called it "Second Survey" data.

And on top of all that, there exists a computer program called Galactic that is a sector/subsector map-viewer for Traveller UWP information. The data in it may not match the rest. I didn't do any checking, but the documentation refers to a new system of generating world data.

In short, there is a hodgepodge of UWP information out there.

--------------

The website posted by inexorabletash, http://www.downport.com/amv/software/Genie.html, has some interesting information on it about the GEnie/Sunbane data.

 

[Qstor2]

what was the article in Challenge that you were referring too? Thanks!

Mike

 

[Frankymole]

Here's a factoid, the files show fewer than 11,000 worlds with Imperial allegiance codes.

8792 in my most recently corrected GEnie data, 8425 in the data I picked up from the T5 playtest, and 9073 in a file I picked up from a website I can no longer remember that called it "Second Survey" data.

Those 8000 or 9000 are mainworlds though. One of the Q&As in (I think) a DGP Digest or Journal said that the 11,000 inhabited worlds of the Third Imperium contain secondary worlds in a system too, so if inhabited systems have on average 1.4 inhabited worlds (main and secondary) then it's right.

 

[GypsyComet]

There was no "one" original source of data. Supplements 3 and 10 were the only fully official GDW sectors published that I recall.

Prior to the publication of AotI and in the 35 sector region of the AotI, yes.

Several sectors were visited for minor or partial coverage by GDW or were adopted by GDW, including bits of Trojan Reach, Gvurrdon, and Reft. FASA's work in Old Expanses and Reavers Deep was also later adopted.

Two whole sectors were published by GDW in Challenge. Both are outside the AotI region: Hinterworlds and Mendan.

GDW also published Diaspora, Deneb, Trojan Reach, and Reft whole for TNE, and the DGP version of Massilia was published by GDW in Knightfall for MT.

 

[robject]

Jim Fetters and I grepped through the genie data, compared them with AOTI's data, and produced some corrected sector files.

File Format
Please note that the file format on several of those files is lightly salted with XML. I considered the minimal amount of XML necessary to aid document organization without changing the overall look of the file.


Joshua's summary:

Some initial UWP cleanup discussion on COTI turned into a full scrub of AOTI data on COTI (Imperial-only worlds), direction approved by MWM. Initially it was to just create an AOTI-pure dataset (i.e. drop DGP-sourced data and HIWG-introduced world names) but it evolved into "just weed out bad data and keep as much good stuff as possible". This skipped regions which had independent cleanup projects - Solomani Rim (CT Supplement 10?), Domain of Deneb (TNE Regency Sourcebook/Daryen?), and Gateway Domain (T20?). This led to Robert's Second Survey files. The results have been fed to MWM who has a ginormous spreadsheet with the additional T5 fields added.

I've got links to many data files, including:

(1) the AOTI 'extracted data'

(2) the corrected genie data

(3) the corrected data with 'marked' worlds - this is for UWP regeneration, which is not canonical, but just an exercise. 'Marked' worlds have a + after their names, which indicates to the regeneration script that the UWP should be retained.

(4) regenerated sector data (noncanonical)

The data is here:

http://eaglestone.pocketempires.com/survey/second/


Note that (for example) Core sector uses some T4 world names, but also keeps some older world names as well. Choosing which name to use was pretty much arbitrary on my part.

 

[robject]

Another Note on the Sec file format

ASSUMPTION

The central assumption of my "XEC" sector format is that parsing UWP lines and parsing key-value pairs are solved problems that don't need to be solved by XML. This assumption is what makes the XML a "light salting", and is an easy way to turn a SEC file into something with basic organization (and NOT a database) without getting in the way of humanity.

The reason I don't care for "database"-style XML in SEC files is because the SEC format is a Dodo Bird of a database format anyway: technology has passed quietly into SQL and Excel. If you want a database, use those. If you want SOAP, again, use those. This old format is as much for humans as machines.

Document

<sec name="Fornast" milieu="Second Survey" desc="AOTI recapture">

The document is surrounded by the <sec> label. This label is the most complicated, because it has processed data in it, and is better when NOT edited by humans. I would probably like the "generated" date in here, as well (i.e. gen="Wed May 10 14:41:53 2006").

(Also, Joshua's files start with the obligatory <xml> directive on the first line; mine typically leave it out (for no particular reason)).

Header

The simple header looks like:

<header>
Sector: Core Sector
Alt:    Sector fG
DOS:    core
Coord:  fG
Ref:    9960 Ring/Ray 62832
Gen:    Wed May 10 14:41:53 2006
</header>

I figure we're all big boys and girls and know how to parse key-value pairs, and I must admit I prefer this over the second format I use:

The second, alternate format uses a "meta" label:

<meta
    Alt="Sector iH"
    Coord="iH"
    DOS="alphacru"
    Milieu="Second Survey"
    Ref="10080 Ring/Ray 62864"
    Sector="Alpha Crucis"
   />

Yuk. Okay, it's all XML, so the parser does all the work for you. But still, it's ugly, and the point of these files is human editability, isn't it? Ah well.

Subsectors

Again, simple is better, because we all know how to break apart key-value pairs.

<subsectors>
A: Apge
B: Perite
C: Ameros

...etc etc...

O: Dunea
P: Saregon
</subsectors>

I have an alternate, all-XML format for this as well, but again, I feel it is overkill.


Notes

A place for cross-referenced world data.

<notes>
ref 0125 Trevor (SM)  "Traveller News Service" 
ref 0140 Reference (TD)  "Reference, Center of the Imperium" 
ref 0523 Dinhe (SM)  "Traveller News Service" 
ref 0531 Cadion (AV)  "Hand of Varian" 
ref 0839 Night (TD)  "The Pilgra System" 
ref 1323 Bogustin (TD)  "Reference, Center of the Imperium", "Library Data of the Core Sector" 
ref 1518 Depot (RS)  "Imperial Navy Depots" 
ref 1524 Shibashliim (TD)  "Cemplas Subsector" 
ref 1623 Velpare (TD)  "Reference, Center of the Imperium", "Library Data of the Core Sector" 
ref 1715 Fornol (TD)  "A Concise History of the Third Imperium" 
ref 1721 Temra (TD)  "Chant Subsector" 
ref 2021 Anther (TD)  "Reference, Center of the Imperium", "Library Data of the Core Sector" 
ref 2118 Capital (RS)  "The Assassination" 
ref 2214 Shudusham (IE)  "Library Data" 
ref 2320 Umgadin (SMC)  "The Spinward Marches" 
</notes>

World Data

The UWP listing, again very simply boxed.

<data>
Irkigkhan       0103 E54536A-2   Lo Ni                   700 Im M4 V
Shana Ma        0104 E351112-9   Lo Ni Po                903 Im K2 IV M7 D
Niizediju       0106 B9C6886-9 N                         704 Im A1 V
Azimuth         0202 B847427-C N Lo Ni                   200 Im M2 V M7 D
Khaur Ga        0203 B234314-F N Lo Ni                   600 Im K1 V
Gekshiiuun      0208 D527524-7   Ni                      923 Im F9 IV
... etc etc ...
</data>
</sec>

That's it. Just a little light salting of XML around existing SEC files and we've got a free level of organization without compromising readability or edit-ability.