first microDem and HOG results

[martinvk]

For the longest time, I've made my routes on plane baseboards. Since I'm modeling Dutch routes, the lack of relief was not a major hindrance but despite its reputation, Nederland is not totally flat so I took the plunge and started a new map with real elevation.

Thanks to my friend Ron (Popeye25) over at trainz.nl I received some DEM data in the form of hgt files . Using a free viewer 3dem, I combined several files and extracted a subset of the data. That was then put into microDem and finally HOG to produce a gnd file. That was then put in a route I prepared in TS2010 without any major issues.

What is the best way to bring the lat - long coordinates with the data? In the past, it didn't really matter where I placed the world origin since one spot was a goo as another. Now that the terrain has elevation, it matters. The original data was tied to lat - long values but they got lost in the translation to the gnd file especially since the subset data was hand cut from the original files and so don't have boundaries on exact lat - long values.

Since microDem is a complex and multi-talented program, could I have combined the files in there and avoided the extra step via 3dem?

A minor issue are sharp ridges in the landscape.

Are these normal artifacts from the conversion process that I will just have to smooth over manually or did I do something wrong?

 

[clam1952]

It's a glitch or probably a bug, can be got round by adding an additional step. You have to go through the creating the route and adding the Mapfile.gnd again. This time before opening the new route with the DEM, create a new 1 board blank route, save it then merge the dem to it, doesn't have to actually touch, I leave a 1 board space. Then after merging delete the additional board and that should solve the problem.
You are lucky Holland is flat as in a mountainous area without performing the "fix" those ridges are huge!

 

[martinvk]

Thanks for the tip. I went back and tried it again and the majority of the ridges appear to have been removed. The area is too big to see in one pass so I'll see later if there are any others hiding in far corners.

Since I had to go back anyway I read more of the microDem help file and found out how to load multiple hgt files. This time I also ignored the advice to have no grid. I opted to leave the Lat/Long grid on but used the Ticks option instead of the grid option. The little black + signs became cross shaped dents in the landscape. By placing the World Origin in one, I can establish the proper origin point for my route.

Now to lay some track!

 

[geophil]

Thanks to my friend Ron (Popeye25) over at trainz.nl I received some DEM data in the form of hgt files .

You can simply download these from the public USGS web site: http://dds.cr.usgs.gov/srtm/

What is the best way to bring the lat - long coordinates with the data? In the past, it didn't really matter where I placed the world origin since one spot was a goo as another. Now that the terrain has elevation, it matters. The original data was tied to lat - long values but they got lost in the translation to the gnd file especially since the subset data was hand cut from the original files and so don't have boundaries on exact lat - long values.

I don't think there is a "best way" without the use of dedicated software. As you already stated, the MicroDEM/HOG approach loses all georeferencing information in the process. This isn't a big problem for US routes in combination with TIGER since MicroDEM happens to support TIGER data and provides the means for accurate map overlays. Outside the US things become more complex. There is a tutorial around, written by Ultimax, on this subject. It's in German and unfortunately I don't have the URL and don't have it on disk any more. This tutorial (and a few others) suggest to prepare the map overlay as an input to HOG. The map overlay is created by taking coordinates from MicroDEM for the selected DEM clipping and adjusting a topo map image to this clipping using an image editor, applying scaling and stretching functions. With the DEM and map overlay as congruent HOG inputs there doesn't seem to be a need to bring geographic coordinates into Trainz itself. That's from the HOG users' perspective.

The way TransDEM does it probably is not suitable for a manual approach. Anyway: TransDEM maintains an exact mapping of Trainz baseboards to the UTM grid. Both the Trainz and the UTM coordinates systems are Cartesian systems with a metric scale, so mapping is easy. The mapping is based on the UTM origin, central meridian of the zone and equator. (MapMaker does it similarly.) TransDEM then picks a baseboard near the centre of the route it creates. This baseboard must physically exist. TransDEM determines the UTM coordinates of the centre of this baseboard and converts these coordinates into lat/long. It then places the World Origin at this spot and assigns the obtained lat/long to it.

As said often before, do not use the World Origin and Trainz lat/long in Surveyor to place objects on a DEM-based terrain. Trainz lat/long values are in an unknown projection. Coordinates will not match the DEM or map overlay, once you leave the immediate vicinity of the Trainz World Origin.

The way TransDEM uses the World Origin is to transform it back to UTM coordinates, to the well-known Cartesian system, and forget about lat/long.

 

[martinvk]

I found that the terrain deformations made in microDEM to register the lat-long points were pretty accurate. In the east-west direction, there is only a 0.03% difference between the dents and Google-Earth. The Trainz coordinates are off by less than 0.11% (12m out of 17,152m)

Thanks to ... I've been able to try TranzDEM. Using the same hgt data files, I made a DEM of the area of interest. Google-Earth provided the images to make georef files. Next time I should not use so high an altitude, 1500m. The resulting ground textures are too clunky and almost useless for guiding the route creation. Smaller, more detailed images would be better. Once everything was prepared, it was successfully exported and brought into TS2010 without any problems.

Looking around the route in Surveyor, I had a hard time finding familiar points of reference in the terrain. After a closer look and some measuring I found that the terrain was shifted about 365m bearing 204 degrees true and rotated 1.9 degrees clockwise. It is a route generally running east-west for about 70km. The west-end is shifted NE by about 780m and the east-end is shifted SW by about 1548m.

Since the same hgt files were used as the source in both programs, I'm guessing that the transformation to the UTM coordinates is one of the causes. The route is on the edge of 32U so some of the raster images had to be converted from 31U

There are a lot of configuration options in TranzDEM. Would using a different grid, Alt-U or Alt-G make a difference or one of the National Coordinate
Systems? Are there other options that I over looked?

Is is possible to rotate the data set before or after exporting? The shift can be taken care of by reprogramming the World Origin values in Trainz.

Even though the Trainz baseboards and the UTM system are both Cartesian systems and so map 1:1, almost all of my data sources are in lat-long coordinates and so having the terrain match that makes it much easier to use. If there was a way to control the rotation in TranzDEM to align with lat-long instead of the UTM grid, that might solve my problem. Other than that, it would be a great program for me.

 

[geophil]

Think of the orange peel. It is simply impossible to project the surface of Mother Earth to a single plane in Cartesian coordinates without introducing all sorts of distortion. A map projection is always a compromise.

Even without a projection there is no such thing like absolute lat/long for a point. Geographic coordinates are subject to the underlying shape model. This could be a sphere, an ellipsoid or (unlikely) the Geoid itself. Spheres and ellipsoid may have different radii and they have different origins. They are called the "geodetic datum". Many of the national coordinate systems in TransDEM are based on different datums.

On the projection side UTM is widely spread, but it's not the only one. We don't know what Auran is using. UTM is zonal, an unknown term in the Surveyor world. Keep in mind the role of lat/long in Trainz. It's definitely not surveying. And don't put too much in it.

UTM and all other Mercator projections are cylindrical ones. There are also other projections like those on a cone, e.g. Lambert. Or the more exotic ones, particularly for smaller countries. The Netherlands, Hungary, Chechia/Slovakia or New Zealand spring to mind.

Let's stick with Mercator. Standard Mercator has the cylinder axis through the poles, Transverse Mercator through the equator, somewhere.

During projection, in TransDEM or in Trainz, you unwrap a sphere or an ellipsoid (big difference) to a another shape, e.g. a cylinder, with a particular orientation. Many parameters. Not a single "one size fits all" solution. And, due to the rather complex math behind it, not the ideal case for guessing type and parameters. That's the reason why I say keep away from lat/long in Trainz when dealing with DEM and map-based terrain.



In TransDEM, internally it is always UTM/WGS84. This is also the coordinate system transferred to Trainz. The national coordinate systems are for viewing and particularly for georeferencing from national sources, manually and automatically. The "geographic" coordinate system is for georeferencing convenience and display only. In TransDEM it is still a UTM system.

You mention UTM zone issues. That is one of the difficulties you have with any map projection to a Cartesian system, which is conformal, but tries to be equal-distance or equal-area as well. Valid with a given error only in a certain range, a zone. TransDEM is aware of this and has built-in functionality to convert from one zone to an other, DEMs, maps, and vector data. This cannot eliminate projection errors, of course. But it applies the same error to all sources and makes them congruent again. This zone conversion looks like a rotation but mathematically is not an affine transformation at all.

For DEMs, zone conversion is easy, either explicitly by clicking on the button, or implicitly when merging DEMs. For maps, it's a bit more indirect. See here for a tutorial to apply zone conversion for maps: http://forum.uktrainz.co.uk/viewtopic.php?f=131&t=5382&sid=4880b6350f584fe016d49fa4b1b90248#p87476

If you want to work with numerical coordinates in Trainz, use the UTM grid (the magenta lines). You may want to activate painting grid labels, too. For spot-on positioning, switch on the Trainz coordinate display in Surveyor (Trainzoptions.txt). Take a one-time note of the offset between UTM and Trainz coordinates. This offset is in metres and remains constant throughout each route.

 

[martinvk]

Thanks for the dissertation on mapping. It was enlightening for sure.

As interesting as it was, I still have a few quibbles. Even though the system used by Trainz is not known, I have found that the coordinates controlled by the World Origin are close enough to real world lat-long values to enable me to use things like topo maps and Google Earth as guides. Using my lat-long reader, I can get a direct reading for any spot in the route.

If you want to work with numerical coordinates in Trainz, use the UTM grid (the magenta lines). You may want to activate painting grid labels, too. For spot-on positioning, switch on the Trainz coordinate display in Surveyor (Trainzoptions.txt). Take a one-time note of the offset between UTM and Trainz coordinates. This offset is in metres and remains constant throughout each route.

Painting on UTM lines takes me back to the days before TRS2004 when lat-long was introduced and I was using rulers to measure and locate objects.

I tried the -loc option and yes I can see the coordinate values for the current cursor position. Since Positive X is south and Positive Y is east., how does that map to a UTM system of eastings and northings?

If I can find a way to have an object read and display this too, it could replace my lat-long reader and provide a way to mark locations for future reference.

Too bad the lat-long values are both offset and rotated so each point in the map had a different offset from the internal lat-long coordinates. No way to undo this?

Since I've already started my current route the lat-long way (all of my reference notes are in lat-long), I will continue it that way. Will see about using UTM in my next route.

 

[geophil]

Painting on UTM lines takes me back to the days before TRS2004 when lat-long was introduced and I was using rulers to measure and locate objects

Let me emphasize that the fundamental idea behind MicroDEM/HOG/TIGER, MapMaker, TransDEM and also Basemaps is to avoid numerical coordinates and instead use the map overlay or tile as the visual guidance to place objects. That's the preferred way of working with these tools but I can understand that this is unfamiliar to you since you took a different approach.

I tried the -loc option and yes I can see the coordinate values for the current cursor position. Since Positive X is south and Positive Y is east., how does that map to a UTM system of eastings and northings?

Yes, the Trainz y-axis is inverted (to achieve a left-hand-coordinate system for 3D operations?). You have to negate the sign again.

An example. Both coordinate systems are in metres. Assume a UTM grid line intersection has the following coordinates, with matching sample Trainz coordinates in the next column:

easting   644000   1440
northing 5772000   3120

The next 1000 m line intersection to the NE would be:

easting   645000   2440 (= +1000) 
northing 5773000   2120 (= -1000)

If I can find a way to have an object read and display this too, it could replace my lat-long reader and provide a way to mark locations for future reference.

One of the options in TransDEM is support of vector data. It's limited to line objects and one layer only with the current version. (Point objects and multiple layers are on the wish list for some time.) Locate your object position on the map or aerial image in TransDEM, draw a short vector line in TransDEM, and export as a Trainz spline. It will appear spot-on. Or, to process your existing data, bring it to a format TransDEM can read, even in lat/long (must be proper WGS84) and import into TransDEM.

Too bad the lat-long values are both offset and rotated so each point in the map had a different offset from the internal lat-long coordinates. No way to undo this?

Again, Trainz terrain is a Cartesian plane. Unwrapping to a plane needs a projection. Each projection is a compromise. TransDEM is UTM. UTM has zones (but TransDEM can convert between adjacent zones). Within each projected coordinate system, lat/long to projection coordinate relation is bijective (unambiguous). Determine the Trainz coordinate system (I would guess something like standard Mercator to a sphere) and we create a transformation.

Since I've already started my current route the lat-long way (all of my reference notes are in lat-long), I will continue it that way. Will see about using UTM in my next route.

The last version of Microsoft Flight Simulator was based on lat/log throughout. It did not have a Cartesian map projection at all. Its underlying model was the WGS84 ellipsoid itself. 3D rendering scene coordinates were computed on-the-fly. The Trainz terrain model is different, it is based on the classic plane like many other simulation architectures. We have to live with it.



An excursus: To illustrate the influence of the model shape of the Earth, try the Map Tile client in TransDEM, with one of the given sample configurations. Acquire a few tiles. Save. Then go to the setup for this configuration and change the model shape from sphere to ellipsoid or vice versa. Acquire the same tiles again. Superimpose and note the difference.

 

[avdbrink]

Hi Martin,

I saw your post on trainz.nl and found this thread on the auran forum.

I was always interested in this way of building routes (Dutch routes) for trainz: with REAL landscape. But I've never managed to make a base map using all these maps and third party programs... I think it's just one step to far for me.

I was wandering if you might want to share the base map you've build of the Netherlands with the rest of the Trainz community, so any dutch guy can build a more realistic route.

Regards,
Arno

 

[martinvk]

Nice to see others are also interested in building Dutch routes, realistic or not. Saying that I made a base map of the Netherlands is a bit of an exaggeration. It is only a narrow band from the outskirts of Utrecht to the German border east of Zevenaar and it is over 310MB. Even compressed, the gnd file is more than 76MB, too big to easily send to anyone.

As I mentioned, there are two methods to getting a DEM gnd file, the two step process using mircoDem and HOG and the one step process using TranzDEM. Each does the job but because of the way I work, I find it easier to work with the gnd made by the two step method. Even though TranzDem's converting to a UTM form might be more accurate and better fit the Trainz baseboard grid, all of my notes and raw data are in lat-long coordinates and so the unrotated and unshifted gnd file produced by microDem works better for me.

Give one of the DEM programs a try. It really isn't that hard. especially if you follow a good tutorial wit step by step instructions.

Working with the resulting baseboards, I have noted a few interesting things. The overall look is pretty accurate with hills and valleys where I expected to find them. Where I have problems is in areas with heavy woods beside open fields and in cities and other build up areas with large buildings. Elevations imported into Trainz appear to be the tops of trees and buildings, resulting in ground that is a lot more undulating than I expected. Once you understand and compensate, it is pretty easy to build a route. If only there was a way to more easily create gentle and smooth slopes without raising or lowering each ground node. I have already created a spline object with 12 parallel tracks that can be placed with different elevations for each end. Then when the smooth function is used all the ground underneath is smoothed together but it is still only an approximation.

 

[geophil]

Martin,

which projection do you use in MicroDEM?