TransDEM issues

[JCitron]

You might find this helpful at some point.

http://www.topoquest.com/find/state.php?state=AK

These are geo-tif format files which will automatically map themselves to the DEM file in TD.

John

 

[steamboateng]

I think that last post belongs in the Post Topic prior to this, John,..................................i.e:. I want a DEM or sumthin' like it!

 

[cheechako]

I suppose, simply stated; TransDEM is a geo-mapping tool..................not an image manipulation tool. I think we can agree on that Stan..........no?

Hmm. Well, it has a vector drawing tool, so it is obviously a little bit more functionality than just a geo-mapping tool. It seems logical to expect some basic image manipulation and alignment tools. However, this is beyond the scope of the program and perhaps with good reason. I know what I'm after is a lot of hard work. I just doubt I'm the only one that has tried this, and just hoped the was a bit more than just "read the manual". Oh well. I will stick with reading the manual.

 

[geophil]

Hmm. Well, it has a vector drawing tool, so it is obviously a little bit more functionality than just a geo-mapping tool. It seems logical to expect some basic image manipulation and alignment tools. However, this is beyond the scope of the program and perhaps with good reason. I know what I'm after is a lot of hard work. I just doubt I'm the only one that has tried this, and just hoped the was a bit more than just "read the manual". Oh well. I will stick with reading the manual.

For all three geo data types that TransDEM can handle it has its own subset of editing/manipulation tools.

For vector data you either draw the lines yourself or import in one of the supported formats, e.g. from a vector map, from you outdoor GPS device and even track geometry from your favourite railroad (presumably confidential). I had a recent email conversation with a TransDEM route builder who succeeded to load the entire railroad network of the 48 states into TransDEM, acquired via The National Map Viever. He noticed TransDEM becoming very slow and we found out what was going on. That vector map was only 1:1 million, not suitable to run trains along those lines, but still quite some amount of data to deal with.

Now, for raster maps/orthoimagery TransDEM also offers some editing and conversion utilities.

Independent of georeferencing there is the re-sampling and the rectangle mask cropping tool, to make the image smaller.

Georeferencing itself applies an affine transformation, based on three points (= 6 unknowns which can be solved this way) to determine translation (x,y), scaling (x,y), rotation (phi) and shear (x). Variations include the simplest case with map corners only (4 unknowns, uniform scaling, no shear) or its ultimate complication with a TIN (a triangulated irregular network) where you can define an arbitrary number of control points and assign geo-coordinates to each of them. The control points will form a mesh and each mesh triangle will have its own affine transformation. Useful for distorted images, like track charts with different scale for the longitudinal and lateral axes.

Then, of course, we have all the geo coordinate system transformations. TransDEM uses UTM/WGS84 internally and for export to Trainz but also understands several other coordinate systems which it converts into UTM, pixel by pixel.

Here is an overview of geo-referencing and coordinate conversion: http://rolandziegler.de/StreckeUndLandschaft/transdemMap_Engl.htm

The different options in summary:

1. Automatic
   • on-line
     • WMS Web Mapping Service. The request defines the boundaries for the image in geo coordinates, the reply delivers the matching image.
     • Map Tile Services. The request addresses a pre-fabricated map tile in a quad tree, the reply delivers that tile. The tile address itself serves as an encoded form of georeferencing.
   • off-line
     • Geo-TIFF. GeoTIFF is an extension to the TIFF raster format, to carry geo-referencing meta data. Has many parameters, like TIFF itself. TransDEM supports a subset.
     • GeoPDF. A proprietary encoding of ge-referencing information into a pdf file. In use by USGS for both modern and historical maps. Quite complex under the hood. TransDEM supports a subset, successfully tested with USGS and Canadian topo maps.
     • World file. A companion file for raster images, industry standard. Holds geo-referencing info as an awkwardly encoded set of affine transformation parameters
     • Google Earth placemark. Another companion file type
2. Manual
   • 3 (+1) points. Defines affine transformation parameters incrementally by entering x and y geo coordiantes for three points of an image.
   • 2 corner points. Simplification of the above.
   • More than 3 points. Complication of the above by creating a TIN of affine transformations, for distorted images.

Finally (we won't deal with DEM editing in this post): How to georeference images with no coordinates at all: Use two instances of TransDEM, best with two monitors. Load a geo-referenced representation of the area in question into one TransDEM window: map tile, WMS, topo map. Load the other image into the second window. Try to find identifiable landmarks on both. Take the coordinates from the first image and make them control point coordinates in the second image. Will require precise, concentrated work and some patience.

 

[cascaderailroad]

I Yam', making progress ... I think ... but I am still a Transdem dummie, but I am learning alot every day !

I tink' I am a slow learner, but I as a child ... I dint' take the short bus, for I walked to school ... And the winters in our area was so cold and icy, that we had to wrap our feet in barbed wire just to get traction on the ice ... And we couldn't afford shoes, and the snows were so deep, that we had to shovel our way to, and from school, and it was all uphill, both ways !

http://cdn.slidesharecdn.com/ss_thumbnails/angermanagementfordummies-130228043029-phpapp02-thumbnail-4.jpg?1362047917
[url]http://improveverywhere-com.zippykid.netdna-cdn.com/images/sw06.jpeg
[/URL]

 

[steamboateng]

Well Roland, I don't quite know what it is that Cheechako is attempting to do. The best I can figure is, he wants to add images to a georeerenced map in TransDEM. To my knowledge he can overlay any number of georeferenced images onto the previous one and save it as a single georeferenced raster map. But any actual image manipulation will have to be done in an external image editor.
It's good to hear that Cascaderailroad is making progress......I suppose before long we'll be seeing Trainz maps of the antarctic ice cap with the Rostov Express breaking dog sled speed records, left and right.......

 

[cascaderailroad]

I have actually thought of making the following routes:

The Antarctican Rwy (is 5 miles long ... do dah, do dah)
Area 51 (maroon lake ... but I was afeared' that MIB's would come knocking on my door, asking lotsa' questions).
The COG Rwy (Mount Washington NH, but I have no slanty scenic railcars, and slanty tank locos).
The Mount Lowe Rwy
The Mount Tamalipais Rwy (Muir Woods)

 

[cheechako]

Well Roland, I don't quite know what it is that Cheechako is attempting to do.

I have scans of old sketches and I want to get them to fit the route I draw in TransDEM. I already said this. So, thanks for your input but if you cannot grasp what I am trying to do, you cannot offer any help except the obvious.

Anyway, much of my work needs to be done outside of TransDEM it seems, and then Roland gave me some tips (like two side-by-side instances of TransDEM) to deal with those images and geo-ref data by hand.

 

[steamboateng]

Chief Mate and I took the Mt Washington cog railway several years ago.................it's nice going up........very spooky going down, as the engineer uses the steam cylinders as a brake.................please don't lose the fires now!
We also did the Muir Woods a gazillon years ago...............honeymoon actually, after spoiling ourselves in Maui and Honolulu...............please don't put a noisy, stinky railway there, you'll definitely p.o. the deer and squirrels...........
Yu can build whatever you wish in Antarctica, .............tired of watching silly penguin auditions for 'Dancing with the Stras!

 

[geophil]

If I remember correctly, steamboateng was the one who first dug out those historical USGS topo maps for his Hosaac tunnel project, long before GeoPDF came into being. He has a bit of hands-on experience with those things.

cheechako, I'm not quite sure what you want to do in Photoshop. Usually you just load the aerial image into TransDEM and start building the affine transformation by adding georeferencing coordinates, more difficult in this case because you'll need some landmarks, but still the same procedure. Those coordinates will sort translation, rotation, scaling and shear for you. No need for Photoshop here.

Anyway, aerial images will have to go to "UTM tiles" in TransDEM because of the limited ground texture resolution in current Trainz versions. And you probably want ground textures, too. That's where the USGS historical topo maps might be helpful.

 

[mezzoprezzo]

@cheechako

You could always place your detailed historical stuff onto basemaps. Being a single flat plane object they won’t stick to the contours of the terrain, but you can raise and lower them above and below the landscape. They are on the DLS in 750 metre and 1km square options.

They are useful for very detailed historical modelling. Although the advice is to use a 1024x1024 sized image on the basemaps, I always use 2048x2048 which gives better detail. (It may be possible to go even bigger but I’ve never tried).

Here the corner of a 1km basemap I was recently working on which gives an indication of the detail obtained from a 1:500 historical map. By working directly overhead in Surveyor, using wireframe mode as necessary, it was easy to achieve some very accurate track laying.

 

[cheechako]

If I remember correctly, steamboateng was the one who first dug out those historical USGS topo maps for his Hosaac tunnel project, long before GeoPDF came into being. He has a bit of hands-on experience with those things.

Well, then I am just not explaining myself well. I need PS to clean up and sharpen the sketches. I don't think any of them agree about north or scale, so I also must rotate and scale. I am quite familiar with PS, so naturally these seem like simple PS tasks.

However, none of this deals with accurate alignment. I've tried basemaps in the past, and I don't see how I will ever get anything lined up with anything exported from TransDEM.

So, in order to make these old images fit, they must be manipulated and geo-refed. The terse answer from steam implies that TransDEM will not let me manipulate these images, so I cannot do this. You are telling me that TransDEM will manipulate the images, but this is behind the scenes based on the geo data points.

I was hoping for ideas about the best way to merge old hand sketches with modern, accurate data. The impression I got from steamboateng was to stick with geo-data and forget everything else. Maybe we're just not understanding each other, because the impression I got from you was that TransDEM is indeed one of the tools I'll use in this process.

 

[steamboateng]

Cheechako, I understand I may came across as 'terse', where humor was intended. I attempt to put things into simple lay terms because I know how intimidating TransDEM terminology can be to the newly initiated! Certainly nothing personal..........I don't know you well enough!
It's my understanding you want to add sketches to a topo map. You can do that..........just the way you describe; i.e. scale and rotate them and over lay them on your map as closely as you can by eyeball, in your image editor. The map, with your modifications applied, must then be georeferenced to apply as an overlay to a DEM. Understand that once any georeferenced map is converted out of TranDEMs' native format, into jpg or bmp, for editing, its embedded geo-data is lost, and the map must again be georeferenced.
You can 'group' georefenced maps into a single overlay, and even stack georeferenced overlays.

 

[geophil]

Well, then I am just not explaining myself well. I need PS to clean up and sharpen the sketches. I don't think any of them agree about north or scale, so I also must rotate and scale. I am quite familiar with PS, so naturally these seem like simple PS tasks.

North orientation or scale are core subjects of geo-referencing.

Did you do tutorial #1 in the TransDEM main manual, the one with manual georeferencing of a Swiss topo map? If not, please do it now as it is helpful for understanding the following.

We start with a plain raster image in a standard raster format. It could be a photo or a drawing. In this case it's a map and in this case it already has a grid printed on it which we use for georeferencing.

Georeferencing is nothing more than mapping certain points on the image to their real world counterparts in the form of geo coordinates.

After we have done that, TransDEM will draw its own coordinate grid (shown in magenta) which tells us it has enough parameters for a georeferencing transformation. At the moment this image is already aligned with the Swiss national geo coordinate system. TransDEM however, uses UTM. UTM north is different from Swiss north (and neither of them is geographic north).

Now we run the transformation and after that, our image will be aligned with the UTM grid. That implies some rotation as well.

The final result consists of the manipulated image and a companion file with georeferencing information: the UTM coordinates of the south-west corner and scale per pixel.

No Photoshop was involved in these actions.

To be continued in part 2.

 

[geophil]

Continued from part 1.

Now the more complicated case with no coordinate information to start with. I use an example I have shown earlier: http://forums.auran.com/trainz/show...ogress-The-Wisbech-Line&p=1201358#post1201358

That's the track diagram image we want to process (http://forums.auran.com/trainz/show...ogress-The-Wisbech-Line&p=1200208#post1200208), a plain image. The drawing seems to be scanned from a book. The location is England, north of Cambridge.

We load this image into TransDEM. In a second TransDEM window we load our reference map or orthoimage to obtain geo coordinates. Here we use a historical British topo map 1:25,000 from the 1950s, accessible via a map tile service, therefore already georeferenced.

We now try to find three identifiable points on both maps than span a triangle of reasonable area. That's the most difficult part. Fortunately, in our example there are enough hedges, fences and ditches to play with. Since the scale in both maps is different, I go for intersections of lines to reduce scale error.

Circles in magenta mark the points I have picked. Compare with the same points on the map above.

Now I start the georeferencing procedure, just as in tutorial #1, and add my three points and their UTM coordinate values.

Once done, the grid is drawn and it looks a bit distorted, rotation and shear apparently in place.

We apply the transformation and let TransDEM manipulate our image. Now the grid is vertically and horizontally aligned. Our image has experienced some rotation and slant just as the previous grid indicated.

With the important task of geo-referencing accomplished we now notice that the left and bottom edge could be a bit tidier. TransDEM has a tool named “Transparent margins” to help here. This masking tool preserves geo-referencing, even if we cut off some of the edge and make the image smaller.

Finally, we load the georeferenced track diagram on top of out topo map:

That's it. Not perfect, but such track diagrams weren't necessarily created with accurate cartography in mind.

All has been done with TransDEM, no Photoshop was needed.

 

[geophil]

@cheechako

You could always place your detailed historical stuff onto basemaps. Being a single flat plane object they won’t stick to the contours of the terrain, but you can raise and lower them above and below the landscape. They are on the DLS in 750 metre and 1km square options.

In TransDEM, the basemaps are called UTM tiles (because they align with the 1000m UTM coordinate grid) and a part of the product since its early beginnings in 2005. Since version 1.2 in 2006 UTM tiles are placed into the Trainz route automatically, thanks to the concept of geo coordinates.

Here are two typical examples (also shown in the TransDEM thread a few days ago):

In the future, UTM tiles will no longer be flat but will have a terra-shaped third dimension:

This new feature will become available in the next version of TransDEM.

 

[cheechako]

Cheechako, I understand I may came across as 'terse', where humor was intended. .

It's all good. Ultimately, I think we all uncovered what I was after.

North orientation or scale are core subjects of geo-referencing.

Did you do tutorial #1 in the TransDEM main manual, the one with manual georeferencing of a Swiss topo map? If not, please do it now as it is helpful for understanding the following.

No, I did not. I only did #7 in that manual; so far I focused on the Trainz one and only skimmed that one. Thanks so much for what you posted here! Part 2 seems to be exactly what I had in mind.

The bad thing is that the more I study my source images, the less I think they will help. The quality of the scans are very poor, nearly all of the data is illegible, and I don't even have a third of the route! I am not giving up hopes of finding better copies of the original maps, but that will take time if it ever happens at all.

I think my next best approach is to fake it with Google Earth. Luckily, there's a tutorial for that too! #2 in the main manual. I might also look at the detailed terrain method once I get through the tutorial.

p.s. I missed the last post while I was writing this. Yeah - if I can get good maps, I like that UTM on the terrain feature a lot! That's another reason I'm looking at the Google Earth method now. I tried the basemaps in the past and had a bad time working in deep valleys.

 

[AntonyVW]

In the future, UTM tiles will no longer be flat but will have a terra-shaped third dimension:

This new feature will become available in the next version of TransDEM.

When will this version be available please?

 

[geophil]

When will this version be available please?

I am aiming for the Christmas / New Year holidays but I cannot make promises.

I think my next best approach is to fake it with Google Earth. Luckily, there's a tutorial for that too! #2 in the main manual. I might also look at the detailed terrain method once I get through the tutorial.

Google Maps orthophotos will provide the same data. Much quicker via the map tile client. There are two catches, though.

Firstly, map tile address parameters expire quickly for Google. You will be up to date, if you set

kh/v=142

for the query prefix in the URL composition dialog (Map Tile Settings for "Google Maps", tyle type "orthophoto").

Secondly, Google in particular only allows you to download only a limited number of map tiles per "session". MS Bing (under its original name Virtual Earth in TransDEM) does not have such a restriction - and often the better data.

The other question is: To what level of detail do you want to go? Couldn't the 1:24k topo map suffice? And if it's an abandoned line, there might by a historical map for it in the USGS collection:
http://nationalmap.gov/historical/index.html
(Also available via The National Map Viewer)

 

[cascaderailroad]

I have been experimenting with Transdem, but I can only get a 15 mile wide raster topo map displaying really big and way out of proportion, the same size as the 60 mile across DEM, and it is way off register, way up in the black right hand corner of the screen, and is no where near the correct size.

I swear I am the most dense person in the world ... everything I do, I am the only one who has great difficulty with things.

I wish a raster topo would automatically register and align with the DEM ... And I am told that some raster maps do this ... but I can not get it to work ... I am doing something radically wrong on my end.

I swear I have undiagnosed Dickxlexia, or ADD or something ... as when I start reading a 73 page tutorial, I slowly feel my mind slipping away into a hypnotized coma, as I slowly read the words: "Now geo-reference the UTM contol point grids with transnoodial 1/3 arc data coordinants, and type those numerals into the Time/Space Transmorgrahpyer input table, and convert them into a PDA ADF file" ... and soon all I hear, it is like hearing Charlie Browns teacher: "Wahhh Waahhh Waaahhh ... Waahhh, Wahhh ... Wahhh Waahhh Waaahhh" ...

If a new Transdem was invented, a version that even the average imbecile (like me) could operate ... Now There ... You could make millions of dollars !

All I want is a set of DEM's from Conway to Johnstown ... and another DEM from Mount Union to Marysville .... and another DEM from Reading to Phila ... and another DEM from Highspire to Phila ... and my little world would be complete ... and I could make many cut down mergable iPortal/portal sections of individual DEM's of a route going all the way from Phila/Reading/Harrisburg/Altoona/Johnstown/Picairn/Pittsburgh/Conway.

And world peace too !