Bad News for TransDem users

[steamboateng]

I don't know if any of you have accessed the USGS National Server lately, but they changed it all around again, and................Geo Tiff is no longer offered as a download format for 1/3 Arc Sec DEM data!
What are we to do with TransDEM now?

 

[p-dehnert]

Have a look at this thread (post #20).

Peter

 

[steamboateng]

I got on Roland's TransDEM forum and got this reply:
http://forum.transdem.de/viewtopic.php?p=1172

 

[geophil]

JCitron notified us about the USGS announcement last December: http://forums.auran.com/trainz/show...ing-DEM-formats-on-the-US-National-Map-Server

At that time I was implementing "ESRI Binary Grid" for the new Australian DEM portal and it turned out to be the same as "Arc Grid" offered by USGS. (ArcInfo is an ESRI product.) Saved a bit of work this way.

 

[Vern]

You can still obtain and use SRTM data (okay not so accurate) until Roland is able to mod Transdem for the new format.

 

[geophil]

To reiterate: TransDEM can read USGS NED data in Arc Grid format since Dec 21st, 2012. USGS retired GeoTIFF on Dec 19th.

 

[steamboateng]

I found out yesterday when I went to download some data.
I'm still a bit upset about all this.........I use USGS data exclusively.
It's a big pia, having to upgrade TransDEM and grease up the PayPal account.
Of course this is no reflection on Roland.
I'm sending the Bill to Obama..........

 

[JCitron]

Mike,

Just an FYI regarding the new Arc format. The files are huge! The good news is they cover a big swath of area. The two I need for the east coast area extend from near Newington, NH across to Manchester then all the way over to near Fitchburg to the South Shore then across the tip of Cape Cod. What I needed out of the whole area happened to have spanned the two DEMs and is about the size of a pinky nail!

Beware of the processing too. I find that if I get my tiles first, save my route. Then import the DEM files one at a time, saving a DEM, closing it then adding them afterwards, I don't have any processing issues. They do take quite some time to load though so hang in there as the drive chunks away and nothing appears on the screen. Once you see the coordinate box, you're golden and the map should appear. I panicked the first time and killed the process thinking something had fried.

John

 

[geophil]

Just an FYI regarding the new Arc format. The files are huge! [...]

Beware of the processing too. [...] They do take quite some time to load though so hang in there as the drive chunks away and nothing appears on the screen. Once you see the coordinate box, you're golden and the map should appear. I panicked the first time and killed the process thinking something had fried.

I guess the sheer size of the files will put off some of the more occasional users. Yes, loading is slow, and it does not give progress feedback at the moment. (Loading is running in the main thread.) Once the data is in memory - the resample dialog pops up - it is easier to control. The UTM transformation makes use of all available processor cores and feeds the progress bar.

I find that if I get my tiles first [...]

Memory can also become an issue. Merging of two 1x1 degree 1/3 arc sec DEMs should work (at least on a 64 bit system) but more than that can be problematic. If more than one DEM file is needed, load the maps first and crop each DEM file to the area of the maps in use.

 

[JCitron]

I guess the sheer size of the files will put off some of the more occasional users. Yes, loading is slow, and it does not give progress feedback at the moment. (Loading is running in the main thread.) Once the data is in memory - the resample dialog pops up - it is easier to control. The UTM transformation makes use of all available processor cores and feeds the progress bar.


Memory can also become an issue. Merging of two 1x1 degree 1/3 arc sec DEMs should work (at least on a 64 bit system) but more than that can be problematic. If more than one DEM file is needed, load the maps first and crop each DEM file to the area of the maps in use.

This makes sense, Roland. I'm running on 64-bit Windows with 16GB of RAM. Using Process Explorer and ProcMon, I noticed that the disk I/O is pretty minimal so all this is done up in the RAM. If I could afford it, I'd throw more memory on the motherboard, but right now that's out of my budget.

I only work with two DEMs anyway. I couldn't imagine trying to merge more than that right now. I've learned the hard way so now I walk away from the computer and let the machine do its thing while it processes the information. I'll try your suggestion for cropping to the map area and see how that works for me.

Thanks,

John

 

[steamboateng]

Thanks for the info guys.
John, what do you mean by "load tiles"?
I only have 6Gb RAM so I guess it's gonna be slow going whence I commence.
Still waitin' on PayPal to contact Roland.

 

[JCitron]

Thanks for the info guys.
John, what do you mean by "load tiles"?
I only have 6Gb RAM so I guess it's gonna be slow going whence I commence.
Still waitin' on PayPal to contact Roland.

Load tiles... When you go the map servers and download the map bits to cover the area/route you want to create a Trainz map for.

Using his tutorial, Section 3 I think it is, you draw a path over the area you want the tiles for. You then connect to the mapping server such as MS TerraServer and download the topomap for that region. Once I get these, I then import the DEM. These files as I said are huge! One DEM, out of the two I needed because I spanned a hangnail's worth of map over into another slice, was 454 MB. The other is about the same. The two slices are, by the way, N43W071 and N43W072. It'll be slow processing. If you need help, let me know I can slice and dice if you want and send the files to you.

Your order is waiting on Roland now. He's asleep since it's about 3:00 am over in Germany.

John

 

[steamboateng]

Ok John, I understand now. I've been relying soley on the UNH site for 'historical' topo maps; georefering them as I go along.

 

[steamboateng]

Well, I fnally got Roland's latest version (2.3.1.2) of TransDEM installed and cranked up this a.m. I downloaded several of the new 1x1 degree DEM files from the USGS National server and fed one to the program.............the program worked well and the DEM map came out just fine..............processing time wasn't long at all. But yes, there is a longer lag time reloading these large maps. Now I gotta hit Roland's manuals again and figure out how to crop these monstrosities!
Despite my #&*%!#$ earlier, downloading a big hunk of data like this is easier than the peicemeal method offered before.