TransDEM Trouble.

C

Chris004

New member
Does anyone know why TransDEM is gobbling up all available memory? When I had only 4GB installed it made sense, and I could work around it (by breaking up jobs into little bits: 2 or 4 tiles at a time etc). I put another 8GB in and was golden for a week or so. But now when I process even just one tile, memory usage immediately jumps to 11.6GB and shortly after that the OS is tripping over its own feet so badly it can barely show me the taskbar. Sometimes I have the patience to leave it for an hour (sometimes it recovers, sometimes not) but mostly I find myself pushing the power button.

Do I need to re-install? Will that even help? Is there a trick to doing a re-install (there's always the thing that'll bite yer butt if you don't know about it, isn't there?)

All suggestions are welcomed with gratitude. I want to try everything I can before I go bothering Roland...


Thanks, People

-Chris
 
If you are working on a large route with large DEM's then Transdem is pretty hardware intensive, a lot going on under the hood there.

My advice is to always run it from a fresh boot of the PC/Laptop and when you've finished shut down the computer and start again before doing anything else (e.g. importing your route into Trainz).
 
My laptop is running out of disc space, because Trainz TRS2006 and it's Local file take up most of my C drive ... I would go out and buy a 2TB External Hard Drive, and put Transdem on it, then you would have plenty of space
 
cascaderailroad said:
My laptop is running out of disc space, because Trainz TRS2006 and it's Local file take up most of my C drive ... I would go out and buy a 2TB External Hard Drive, and put Transdem on it, then you would have plenty of space
Click to expand...

Speaking of which, I have done just that. I got a case from Insignia for $37 bucks at BestBuy. (Well $36.99 with no tax cuz it was up in New Hampshire where there's no sales tax). I put a leftover hard drive in it, complete with two blood blisters and screws dropped about 21 times on the floor. Yup, I spent more time chasing screws than I did putting the drive into the case, which then pinched me as it slid together. Other than my clumsiness, the case is made well and even comes with its own tiny black screw driver to help do the job so no additional tools are needed!

I use this drive, which is USB 3.0 as my TransDEM Data drive. It has plenty of space to do what it needs to do for my TransDEM files.

Speaking of TransDem, Chris you might be running the 32-bit version which runs out of memory easily. The 64-bit version, if your system will support it - it needs to be running a 64-bit OS such as Windows 7 64-bit, and not 32-bit, runs more efficiently as it makes use of the system resources better.

John
 
gawpo50 said:
Tell us what version you have and whether it is x32 or x64 version.
Click to expand...
Version: 2.5.1.0 64bit

I ordered it in December.

The weird part is it had been working, within its limits. Number of base boards wasn't that critical, even with large raster files being used for ground texture. It would be gone for quite some time but would not paralyze the OS - I could tab over to TaskManager and see that it was using CPU.

Doing Tiles I need to do only a few at a time when I'm using the enormous raster maps and making 4096 tiles, but again it had been able to do this.

My last total lock-up (which now happens every time I use TD) was a test 7x5 baseboards, using 4 or 6 64x36 pixel raster maps. Had to shut down with power button but route was made (as far as I could tell in surveyor). I then made a single tile - same thing.

Just before the OS stops responding memory usage goes from almost all to all, cpu is only working on service tasks (no green bars over the red ones) and then even that drops to 5 or 6%. I figure OS is doing nothing but trying to move stuff from virtual RAM(disk) to real RAM just so it has some code to try to run.

This is exactly how it behaved when I had only 4GB installed and I pushed TD beyond it's limits. For a week or two after I put another 8GB in it worked fine....

Hope this helps
 
Chris004 said:
Version: 2.5.1.0 64bit

I ordered it in December.

The weird part is it had been working, within its limits. Number of base boards wasn't that critical, even with large raster files being used for ground texture. It would be gone for quite some time but would not paralyze the OS - I could tab over to TaskManager and see that it was using CPU.

Doing Tiles I need to do only a few at a time when I'm using the enormous raster maps and making 4096 tiles, but again it had been able to do this.

My last total lock-up (which now happens every time I use TD) was a test 7x5 baseboards, using 4 or 6 64x36 pixel raster maps. Had to shut down with power button but route was made (as far as I could tell in surveyor). I then made a single tile - same thing.

Just before the OS stops responding memory usage goes from almost all to all, cpu is only working on service tasks (no green bars over the red ones) and then even that drops to 5 or 6%. I figure OS is doing nothing but trying to move stuff from virtual RAM(disk) to real RAM just so it has some code to try to run.

This is exactly how it behaved when I had only 4GB installed and I pushed TD beyond it's limits. For a week or two after I put another 8GB in it worked fine....

Hope this helps
Click to expand...

Same version as I have. Most of the route files I work with are up to 4 gb in size and I don't have problems, that's around 7 or 8,000 baseboards. I really don't have a clue why you are getting lockups. Can you post any error msgs it is giving you. Maybe that will point one of us in the direction where we might be able to help.
 
I would check on disk space. How much disk space do you have left, Chris?

TransDEM needs working space and so does the OS. Even with 8GB of RAM, you're going to crash if there isn't enough room to move stuff around in.

John
 
Thanks, Guys, for taking the time to help.

Main mystery solved: Why did it work and then stop working?

I was doing some very low-res raster maps for baseboard texture (72x30 for pics taken 1km up) so i unchecked "use jpeg for texture files" on the tile-export menu. Rechecked that box. Problem solved.

We'll see how many tiles I can do at one time using the 4k UHD jpegs, but I know I can do at least one...

Still no reason this task should use anywhere near 10GB but...

Something I discovered by accident:
You have to move or zoom the map after using a rectangular filter. You end up with a selection area of -1 x 0 tiles if you don't, even if you draw another rectangle.

After I got tile creation working again I noticed TD keeps the 6GB or so that it uses. When I force a map redraw (with a move or zoom) it then releases that block of memory which leads me to believe the 2 bugs are somehow related. I'll do a little more research before I tell Roland.

On another note, I love using the task manager performance tab on my other monitor; when importing a DEM I can watch all 4 processors go to 100%. Excellent utilization of available resources.

So, now I can get back to work. This route could be really good...


Thanks again,

-chris
 
That would make sense. A JPG is a highly compressed image format which allows for more information taking up less space. By unchecking that option, you used up all your reserved memory and the program crashed.

Task Manager is great for stuff like this. If you are interested in more details, you should run Process Explorer and RAM Map from Microsoft. You can download these two free utilities from here: https://social.technet.microsoft.com/Search/en-US?query=system internal&ac=2

Process Explorer gives a quite a bit more details on which threads are occupying memory and operating. The extra advantage of this is the ability to pause processes which is great for killing viruses and cleaning up the machine. RAM Map is a RAM utilization utility which shows how much memory is being used by the system and where.

I'm glad you resolved your problem though as that was a good one as we say in the IT world.

John
 
JCitron said:
That would make sense. A JPG is a highly compressed image format which allows for more information taking up less space. By unchecking that option, you used up all your reserved memory and the program crashed.
Click to expand...

Been doing some math on that and there's still no way this should use 10GB: The huge tiles are 16Mpixel (16,777,216). even if we assume an 8-byte WORD
(such overkill esp. since we're down to 256 color images by this time) that's 134-1/4 MB. I did not make 81 tiles at once (24 seems to be the limit) (and if I did I would hope tiles would be written to disk once they were done...)

JCitron said:
Process Explorer ... the ability to pause processes ...
Click to expand...

Shiny! Thank you, John.

JCitron said:
RAM Map is a RAM utilization utility which shows how much memory is being used by the system and where.
Click to expand...

That should just about take care of it. Awesome.
 
I'm glad you found those utilities helpful. They're part of my toolbox when checking things out.

Regarding the file sizes and memory use. This is only a tiny portion of the picture. The program still needs room to process the files and load them along with other memory overhead just to work. On an unrelated program, I ran into a similar issue. This was actually a very large database which required building queries in Excel and Crystal Reports for reporting purposes. If the query was too large, the programs would hang. The issue was due to the query builder in ODBC running out of memory as it tried to build the reports by downloading the information. With smaller database queries, things were fine, but if we went just a tiny bit too large, POOF! that was it and the program would crash.

John
 
You must log in or register to reply here.
Back
Top