HD terrain and TransDEM textures

martinvk

since 10 Aug 2002
One of the really nice features in TransDEM is that it can paint the ground with topographic details: roads, buildings, vegetation, etc. which helps when it is time to make the route look lived in. In my case that is a long way down the track.

Then HD terrain came along and I thought it would be great to make the ground smoother and less jagged. Well when the conversion was completed all those nice topo details were gone. Luckily I had saved the route just before so it was easy to go back.

Now the question is, did I do the conversion to HD wrong, was there something I could have done to preserve the painted details or do I have to wait for TransDEM to change?
 
Martin, you did nothing wrong. This was my first complaint about the HD Terrain texture limits. What good is it if we can't use it for prototypical routes with embedded textures from TransDEM? My statement fell directly into the inevitable bit bucket and lost in the black hole.

The alternative I think is to use UTM tiles, but that presents a problem with huge routes and also with those that require excavating out areas where railroads have been replaced by dammed up waterways. I've run into that on a recent project where the railroad is now buried 400 feet under a reservoir and a dam. I was lucky I could trace out where the route was because the topo-map had the existing railroad grade marked on it even where the water is today. With the HD terrain texture limits turning the lines into mush, that becomes impossible to do.
 
Martin, you did nothing wrong. This was my first complaint about the HD Terrain texture limits. What good is it if we can't use it for prototypical routes with embedded textures from TransDEM? My statement fell directly into the inevitable bit bucket and lost in the black hole.

The alternative I think is to use UTM tiles, but that presents a problem with huge routes and also with those that require excavating out areas where railroads have been replaced by dammed up waterways. I've run into that on a recent project where the railroad is now buried 400 feet under a reservoir and a dam. I was lucky I could trace out where the route was because the topo-map had the existing railroad grade marked on it even where the water is today. With the HD terrain texture limits turning the lines into mush, that becomes impossible to do.
It certainly is a real dampener isn't it ? Given that the most likely users of HD would be those who are keen to make accurate , more realistic routes , to not be able to take advantage of the transdem indicators is a major downside.
 
hmm, so it will be the third option after all. I wonder though, one of the TransDEM files that needs to be loaded is theTransDEM Ground Textures Vn.cdp where n is incremented with each new version. In CM I see a whole series of Texture Env TransDEM128 000 to TransDEM128 127. each one paints a different color on the ground.
I painted the ground in a new 10m grid baseboard using some of those. After converting the HD, the manual painting survived so what is the glitch when it comes to a TransDEM painted ground. Is it the 16 textures? When I manually add more texture colors, older ones disappear if I go over the limit. So can TransDEM be told to only use a limited set of colors? Might lose some of the fancy gradations but something is better than nothing.
 
hmm, so it will be the third option after all. I wonder though, one of the TransDEM files that needs to be loaded is theTransDEM Ground Textures Vn.cdp where n is incremented with each new version. In CM I see a whole series of Texture Env TransDEM128 000 to TransDEM128 127. each one paints a different color on the ground.
I painted the ground in a new 10m grid baseboard using some of those. After converting the HD, the manual painting survived so what is the glitch when it comes to a TransDEM painted ground. Is it the 16 textures? When I manually add more texture colors, older ones disappear if I go over the limit. So can TransDEM be told to only use a limited set of colors? Might lose some of the fancy gradations but something is better than nothing.
It is definitely the 16-texture limit because I saw the difference on the baseboards as the conversion process was going on. There were also a gazillion warnings/errors about the conversion process not processing due to the texture limit.

I need to experiment with that myself. I tried using those textures once but never saw anything different; maybe now there is. If this works, then I'll be happy.
 
I see in TransDEM where the ground texture is specified, the option "custom colour Textures is used referencing a TransDEM128_mapping.txt file. In that file are 128 sets of RGB values and a kuid.
If the file was carefully trimmed to only include the 16 most useful colours, would that work?
 
I get around this by making sure I have Gotten all the information I need from the textures BEFORE I do the HD conversion. All I really need is building locations, as I have TransDEM place all the roads and tracks.
 
Hi All
Unfortunately Single Pass Terrain rendering does limit how many ground textures you can have per 'tile' (aka, a baseboard in Trainz); this is a limitation of Single Pass Terrain rendering on currently available hardware and not something we have just plucked out of the air. In Trainz, the maximum we can provide is 16 textures per tile. Going beyond this is simply not an option at the current time.

So the option here is that you either need to use much less efficient multi-pass rendering for the terrain (ie the 5m/10m grid terrain we used), or the much more efficient (but with a limited number of textures per tile) single-pass rendering for the terrain. Sticking with the 5m/10m grids long term aren't an option for us, so we have introduced the Single Pass Terrain so we can provide a finer grid, but with the trade off of limit textures per baseboard.

If you are generating your own routes with TransDEM, then you may find it much more useful to use basemaps for the wanted data, and have very limited data displayed on the terrain.

I haven't tried it myself, but if TransDEM allows you to specify which ground textures are used to represent map data, you could specify actual ground textures you plan to use on the route for the 'base' textures. So instead of, say, white for the 'blank' areas of the map data, specify a grass texture you like; for the track lines specify a ballast texture you intend to use, for roads specify a ground texture you might paint around the roads, and so, whilst working to keep the number of textures as few as possible. Keep in mind that all of those textures you choose will count toward the 16 limit.

One trick I used recently whilst testing a DEM route with HDT was, after making out roads/etc with splines in the area I was working on, to paint a base dirt texture over that entire area of the route. Then paint in the other textures I needed on top of that.

It's worth noting that Single Pass Terrain ground texturing in most games is designed around using having specific textures for individual ground types, rather than multiple textures blended together. So a good start point for a route is to make a blank baseboard, or several baseboards, (you could either make a new route, or just add them off the edge of the existing route), and search for ground textures of various ground types that you would want. So you would paint down your desired 'ballast' texture, your desired 'grass' texture, your desired 'concrete' texture, and so on. This will then provide a handy reference as you need it. Or you can make an asset filter that shows only those textures, or make your own written list, or however you would prefer to reference them.

Not useful for TransDEM data unfortunately, but still relevant to the 16 texture limit, is the colour effect layers. Remember that you can then use the colour effect layer to create colour/shading variation where previously you would have used multiple ground textures.

Just as an example, and keep in mind that this is very much a work in progress route, and is also me learning how to use both HD Terrain (with the 16 texture limit), and a Colour Effect layer, I've included a couple of screenshots from a route with the colour effect layer turned on, then turned off, to show how it can be used to create extra variation to the ground textures within that 16 texture limit.

rNMVBtI.jpg


F7kb8f9.jpg


Color effect layers are, thankfully, quite easy to use. You should be able to get it up and running by following these steps:
1) Create a new effect layer, of the 'color' Effect type (if you don't already have one), and then save. You should generally be fine with default settings on this effect layer.
2) Select the color effect layer in the Layers Palette (this should then also select the appropriate brush automatically)
3) Choose a colour to paint in the 'Color' box in the Tool Options palette (I would suggest starting by just adjusting the brightness slider to get a feel for it; using this you can just create alternate 'shades' over the ground).
4) Start painting

Note that by default the Color effect layer uses a 10m grid; which is what I've used in those screenshots. Personally I've found this sufficient, but for something like a diorama route you may want a smaller grid. I would suggest leaving the 'default value' set to mid-grey (0.5,0.5,0.5 in this case) as this will not change the colour/tint of the route, and allows you to instead paint what you want where you want.

Hope this helps provide some ideas/options, and some clarification.

Regards
 
Fundamentally, TransDEM uses an old Trainz file format due to nothing being shared with 3rd party utility creators anymore. This means TransDEM is limited to the 5/10m grid and an old methodology for textures.
 
Fundamentally, TransDEM uses an old Trainz file format due to nothing being shared with 3rd party utility creators anymore. This means TransDEM is limited to the 5/10m grid and an old methodology for textures.
Correct, however my reply was specifically to provide information for those intending to convert routes to HDTerrain that have been generated using TransDEM. If you are generating a new route, with the intention of converting it, then finding methods to assist with this will make the job easier.

Unfortunately giving the terrain formats to 3rd parties isn't an option for us, at least at the moment. However we are aware of the desire for people to be able to import DEM data into Trainz, and hope to look at options for this in the future.

Regards
 
What Zec is suggesting is very close to what artists have been doing with limited palettes for a long time. Here is a video where using just two colors plus black and white the artist is able to create a wide range of shades to use while painting a portrait. The color effect layer(s) can be used in much the same way.

 
I have been looking at his forums for an answer, but suspected as much, due to answers on previous file formats. I understand their reluctance to release file formats as things maybe still in flux, and if third party utilities create errors, users to the game tech support. May even have licensing issues.

A four baseboard HD Terrain route has over 132 million height points, thus requiring heavy compression. That has made decoding the file formats, much more harder. Converting from 5/10m terrain to HD, doesn't add more detail, but allows more detail to be added. At this time all that additional height detail data has to be added by hand.
 
I have been looking at his forums for an answer, but suspected as much, due to answers on previous file formats. I understand their reluctance to release file formats as things maybe still in flux, and if third party utilities create errors
I don't understand their reluctance at all. TransDem is the most useful tool available for building prototypical routes. The author of TransDem is an expert in his field. What an advantage Trainz would have over its competitors if they would only co-operate. Seems a missed opportunity to me to dismiss the talents and professionalism of GeoPhil.

N3V say that they will look at DEM at some point in the future. They would need to allocate resources to do this (or outsource it) and support the many different types of DEM data available. Their history of delivering products that work first time out of the box is not very good. When deficiencies are identified by end users they may get fixed or put on that to-do list hoping that end users will forget.

Well here is an opportunity to get one item off their to-do list that will work, and at no cost to themselves. Make use of it, market it stating that Trainz can support DEM with the use of a trusted third party tool and give the competitors something to worry about. I won't say steal a lead on the competitors because Railworks supported DEM terrain from day one, and it worked without flaws. It is limited today in the formats it supports. This is where N3V could take advantage if it co-operated. I'm sure an NDA on the file formats could be reached.

We are told, many times, that N$V are a small company with limited resources, but sometimes they just don't help themselves.

John
 
I think the issue is who has to do the support, when problems arise. Maybe licensing of the format also. Third party programs can cause errors, then again third party programs without documentation's will cause many more errors. I've suggested an intermediate file format, that could be documented, and parsed by CM for errors. Otherwise I see things the same way as you do, and have said so. They need 3rd party utilities and need to provide support to encourage third party utilities. One problem might be documentation of the file formats. We could get things a lot simpler and automated if third party utilities were robust and encouraged.
 
And it's entirely possible N3V licensed this technology from a 3rd party, and isn't able to divulge any details due to the license agreement.
 
...Sticking with the 5m/10m grids long term aren't an option for us, so we have introduced the Single Pass Terrain so we can provide a finer grid, but with the trade off of limit textures per baseboard.
Oh, so the 5m/10m grid is going to be phased out then Zec? When will that be? Because, with the ridiculous file size, ludicrous texture limit and inability to use TransDEM (the most valuable add-on that Trainz has ever had!) I can't say I'm terribly enthusiastic about HD. In fact, at the moment I'm feeling very wedded to 5 and 10m grid, so I hope it continues for some time to come!

Paul
 
.... "Because, with the ridiculous file size, ludicrous texture limit and inability to use TransDEM (the most valuable add-on that Trainz has ever had!) I can't say I'm terribly enthusiastic about HD."....
Couldn't agree more there. I don't see anything positive in this upcoming change at all. Falls right in the "unusable" category for me
 
As far as I'm concerned, the direction N3V is taking is a total disconnect from their customers. It's obvious that no one in "marketing and development", note the quotes, has paid attention to what's been going on here for a very long time.

Without prototype options, for a product that's supposed to be all about realism, what good is it?
Is it to make yet again another version of UK Midlands or City and Country with HD Terrain and PBR textures?
 
its a bit one step forwards, two steps back game isnt it ? As with previous iterations, we get a new feature that is very nice in theory but it renders other features inoperable. More realistic terrain management is nice, but the tradeoffs make it less inviting to use , IMO if NV3 had polled users about this feature and its downsides , they may well have informed them that it was going to be of limited use. If some one like me who models ultra rocky routes probably isn't going to use it due to its file size and lack of DEM support , who is ? I imagine it will be the preserve of those who make shorter routes and those with top end hardware. I hope I'm proven wrong. Another issue is that downloading gigabytes of HD routes from the DLS isnt going to be a fun process either unless they can up the speeds to those that one gets when downloading new versions of the game or bundled routes.
 
Back
Top