Persistent bug in 2010

[RPearson]

Bill - Are you using 5m grid baseboards?

Looking at the screenshot in the 1st post I can see a bunch of cruciform openings that look like the terrain is breaking down on the 5m grid boundry. Some or all of the 5m center and mid side vertices of the 10m sq in the center of each cruciform are at 0 elevation. The 4 corner vertices of the 10m sq are at the correct elevation.

HOG and TransDem write out gnd files using different version of the gnd file format. HOG still uses the old UTC gnd file format. TransDem uses a newer version maybe the latest. Since the terrain initially gets into Surveyor without any spikes using both of these programs I'd lean towards a bug in Surveyor. But strange things often happen with Trainz you never know.

Bob Pearson

 

[HiBaller]

No, Bob. I'm still using the old 10M grid. Peter and I have isolated what we think may be part of the problem. It has to do with a texture that uses Targa files (2) that are 1024x1024 (4098K) each. The Replacement tool might be having trouble getting around that size of a file.

In addition, my investigations show that scattering textures by holding down the "[" key and swooshing the mouse around may be leaving very tiny areas of "no color". When this texture is replaced with another, those tiny areas end up with little black dots on them. Since there is no color, then the Save routine converts this no-color area to a zero height for some reason.

Heck, I'm just guessing here, but I'd bet it has to do with the Replace function since that is new to 2010.

Bill

 

[narrowgauge]

Bob

I'm not inclined to hold the texture at fault because I have had the spikes where that texture has not been used.

I think it is the Replace asset tool, both Bill and I have been using that for replacing very large areas of ground texture, and the spikes appear to be more numerous in those areas. I initially used a particular ground texture for all my section of the route, as Bill did, spinning while applying. I then overpainted forest and river gully areas with a dark green. No problems to that stage for either of us. Both Bill and I replaced that initial colour with another one, and I also experimented with othe colours for the forests always using the Replace tool. That is when the 'cancers' started.

Bill commented about the black areas after a Replace, if they are left they result in a spike, if they are painted over before saving there is no spike.

Peter

 

[cvkiwi]

Blue Board

Yep I get Blu Boards when Im dashing along a layout checking out the assetts on that layout Im not a happy man and this is in my nice new 2010
and there is no traffic moving when I put doun a road but in 2006 I dont have that problim.

 

[narrowgauge]

Forget my drivel about the vertices. I just used native wit to see what i was talking about.

1. The spikes do centre on a vertex

2. The spikes involve 4 x 10 metre squares so the 5 metre grid is not likely to be the culprit.

3. On a new route, I raised the ground by 30 metres. I painted it with the Rough Grass texture, spinning and swirling like a mad thing. Finally I replaced the texture with another but no black spots appeared. That texture would not have been used with spinning, it is the new type with Normal mapping, and spinning destroys the normals effect.

4. The sections that Bill and I are working with are probably about 250-300 boards so the area being replaced may be a factor to consider.

CVkiwi. Your problem may not be the same as the one we are discussing, can you provide a screenshot of the effect.

Peter

 

[RPearson]

1. The spikes do centre on a vertex

2. The spikes involve 4 x 10 metre squares so the 5 metre grid is not likely to be the culprit.

In Bill's 1st post the screen shot definitely shows patterns that involves a 20x10 cut n-s and e-w. Maybe this is different from yours. [The 5 missing vertices are all on the 5m grid. The verticies on the 10m grid are all where they should be. See pic below. Since this is a 10m baseboard I don't understand why that should happen. EDIT- This isn't correct. I got the 5 and 10m grid lines mixed up. The pattern I saw is consistent with 1 central vertex displaced and the terrain folded along the diagonals of the 10m grid squares.] Guess I'll have to open the hex editor and work out the new format as Auran isn't making them available anymore. Got to do it sometime - Roland bit the bullet awhile ago so maybe now's a good time for me.

3. On a new route, I raised the ground by 30 metres. I painted it with the Rough Grass texture, spinning and swirling like a mad thing. Finally I replaced the texture with another but no black spots appeared. That texture would not have been used with spinning, it is the new type with Normal mapping, and spinning destroys the normals effect..

I tried the same with a couple of ordinary textures and also with scenery objects and track but didn't see any spikes. I also tied appending 10m and 5m baseboards and saving and no spikes apeared.

Bob Pearson

Sorry the pic I posted here was not correct. I got the 10 and 5 m grid lines mixed up. When I tested a few manually made spikes on a 10m grid board I get a pattern similar to what Bill posted. The wire frame distorts the results somewhat as the 10m squares get folded on the diagonal.

 

[JCitron]

I've had the black holes in other versions besides TS2010 so this goes way back. It appears to be a memory-related issue with the program.

I've had the spikes and hole issues too with my route, and they seem to happen randomly. At first I attributed it to merging routes, but then it happened after saving another time. The spikes and holes seem to happen along the seams between the baseboards. Another "feature" that seems to be related to the same issue as the spikes and holes, is missing textures directly along the baseboard seams. This bug seems though to be more relating to merging and less of a random incident.

Now, when TS2009 was in Beta I and Beta II, I reported the spiking/holes and the missing textur-issue to Auran. The reply back was they are aware of the problem, eluded that due to the lack of time that they couldn't fix it at that time, and it appears that they never did solve them so the same issues that appeared in TS2009 have now become basecode bugs that will be in all future versions of Trainz.

John

 

[HiBaller]

Oh, wonderful. I can't tell you how much better that makes me feel.



Bill

 

[JCitron]

Oh, wonderful. I can't tell you how much better that makes me feel.



Bill

Exactly my feelings too.

Perhaps if we squeak a little louder on these [problems, we can get these issues looked at again. The reason I'm saying this is because I've received some promising news from Zec regarding the merging issue.

John

 

[clam1952]

Oh, wonderful. I can't tell you how much better that makes me feel.



Bill

Have you tried merging the DEM with a blank board saving reopening it and then deleting the blank board? That solved the merging problems in 2009 with the sharp ridges and troughs along the board edges. May be worth a try?

 

[narrowgauge]

Clam1952

Neither of these boards have been merged unless it was done as part of the DEM process, I don't know enough about that to guess. In any case they appear at random, not on board borders.

What we did find is that when the black markes appear after a Replace move, if the mark is selected and retextured, a spike does not get made. On my section I replaced the two broad area textures with very light coloured ones that showed the black marks, After 'killing' those potential spikes, I repeated this process a couple of times and ended up with a spike free board.

Peter

 

[geophil]

Whatever happens in Surveyor, the gnd file, which holds the terrain and all the elevations, does not store any assets (apart from ground textures) or link to them. All relations to other assets are by coordinates. It seems strange that replacing assets should change terrain elevation (or reduce it to 0).

The 5m and the 10 m grid are structured in a similar manner. There was a relevant format change in the gnd file for TS2009, to accommodate the 5 m grid, and a number of attributes were changed and some removed but the basics are still the same.

The only influence TransDEM may have depends on the original DEM and any post-processing to it. If the DEM has holes (void points) these may end up as zero elevation in the TransDEM-generated Trainz terrain. Re-sampling a DEM, in particular re-projecting, with another program (not TransDEM) may produce such holes. Re-sampling in TransDEM is designed to prevent such holes.

 

[HiBaller]

I have gone back to the absolute original TransDEM-created route. It contains no voids or anything like that. It did have a mapped terrain cover showing roads, lines of elevation, and the like. When I got my third of the route to work on I laid down tracks according to the cyan-colored track route and then began painting (not replacing) over the original baseboard texture.

We did find that the TransDEM contour representation was no where near what it should have been. Granted, elevations for that part of Australia are hard to come by, but in some cases the lines of contour showed a valley and the DEM would either show a flat area or even a small hill.

We ended up pushing and pulling hills into shape. That shoud have had no effect on 'holes' or 'spikes'. It was when I got to my 10th iteration of the route that spikes began to show up. I encountered one right along the right-of-way where I had been over and over again loads of times. It should not have happened.

I repaired the spike and continued. When I ranged afield of the tracks is when I began to spot more spikes. The areas affected were areas where I had painted (not substituted or did a 'whole board at a time') a brown grassy texture. We decided later to replace it with a texture created by Peter which was a darker (and greener) grass. Each time I substituted the old browngrass texture with the BFR texture I would end up with spikes about 50% of the time. It wouldn't happen often enough to be called repetitive, but enough to cause gnashed teeth and swear words.

We know now that if we substitute textures to go over the whole map looking for little 'black holes'. If we find one (or more) we can paint over them with the surrounding texture and when we save the route no spikes are formed. If we forget to do this and save without covering the black holes we get spikes at every location of a black hole.

This indicates that somewhere in the save routines is a process that "sees" those black holes and moves the relevant vertex down to ground zero. It isn't until the Save that the spikes are formed. I agree that they are ready to be formed by the appearance of the black holes, but these holes should have actually been completely covered by the replaced texture and weren't. In my tests, I focused directly over a patch where black holes were formed by the replace procedure. The original texture was completely covering the ground - no gaps or thin spots. When the replace was done - the black holes appeared for no discernible reason.

So, if it were I, I would look into both the substitute and the save routines for perhaps a double fault.

Bill

 

[narrowgauge]

Geophil

does not store any assets (apart from ground textures) or link to them. All relations to other assets are by coordinates. It seems strange that replacing assets should change terrain elevation (or reduce it to 0).

This agrees with what we find, all scenery and track assets remain but vertex height data and ground texture data lost, Remember, it was replacing ground textures that appear to be the originator of the problem.

It is possible to recover the route, painting over the black spots and saving removes them, strange however, that doing this by replacing with a light ground texture and looking for black spots, saving and replacing again eventually ended up with a route with no problems. This was despite the repeated use of the Replace asset function which originally seemed to be the cause of the problem.

All my instances appeared where where I had changed contours. Typically where I had 'gouged' valleys for river courses, spikes would appear in one side wall of the valley. Perhaps the Topo tools have a small problem?

Peter

Peter

 

[HiBaller]

Like most of the anomaly's found in 2010, without investigation by the very programmers that created the routines no answer will be found. We can make educated guesses but that's all they are - guesses.

I've put in a help desk chit, but have not heard anything back on it at all. I do think I have enough information to get rid of the spikes I already have and hopefully they will not return.

Bill

 

[HiBaller]

Well, shoot. Now I know why I had so many spikes on my section. I was thinking about it and remembered that I had used copy/paste for large sections of texturing. Naturally, if I picked up an incipient 'hole' and pasted it down, the information would go with it.

Does that mean anything to anyone? Perhaps it's the texture that's causing the spikes and not any DEM/elevation data?

This is making my head hurt!

Bill

 

[martinvk]

Sounds like a theory but I'm not sure what an incipient hole is. I always thought the grid was fixed until a node was raised or lowered manually. On the other hand, displacement maps can do that too so perhaps investigations should look in that direction. Are your textures acting like extreme displacement maps causing negative spikes to appear?

 

[HiBaller]

I should have identified what I meant by 'incipient hole'. It means the scattering of those black spots that will turn into reverse spikes when you save the route.

What I meant was if I outlined an area of texture I wanted to copy - and it contained some black holes - then when I pasted it down in a clean area the black holes would cause new spikes.

I tumbled onto this when I looked carefully at a group of reverse spikes and noted that they were arranged exactly like some that I had just repaired. Unfortunately, I couldn't re-paste to remove the spikes. They were there to stay.

Bill

 

[geophil]

Does that mean anything to anyone? Perhaps it's the texture that's causing the spikes and not any DEM/elevation data?

The gnd file has three variants to store elevation and texture for a given ground vertex, 1: single texture, with elevation zero, 2: single texture, elevation unequal zero, 3: multiple textures, any elevation. (There is no "no texture", the grey or grey/orage grid is created by a default texture.)

Perhaps something goes wrong when switching from one variant to another?

 

[HiBaller]

Could very well be. When I puzzled out the TRK and GND files for my programs it was pretty straightforward. Since the format was changed for 2009, and yet again for 2010, I simply don't know what is going on inside those files. I can see that the basic structures are relatively unchanged, but the structures are now larger than before. Most of it is undoubtedly taken up with what layer an object is on and the like, but I am sure that other factors were thrown into the mix too. I've gone about as far as I can go now. I know how to avoid the spikes and can continue on with the whole point of Trainz - creating and releasing new routes.

Bill