Questions on layers

I am working on a large route that consists of four routes, each of which was generated by TransDEM and merged together. As you know, to do this, each route layer name has to be unique, so you need to rename the route layers first. TransDEM also creates three layers, two of which are named Vector and the other UTM. I forgot to delete these two on one of the routes, so now I have a total of 6 route layers.

What I want to do is delete the Vector and UTM layer, leaving four route layers. Then I want to merge three of the route layers into the fourth. Easy to do and I have done this many times. But when I try to delete the UTM and Vector, or when I try to merge a route layer into another, the progress bar gets to about 40% complete and then just stops going further, even if I let it run overnight. Any idea what is wrong?

Also, I noticed for the first time that, in the merge layer window, there are four colored circles (gray, yellow, red, and green) each of which can be toggled on or off with a mouse click. I have been unable to find anything on the wiki that explains this. Anyone know what these do? Thanks.
 
I know very little about Layers, but the gray, yellow, red, green circles could be similar to what you see in Content Manager when it is submitting assets to its database. They are like filters for the output of the validation process, corresponding respectively to statements, warnings, faults and acceptance of the asset.
 
schweitzerdude - Most likely the merge process has run out of both physical memory (DDR4) AND page file (virtual) memory 'headroom'.
Recommend increase your virtual memory to something like 3X your current amount of physical RAM and try again. If there is sufficient memory, then the process should complete - and complete faster.
I had to temporarily increase my page file memory to 90Gb (of free disk space) to allow one such Trainz process to complete reliably.
You can always reduce this amount afterwards for normal operations.
 
schweitzerdude - Most likely the merge process has run out of both physical memory (DDR4) AND page file (virtual) memory 'headroom'.

Your recommendation makes sense, and I tried it. I have 16GB of physical memory, so I increased virtual memory on my hard drive (which is only 12.5% full) from about 21GB to 48GB. Unfortunately, it had no effect on the merge/delete layer process.

I am guessing that there is some limiting factor in the software itself which prevents the process from completing. The route has a very large number of baseboards although 95% have no development at all. When you build routes from TransDEM, it's very easy to make the route too big, thinking "well I can always delete baseboards later."

Thanks for your suggestion.
 
Did you monitor your memory allocation/ usage whilst performing the latest merge attempt?
Suggest use Task Manager's Performance tab whilst carrying out your next attempt to see just how much memory the process consumes as it progresses.
You may need to boost the virtual memory allocation to ~75Gb to ensure completion of the merge task.
Also, did you reboot your machine after resetting the Windows page file allocation before restarting your merge?
You will be amazed by just how much memory is utilized by the Trainz merge routes process.
Cheers! PC.
 
Last edited:
When merging two routes it may take a while because it needs to check for all the duplicate asset names you may have used. Junction 12895 might have been used in both routes so causing a duplicate in the merged route. Trainz will not like duplicate names as it gets confused about their proper location.
 
To stagecoach: Good advice, except one route had all the usual development, while the three routes created by TransDEM and then merged were bare terrain with nothing else.

To PC Ace: Yes I did restart. I increased virtual memory max to 75GB as you suggested. Deleting/merging layers still stalls at the same place as before on the progress bar at about 40%. Looking at task manager/performance tab, I see CPU 19%, Memory 43%, GPU 22%, and all drives at 0%. I don't know how to interpret this, however.
 
Ok. This looks like a job for monitoring the 'Show logs' stream in a concurrent or adjacent Window whilst you attempt the next merge.
Leave the page file memory at 75Gb for the moment, as you still might need most of it once the block is identified.
Watch the stream of log entries as the merge proceeds and look to see what is happening/ last message(s) before the stall.
It should help you to narrow down the suspect assets/ process.
 
Ok. This looks like a job for monitoring the 'Show logs' stream in a concurrent or adjacent Window whilst you attempt the next merge.

I tried finding the "Show Logs" in Task Manager and was unsuccessful. I don't know where else to look. This is why your username is "PC Ace" and mine is not.:)

Can you tell me where to look for this? Thanks.
 
Yup. Peter (above) is right. Go to the Developer menu in Trainz Launcher and choose 'Show logs' from the menu list. (Suggest clear them first - 'Clear logs' to get a clean stream for subsequent analysis).
I open this up in a second monitor so I can watch the log stream unfold during merge runtime, but if you have only a single monitor, this can simply be in a new Window set off to the right of your screen.
My apologies for the ambiguity.
Your PC allows for multitasking of many separate processes/ threads/ programs, especially if you have large amounts of physical memory or supporting page file/ virtual memory.
This allows for situations like this where it is desirable to have Task Manager, Trainz 'Merge Routes', and 'Show logs' tasks running simultaneously in separate Windows allowing you to monitor the Merge Routes process and memory utilization as the task proceeds.
(another) Peter, aka pc_ace
 
Thanks PC_Ace and pwjohnson for the advice.

I opened the Developer/Show Logs window. Cleared what was there. Began the merge and let it run with the Show Logs window open as well, being careful to not do anything in Trainz so as to avoid extraneous info in the Show Log window.

I let it run for about 6 hours. It stalled at about the usual 40% completion. The Show Log window remained totally empty.
 
schweitzerdude - You're right, the 'Show logs' window is not terribly verbose during the route merge process - I only get a few lines on a fully completed merge.

Did you get any lines similar to this in the log at all?

+ <kuid:220281:100002> : <kuid:220281:100002> opened for edit
+ <kuid:220281:100646> : <kuid:220281:100646> opened for edit
? <NULL> : WorldCel::Merge
- <NULL> : TrackStretch id=918 detaching from invalid owner object id=1422
+ <kuid:220281:100754> : <kuid:220281:100754> opened for edit
+ <kuid:220281:100754> : Asset <kuid:220281:100002> successfully cloned to <kuid:220281:100754>

Note that the TrackStretch entry above indicates an error condition resulting from my test merger.

I renamed any layers that were identified as duplicate names before the merge process would allow continuation (route_layer, and route_layer1, etc.)

Evidently, your layer merge issue requires further resolve and maybe requires some assistance from N3V QA.
Suggest complete a bug report via this link https://n3vgames.typeform.com/to/xRdryu and see if QA can assist.
 
schweitzerdude - You're right, the 'Show logs' window is not terribly verbose during the route merge process - I only get a few lines on a fully completed merge.

Did you get any lines similar to this in the log at all?

/QUOTE]

No log entries were generated at all during the six hours of the merge attempt. I agree that the next step is a bug report.

Thanks for all your help.
 
Have you tried merging all the duplicate layers first rather than keeping them separate with new names?

I have found through experience that the fewer the number of layers, the more successful the merge process is. As always anything with Trainz is pure KISS.
 
JCitron: Good question. Unfortunately, I did all the route mergers (three into the fourth) and didn't do any layer mergers until a few days ago, at which time I had six route layers.

I think what you are suggesting is that I should have done layer mergers right after each route merger.

With that in mind, my next plan is:

1. Save MyRoute as MyRouteTrimmed.
2. Delete all undeveloped baseboards from MyRouteTrimmed. This will get rid of 95% of baseboards.
3. Attempt to merge the six layers in MyRouteTrimmed into one.
4. If successful, I will, for each of the three TransDEMRoutes (which I still have, and each is undeveloped):
4A. Delete the Vector and UTM layers.
4B. Rename the one remaining TransDEMRoute layer to a unique name.
4C. Merge the TransDEMroute into MyRouteTrimmed.
4D. Merge the single TransDEMRoute layer into the one route layer of MyRouteTrimmed.

Doing this means I will never have more than two route layers at any one time.

If this doesn't work, I can either:

1. Live with it.
2. Do a bug report and see what N3V suggests.

Thanks for your comment.

EDIT: My idea did not work. My guess is that the route is corrupt.

I am thinking about using Surveyor 2.0 to copy rectangular chunks of the route to the Scrapbook and then paste to a new, blank route. I don't know how big the Scrapbook chunks can be so it will be very experimental for sure.
 
Last edited:
There is a possibility that the route is corrupt. I've had that happen not long ago. The symptoms were similar but not quite the same because the route opened up after processing for a very, very long time. When the route finally opened up, the objects were missing. Tracks, roads and textures were there but nothing else. A retry of the merge worked fine. What I'm thinking now is that this is the same issue except manifesting itself in a different way. I would definitely submit a ticket for this.

My thought is to use fresh data again from TransDEM to generate new routes and then try merging again. For my use, I found that none of the TransDEM layers are necessary for the routes. You may, just for kicks, try removing them completely.

This is my merge method for large routes:

1) Open the route in Surveyor and cut it up into smaller parts.

Delete all the unnecessary layers.

Save that as Part A

Repeat for Part B and C, or how many that are needed.

2) Open up Part A again and merge in Part B.

Save that as Parts A and B.

3) Merge in Part C, etc.

If all goes well, call that the final route name and delete all the interim pieces. I've actually saved them to CDPs in case I whiffed, and I needed to restore that section again. That saved my backside a few times when setting up the base routes because during my trimming process, I trimmed a baseboard or two too many.
 
JCitron

I agree that the TransDEM layers (2 of them) are not needed, and in fact I deleted them from two of the TransDEM routes, but not the third. Going forward this is a good practice.

My experimentation with using Scrapbook to copy and paste to a new, blank route is having mixed success. Although it works, if you try to copy and paste too large a chunk, nothing happens, even if you let it run overnight. Small chunks work fine, but it becomes impractical. On the other hand, doing this provides an opportunity to rearrange chunks of route, which I can do because it is fictional, not prototypical. Fortunately, I have Surveyor 2.0 which is the only way I can think of to move a chunk of development from one route to another starting fresh with a distinct layer. Using Classic to copy and paste to new, blank baseboards and then delete the original baseboards doesn't fix the layer problem in my opinion.

Starting a bug report or help desk ticket becomes a problem because N3V will ask me to send them the route. The last time I did this for another issue (ground textures), I ran up against the limitations of max file size for gmail and Google Drive. And it was for the same route, but severely trimmed back to lower file size. I was able to fix the problem myself with the untrimmed route. N3V QA team told me they had never seen the issue before.

Having two different, unsolved bugs for the same route supports the idea that the route is corrupt and it's time for me to move on.

Thanks to you and all others who tried to help.
 
Last edited:
JCitron

I agree that the TransDEM layers (2 of them) are not needed, and in fact I deleted them from two of the TransDEM routes, but not the third. Going forward this is a good practice.

My experimentation with using Scrapbook to copy and paste to a new, blank route is having mixed success. Although it works, if you try to copy and paste too large a chunk, nothing happens, even if you let it run overnight. Small chunks work fine, but it becomes impractical. On the other hand, doing this provides an opportunity to rearrange chunks of route, which I can do because it is fictional, not prototypical. Fortunately, I have Surveyor 2.0 which is the only way I can think of to move a chunk of development from one route to another starting fresh with a distinct layer. Using Classic to copy and paste to new, blank baseboards and then delete the original baseboards doesn't fix the layer problem in my opinion.

Starting a bug report or help desk ticket becomes a problem because N3V will ask me to send them the route. The last time I did this for another issue (ground textures), I ran up against the limitations of max file size for gmail and Google Drive. And it was for the same route, but severely trimmed back to lower file size. I was able to fix the problem myself with the untrimmed route. N3V QA team told me they had never seen the issue before.

Having two different, unsolved bugs for the same route supports the idea that the route is corrupt and it's time for me to move on.

Thanks to you and all others who tried to help.

I've run into the same issue sending routes to N3V.

I have had the mixed results with S20 as well when copying chunks. The issue is memory, meaning RAM. Even with 64GB in my system, I've run right up against the wall when doing that which means like you the chunks have to be smaller and then that becomes cumbersome. It's a great tool for mixing and matching parts though. I did that the old-fashioned way many years ago with my old route and that was a long arduous process when trimming out what I didn't want and leaving the other parts.

I agree the route sounds like it's been corrupted in some way. Do you have any backups? I got lucky with mine and did that happy dance when I found it, well figuratively that is!
 
All the backups would be corrupt as well since I never tried layer merges until very recently. With 16GB physical RAM, up to 75GB more virtual, I thought I could Scrapbook larger chunks. And smaller chunks means more work reattaching track and road splines after the paste. And you have to accurately predict which baseboards in the new route are 5m vs 10m grid so you are not left with a lot of retexturing. Even 2x4 baseboard chunks with a modest amount of development take 10 minutes to paste. And high development chunks left to run overnight never complete the paste at all.

Other than the layer merge problem, the route works perfectly. So, I will likely live with six route layers. But I will submit a bug report and email a very trimmed version to N3V to see if they have a solution.
 
Last edited:
Back
Top