PDA

View Full Version : Merge route!?



escd84
August 28th, 2018, 12:03 PM
Has anyone already tried this in TRS19? For me, there are always exactly the 2 ways it ends. A to B and the game crashes from B to A and the game hangs forever.

Edit:

Something like that CPU 24%, RAM 43% and GPU 65% all the time when the game hangs on.

JCitron
August 28th, 2018, 12:27 PM
Is the program being busy, which it sounds like rather than hanging. Give it a few minutes and see what's happening. A very large route or routes can take a very long time, for lack of technical explanation, to merge together.

I had no issues merging Clear Lake Lumber into my test single-baseboard route either way.

Why your's crashed going one way but not the other is odd, but it could be something to do with route sizes, something corrupted, etc.

escd84
August 28th, 2018, 01:16 PM
So I wait 1 1/2 hours, nothing changed. I tried to merge 2 blank baseboards I created in TRS19, so both are build 4.6

JCitron
August 28th, 2018, 01:33 PM
I just tried this and there was no problems merging.

escd84
September 2nd, 2018, 08:45 AM
Found a cause: With TransDEM generated maps the *name*_transdem_info must be deleted for it to work. Then it works within a few seconds. No idea if Geophil and TransDEM should be informed as well?

For the crash when merging empty baseboards, I have not found a clue. Bug report made.

Edit:

I have no idea how a difference from a grid comes about with the baseboard transitions

https://img2.picload.org/image/dliaigai/2018-09-02154621.jpg

https://img2.picload.org/image/dliaigaw/2018-09-02154636.jpg

JCitron
September 2nd, 2018, 09:46 AM
TransDem routes use a different baseboard texture.

celje
September 2nd, 2018, 09:18 PM
My experience with merge is not good. I tried to merge DBtogether- France and Sulitjelma routes, both clean, build 4.6.
When i tried merge small route to big then i have CTD after about 20 min. Then i tried to merge the big route in small one, then process isn't finished in 3 hours, so i gave up.

PC_Ace
September 2nd, 2018, 09:50 PM
celje - Recommend temporarily increasing your swapfile/ virtual memory allocation to an amount about twice what your physical RAM is.
This should give you the memory headroom for this (obviously lengthy) process to complete, since it is evidently swapping out to pagefiles already.
Note that new swapfiles can be added to any HDD and not just your boot disk.
(Control Panel, System, Advanced System Settings, Advanced, Performance, Settings, Advanced, Virtual Memory, Change... etc. Save and reboot to apply your new virtual memory settings)
The other ingredient needed here is Patience. Merges of large routes take time...

celje
September 2nd, 2018, 10:06 PM
celje - Recommend temporarily increasing your swapfile/ virtual memory allocation to an amount about twice what your physical RAM is.
This should give you the memory headroom for this (obviously lengthy) process to complete, since it is evidently swapping out to pagefiles already.
Note that new swapfiles can be added to any HDD and not just your boot disk.
(Control Panel, System, Advanced System Settings, Advanced, Performance, Settings, Advanced, Virtual Memory, Change... etc. Save and reboot to apply your new virtual memory settings)
The other ingredient needed here is Patience. Merges of large routes take time...



PC Ace, thank you very much for advice. About virtual, I have now Initial 16384 mb and max 32786, free space on HD 3TB. Can you suggest a different setting.

Tony_Hilliam
September 2nd, 2018, 10:20 PM
It is quite possible this issue is specific to the routes you are merging - please provide the KUIDs so someone can check if it happens for them to eliminate hardware configuration as a factor.

PC_Ace
September 2nd, 2018, 10:21 PM
What size is your physical RAM (i.e. DDR3 or DDR4?)
For example, I have 16Gb of DRAM so I would temporarily set mine to 32Gb on two drives (i.e. boot drive and including the drive that houses T:ANE or TRS2019) just to be sure.
Your available CPU cores and hyperthreading play a big part in this too, as does any fragmentation of the files contained in the route - suggest run a defrag on the drives you're planning to use as virtual memory first in any event, plus the drive that contains the routes. And don't run much else during this process except perhaps Task Manager or some other memory monitoring tool.

I don't know about the route size of DBtogether - France (etc.) but understand that it is quite large. How many megabytes?

In the past, I have watched some T:ANE processes (like merges and Full Database Repairs) take up more than 40Gb of pagefile memory.

celje
September 2nd, 2018, 10:39 PM
It is quite possible this issue is specific to the routes you are merging - please provide the KUIDs so someone can check if it happens for them to eliminate hardware configuration as a factor.



Thank you Mr. Hilliam
It's not possible to provide the kuids, because DBtogether-France route for TRS19 is not yet published. I'm working on it now. I will try with higher virtual memory.

regards
celje

celje
September 2nd, 2018, 11:03 PM
What size is your physical RAM (i.e. DDR3 or DDR4?)
For example, I have 16Gb of DRAM so I would temporarily set mine to 32Gb on two drives (i.e. boot drive and including the drive that houses T:ANE or TRS2019) just to be sure.
Your available CPU cores and hyperthreading play a big part in this too, as does any fragmentation of the files contained in the route - suggest run a defrag on the drives you're planning to use as virtual memory first in any event, plus the drive that contains the routes. And don't run much else during this process except perhaps Task Manager or some other memory monitoring tool.

I don't know about the route size of DBtogether - France (etc.) but understand that it is quite large. How many megabytes?

In the past, I have watched some T:ANE processes (like merges and Full Database Repairs) take up more than 40Gb of pagefile memory.



Specifications
RAM 16 gb, i7 4790, CPU cores 8

DBtogether-France 200mb, Sulitjelma 50mb

I defraged, i will try now with 80000 mb initial and max same value.

PC_Ace
September 2nd, 2018, 11:15 PM
Heh! Good luck with that. Costs nothing to experiment with swap file space and with 3Tb available, you've got a bit of headroom there. :)

Expect that it WILL still take some considerable time to complete - especially if your route is over 100Mb.
4 cores and 8 threads will certainly help - (I have the i7-4790K running at 4.6Ghz btw) - but such merges often involve serial rather than parallel threading processes.
Now give it a try and monitor your pagefile and memory use using Process Explorer or Task manager or similar. Should be interesting.
Remember - if it's not throwing up errors, it's probably merging, so Patience and time allocation will be the key ingredients hereinafter.

If successful, you can revert your virtual memory settings to something more sensible afterwards...

Tony_Hilliam
September 3rd, 2018, 12:40 AM
A normal route merge with ~200MB routes should take less than a minute. We have identified an issue with TransDEM route merging so it sounds like your issue will fall under that category.

celje
September 3rd, 2018, 04:50 AM
I'm optimizing the disk now, it will last a couple of hours still.
But i have question. In CM of TANE the routes for TRS19 are shown as ''Incompatible''. In CM of TRS19 the new routes are not shown yet, so the only way for download is use DLS with ''Download with Content Manager''. Is it possible that this cause the problems.

celje
September 3rd, 2018, 09:05 AM
Unsuccessful, merge don't work

JCitron
September 3rd, 2018, 09:42 AM
Incompatible with T:ANE means the route is at a different Build version now for TRS19, and therefore cannot be downloaded into T:ANE. With the changes made with TRS19, I don't think that the content can be down-versioned.

celje
September 3rd, 2018, 11:35 AM
Incompatible with T:ANE means the route is at a different Build version now for TRS19, and therefore cannot be downloaded into T:ANE. With the changes made with TRS19, I don't think that the content can be down-versioned.



Yes of course, it's not possible to download from CM of TANE, but question is why the route is not shown in CM of TRS19. When you open CM of TRS19 there is no new assets.

JCitron
September 3rd, 2018, 01:12 PM
Yes of course, it's not possible to download from CM of TANE, but question is why the route is not shown in CM of TRS19. When you open CM of TRS19 there is no new assets.

That's odd. Is something corrupted in the route (I truly hope not!). Have you tried a backup of the route? Even one that's older maybe good for testing just to see what's going on.

Can you make the unfinished route available for testing? I would like to take a look at it and see what's going on. I don't need your content as I will just try the merge of the routes.

If you can do that, I'll test it privately and see what's going on. Just send me a link via PM.

My system has gobs of memory and disk space so with that I can rule out hardware.

celje
September 3rd, 2018, 03:06 PM
That's odd. Is something corrupted in the route (I truly hope not!). Have you tried a backup of the route? Even one that's older maybe good for testing just to see what's going on.

Can you make the unfinished route available for testing? I would like to take a look at it and see what's going on. I don't need your content as I will just try the merge of the routes.

If you can do that, I'll test it privately and see what's going on. Just send me a link via PM.

My system has gobs of memory and disk space so with that I can rule out hardware.



Hello I'm sorry I'm late.
Maybe i wasn't be clear enough. I have no problem with my route. In Tane i made cdp and transfer it in TRS19 without problem. Same i did with session. Work without problem. It had build 4.5, i change in config on 4.6.
For merge i download from DLS new route by Vern, named Sulitjelma from 2. September. When i'm searching for reasons why merge don't succeed i thought that problem is maybe in this route which is not shown on CM of TRS19, but only on DLS. Truth is that new assets are not shown yet in CM of TRS19.
I will send you PM.

JCitron
September 4th, 2018, 11:04 PM
Hello I'm sorry I'm late.
Maybe i wasn't be clear enough. I have no problem with my route. In Tane i made cdp and transfer it in TRS19 without problem. Same i did with session. Work without problem. It had build 4.5, i change in config on 4.6.
For merge i download from DLS new route by Vern, named Sulitjelma from 2. September. When i'm searching for reasons why merge don't succeed i thought that problem is maybe in this route which is not shown on CM of TRS19, but only on DLS. Truth is that new assets are not shown yet in CM of TRS19.
I will send you PM.

The new route has nothing to do with the crashing. I checked your route and can't merge your route into another, though I merged a single baseboard into your route, but not the other way around. There's something wrong somewhere which I can't find, but I did come across these here with errors in the Developer _Show Logs as having errors.

<kuid2:122285:4043:20> JK Point 10-CL Curved Left
<kuid2:122285:4051:20> JK Point 06-LH Left
<kuid2:122285:4038:20> JK Point 20-XX Crossing
<kuid2:122285:4030:20> JK Point 10-LH Left
<kuid2:122285:4031:20> JK Point 10-RH Right
<kuid2:122285:4036:20> JK Point 10-SD Double Slip
<kuid2:122285:4032:20> JK Point 10-YP Y Point


- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade01-texture' not found on asset '<kuid2:122285:4029:20> "JK Way Gauge 3T-5.0-FT (1.5m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade02-texture' not found on asset '<kuid2:122285:4029:20> "JK Way Gauge 3T-5.0-FT (1.5m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade03-texture' not found on asset '<kuid2:122285:4029:20> "JK Way Gauge 3T-5.0-FT (1.5m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade04-texture' not found on asset '<kuid2:122285:4029:20> "JK Way Gauge 3T-5.0-FT (1.5m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade02-texture' not found on asset '<kuid2:122285:4030:20> "JK Point 10-LH Left"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade03-texture' not found on asset '<kuid2:122285:4030:20> "JK Point 10-LH Left"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade04-texture' not found on asset '<kuid2:122285:4030:20> "JK Point 10-LH Left"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade01-texture' not found on asset '<kuid2:122285:4022:20> "JK Spacer 10-5.0-FT (9.14m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade02-texture' not found on asset '<kuid2:122285:4022:20> "JK Spacer 10-5.0-FT (9.14m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade03-texture' not found on asset '<kuid2:122285:4022:20> "JK Spacer 10-5.0-FT (9.14m)"'
- <NULL> MeshObject::SetFXTextureReplacement> effect 'blade04-texture' not found on asset '<kuid2:122285:4022:20> "JK Spacer 10-5.0-FT (9.14m)"'
.
.
.

etc.

There's too many to post here.

From what I recall these have 'problems' with T:ANE and have had problems since the earlier days, and now they could be causing crashes. I would remove these and give the merger a try, which I will leave up to you.

There's one other thing I tried, which I thought might help is to replace the track since the track on the route is quite old.

JR CS Concrete 2 Grey Kuid:334896:38048, which seemed like a close match to the old MP concrete track. I also replaced the occasional Auran Track with the same concrete track just because to rule out anything related to that.

And finally you have some double-placed signals in some locations such as here:

Altenstadt

Signal:
Sig LMS home lattice 20ft RH kuid2:330374:1042:1

When I installed the route, I was only missing 2 dependencies. This what happens when we've used Trainz for so long, apparently we gather nearly the whole DLS on the hard drive.

I checked the route in both TR19 96000 and 96191 and both did the same crash so it's not version-specific. Outside of these things I've found, the route is quite large, which means there's lots of content and it could very well be something else causing the problem. I can't say it's the route size that's the problem because I have a route just as large and that does not exhibit these issues.

celje
September 5th, 2018, 12:56 AM
JCitron

Thank you very much for all your efforts you put it in the search for an error.

I will delete this kuids and change the track. This track is good looking without moire effect so i will try to keep it. Interesting is that route (session) work without problem (except script errors) as stand alone and interesting is too that merge work in TANE.
I hope that N3V read this and that they will try to help. Maybe they have any suggestion.

Again, thank you very much for your help.

regards
celje

JCitron
September 5th, 2018, 11:35 AM
JCitron

Thank you very much for all your efforts you put it in the search for an error.

I will delete this kuids and change the track. This track is good looking without moire effect so i will try to keep it. Interesting is that route (session) work without problem (except script errors) as stand alone and interesting is too that merge work in TANE.
I hope that N3V read this and that they will try to help. Maybe they have any suggestion.

Again, thank you very much for your help.

regards
celje

Ah, you're welcome I'm glad I could help in some fashion by at least getting you in the right direction. The route is really nice, and I can see where there are things being changed to work to keep everything consistent. Like any route in progress, there are lots of details which will need to be looked over again and again to get rid of the bumps and holes, both literally and figuratively with such things as trees in the way, floating track, quirky junctions and everything else we find in routes.

The route will work as is as a standalone route, but that's not your intent so it's something that needs looking into in detail. I agree have N3V look at it. Send up the link you gave me to the QA Team via a bug report.

https://n3vgames.typeform.com/to/xRdryu

Explain in detail what's going on, what has been found, etc. With that information and route in hand, maybe they can take a look at things in a different way than we can. They have debugging tools and stuff that can read the code that we can't. During the testing I did receive some Assertion errors in 96191, but didn't capture them. You should send those along with the route because that maybe an answer to your TRS19 problem as well.

In part what we're running into here in TRS2019, as much as it's based on T:ANE underneath, is this really isn't T:ANE and these additions and changes can cause all kinds of incompatibilities which need to be worked out. This is why we need to report the errors to the QA Team to work through and get the bugs over to the developers.

celje
September 5th, 2018, 01:32 PM
Hello again JCitron

I sent your post, link and my observation to help desk one hour after you wrote what you find. I really don't know if my route is causing the merge problem. Meanwhile i tried to merge builtin route Niddertalbahn and Sulitjelma. i leave merging more then one hour and merge didn't finish so i close merging. Then i return to TANE and try merge there, no problem, so my PC seems to be without faults.
Two conclusions i see:
- something is wrong with TRS19
- still remain using TANE for merge and then transfer to TRS19, but this is limited for build 4.5 routes only
And yes I'm tired now.
JCitron thank you again for help.
regards
celje

celje
September 7th, 2018, 01:14 PM
After elimination all 'JK' contents from route, merge still don't work.

celje
September 8th, 2018, 02:11 AM
I hope that N3V will be so kind and will try to find out the mistake

JCitron
September 8th, 2018, 11:06 AM
I hope that N3V will be so kind and will try to find out the mistake

Yes. I agree. They have the analysis tools that we don't have which will help find this stuff, hopefully.

ish6
September 8th, 2018, 01:28 PM
Merging routes, for me, is a nightmare -- I recently began updating my Marsz route, and it's a monster of a route; And yet when I have tried to merge marsz modules to it I get lost. Etc … We need more features when dealing with merging routes!

Just my 2 cents!
Ish

escd84
September 8th, 2018, 02:41 PM
Mayby this is worth a try? Check if there are more txt files in the content folder than config.txt. Had as described above also merge problems. Later I try for myself, for me no merge worked where, for example, an Info.txt or another txt file except the config.txt was there.

celje
September 9th, 2018, 03:58 AM
Mayby this is worth a try? Check if there are more txt files in the content folder than config.txt. Had as described above also merge problems. Later I try for myself, for me no merge worked where, for example, an Info.txt or another txt file except the config.txt was there.



Can you explain where this txt. may i find. Inside TRS19\data\ i have patch-config.txt, is this?

escd84
September 9th, 2018, 04:36 AM
I looked at the content folder of Sulitjelma and the other routes you mentioned and everything is all right. Also merge the mentioned routes (https://img1.picload.org/image/dlprgdiw/2018-09-09111149.jpg) worked for me in build 96191. I just got an error message (https://img3.picload.org/image/dlprgaor/2018-09-02184209.jpg) but in less than 1 minute everything was done.


Can you explain where this txt. may i find. Inside TRS19\data\ i have patch-config.txt, is this?

In CM, right-click and open in Explorer. Then you are in the content folder of the routes where everything is which is important. Did you get any error messages from the CM or from the game?

As JCitron says, wait and see what N3V says. Maybe they can find the mistake.

celje
September 9th, 2018, 05:42 AM
Thank you escd84 for your test. I have build 96000.

regards
celje