Stuck at load when merging modules of 1 MB or larger file size

[STLSF4003]

As a benchmark, I just did a trial run merging two of my largest routes - 65MB and 55MB (baseboard counts "huge") - and the process took less than one minute (including resolving a layer name conflict).

Then, I have no idea what's going on! I tried again, and after all that work, it still just infinitely loads after hitting the merge button!

I'm starting to get the feeling that I must have screwed up at some point when I was building this PC.

Not enough RAM?
Underpowered CPU?

I have no idea anymore!

 

[pware]

Not enough RAM?
Underpowered CPU?

I have no idea anymore!

You have not mentioned your CPU stats.

Your 16GB RAM should be more than enough. The only "failed merge" case that I have heard of that was resolved by increasing the available RAM from 16GB to 32GB was a user who was merging routes of 100s MB each. In that case doubling the RAM reduced to merge time from 10+ hours to under an hour.

None of my merges have ever taken longer than a few minutes.

It is possible that the problem may be with the route or routes themselves. A high number of textures used and their types (e.g. PBR), an ultra high poly count on some of the assets are two possibilities that come to mind.

 

[STLSF4003]

You have not mentioned your CPU stats.

Your 16GB RAM should be more than enough. The only "failed merge" case that I have heard of that was resolved by increasing the available RAM from 16GB to 32GB was a user who was merging routes of 100s MB each. In that case doubling the RAM reduced to merge time from 10+ hours to under an hour.

None of my merges have ever taken longer than a few minutes.

It is possible that the problem may be with the route or routes themselves. A high number of textures used and their types (e.g. PBR), an ultra high poly count on some of the assets are two possibilities that come to mind.

I'm running an AMD Ryzen 5 2600 Six-Core Processor 3.40 GHz.

I don't think it's a texture issue. The module was made in 2004, so the ground textures shouldn't be too taxing by today's standards, and the buildings look pretty simple as well.

Here's the kuid of the offending module if you want to give a once-over: <kuid:137353:100466> Perhaps you'll notice something I haven't.

Also, Thank you for helping me try and work this out!

 

[JCitron]

I've merged much, much larger than that without any issues. It took about 3 minutes to complete. One route was 350 MB and the smaller one was about 30 MB.

 

[STLSF4003]

I've merged much, much larger than that without any issues. It took about 3 minutes to complete. One route was 350 MB and the smaller one was about 30 MB.

Then my system must be lacking somewhere, or this route is just plain FUBAR.

 

[JCitron]

Then my system must be lacking somewhere, or this route is just plain FUBAR.

I think it may be the route. Try merging two simple routes of your own. If that works, then you know where the problem is.

 

[pware]

Here's the kuid of the offending module if you want to give a once-over: <kuid:137353:100466> Perhaps you'll notice something I haven't.

I just checked that out. There is nothing in the route that stands out as being too large or troublesome. But I did have a missing asset <kuid2:30671:27226:1>which, I suspect, is a texture.

I successfully merged it with my 65MB route and while it did take a little longer (about 1 minute) the merge was successful. The culprit may be with the 6.1 MB merged routes. Try merging that with other small routes to see what happens.

 

[JCitron]

Try merging the other way around. Smaller into bigger works better, sometimes.

 

[STLSF4003]

I just checked that out. There is nothing in the route that stands out as being too large or troublesome. But I did have a missing asset <kuid2:30671:27226:1>which, I suspect, is a texture.

I successfully merged it with my 65MB route and while it did take a little longer (about 1 minute) the merge was successful. The culprit may be with the 6.1 MB merged routes. Try merging that with other small routes to see what happens.

Try merging the other way around. Smaller into bigger works better, sometimes.

Okay, first, I tried merging the module with the smallest project I had ( 301.2 KB ), and it still refused to merge! But, as you've pointed out, joining a smaller route onto it and working from that point works like a charm.

So, while y'all have helped me find a workaround to the issue, of which I am truly thankful for your help, I'm still convinced that it's a hardware issue on my end that's causing this route and a few others not to play nice when trying to merge them. I might need to see about a CPU or RAM upgrade down the line.

All the same, So long as I don't go overboard on ground textures and overly high ploy buildings, I should be able to get my project ready by Christmas.


So once again, Thank you, JC and Pware, for all your help!

 

[pware]

But, as you've pointed out, joining a smaller route onto it and working from that point works like a charm.

Further testing in the latest beta build of Trainz Plus (123315) reveals that when merging two routes the merge process never terminates (i.e. has to be terminated by Task Manager) if the first route has fewer baseboards than the second route. This applies even when the first route is 1 bare baseboard and the second is 4 bare baseboards. Merging by starting with loading the larger route first then selecting the smaller route to merge into it works without this problem.

 

[pware]

I made a bug report on the merging issue, with sample routes and copies of assertion messages. N3V QA got back to me with the news that the problem has been fixed and the solution will appear in the next update.

 

[pware]

My question is if this has happened to anyone else before, or is this somehow due to having built the Rig I'm running Trainz on myself as a first-time project?

Merging takes a bit more disk space than the two routes do individually due to the temp files.

Just tested the merge process (larger route merged into smaller route) in the new SP3 update and the problem seems to have been fixed.

 

[STLSF4003]

Right, the issue is back; this time, it's one of my own modules giving me trouble.

But I might have made a breakthrough at the root of the issue.

Note the red x'es below where it says: WorldCel::Merge

This same thing has shown up in the logs every time I've had a merge hang up on me.

So, dose anyone know what the mess of code means?

 

[JCitron]

I have no idea what that means. The error messages are written in "programmer".

With the detaching message, I would try a delete missing assets first on the route you want to bring in. Trainz may be searching in memory where those things might be but can't find them. They're not missing in the sense of a missing object but something may be missing internally in one of the tables. I have found this fixes a lot of weird things by resetting tables and getting things back in order.

A backup is always a good thing prior to trying something like this, so do one of those first just in case things don't go as planned.

 

[STLSF4003]

I have no idea what that means. The error messages are written in "programmer".

With the detaching message, I would try a delete missing assets first on the route you want to bring in. Trainz may be searching in memory where those things might be but can't find them. They're not missing in the sense of a missing object but something may be missing internally in one of the tables. I have found this fixes a lot of weird things by resetting tables and getting things back in order.

A backup is always a good thing prior to trying something like this, so do one of those first just in case things don't go as planned.

And we're back to square one. Turns out that the errors in the log had nothing to do with the issue.

Here's the kicker, though: the route I'm trying to merge is only 184.7KB in size, and the route I'm adding it onto is 2.0MB. So, on the principle of adding a smaller section to a larger route, it should merge quickly, BUT it still hangs!

And to throw an even bigger WTF into the mix, I've got another merge testing project that now stands at 9.9MB...And I can still add another module that's 2.4 MB without issue.....

 

[JCitron]

This could be something wrong with the route you are trying to merge, or the one you are trying to merge into.

 

[STLSF4003]

This could be something wrong with the route you are trying to merge, or the one you are trying to merge into.

And that right there is one thing I really hate about this game. It will flat-out not tell you if there's a merging issue with your route.

Oh sure, it'll throw up a ton of warnings and errors if an asset so much as looks at the game sideways, but not a solitary peep from it when there's a merge error or if you lack the needed Vram/RAM to perform such a task.

 

[JCitron]

And that right there is one thing I really hate about this game. It will flat-out not tell you if there's a merging issue with your route.

Oh sure, it'll throw up a ton of warnings and errors if an asset so much as looks at the game sideways, but not a solitary peep from it when there's a merge error or if you lack the needed Vram/RAM to perform such a task.

This has always been an issue with this program. No errors or warnings when something times out. We can sit there for hours waiting while a progress bar looks like it's doing something and the log looks like it's processing, but in the end the only way to find out is to check Task Manager. When there's zero disk activity and the memory and CPU are sitting with minimal activity, the only recourse is to end it all and run a DBR. I've noticed in my experience with this that the network activity continues but everything else is dead. I mentioned this to the QA Team and that was ignored as usual.

Maybe it's me but after 20 years of watching a program slowly evolve into crap with the same errors over and over, it might be my time to hang things up before I start saying things that I shouldn't. I no longer trust the operations, and after my data fiasco, I've been left with trepidation every time I start the program up as I wait for another thing to not work as intended. Each start up on a different day brings different results, making me leery of running the program.

 

[STLSF4003]

This has always been an issue with this program. No errors or warnings when something times out. We can sit there for hours waiting while a progress bar looks like it's doing something and the log looks like it's processing, but in the end the only way to find out is to check Task Manager. When there's zero disk activity and the memory and CPU are sitting with minimal activity, the only recourse is to end it all and run a DBR. I've noticed in my experience with this that the network activity continues but everything else is dead. I mentioned this to the QA Team and that was ignored as usual.

Maybe it's me but after 20 years of watching a program slowly evolve into crap with the same errors over and over, it might be my time to hang things up before I start saying things that I shouldn't. I no longer trust the operations, and after my data fiasco, I've been left with trepidation every time I start the program up as I wait for another thing to not work as intended. Each start up on a different day brings different results, making me leery of running the program.

Hit the nail right on the head! Though careful with those disparaging remarks towards the game/devs, for the bootlick brigade lurks around every corner...

In all seriousness, though, crap like this makes me wonder why I even bother with this module route project. If I'm having this issue, who's to say how many other users will download the modules, have the same problems, and then be ignored by our overlords?!

 

[STLSF4003]

OK, I still can't get the rat to merge into any route whatsoever. However, I can merge modules onto it, so I'll just have to roll with that because I'm running out of time here.

Though I think, as odd as it may sound, that the track work is what is causing the game to hang itself on the merge. After I removed the yard lead and deleted one of the yard tracks, I could finally add modules (within reason) to it.

This is how the module looks as it is now. Still very much a work in progress

I'll finish this up, post the showcase route on the 24th/25th, and then run some experiments on this module at a later date to try to crack the code.