@JCitron - Massive Route Building!

[ish6]

@JCitron

Hello John,

Sir, I have a curious question -

Let's say you have a route with 500x500 boards, basically one huge square! So now you lay tracks, and you drive the train, focusing only on the area that your are driving the train, so, my question is: Will the surrounding unseen areas tax the PC's ram? (Does that make sense?)

(Of course, anyone is welcome to post their comments, also! :wave

Thanks, John, in advance! :wave:

Ishie

 

[pcas1986]

For some reason I always thought that Trainz loaded the entire route first and then the assets in view. But, assuming that it takes you ten seconds to create each baseboard, then it would take 28.94 days to create just the baseboards. But by then you will have been divorced, probably died of hunger and thirst, and Windows would have crashed anyway so you would have lost all your work. :hehe:

 

[mjolnir]

Ish, as I understand it, the entire route gets (got?; I'm not sure how T:ANE handles the graphics of this) loaded into RAM, but only the bits of the route that are in the immediate view of the active camera get processed. What is in the view of the active camera will depend, for example, the mode (cab view will show bits of the interior of the asset which you'll not be likely to see much of in aerial view, for example, and anything that is behind the camera will not show at all. So I would expect after the initial load, there would not be any delay on account of disk access.

It would also take into account, besides RAM, and CPU speed, the specifications of your graphics card: how much G-RAM and processor power you have there. One of my old favorite routes was the Tidewater route from TPR, built in in 2004 and 2006, as I recall. It seems to me that that was a fairly large route, perhaps 50 x 500 boards (though I never counted, and don't have time to look it up at the moment). At 50 x 600, it would be 1/10th the size you propose, and it did not take (even under the slower speed hardware I had in the 2006 days) 2 days to load.

ns

 

[JCitron]

I agree with the above as well.

What we see is what gets loaded as this would make more sense. I believe the program uses what is called back face culling so that if there are hills and mountains, for example, only the fronts which are facing the camera or our eyes are what gets rendered in the scene. Many, many graphics rendering programs work this way due to the reasons that Paul has stated... Maybe when personal computers operate at 100GHz with 40TB of RAM and video cards with 100GB of RAM on them, might we have a fully rendered scene at all times. Then again we'd find a way to bring that system to its knees.


John

 

[ish6]

Hi John, guys....

Thanks for your answers!

My Marsz layout is just one huge square, with corridors sprinting outwards here and there, and these corridors are very huge and long! -- Each auto-save takes 2 minutes! Most of it's just plain boring vas lands! --- hills, valleys, mountains, etc etc ... Tracks run long distances between bases, however, the layout is squarely huge simply because of the aerial units! These have been set to not run the length of the track, so to cut time drastically between bases and outposts!

So, back to the original question: my understanding from reading the above posts is that while focusing on a specific train or aerial unit or activity within a specific bases, the surround areas will NOT affect game play, not matter the size, because they are not present in our monitor! ---- hope that makes sense! (asking this face to face sounds better than typing, LOL)

In case you are curious, guys, have gotten a chance to test these theories out! Route is being constructed, as well as contents!

Anyhow, thank you guys --- anything at all, posts way... anything related to huge layout please use thread, just to be informed in case I miss juicy info!:wave:

Take care
Ish

 

[JCitron]

Hi Ish,

I sure have tested this out on many occasions. I am currently working on a whopper of a combination of DEM and hand-carved route which actually started back in late 2003. The route is about 250 miles in size, not counting the new un-textured sections. The baseboard areas range from 6 baseboards wide to more than that in other areas. In the newest area, the imported DEM of Down East Maine, this is 65 miles long and many, many to be trimmed off baseboards wide. Though this section hasn't been completely landscaped yet, it still loads up.

If it weren't for the back face culling and in-view rendering, I may as well as wait a month before the route loaded. Driver might even be a bit faster when in a driver session because the inside cabview actually cuts back on what outside information needs to be drawn on the screen.

A good example to try is Deremmy's East Kentucky 2.0 route. This is a substantial route with lots of hills and ins and outs. Without the hidden geometry, it would take forever to load. There are many other examples like this.

John

 

[ish6]

Good Saturday Afternoon John,

Have you test it the above in TANE, sir, or only in T12?

For all the features, etc that TANE has, the one only that I am curious is how will large routes operate in TANE, since it's 64 bit only, and we all know what that means for ram! :wave:

Here's the picture of the Marsz route, sir, which is still yet in development! It's too large to be displayed in Map view in trainz, because there are more regions outwards that stretch to very wide areas; So far it operates smoothly -- It's the saving issues that got me nervous, in that, it takes up to 2 mins+, and I get scare it might freeze! :'(

--so, the light areas are were the tracks are set -- the dark brown sections are rough landscapes -- Only Aerial units can fly over them!

The picture below is just to give you an idea, sir, of what I am working with, etc ... just seeking my limitations with the route size!
Happy and thankful for your input in all, it's all very valuable, indeed


*** I hope that in future trainz products N3V implements a series of features that will a give us every single piece of info from our routes, especially baseboard counts!

Thanks for the info, etc sir
Ishie

 

[Pladroid]

Hi Ish don't worry about the saving time as you can see from my sig I have worked with a huge route, which I had to clone sometimes and save it. This used to take from 15 to 30 minutes depending on whether I made changes or not, I just left it to save while I went and made a cuppa it always worked you just have to give it time and don't worry about it.

 

[JCitron]

Ish,

Right now saving stuff in TANE is a bit of a hit or miss, meaning stuff will quit unexpectedly or sometimes take extra long and is very unpredictable even then. Currently my route only takes perhaps 30 seconds to a minute to save in TS12 so it's probably not as big file wise as we think. (I'll have to look one of these days in CM and find out). So we should wait to find out about this in TANE. As Paul mentioned he goes off and makes a cup of tea as it saves. I've done this myself with testing in TANE, but I have never had his issue and had to clone anything.

Going forward TANE should handle the bigger file sizes, as I mentioned in your blog, much better than TS12 due to the inherent nature of a 64-bit file system and operating system. Once the save bugs have been worked out, this will definitely be put through the hoops.

John

 

[ish6]

Hi Ish don't worry about the saving time as you can see from my sig I have worked with a huge route, which I had to clone sometimes and save it. This used to take from 15 to 30 minutes depending on whether I made changes or not, I just left it to save while I went and made a cuppa it always worked you just have to give it time and don't worry about it.

Good Morning Paul --

Wow, that's a long time, sir! Thank you for your comments!

You saved me from having a few heart attacks! :hehe: --- My PC is in the living room, so, when I do save I turn my chair around and just watch whatever's on TV!

Take Care
Ishie

 

[ish6]

Ish,

Right now saving stuff in TANE is a bit of a hit or miss, meaning stuff will quit unexpectedly or sometimes take extra long and is very unpredictable even then. Currently my route only takes perhaps 30 seconds to a minute to save in TS12 so it's probably not as big file wise as we think. (I'll have to look one of these days in CM and find out). So we should wait to find out about this in TANE. As Paul mentioned he goes off and makes a cup of tea as it saves. I've done this myself with testing in TANE, but I have never had his issue and had to clone anything.

Going forward TANE should handle the bigger file sizes, as I mentioned in your blog, much better than TS12 due to the inherent nature of a 64-bit file system and operating system. Once the save bugs have been worked out, this will definitely be put through the hoops.

John

Good Morning John --

Since Marsz will most likely keep growing simply because it's hard sometimes to stop when it's too much, I am looking forward how TANE will handle all of this! I think massive route building should be the norm, for example, look at this Amtrak map / links below: --

I have not installed marsz into TANE because there's way to much contents that goes along with Marsz route!
Hopefully, with TANE'S help, I can really expand even further away to regions.... one would think what's there, but imagine having 20 different ore mining companies operating at the same time! Would a 64 bit system handles that much processing? We will see, sir! :wave:

Thanks, as always, John for your input, sir
Ishie

or this:

http://www.trainwacko.com/images/LargeAmtrakSystemMap.gif

 

[dougtrain]

Just a point John, to back what you are saying. If I have it right then I think the new engine in Tane will work with more RAM and more CPU cores. The way we do things now in 2012 is very limited by today's standards. As you say, it will be 64 bit and it all helps.

Doug

 

[JCitron]

Just a point John, to back what you are saying. If I have it right then I think the new engine in Tane will work with more RAM and more CPU cores. The way we do things now in 2012 is very limited by today's standards. As you say, it will be 64 bit and it all helps.

Doug

Spot on, Doug. Exactly!

Using the new 64-bit version of Content Manager in TANE, I was able to import a 1GB CDP (my fault when I created it...) in without a problem. I tried the same with TS12 version of Content Manager and received an immediate error about being unable to read the .cdp file.

There will of course still be limits to the size of a route; it's just they are much larger than we are used to now and this opens up a whole new world for us.

Keep in mind that our hardware resources are still limited, and that the theoretical numbers are a lot, and I mean a lot larger than we can physically fit or afford in our computers. With Windows 7 Ultimate 64-bit and up, the maximum memory tops out at 192 GB! which of course is more than we could fit on or motherboards, or afford!

Copied directly from Microsoft's TechNet article on hard drive partitions while using GPT-formatted partitions which allow large disk sizes, here's an article that explains about the theoretical disk sizes...

How big can a GPT disk be?

In theory, a GPT disk can be up to 2^64 logical blocks in length. Logical blocks are commonly 512 bytes in size. The maximum partition (and disk) size is a function of the operating system version. Windows XP and the original release of Windows Server 2003 have a limit of 2TB per physical disk, including all partitions. {Note: This is a 32-bit OS like TS12 is a 32-bit program}

For Windows Server 2003 SP1, Windows XP x64 edition, and later versions, the maximum raw partition of 18 exabytes can be supported. (Windows file systems currently are limited to 256 terabytes each.)

18XB (or is it EB for exabytes).... That's 18,000,000 GB, Windows is currently limited to only 256 TB (terabytes) or 256,000 GB of space. Awe, I feel cheated!

Keeping this in mind, there are probably program limits though which would prevent something this huge, besides who could afford all those 4 or 8 TB drives to make an array that large, and route building, both human-wise and program code wise will probably top out somewhere.

The thing is we need to test and push if we can to see exactly what the limit will be, which I think will be quite high.

John

 

[dougtrain]

John- Something else came to mind when I was reading your last post. Will we be able to use raid and crossfire to any useful purpose do you think? It was a no-no so far but I wonder if it will be a goer in Tane...

Doug

 

[pdkoester]

John- Something else came to mind when I was reading your last post. Will we be able to use raid and crossfire to any useful purpose do you think? It was a no-no so far but I wonder if it will be a goer in Tane...

Doug

RAID is just a configuration of the hard drives. You should be able to utilize that without any problems.

 

[Pladroid]

John- Something else came to mind when I was reading your last post. Will we be able to use raid and crossfire to any useful purpose do you think? It was a no-no so far but I wonder if it will be a goer in Tane...

Doug

Hi Doug I am using a crossfire configuration at the moment and TANE seems to be using it but I am sure it could be optimised better as I can't see the crossfire logo (which usually shows that it's working in crossfire mode) yet.

 

[dougtrain]

Hi Paul.
I will remain neutral on this as I just about recall snippets on this subject. Looks like crossfire works then but I think there were issues or problems with Raid (striping). This was all based on the old engine with serious limitations on RAM and CPU. With better options in TANE and computers with plenty of grunt then this would be an asset.

 

[JCitron]

Hi Paul.
I will remain neutral on this as I just about recall snippets on this subject. Looks like crossfire works then but I think there were issues or problems with Raid (striping). This was all based on the old engine with serious limitations on RAM and CPU. With better options in TANE and computers with plenty of grunt then this would be an asset.

The new process is totally different in Win 8 and up. Instead of using the Disk Management console to extend volumes, and face the performance issues found with Windows 7 and Windows 8, the drives are now handled by the Storage Manager which appends volumes easily. The underlying engine too is based on the new server code and is totally different than the older code.

TANE is supposed to support Crossfire and SLI. There was quite a push for this in the beginning. TANE CE may not perform well at the moment, but it doesn't overall no matter what system you're running it on.

John

 

[ish6]

Hello John, and fellow trainzers!

A question -- when you are working on large layouts do Trainz suddenly freezes on you for about 2-3 secs, etc?

Of does this happens, regardless of map size?

Thanks in advance!

Ish

 

[JCitron]

Hello John, and fellow trainzers!

A question -- when you are working on large layouts do Trainz suddenly freezes on you for about 2-3 secs, etc?

Of does this happens, regardless of map size?

Thanks in advance!

Ish

Yes, it happens all the time regardless of map size. It's due to TADDaemon and Trainzutil doing stuff in the background. I noticed this happens when I've moved locations briefly, or swapped categories as TADDaemon will post a message about "writing database" whatever that means if you have the console open and you watch the messages. It might be a typical N3V generic message that has many more things going on underneath than what we see.

I found that if I defrag my hard disk before using Trainz, (don't do this with an SSD!), I get better performance, but the process still happens.

John