Need Strategies for Downsizing

Hi! Like a lot of folks, I am sure, I have spent the last few years playing "pack-rat" with my two versions of Trainz. To the point where the two together now consume about 2TB of disk space. For that reason, they are on a 4TB WD external USB drive, where I thought I would have lots of room, but it turns out the external drive is pretty slow. Last night I was trying to create a session in TS2019 and the content picker sat and spun the whole evening, and I was not able to use it until this morning. I could just see "precashing" and "compiling scripts" flashing back and forth at the top of the screen all night.

I am sure it is time for an EDBR, which may take all weekend with this drive. My thought is to move TS2019 back to the system C: drive, where it would be faster, but I would also like to downsize it, to make it lean and mean. But I hate to get rid of anything awesome. Strategies MIGHT be:

- Uninstall and then go with a fresh install, only pull stuff in as needed.
- Delete some content, but I can't decide what. Hand-picking would take forever, but trying to delete by version or type (Like modified) could lead to missing assets in lots of places.
- I could delete installed from DLS, and use N3V for a backup in that regard, but that will limit me trying to pick assets for my routes. Thumbnails in DLS are too often not representative, and I need the 3D asset preview to know if things are what I want to use.
- It would be REALLY nice if I could delete Payware obsolete, Built-in obsolete, and base obsolete. I have a TON of that stuff.

Anyway, looking for suggestions for trying to speed things up a little. After losing a whole night where I had time to create something, now I have to wait for another night, and maybe lose the whole weekend to an EDBR in the bargain. Thanks for ideas in advance!

Use a SSD, 1TB are reasonably priced now. Put the build(s) you use the most on there. Store the other stuff on your large hard drives. Research will show that large HD's have issues with performance.

Put the program its self on your C: Drive and upgrade that to an SSD.

Put in an internal large spinner. I got some enterprise-level NAS drives with huge caches. These 8-TB drives cost about $200 each and I've seen them for less recently. I went with the Seagate rather than Western Digital and I'm glad I did. WD got in trouble for using slow drives when they should have been the fast ones.

The problem with external devices is they are bound by the USB 3 interface which is a bit slower than the internal buss. External drives also tend to be slower drives as well with much smaller caches. It's not unusual to see a 5400 rpm drive in an external enclosure. They're great for storage of pictures, documents, and downloads, but not good for continuous access. I have also found they tend to get really, really warm under lots of extended use and that causes the drives to actually run poorly and slow down, and possibly degrade the drive its self.

At the moment, I am copying 1.5 TB of data from a 2TB older consumer-grade drive to my 8-TB 7200 RPM Enterprise NAS-quality drive. I believe these have 256 MB caches in it versus 16 MB. The process will take about 5-hours to copy this data at about 325 GB per hour.

The reason for putting the data on the spinner is for reliability. I know SSDs are getting better, but if they die, they're dead completely. A spinner has warning signs such as slower read/write speeds and even if the drive starts running slower, we still have a chance to back up the data off the device before it completely snuffs out. Ideally, we want our data on the fastest device possible. This helps with the caching, as well as data access. If 8 or 10 TB SSDs cost $200 with the reliability and lifespan of a hard drive, I would go for those, but at their current price for a 2TB consumer-grade SSD, I'll stick with a large spinner for now.

As always, I recommend disabling real time antivirus scanning of the Trainz data and program folders, and perform frequent defragging. The SSD should be trimmed at least weekly, or according to the manufacturer's recommendations, which turns out to be weekly on the two devices I have. Setting the system up for automatic defragging is great, but that only works if the system is left on so plan on a weekly routine yourself. If you do a lot of data copying, Trainzing, or downloading, I recommend defragging more often. I noticed this really, really, really helps with the caching and compiling script process as well as database repairs.

If a drive has a 256 MB cache, it almost certainly uses shingled storage. Time will tell whether shingled is more reliable than SSD. I would back up your SSD at least once a month, maybe more under heavy development activity.

I suggest that some research on large capacity hard drives be undertaken before using them internally. Certainly not for the system drive.

These do have 256 MB cache, but are actually meant for a NAS. Shingled storage isn't recommended for NAS use due to poor performance. I'm not using these in a RAID, although I could, so maybe this doesn't matter, but if they're NAS-quality drives, I don't think they would use shingled storage methods.

I can't complain about my hard disks so far. They've performed well and consistently across the drive with no apparent slow down in performance.

I agree with you on the SSD.

JCitron said:

These do have 256 MB cache, but are actually meant for a NAS. Shingled storage isn't recommended for NAS use due to poor performance. I'm not using these in a RAID, although I could, so maybe this doesn't matter, but if they're NAS-quality drives, I don't think they would use shingled storage methods.

Click to expand...

Western Digital used SMR on their NAS drives. My point is, the OP should use due diligence in selecting which drive and where and how installed for large capacity drives.

Thanks guys, I appreciate the tips. I have 1TB available on my internal system drive, which is a 2TB spinner. But my TS2019 is almost 1TB now, and I hate to use up my whole system drive. Wishing now I had opted for an SSD internal drive instead of the Optane cache, because the Optane is fast once loaded, but it isn't resident, it has to be loaded every time. I have looked at 2TB and 4TB SSDs, but expensive. I do back everything up to an 8TB external I only plug in for backups, so I am less concerned about failures.

I think if I opt for an internal drive, it will be 4TB, and I have to decide between a NAS-quality drive like you mention John, or shell out big time for SSD. I have had spinners go bad all at once too though. All of a sudden one day they just can't read anything and they are done. I don't know what causes it, but it has happened to me a couple of times. I have an external 300GB Seagate spinner that quit and nothing would read it, but lo and behold, my local TV over-the-air receiver box can use it to record TV! weird. And my old laptop, I came home from vacation and the drive could not boot, could not even read. The local computer shop could not read it, and it would cost hundreds for file recovery. Fortunately everything was backed up except for the pictures and videos I had offloaded from my phone before leaving on vacation. Unfortunately I just considered those lost.

Thanks again guys!

Forester1 said:

Hi! Like a lot of folks, I am sure, I have spent the last few years playing "pack-rat" with my two versions of Trainz.

Click to expand...

Thanks for starting this thread. I recently bought TRS19 and have been transferring GBs of non-railroad stuff to external drives right off the bat. Learning quite a bit from the responses. Need a better laptop or perhaps build a custom portable system.

OddRails said:

Thanks for starting this thread. I recently bought TRS19 and have been transferring GBs of non-railroad stuff to external drives right off the bat. Learning quite a bit from the responses. Need a better laptop or perhaps build a custom portable system.

Click to expand...

I agree research is good.

I just looked up my drives again, and they do not have SMR according to this article here.

https://blocksandfiles.com/2020/04/15/seagate-2-4-and-8tb-barracuda-and-desktop-hdd-smr/

OddRails, I just pulled the plug on my daily DLS downloads and started an EDBR. I am probably out of it for the weekend until that completes, but I am thinking I should have just bit the bullet and done a backup/uninstall/reinstall to my system drive and made every effort to run a lean shop. And give up on the daily downloads of everything that sounds like yummy candy. I may do that yet. Maybe my fairy godmother will drop a 4TB internal drive into my lap somewhere along the line, but not likely. I am going to have to live in the real world for now. Best of luck in your efforts!

John, "At the moment, I am copying 1.5 TB of data from a 2TB older consumer-grade drive to my 8-TB 7200 RPM Enterprise NAS-quality drive. I believe these have 256 MB caches in it versus 16 MB. The process will take about 5-hours to copy this data at about 325 GB per hour."

Can you tell me the model number of this drive? You mentioned Seagate, I believe... I don't have the budget probably, but I would like to know which drive I am lusting after.... :udrool: Thanks!

I have a 1tb WD SSD. I have it set up with 2 logical 500 GB drives. I have TRS19 build on one and TANE build on the other. Programs are on the C: drive. I think that the export to cdp is your friend. Query whatever you don't really need and export to large cdp collections. Then shoot them out to your backup drives then delete. You can search them with cdp explorer if necessary. Or move entire unused builds out there and point trainz settings there and use the CM.
If you happen to follow the thread on multi tracks, I had over a thousand commodities (not built in) in my installation. I have cdp'ed and deleted them. I have had to download 3 so far. They are not very large, but point being that I probably don't need 2/3 of the assets I have now.

Thanks Pitkin, that is just the kind of strategy I am looking for. The CDP exports is a great Idea. I may do them by type, so if I am looking for something I can look by type. Did you use a similar strategy? I do not know that I will ever get either version down to 500GB , but it would certainly aid in the duplication of assets between the two if I only loaded what each one needed at a given time. That is definitely a target I can start aiming for, and I can live with a smaller internal drive. At least for a while.....

Forester1 said:

John, "At the moment, I am copying 1.5 TB of data from a 2TB older consumer-grade drive to my 8-TB 7200 RPM Enterprise NAS-quality drive. I believe these have 256 MB caches in it versus 16 MB. The process will take about 5-hours to copy this data at about 325 GB per hour."

Can you tell me the model number of this drive? You mentioned Seagate, I believe... I don't have the budget probably, but I would like to know which drive I am lusting after.... :udrool: Thanks!

Click to expand...

Yup it's a Seagate.

Here you go: ST8000VN0022-2EL112

You can get them at New Egg for about $200 now. I paid a lot more a year ago.

My data copy just finished about 5 minutes ago. My sustained writes were 88 MB/s. Total time was about 5 hours for the lot. I'm now defragging my drive.

Forester1 said:

I may do them by type, so if I am looking for something I can look by type. Did you use a similar strategy?

Click to expand...

Yes, strategy is to use the CM. Give it a little thought, look at the types, For example, for commodities, on the dls, built-in, base, and so on. You are trying to gain space, the results do not have to be perfect. Another approach is to sort by file size. Some creators are uploading assets that have ridiculous file sizes for a minor structure.

Seagate are using SMR on drives now as well, came up on Ten Forums a few weeks back, they were conveniently not publicising the fact clearly, I got caught with one 2TB with SMR, not the end of the world as it's just being used as storage.

Curiously the SMR drive can't be benchmarked on the Userbenchmark says it's using a cache so values are inaccurate. I use 2 4TB Seagates Externals for Trainz backups not SMR, the backup is backed up. Just took me 7 minutes to do a mirror backup of TRS19 plus at 343GB before applying the new update. Obviously the initial backup takes a while longer, but mirroring is quick. I use Create Synchronicity, free with a simple non brain taxing interface and reliable.
Any problems I can just reverse the mirror to return things to previous and I can also run Trainz from the backup disk in an emergency.

Have to keep my Trainz data down a bit as I'm using 3 500GB SSDs for TANE, TRS19 and TRS19 Betas, 1 x 240GB SSD for the OS and a 2TB Toshiba spinner for content creation stuff. Other PC is just as with drives bad but has smaller SSDs and 3 x 2TB spinners all Seagates.

I've already thinned things out a bit and any third party stuff I have safely backed up on yet another 4TB external drive.

I've been using hitachi enterprise drives that are used for servers, they are some of the most reliable according to those who investigate these things, I have had nothing but trouble with seagates and avoid them at all costs. The hitachis , even tho spinners , never seem to have any issues ( so far ) and are also fast and if you look around are not exorbitantly priced . Beware tho, some get sold on from server work, check the year they were manufactured if the deal seems too good to be true.

Thanks again everyone, This is the kind of stuff I waned to know. I am working from home today, but checking in when I can. I really appreciate it!

Never had any problems with Seagates, I have had with WD though, think it's luck of the draw / where you get them from.

Same here with Seagate drives. That's why I recommended the ones I use. I had an awful mess with WD drives a few years ago and I've shied away from WD since. Many years before, I was tasked with upgrading a bunch of machines for the company I worked for at the time and it was pure hell as it became a repeated replace process after the upgrade. In this case, I had no choice on what drives I wanted because I wasn't ordering components and got stuck with what I was given to use. The company also used McAfee antivirus if that says anything. (Shudders).

Many years before that, at another company, I went through pure hell with IBM and their SCSI Death Star drives. The company used them in a server-product they sold with an imagesetter and RIP. The drives died almost daily and field service would ship out a new imaged drive and the customer would return the croaked one. It got to a point where I was on a first name basis with their customer service center.

Getting back to the problem in hand, we actually face two issues here. First of all, it's not an easy task to clean up old assets. If Content Manager had a better way of displaying dependencies, we could pare down the amount of content we have because some old stuff may no longer be used.

The other issue is the size of the assets. What was once a a few megabytes, is now turning into tens of megabytes and up. This content adds up pretty fast, and this doesn't count the DLC which can really take up a lot of space if those packages are installed.