TS12 SP1 HF4.....validation, validation, validation..so when can I get to use the sim

[NICKEL]

I agree the update DLC is causing me 10000 validation of assets as it cannot download one of the files properly. I get 2 errors on it each time I have tried to download it. Meanwhile it is validating the two other files. After a few tries to get the number two file of Murchison it has started downloading the 1 and 3 files again and again validating all of the assets.

I have in last couple months also had the spontaneous validation of large numbers of assets for no apparent reason. This causes all sorts of delays and non-responding windows. Without the process window to tell me what is going on I would have figured out I have a broken installation.

Sure would be nice to get this fragile DB to work better. Between asset validation and the dreaded system EDR on start up I sometimes spend more time waiting than actually doing anything. Hopefully the next version will be better.

I can see where somebody new using TS12 would give up and say the software doesn't work reliably and is always freezing on me. Need more visual indication of what is going on underneath to the new or casual user. Keep them informed.

Trying to download the DLC files again.

 

[Kennilworth]

Hi

I keep getting error messages popping up on screen when I start Trainz (TS12 SP1 HF4) from the startup screen telling me that a built in asset is faulty and that I need to do an EDR. This has been going on for at least a couple of months and despite numerous EDRs the problem persists. About three weeks ago I'd had enough of it and after backing up my content I reinstalled TS12 and installed all the patches. I imported all my content again and ran an EDR.

For the first two or three days it all ran fine and then this issue cropped up again. Last week I needed a second clean install to test a session out so I installed TS12 and started to patch up. After one of the patches completed (sorry I can't remember which one) one of the messages was that kuid -3:11060 had an error, however checking CM doesn't show this kuid as being installed. Checking in TS2010 shows this kuid as being Passengers which appears to have been obsoleted.

This has left me wondering if it is this rogue kuid that is causing these validation issues that a lot of people, including myself, have been having?

The new installation has run faultlessly for almost a week so far but there is very little additional content installed yet, apart from DLC, as I am just using it for testing the session for someone else.

Alf - I downloaded Murchison 2 into this new install without any problems and it appears to be working correctly on the couple of sessions that I have tried so far.

Regards

Brian

 

[NICKEL]

Well finally got all of the Murchison files to install properly. I would say about 6 to 8 tries before they installed without errors.

 

[johnwhelan]

Sorry to hear that you're having a bad day, but here are a few tips that might help:

* DLC is unrelated to content validation.
* If you're talking about "validation" messages in the TADDaemon console window, I'd suggest simply hiding the console window and continuing about your work. The console window is for development purposes only and does not have any meaning to the average user.
* If you're talking about a "database repair" prompt when attempting to start Trainz, that typically means that you did not exit Trainz cleanly on the last run. Unfortunately, all you can do here is wait for the repair process to finish up.

hth,

chris

Just a comment from a mailing group Trainz does seem to have bad hair days. Typically out of thirty or so active users I see someone complaining their Trainz is acting up in such a way about twice a week. Ie it can’t be used when they would like to use it.

My own system is fairly stable but that has ECC memory, and a UPS to smooth out any power glitches first. I’m probably tempting the gods by saying so.

Years ago I was taught that programmers think speed is the most important thing, and end users always put reliability first. I think we have old fashioned end users who would simply like to use the product.

I know Oracle software had the reputation for dying when upgraded, I think it was version 8.03, around 30% of the upgraded databases crashed. It took years for their reputation to recover.

If you feel that Trainz needs to be run on a system with ECC memory and with a UPS that protects against brownouts you should say so in the requirements. If not then it should be “suitable for the purpose” and we shouldn’t see this happening so often.

Cheerio John

 

[WindWalkr]

Years ago I was taught that programmers think speed is the most important thing, and end users always put reliability first.

I'm not sure which programmers this might be, but anyone worth their pay will understand that there is a balance between various demands, and will act accordingly. Anything which destroys performance (or destroys reliability) can be considered a bug. Anything which is a small impact on performance (or a small impact on reliability) is an avenue for improvement. And of course there is more to life than 'speed' and 'reliability'- graphics, features and usability are great too.

If you feel that Trainz needs to be run on a system with ECC memory and with a UPS that protects against brownouts you should say so in the requirements.

I certainly don't. But I do accept that if my machine goes down unexpectedly while files are being manipulated, there there will be some amount of repair necessary.

chris

 

[Fabartus]

When you think about it, it probably doesn't come as a surprise to hear that we do this a lot. The pre-alpha environment tends to have a lot of crashes, and we tend to run with a lot more content than the average user.

For example, my regular TS12 SP1 install at work has the entire DLS installed, in addition to all of the DLC. I completely agree that rebuilding the database is painful (with that much content, very definitely a leave-it-go-overnight operation.)

chris

Do you even have a clue that the machine just goes off and does this JUST WHEN we FINALLY HAVE A COUPLE OF PRECIOUS FREE HOURS WE'VE LOOKED FORWARD TO SPENDING WITH TRAINZ?

It sure sounds like you don't care a hoot about the way your code impacts the timing of THAT! It may happen after importing assets across Trainz installs, or loading a big batch of CDPs, particularly ones with a route or two, then relaunching CM. FYI, this in my admittedly limited TS12 experience, seems to happen most often the next CM boot--possibly only after running a QDR. Or so it has in my limited experiences, with TS12. I know I was shocked and P.O'd when it ran for no good reason--nearly three hours just gone from my schedule. SO the more urgent question is WHAT TRIGGERS THIS behavior?... IIRC, the only two times I definitely recall seeing it has been after running a QDR (after such data importing activity) then quitting out normally, to get bitten by it the next time I boot CM. (I don't turn my machine off, ever. Easier on the chips thermally, and I only reboot only after TADdaemon has been off the Task Manager processes list for at least a minute. I give TADdaemon absolutely zero room to mess me up!)

Acknowledging this as a problem would indicate you actually respect the communities time, unlike your answers above, nor your normal answers. // Frank

 

[gremlin1812]

I've no idea what might be causing this to repeat on your install. If you're interested in getting it resolved and are willing to help with their investigation, I can pass your contact details on to our QA team. For that to work out, you'll probably need to be ready to download up to ~20 GB of data (ie. several separate test installations.) PM me if you're interested.

chris

G'day, this should be interesting and I will do what I can to help sort this out even though I am sceptical I will do what I can. PM sent.
Regards
Barrie

 

[WindWalkr]

It may happen after importing assets across Trainz installs, or loading a big batch of CDPs, particularly ones with a route or two, then relaunching CM.

This is expected behaviour. The game will begin background validation of all newly installed content. You can continue to use your computer while this is happening (admittedly with a performance penalty due to the background activity, which will vary depending on the performance of your computer, especially drive speed.) If you want to force the issue, you can also view the 'all' filter in CM, as this will require the content to be fully validated before it can continue.

I know I was shocked and P.O'd when it ran for no good reason--nearly three hours just gone from my schedule. SO the more urgent question is WHAT TRIGGERS THIS behavior?

Indeed, that's a very good question.

chris

 

[Kessica]

SO the more urgent question is WHAT TRIGGERS THIS behaviour?

Indeed, that's a very good question.

chris

This is the real question that needs to be answered. I trust we will get a good in-depth answer.

 

[johnwhelan]

This is the real question that needs to be answered. I trust we will get a good in-depth answer.

Wouldn't a good in depth solution be so much better?

Cheerio John

 

[dnevans8]

This is expected behaviour. The game will begin background validation of all newly installed content. You can continue to use your computer while this is happening (admittedly with a performance penalty due to the background activity, which will vary depending on the performance of your computer, especially drive speed.) If you want to force the issue, you can also view the 'all' filter in CM, as this will require the content to be fully validated before it can continue.

chris

Chris - The performance hit unless you are using some sort of super computer results in the computer being unusable for anything even browsing the internet which becomes very slow and appears to lock up at times so you cannot continue to use your computer as you suggest! My computer is a quad core I7 3.4 GHz machine with 12 GB of memory so it is no slouch except when it comes to Trainz operations.

Like everybody else my machine launches into a validation every so often, fortunately not as often as others are suffering. I have found that if it gets past the initial launch then clicking on the All tab so far has never launched a validation. The only time a validation occurs other than at the start is if I have had to do a QDR or EDR after adding updates to DLC.

 

[nicky9499]

My main install is 49922, so I don't have #SP1problems, but the validation issue has been around long before SP1. Whenever I download some new stuff from the DLS or import a bunch of CDPs, upon the next startup Trainz will (appear to) hang for a lengthy period of time anywhere from a half hour to several hours. The time spent locked up is proportional to the content just installed. Which brings me to my question - and this also applies to the issues people above have with SP1.

Whatever is being done during these lockups - validation and whatnot - why is it not being done at the same stage as when it's installed?

Surely CMP does a fair bit of processing when assets are installed; it has an option for "hardware accelerated texture compression during install" and the process windows are plenty busy when installing stuff. Instead this process is broken up into 3 stages: install, commit and locking up (aka validation). No, validation will only occur when you want to launch Trainz. You can install content then leave Trainz alone for 3 months and it will only start validating right when you fire it up.

In the real world that like sending your car to the workshop for couple days to install a new motor, get it back and only when you want to drive it, get a call from the mechanic saying sorry we'll need 3 more hours to tighten all the screws.

 

[WindWalkr]

My main install is 49922, so I don't have #SP1problems, but the validation issue has been around long before SP1.

Correct, it's been around since at least 2006.

Whenever I download some new stuff from the DLS or import a bunch of CDPs, upon the next startup Trainz will (appear to) hang for a lengthy period of time anywhere from a half hour to several hours. The time spent locked up is proportional to the content just installed.

The difference with SP1 is that the game will (in many more cases) remain interactive, rather than stalling completely during these operations.

Whatever is being done during these lockups - validation and whatnot - why is it not being done at the same stage as when it's installed?

It's a "lazy" process- it's done when the information is first required, not when first installed. The main reason that it's done this way is you will typically invalidate the information multiple times over during a regular installation, so doing it immediately would make the whole process several times slower.

SP1 adds a little bit of smarts to the process which allows the game to proactively begin cacheing this information immediately, as a background task. This task is limited to a single thread of execution, whereas the foreground tasks may occupy 4-8 threads of execution, however if you're limited by disk access then this can still make the computer seem a lot less responsive.

No, validation will only occur when you want to launch Trainz.

Prior to TS12 SP1, validation occurred on demand, which was typically:

1. When you changed filters in CM, especially if you selected the wider-scope "all" or "installed" filters.
2. When you opened the routes menu in Trainz, or one of the Surveyor tabs.

The first one is a big pain, but the second makes it look like the game has hung, which is why SP1 is a bit more proactive about this.

You can install content then leave Trainz alone for 3 months and it will only start validating right when you fire it up.

Well, obviously it can't finish the validation while it's closed. Leaving it finish its thing post-installation and it shouldn't* need to validate again later.

chris


*: Some of the people in this thread are hitting a case where it apparently does. This obviously shouldn't be necessary, and our QA team are investigating.

 

[nicky9499]

Hi Chris,

This issue has been around for so long that it doesn't really affect me anymore, and quite a few others too I suspect, because we already expect it. After installing assets, I will start up Trainz when I don't expect to use the computer just for it to hang there, oo the next time I want to use Trainz, everything's good to go. I understand what you mean by having subsequent assets in an install queue invalidate previously validated assets. Why not have a temporary list of assets just installed, and validate these at one go once commit is complete? Basically what we have right now, but the mechanic tightens his screws once he's done with my new engine.

T2 would do well to not incorporate this "feature".

Nicholas

 

[WindWalkr]

Why not have a temporary list of assets just installed, and validate these at one go once commit is complete?

That's what you have right now.

chris

 

[PWeiser]

My attention was directed to this thread by an external user group, and I wish to declare bystander status.

This is because I have not "progressed" beyond TS2009 SP2, that is, TADDAEMON is not spoken in my home. About the most serious delay this setup produces is a long multilingual "wait" page when I shift between compatibility and incompatibility mode while the game marks or unmarks incompatible assets, never more than a few minutes.

This is not to say TADDAEMON is a solution to a nonexistent problem, only that the problem it tries to solve is one perceived by programmers (restricting the universe of data with which they have to deal) rather than customers (who want timely, reliable and predictable operation with the content they know and like).

(Turns over the soapbox... don't slip on the soap!)

 

[Fabartus]

Surely CMP does a fair bit of processing when assets are installed; it has an option for "hardware accelerated texture compression during install" and the process windows are plenty busy when installing stuff. Instead this process is broken up into 3 stages: install, commit and locking up (aka validation). No, validation will only occur when you want to launch Trainz. You can install content then leave Trainz alone for 3 months and it will only start validating right when you fire it up.

In the real world that like sending your car to the workshop for couple days to install a new motor, get it back and only when you want to drive it, get a call from the mechanic saying sorry we'll need 3 more hours to tighten all the screws.

This was very pithy! Kudos on a neat analogy.

But!!! I seem to recall looking for errors and warnings then doing a commit in groups, perhaps 6-10 at a time, sometimes a screenful. (I don't auto-commit generally.) Surely processing the asset for errors and warnings is some kind of validation? Now for me that's the long road for shuffling stuff promoted from TS06, or the TCs, into SP3+SP4 versions of TS09, into two stages of TS10, then into TS12, as I study this software's progression because I need to write accurately about it. Further, unlike most of you, I avoided the upgrade pains and errors of downloads and the DLS happy to spend what little Trainz time I had learning to get the most out of TR06. This was mostly because my oldest boy had cursed up a storm when downloading and error fixing back when I was really just beginning to get into Trainz myself. Because of that weird path, I'm just now going through the perils and pitfalls of non-DLS 3rd party content as well.

The point on that is I'm wondering if importing CDPs is CAUSAL, and particularly when they references assets NOT on the DLS... such as Jointed Rail Trucks or other missing assets. CDPs, unlike cross-loaded repaired local assets do get auto-committed, so such third party content might be in an timing/causal association (Suspecting it might have been the day after installing TPR's Clinchfield route, the one time I suffered 3 hours waiting.), and causal relationships are rather important to engineers like myself or Windwalkr.

Speaking as a software engineer myself, the community can likely best be of help on this needed fix by carefully documenting when it happens, how long it delayed you, and what you had done in the previous 2-5 CM batch operations. If a few want to experiment with imports across Trainz versions, and via 3rd party content, I suspect any triggering circumstance will help N3Vs staff hone their aim soonest. Otherwise they're surfing a sea of digital soup wondering where that airliner went down. Like that sad case, actual cause and effect breadcrumbs... (debris) would help clear the confusion of where to look in detail. // Frank

 

[johnwhelan]

So the idle thought would be when Trainz installs if there are more than 2 cores present also installs a program that starts when windows starts, runs a single thread at low priority that checks the database in the background whilst the machine is used for other tasks.

Cheerio John

 

[Fabartus]

So the idle thought would be when Trainz installs if there are more than 2 cores present also installs a program that starts when windows starts, runs a single thread at low priority that checks the database in the background whilst the machine is used for other tasks.

Cheerio John

Well and good, but it'd be far better to validate and commit just the one time to a fixed JA analog, and skip all this re-checking rechecked checked rechecked committed imported assets. In other words, I find it inexcusable that they aren't tracking that a data set is validated so they can skip re-validating it again.

They need a 'dirty set' of files recording adds and deletions, a backup and backup to that, to branch to if integrity seems busted, a transaction journal linked to basis files (\local) changes and it's backup, and a set of CRC lookup table indexes to record base CRC signatures in \local and in the assets.tdx and of the assets.tdx backups which should be a copy generated when booting the lot, leaving the source as validated and undisturbed. Looking up the CRCs and/or taking entirely new signatures on everything in \local takes only a few minutes and in general case, if used properly, a CRC signature comparison of what we started the day with from the last time the TADdaemon wrote the database to what you are booting currently would and should take place faster than they can draw the CM page.

The other question is what do they mean by validating at this boot time... is it waiting to check dependencies down 2-3-or-four levels as it apparently doesn't (and perhaps should) when making initial checks for errors and warnings. Overall, that that Manual check by me is not validating stuff 100% and at least issuing a warning that this or that missing dependency is not to be found I find to be both bothersome, and the most likely task this off-line processing is conducting. Perhaps Chris can illuminate us as to exactly what validation of already validated once local content this thing is about.

 

[CaptEngland]

Is it just me, or do most of us have this same problem hit at roughly the same day/time frame?

My version has just died on me in the past 15 minutes.

Regards.
CaptEngland.