TS12 SP1 HF4.....validation, validation, validation..so when can I get to use the sim

Is it just me, or do most of us have this same problem hit at roughly the same day/time frame?

My version has just died on me in the past 15 minutes.

Regards.
CaptEngland.

I can't seem to pin it down to any particular time. I was just in TS12, exited to do something else, and have returned back in with no issues today. Right now it's 17:10 local (EST), or 22:10 GMT I think.

We need to document what causes this so we can report it and have N3V fix it if they are so inclined, and I'm sure they will.

Some of the things we should look for, but not just these things...

1) Does this occur after importing CDPs?

2) After reinstalling DLC? --- I've actually seen this already. It may not be the DLC, per se, but more of the adding of content to the data pool that needs validating.

3) Does it happen on a whim due to a program bug, which is also possible?

4) Was there a crash on exit from the program or CM? We need to check Windows Event Viewer for this, if there is no reported errors.

and many other causes not listed here.

John
 
1) Does this occur after importing CDPs?

Yes.

2) After reinstalling DLC?

I'll leave this to the folks who run SP1 regularly.

3) Does it happen on a whim due to a program bug, which is also possible?

Not as far as I can observe. SP0.

4) Was there a crash on exit from the program or CM?

Nope. If there was then there will probably be a QDR on next start.

This "lazy" validation behavior, as Chris has put it, is by design and not really a bug or flaw. A poor design, but at least once can anticipate and work around it. Trainz in it's current state has got far bigger problems.
 
I think I posted this somewhere else.

My install would frequently do a 12000 some asset validation and I could not figure out what it was causing it. The solution came to be that since I have raildriver installed, it will open-for-edit engine specs for every engine I drive when I drive them (no it doesn't even have to be used or connected). Now when I would enter CMP and do any installations (which I do a lot being a content developer) it would commit those items back because of the automatic commit setting in options. This meant that every asset requiring those specs had to be re-validated and would mark as such. Now don't get me wrong here. I am not saying that this can only happen if you have a raildriver. I am saying if you open for edit a file at all and have this setting in CMP that automatically commits the files back then you will get validations that you may not be prepared for. This can lead to the apparent lock up at the routes screen or the surveyor tabs. I would suspect something very similar is what is going on here. Since I disabled that setting, there have been no unexpected validations - only expected ones.
 
I think I posted this somewhere else.

My install would frequently do a 12000 some asset validation and I could not figure out what it was causing it. The solution came to be that since I have raildriver installed, it will open-for-edit engine specs for every engine I drive when I drive them (no it doesn't even have to be used or connected). Now when I would enter CMP and do any installations (which I do a lot being a content developer) it would commit those items back because of the automatic commit setting in options. This meant that every asset requiring those specs had to be re-validated and would mark as such. Now don't get me wrong here. I am not saying that this can only happen if you have a raildriver. I am saying if you open for edit a file at all and have this setting in CMP that automatically commits the files back then you will get validations that you may not be prepared for. This can lead to the apparent lock up at the routes screen or the surveyor tabs. I would suspect something very similar is what is going on here. Since I disabled that setting, there have been no unexpected validations - only expected ones.

I will uncheck the auto-commit and see if that works. Very interesting... Thank you for that possibility to check out.

John
 
I think I posted this somewhere else.

My install ...SNIP... I am saying if you open for edit a file at all and have this setting in CMP that automatically commits the files back then you will get validations that you may not be prepared for. This can lead to the apparent lock up at the routes screen or the surveyor tabs. I would suspect something very similar is what is going on here. Since I disabled that setting, there have been no unexpected validations - only expected ones.
Seems to me being a paranoid programmer type, I'd figured I could spare the energy to manually commit meself... but then to my surprise I was waiting every time I booted either CM or past the launcher into the Main Menu... or stuff wasn't in Surveyor to take a look at...

The one incident I think I remember-- I think I wanted thumbnails for a new asset. After not finding whatever and re-entering CM I found worse, iirc, I'm pretty sure I had a bunch of folders I'd believed I'd closed and committed still in \editing to my surprise. I could be mis-recollecting that, as it was 4-5 months ago and I was bouncing from one version to another a lot back then, but I've had the auto-commit option on ever since and haven't seen the missing assets when expected, nor many big delays. I do tend to keep changes down to 15-30 or so, and tend to reboot CM after such a group, at most a routes extras, which tends to be a bigger chunk. I'm also wondering if I've been immunized against this a bit because I'll take advantage of Last Search and close CM when refreshing my coffee or blowing sanitaries, as we submariners put it. With my coffee intake those trips happen with dreary regularity, perhaps once an hour.
 
If you're interested in getting it resolved and are willing to help with their investigation, I can pass your contact details on to our QA team.
Chris, isn't that Program engineer speak for, "we are going to give you the run-around and blag our way through it like we always do?" No funds no fixes...
 
One thing I've found during these validation processes is that if your in CMP and you click on another filter icon the process in the Taddaemon window speeds up , Instead of validating files one at a time it will do up to 3 at a time until the page has refreshed itself in CMP . I've observed it skip through nearly a thousand items in little under 30 seconds (sadly it doesn't do it all the time though :( ), So my question is if it can run at that speed then why doesn't it run at that speed all the time ? 20,000 items is a pain but if it could clear through those in a few minutes then I'd put up with it !
 
Chris, isn't that Program engineer speak for, "we are going to give you the run-around and blag our way through it like we always do?" No funds no fixes...

No terse riposte yet Chris? That's not like you at all!!!!... I must have caught you on an off day, or is it getting too near the truth for one of your dry witticisms?
 
No terse riposte yet Chris? That's not like you at all!!!!... I must have caught you on an off day, or is it getting too near the truth for one of your dry witticisms?

Yeah, i'm totally give you the runaround. Sorry about that, you caught me. ;-)

(Seriously, what kind of answer did you expect?)

chris
 
Yeah, i'm totally give you the runaround. Sorry about that, you caught me. ;-)

(Seriously, what kind of answer did you expect?)

chris

How about addressing what is going on, and why 'validating' Errors and Warnings then committing is different from this, or that you believe them to be the same?

Or whatever you think on this given the community experiences, including, if you please, any reaction you have that this or that seems more or less likely. You know, An intelligent dialog where you don't go off and just further alienate people with that childish passive-aggressive avoidance stuff. You know. Like an adult.

Heck, you might even suggest we look for this and that and pin something down for you to mull over further. The problem with you is you treat these guys as adversaries instead of co-workers. Granted a Railway worker might describe things badly and be ill-informed, but it is YOUR FAILURE to make it simple enough, turn-key enough to grow the market share, not his. These people have 10,000 eyes and you just blow them off instead of seeing them as a resource.

You think he's baiting you because he's pleased with your past behaviors? You never did answer my post back in June on Performance issues. Seems to be a pattern of yours in the communities mind. I can tell you THAT's the case with the 40+ of us in Yesterdayz Trainz. Let me quote a few:

That’s why I’m not investing in Trainz any further. No more money from my pockets or time wasted trying to show them their ridiculous bugs all in the name of helping them with their lousy, poorly designed software. I have a stable system with 2012 and I’m going to keep it that way; it provides a nice amusement when I’m on the road but not much more. I’ve found bugs and problems with this software that makes me cringe and while the new kickstarter project is interesting but I don’t think it will bring much to the table other than empty promises. Professional software teams don’t put out releases that are worse than previous releases, but N3V does. Professional software companies listen to their customers but N3V ignores them. I have a version of TRS2006 that performs better than Trainz 2012 and that’s sad if you’re looking to build momentum in your customer base. Compare that with any of the MMORPG games out there or Steam and all of its momentum and comparing them to Trainz 2012 it looks like it’s in some sort of time warp from 10 years ago. The Kickstarter project seems to be a last gasp for N3V to get more money out of us but until they change internally, to really listen to their customer base on problems and suggestions, I’m afraid I’m out. Besides as I’ve read the articles/postings on the Kickstarter get ready for more incompatibilities with all those assets you’ve created and DLC you’ve paid for. If some other organization took over Trainz then maybe I’d buy a new version or another FCT but I won’t hold my breath for that day to arrive.

This issue that Frank brings up is a long standing issue with Trainz/TRS for a long time and if they’re not going to fix it going back, what 10 years now with TRS2004 or so for me, do we really expect them to fix anything else other than what they feel is “important” maybe more “speed trees” perhaps? LOL


Or how about:

My personal view is that Trainz has evolved rather than been planned out and I suspect the programmers have as well. The good part is we have something that works on a good day. The regrettable part is I think the programmers are a bit isolated and some of their ideas are rather shall we use the expression not quite mainstream the problem is nudging them to something that works.

Cheerio ...

Or:

The problem is most of their code relies on the old index methods against the files out on disk and the rest relies on the SQLite database. It’s a problem of inconsistency in data stores and really the amount of thrashing they put the disk subsystem through is pretty awful. You could show lots and lots of diagrams and arguments to Chris but in the end I’m afraid he’s not interested in hearing opinions outside his own dev. Team. This can all be fixed pretty easily and would actually help Trainz but as a comedian has said “You can’t fix stupid!”

Or:
On the technical side of things, I’ve been thinking that perhaps this is a program bug more than actual data corruption that is triggering this revalidation process. The intended purpose of this occurrence is to validate data corruption due to database anomalies. These glitches could easily because by the end-user performing a system reset, a program crash, or a hardware fault somewhere. Since the database wasn’t closed properly, the database is flagged as “dirty” and TADaemon then needs to verify the integrity of the underlying database records. Unfortunately in our case, the implantation is bugged so TADaemon has a hissy fit when things don’t appear right to it.

Now having said this, and being the paranoid former IT guy and do over think things quite often, I wonder if there is perhaps some database issues that the program is picking up, but being a Chris Bergmann project, there is no feedback for the end-user so we don’t know why this validating process is triggered. From my experience, this has happened after performing a database repair, whether an EDR (a very, very rare occurrence at that), or an QDR. Perhaps in this case, this is part of the database repair process, but is not communicated properly from the interface. Other times this has kicked in after adding assets from CDPs. I recently downloaded some content from Jointed Rail, and sure as shootin, I got a long dreaded data validation afterwards. I also got this after installing the Murchison update, or rather bug fix. Maybe this is how new non-DLS content is added to the current database. We surely don’t know because there is no communication back to us.

Anyway, if this is a programming buy, the first thing I would do, on Chris’ part would be to take a look at his “perfect” code and see what’s happening. Maybe it is a false trigger and their code needs a bit of patching to solve that. Maybe the very nature of the database engine is corrupting its own data. My gut feelings is this is a linear database file so everything gets appended to at the end, so any changes, updates, or transactions require a long read, write cycle, and there is not enough time allocated for the process.

And then there was the pithy:
As Churchill said �If you have an important point to make, don't try to be subtle or clever. Use a pile driver. Hit the point once. Then come back and hit it again. Then hit it a third time - a tremendous whack.�

Sometimes in dealing with our friends from down under a whack may be more beneficial. For a long time the database inconsistency problem or �rebuilding the database� or downloading assets has probably consumed more time and resources than normal people would consider bearable. The current designs for both of these subject areas are terrible and the lack of focus on fixing them represents a bit of arrogance on the part of N3V towards its user community. Until you change that arrogance you won�t change Trainz in any real or meaningful way.

ALL THOSE on THIS TOPIC in the past 36 hours... most from people who'd laugh to think your demonstrated work qualified you as their peers. In short, all from technical professionals who have a bit more experience in large ponds as small fish growing wiser without fullfilling the Peter principle so young as you; Care to alienate more customers? Keep up the stonewalling. YOU DON'T Want to see my initial post--thanks to those guys, you got the kind and gentle version. I'm convinced time after time you guys have taken a path convenient to you at the expense of the user community.
 
"The heat was felt from miles away. Many had to be evacuated. The cause? Somebody called someone out on a forum.."
 
How about addressing what is going on

I'm not really sure what more information I can give you at this point. Some of you are reporting something that (as far as we can tell, from the varied and sometimes vague reports that reach us) should not be happening. That's not a dig at any of you, but there's very little meaningful information that is user-visible here, so the amount of useful data we can get from your posts is minimal. It's not something that we can currently repro in-house, so (as above) our QA lead is talking to a few people and asking them to assist us with testing.


..and why 'validating' Errors and Warnings then committing is different from this, or that you believe them to be the same?

What's being referred to here is a background validation process that only shows up in the diagnostic logs. This is the same process that is run when you use 'View Errors and Warnings', when you install new content, when you the current version of dependencies, and so on- there are many legitimate reasons for it to occur; we have no concerns at all about the messages showing up in the logs; the question is simply whether there are some cases where unnecessary validation is occurring, and if so: why?


You know, An intelligent dialog where you don't go off and just further alienate people with that childish passive-aggressive avoidance stuff. You know. Like an adult.

It's responses like this that typically encourage me *not* to take that path. Pot, kettle, black, etc. I know you don't like my way of doing things, and honestly I don't really care. I'll happily give information when there's something constructive to say. If you're just going to snipe then i'll either ignore you, tell you off, or snipe back.


The problem with you is you treat these guys as adversaries instead of co-workers.

No; you'll note that where somebody has a meaningful question, I will happily answer them. If someone is being purely adversarial, then I'll still answer them accurately as well, though I won't go out of my way to make it nice on them. I'm not a PR person and I don't pretend to be a PR person. I'm here to deal in facts, and if sometimes people don't like the facts then that can be hard to deal with.


To reply to your assorted notes:

..Steam and all of its momentum and comparing them to Trainz 2012 it looks like it’s in some sort of time warp from 10 years ago.
You really think that comparing a multi-billion dollar company to a million-dollar one is even remotely reasonable? While I'd love to own something like Valve, I'll be the first to admit that we're not in their league when it comes to cashflow, employee count, market share, and many other relevant statistics.


The Kickstarter project seems to be a last gasp for N3V to get more money out of us but until they change internally, to really listen to their customer base on problems and suggestions, I’m afraid I’m out.

Well, I'm sorry that you feel that way, but I'm glad that many other users have supported us. TANE development is coming along very well, with you or without you.


My personal view is that Trainz has evolved rather than been planned out

Absolutely true. Or more accurately, there was a plan, and it did not survive the encounter with the enemy (to steal a good phrase.) Each individual version of Trainz is carefully planned out, and certain systems have long-term plans, but to claim that we know exactly what we'll be doing in another 10 years is completely rubbish. We'll take it as it comes. Generalisation is easy (we'll improve graphics, make the DLS more robust, add some key gameplay features, etc.) but as to what we'll be doing on this day in ten years time? No idea.


The problem is most of their code relies on the old index methods against the files out on disk and the rest relies on the SQLite database. It’s a problem of inconsistency in data stores and really the amount of thrashing they put the disk subsystem through is pretty awful.

This is fairly accurate. We suffer pretty hard for maintaining on-disk compatibility with a system that was written at a time when 10 baseboards was a big route, and 256MB was a lot of memory, and direct filesystem access to the custom content was a requirement by our users. I don't think that anyone is under any illusions about the need to overhaul this, although ironically it's going to happen at around the same time as hardware actually makes this approach a lot faster (no more seek delays, yay!)


You could show lots and lots of diagrams and arguments to Chris but in the end I’m afraid he’s not interested in hearing opinions outside his own dev. Team. This can all be fixed pretty easily and would actually help Trainz but as a comedian has said “You can’t fix stupid!”

It's true, you can't.

(See how easy that was. Still wonder why I don't bother repeating answers that have been explained several times over?)

Unfortunately, it's very easy to say things like "fixed pretty easily" if you're not the person responsible. Underestimating the effort required to maintain a system which involves many, many stakeholders with varying requirements is a very common issue, and programmers tend to be the worst at this. Ever heard something like "Oh, that old crud? Just rewrite it, it'll only take a few weeks"? I assume that you're familiar with the fallacy in that line of thinking.


On the technical side of things, I’ve been thinking that perhaps this is a program bug more than actual data corruption that is triggering this revalidation process. The intended purpose of this occurrence is to validate data corruption due to database anomalies. These glitches could easily because by the end-user performing a system reset, a program crash, or a hardware fault somewhere. Since the database wasn’t closed properly, the database is flagged as “dirty” and TADaemon then needs to verify the integrity of the underlying database records. Unfortunately in our case, the implantation is bugged so TADaemon has a hissy fit when things don’t appear right to it.

Close, but no.

If there's regular background validation occurring for no good user-initiated reason, then there's definitely a bug.

That has nothing to do with "dirty" flags or bad data or anything else. This is probably confusion on the speaker's part between database repair and validation.


From my experience, this has happened after performing a database repair, whether an EDR (a very, very rare occurrence at that), or an QDR.

Definitely valid cases for it to occur, and not considered a problem.


Other times this has kicked in after adding assets from CDPs. I recently downloaded some content from Jointed Rail, and sure as shootin, I got a long dreaded data validation afterwards. I also got this after installing the Murchison update, or rather bug fix.

Definitely valid cases for it to occur, and not considered a problem.


Anyway, if this is a programming buy, the first thing I would do, on Chris’ part would be to take a look at his “perfect” code and see what’s happening.

Again, no claims on my side that our code is "perfect", nor that our code is "my code". Are you sure it's me who is being the negative one, here? Sounds to me like somebody has a grudge.


My gut feelings is this is a linear database file so everything gets appended to at the end, so any changes, updates, or transactions require a long read, write cycle, and there is not enough time allocated for the process.

His gut feeling is very, very wrong.


The current designs for both of these subject areas are terrible and the lack of focus on fixing them represents a bit of arrogance on the part of N3V towards its user community.

Perhaps. However, you should also keep in mind that:

* In this thread, well in advance of your post, we've already identified specific issues that we'd like to investigate further.
* We have a major project underway to overhaul the Trainz engine, specifically to bring it up to date for modern workloads.

If that's "arrogance" and "lack of focus" then so be it.


In short, all from technical professionals who have a bit more experience in large ponds as small fish growing wiser without fullfilling the Peter principle so young as you

In short, a comment like that makes you an ass and comes across more than a little ageist. Not saying you're stupid or don't make occasional useful input, but turning everything into personal attacks against myself and my colleagues doesn't help anything and has actively lead to you being banned from our wiki. I find that a real shame personally, because you were one of the more active wiki editors and we want to see more wiki activity, but if you spend as much of your time attacking what we do as you do making yourself useful, then we don't have much time for you.

chris
 
Last edited:
"Although all seemed lost, the heat was put away in a quick flash, saving many, and preventing a large scale evacuation. Though burn injuries still seem visible on some"
 
Whoa there, Frank. :eek:

That's certainly not what I'd expect from you! And banned from the Wiki as well. Oh dear. If Chris does not mind me asking, for how long is Frank's ban? I sincerely hope it isn't permanent.

ATSF854, well done with those quotes. Very fitting indeed, and beautifully executed.

Kieran.
 
Wow, I had really hoped to find some engaging Thought on the issue by this point.

In short, a comment like that makes you an ass and comes across more than a little ageist. Not saying you're stupid or don't make occasional useful input, but turning everything into personal attacks against myself and my colleagues doesn't help anything and has actively lead to you being banned from our wiki.

no words...
 
I'm not sure in my own mind that Kickstarter is the way forward, I note one company got kickstarted to the tune of $2 million then the company was sold for rather more to another company.

At the moment we have something reasonably functional in TS12, if Tane goes more along the DRM route then we move into do I trust N3V and do I need it?

It's unfortunate when things become confrontational. Together we stand and divide we fall is especially true for Trainz, to succeed it needs both N3V and the community and it needs trust.

I think what we have done in the thread is identify a problem that impacts end users. Whether N3V can correct it is another matter. The community has not identified a solution.

If TANE is a success people have jobs. If it isn't existing TS12 users aren't really impacted that much which is a little stressful on N3V and I think we have to recognise this is coming out in the posts.

Cheerio John
 
Well Chris,

If you say that N3V has identified areas of concern, please let us know that. Keeping things like that a big secret tend to get the community riled up and thinking nothing is being done. A little bit of communication goes a long way. Much of this, including the ill feelings, appears to be lack of communication on the part of N3V. If people responded back often, rather then when put into a corner like this, we wouldn't be in this position. If the program interface had more dialog messages, it would be more reassuring to know that we have to wait and something hasn't gone tits up with the system. This has always been an issue and we wish that this could be addressed. Perhaps this could be an unmentioned goal for T:ANE.

If adding new content requires a validation, instead of validating the whole database all over again, why not validate just the new adds? This appears to us to be one of the issues even if it's not. How about a dialog box too that says something like "Please wait... New assets have been added and are being validated" or something along those lines?

What has prompted people to look at the database console windows is the poor performance. It also tells us when it's safe to shutdown our systems, and do other things, but mainly because we are being slaughtered with not responding screens, the great pauses, and unwarranted stutters which we could not account for. As you know it's not good to kill programs without closing data files first, and being database driven makes this worse. In our case this prompts a long database repair process which would only complicate matters as people then kill the database rebuilds because they want to use the program.

The consoles give us a tool to help you. As you can see we not all a bunch of 12 year olds playing with trains on the computer, and many of us come from technical backgrounds which span many decades. Please leverage our experiences as technicians, programmers, copywriters, editors and other resources. We are a community and are willing to help to make this program even better. To paraphrase Frank. "We're 10,000 eyes out here looking at the program every day." Sometimes when an individual or a team works very close to a project, the individual tree becomes the focus rather than the forest. Having our eyes do the looking and spotting, we see can see many things that you guys don't, and we are more than willing to help.


John
 
Back
Top