PDA

View Full Version : Route Load/Save Changes



WindWalkr
January 26th, 2016, 01:15 AM
Hi guys,

This one is mainly for the script developers.

We're currently working on improving the route load/save process and there will be a number of changes visible to script as a result of this, which we need you to be aware of and which we'd like feedback on.

The changes are intended to resolve a number of outstanding concerns with loading and saving routes, and includes the capability of loading splines and scenery objects from the route for the local area only, similarly to how we handle ground tiles. This has been on the wishlist for quite a while, and has been gradually rising in priority as route size and object density has increased.

The upsides of these changes include:



Lower memory footprint. (Not an immediate concern now that we're 64-bit, but still useful.)
Faster loading times. (Only loading data in the area around the camera, rather than across the whole route.)
Faster save times. (Only having to update the modified areas, rather than resaving the whole route.)
Faster streaming of data from the route files while moving around the route.


To be clear, this doesn't really impact on frame rate. It also isn't entirely new when it comes to the ground tiles, which is where a lot of the data cost comes from in a given route, although we are further improving how that is handled. Finally, it's not new in terms of resource (mesh, texture, sound) loading- it primarily affects the knowledge of the existence of the item, and a few stateful concepts such as script.

The reason I'm bringing it up at all is because of a number of changes to the runtime state which will be visible to script. Many of these changes are likely to negatively impact existing scripts. Some examples might include:



Track Searching beyond the visible portion of the route may not work reliably. Only loaded splines will be searched.
The queries which return all objects of a given type on the route will not work reliably. Only loaded objects will be returned.
Rather than being loaded once at the start of a session, and saved/reloaded only across an explicit "Save" operation, scripted objects may be loaded and unloaded/saved repeatedly as they move in and out of relevancy to the user. (It should be stressed here that relevancy to the user is similar to, but not identical to, the set of visible objects. Items may become relevant through a number of mechanisms including proximity, simulation mechanics, and script queries.)


If you work in either of these areas, I'm sure that you can immediately see how this is going to be a problem.

We've been discussing internally how best to handle this, and it's clear that there are going to be compromises. I'd like to get this out in the open and make sure that the trainzdev group has their chance for input before we make any final decisions. There are a number of strategies that we've discussed already:



It's clear that there will be changes. There is no magic bullet that will hide these changes completely from the script. Our aim is to take reasonable steps to minimise the impact, and where scripts changes are required, to minimise the changes.
We recognise that any changes are going to take time to make. It's possible that we'll need to adopt a "compatibility mode" option as we have used for other transitions in the past, or some similar mechanism perhaps on a per-route or per-module basis, to ensure that content creators have sufficient time to adapt to these changes.
There are situations where a track search which extends beyond the currently loaded content is strongly desirable. We may address this by adding an asynchronous track search, which will offer similar capabilities to the current search but which will run as a background operation, loading data as required in order to give full results. It will be up to the script developer to balance the extent of the query against the desirable performance goals. For example, searching every track on a large route could take a minute or more depending on disk performance.
Similarly, train navigation commands that extend over significant distance may take longer to process. This will move from a synchronous process to an async process.
We may keep certain content loaded despite lack of proximity. This will ensure that the content responds immediately to any queries. This is likely to be limited to very specific items which we are sure will not be particularly numerous on the route. For example, Industries are likely to fall into this category, but junctions and signals are not.
We may consider alternative ways of filtering certain items. This would allow us to quickly load subsets of items for some criteria other than location on the route. An example of how we might use this would be to replace the current TrackMark selection menus (we obviously don't want to have to load an entire route just to list the TrackMarks.) We might for example allow the TrackMarks to be manually subdivided into a number of groups, and then load the groups on demand. (Alternatively, TrackMarks could fall into the above persistent-data category, but then we'd need to get serious about discouraging unnecessary TrackMarks and/or separating TrackMarks into a small number of persistent marks and a larger number which are loaded based on proximity.
It will become imperative for scripted assets to initialise quickly, since this will be happening regularly as the player navigates around the route. It's likely that script timeouts will be tightened for most asset types, and assets which perform poorly in this area will need to be optimised or rewritten to perform lazy or asynchronous calculations rather than attempting to do everything up-front.
It's likely that we'll need to keep data surrounding any moving Train loaded, along with the Train and its Vehicles. This ensures that physics, signalling, junctions, etc. continue to operate as expected. Stationary vehicles with may be excluded from this, however it's likely that the Train itself will still be loaded.
Interlocking towers will continue to work as they do now, and will be a suggested "go to" for resolving some of the concerns raised here, since they help to collapse larger parts of the route into a single logical entity.



Please feel welcome to comment on the pros/cons of each, or to offer your own suggestions on alternatives that you'd like to see discussed.

thanks,

chris

pcas1986
January 26th, 2016, 02:50 AM
There were a couple of thoughts that come to mind. How many of those who develop assets of this type participate in TrainzDev, and what can those who participate here do to help?

I think I've written just one test asset of this nature but had no particular need to publish it nor develop it further. However, I'm willing to participate in identifying and testing assets that may be affected.

You might need to move slowly and carefully with this one. :o

WindWalkr
January 26th, 2016, 04:19 AM
How many of those who develop assets of this type participate in TrainzDev

That's a very good question. I know there are a few, but how well represented this area is, I have no idea. That's something that the group will need to manage.




.. and what can those who participate here do to help?

At this stage, it's just a discussion item. Put forward any thoughts you have, any concerns that haven't already been mentioned, etc. As you note, we'll need to make sure that we have enough representation here of the topics that will come up. We'll need to make sure that we have lines of contact with the various groups who work in these areas, some of which will probably be foreign-language groups.

When we restart the seeding of test builds, things will move from theoretical discussion to the practical side of things- finding problems that we haven't identified in discussion, testing to see how much content is affected, testing various workarounds, etc.

chris

pguy
January 26th, 2016, 06:23 AM
Hi.

I may understand all the benefits of this approach, but that will be a huge break in backward compatibility for quite a lot of sessions.

Most rules and driver commands builds lists of all available assets of some type on the route for user parameters selection and introducing retrieving only loaded objects at local scope will be a major break and make a lot of them unusable without any changes. And currently there are 363 rules and 282 driver commands available commands (base, builtin, on DLS). That's quite a lot to be looked at and updated. And some author's are no longer there ; some have encrypted scripts and if we don't have access to sources, it will be hard to update ...

Just discovering this, at least you certainly need to have for some times a dual mode (or compatibility mode) where sessions not updated will still be able to access all assets and objects at a global level (for backward compatibility) and you can have a new local scope mode with better performance for new and updates sessions adapted to this new scripting constraints.

I would also suggest that even if object access will be limited to a local area scope in the new session mode, you keep some indexes containing the names of all objects of some types in the route at a global level. It happens quite often that you retrieve a list of objects of some type from the world static class object, but only to scan the list to rebuild a new list of the objects names to be used for user global selection. If it was possible to retrieve a list of names, may be meeting some filtering criteria, directly from some native world routines and without loading these objects (as we need only their names) it may help a lot for many rules and driver commands.

About rules and driver commands and also scripted trackside objects scanning tracks in scripts, many of them are related to signals or to path rules. Interlocking Towers may replace path rules but for signals many scripted signals will scan the track to search for next junction direction, speed limit trackside objects ... and sometimes for very long stretch of tracks. In France, main lines will have automatic block system about 1.5 km length and you scan only for the next 4 to 8 blocks ( that may go to a 12 km distance ahead). For hight speed line, you have still 1.5 km blocks length but with the hight speed and the time to stop you will look ahead much longer. For secondary line, signals are more spaced : it can be 15 km between two signals with a repeater 1.5 km before the main signal and you can scan track along 15 to 30 km finding objects. And of course, what exists in France with such distance may exist in other countries with other distance in mind.
I would suggest that if you take the step to limit the distance scope to see objects, this limit should be adjustable depending on the context (country, usage, ... ). Sometimes a session may work with only a 6 km scope. But some other times you will need a 30 km search scope ... I believe it may help if in the new session mode, you request the session creator to give the local scope distance he will need.

That will be all for only a morning of reflexion. I will post more when I will have reviewed some of my rules and driver commands and also some other commonly used rules and commands ...

Regards.
Pierre.

WindWalkr
January 26th, 2016, 10:56 AM
I would also suggest that even if object access will be limited to a local area scope in the new session mode, you keep some indexes containing the names of all objects of some types in the route at a global level. It happens quite often that you retrieve a list of objects of some type from the world static class object, but only to scan the list to rebuild a new list of the objects names to be used for user global selection. If it was possible to retrieve a list of names, may be meeting some filtering criteria, directly from some native world routines and without loading these objects (as we need only their names) it may help a lot for many rules and driver commands.

This is certainly possible to a limited degree, and worth discussing further. It doesn't help solve the compatibility issue, but may make life easier for future script development. The main question would be which names and/or other data would be useful to index in this fashion, whether we can introduce any logical groupings, etc. We've known for a while that the current trackmark UI is reaching its practical limit, so this may be as much a UI question as a data management question.



I would suggest that if you take the step to limit the distance scope to see objects, this limit should be adjustable depending on the context (country, usage, ... ). Sometimes a session may work with only a 6 km scope. But some other times you will need a 30 km search scope ... I believe it may help if in the new session mode, you request the session creator to give the local scope distance he will need.

An interesting idea, but I don't think this is necessarily practical. Because of the very nature of async streaming, it's basically impossible to make guarantees about how far a given script will be able to "see". Obviously we can tune how much data to load, but any significant changes in relevancy will take time to be streamed in, and we need things to be able to keep operating during that period.

One of the upsides of the signalling system is that it doesn't need to update unless something has changed, so we can perhaps leverage this to ensure that the necessary data is preloaded. For example, and I haven't fully thought through the problems with this: when a signal becomes flagged as requiring a state refresh, we could at least ensure that the area that it is currently monitoring is fully loaded before calling the script to update the state. In the worst case, the state refresh could still require following a new path in the track graph which might lead into unloaded territory, but at least we'd be minimising the likelihood of that.

For dealing with signals that rely on a synchronous track search, it might be reasonable to detect that the tracksearch ended at a streaming boundary, and trigger further streaming followed by a secondary refresh of the signal once the new data is available.

thanks,

chris

pguy
January 26th, 2016, 02:16 PM
This is certainly possible to a limited degree, and worth discussing further. It doesn't help solve the compatibility issue, but may make life easier for future script development. The main question would be which names and/or other data would be useful to index in this fashion, whether we can introduce any logical groupings, etc. We've known for a while that the current trackmark UI is reaching its practical limit, so this may be as much a UI question as a data management question.
...
chris

About huge routes with too many trackmarks or other trackside objects to be currently displayed by most common driver commands or generating script timeout due to too long processing, I personally think that the easiest and less painful solution would be to introduce some logical grouping facility, where you can for each trackside object in its object properties assign the trackside object to a logical group. Objects belonging to a logical group should have their name local and unique in their logical group and should have a global name of <logical group name>.<local name in the group> (you can use . dot as separator or any special separator you prefer ... ). And if you introduce a logical group filtering capability in static world class, so that you can limit the results of GetTrackmarkList(),GetSignalList(),GetJunctionList (), GetInterlockingTowerList, ... to only the objects belonging to a logical group or to a list of several logical groups it should be possible to update most of the common driver commands and rules so that they work only with the filtered objects and no longer with the full list of available objects on the route.
And if you add to world class a GetLogicalGroupList() giving the list of all defined logical groups, it should be quite easy to make a rule where the session creator can tick the logical groups he wants to work with, then set the adequate world filter and then the session will work with only the selected logical groups forgetting all the unticked group objects. And giving the route or session creator the choice of what logical grouping it will choose offer the possibility to group objects either by logical location, by logical function, ... or by any other logical usefull grouping idea.

I really think that for huge route we need something like that to be able to keep a limited list of objects to manage during the session even if the huge route has much more available objects.

That's only my own thoughts. May be other scripters or developers have some other ideas ...

Regards.
Pierre.

TRam__
January 28th, 2016, 03:32 AM
Track Searching beyond the visible portion of the route may not work reliably. Only loaded splines will be searched.
The queries which return all objects of a given type on the route will not work reliably. Only loaded objects will be returned.
Rather than being loaded once at the start of a session, and saved/reloaded only across an explicit "Save" operation, scripted objects may be loaded and unloaded/saved repeatedly as they move in and out of relevancy to the user. (It should be stressed here that relevancy to the user is similar to, but not identical to, the set of visible objects. Items may become relevant through a number of mechanisms including proximity, simulation mechanics, and script queries.)


Just ffuuuuuu! Yes, in some rarely cases it would be nice to get scripts loaded only when they are in the visibily area (passenger stations for example), but in most other cases it will lead to comlete session breakdown!

Currently trainz is the unique train simulatior, where the interaction in proceed all over the route, and it one of its best benifits, but you just want to spoil that. The interactions of AI-driven trains among them and with signals/junctions/rules/commands are worth.

The not-all objects loading & incorrect trackstreach will broke my alternative of "Interlocking Towers" (zxPath system) with no ability to fix up, so i will have not to support next versions of TANE.


(we obviously don't want to have to load an entire route just to list the TrackMarks.)But how to set up a "timetable" for a loco, that is on other side of map?

So, in my opinion, that should be a funtion, something like "disable physics" of traincar vehicles, and that has to be up to scripter to choose if he want increase perfomance with not-loaded objects. The objects that definitly have not to participate in this "script disabling" are:

- traincars
- junctions
- signals
- triggers
- trackmarks
- commands/rules
- libraries

An option of disabling/enabling scripts (or may be script threads) is also worth for "scenery with thracks".

WindWalkr
January 28th, 2016, 03:49 AM
..in most other cases it will lead to comlete session breakdown!

This isn't really a fair statement. It's definitely true that if you grab any given script based on today's behaviour, and simply use the script in a new context without updating it to suit that context, you're reasonably likely to run into problems. Obviously this will affect certain types of script more heavily, and certain types not at all. Scripts which attempt to generate lists of every object on a route are much more likely to be adversely affected.

But all that said, there's no fundamental reason why it has to be that way. It's just the way it is now. Yes, it means that we'll need to revisit the scripts which rely on the current behaviour and either update them or find alternatives, but there's certainly no reason to believe that such an update is impossible.

As pcas1986 said above, this is obviously an area where we'll need to tread carefully and this discussion is part of that. Our role here is to identify the various problem areas- both technical, such as which script functions will stop working effectively and will need replacements, and also community, such as foreign-language community groups who will need help understanding the changes and communicating their concerns.



Currently trainz is the unique train simulatior, where the interaction in proceed all over the route, and it one of its best benifit, but you just want to spoil that.

Not at all. We want to ensure that computer resources are spent on things that are relevant to the player. The exact definition of relevance is debatable (that's one of those technical questions that we'll need to work on) but the main point here is that scripts will have to start being a little bit smarter about what they ask the game to do. Broad "give me all the data about the entire route" type queries will no longer be practical. That doesn't mean that you can't perform logic across an entire route, but it does mean that we will need to consider partitioning that data to some extent.

chris

TRam__
January 28th, 2016, 04:09 AM
WindWalkr, for exaple i need to check if a track is clear for a train, that is driven by AI and user is futher than 20 km distance. Yes, later user will be nearer, but that train choose occupied track because the taincars on that track haven't loaded. And we got two trains "one inside other". Yes, you can say that "the AI wouldn't work on that distance", but how to check its path?

The example of semy-behavior (because at server all trains are "in correct positions") we currently have in multiplayer. Just look at this: https://youtu.be/-T_AhdCoJ5w?t=37m2s (to 38 - 55) . The signals are synchronised to server, so their green lights show that there are no that train in that position at server.


Broad "give me all the data about the entire route" type queries will no longer be practical.For any interlocking system it is the first question. "Get all signals to sort them into station groups", or "get all juctions/tracks to attach the occupation state and occupation direction".

WindWalkr
January 28th, 2016, 04:27 AM
But how to set up a "timetable" for a loco, that is on other side of map?

We know for sure the Trains will remain loaded even if they are in areas that are not relevant to the player. It's unclear whether Vehicles will (they probably won't, but it's not definite.) This suggests the following possibilities:

* If the Vehicles are kept loaded, then there is no change to the current behaviour in this area. Otherwise;
* The timetable could be set up based on the Train or Driver, not the Locomotive;
* A Library could be used to track a list of eligible Locomotives on the route. Although it can't access them directly, it could store whatever metadata you need, and it could track which Driver or Train the locomotive is associated with. You use your stored data to communicate with the user, and you use the Train or Driver to communicate with Trainz;
* You could have the user select the Train first, and then perform an async query on just that Train in order to determine which locomotives are available;
* You could perform an async query which is partitioned in some other fashion, such as we were discussing with TrackMarks above.

There may be other options, those are just a few that come to mind.



The objects that definitly have not to participate in this "script disabling" are

Unfortunately it's not feasible to have bulk objects excluded in this fashion. To cover your comment by class:

* traincars - As above, this is undecided. Keeping them loaded would be nice, but may prove problematic for a variety of technical reasons.
* junctions - Will not remain loaded.
* signals - Will not remain loaded.
* triggers - Will not remain loaded.
* trackmarks - Will not remain loaded.
* commands - Will remain loaded.
* rules - Will remain loaded.
* libraries - Will remain loaded.
* scenery-with-track - Will not remain loaded, with the possible exception of Industries.

Obviously nothing's completely set in stone at this point, but I want to stress that we're not going to simply allow large categories of objects to remain loaded because that defeats the point of what we're trying to achieve. What we will consider is additions (either native changes, native script interfaces, or even script helper libraries) that help to mitigate the effects of the objects being unloaded. For example, it might be possible to synchronously search the path between two Trains, but it will not be possible to immediately access script objects for each signal and junction on that path.

To give you more specifics about how your zxPath system could be adapted, I'd need more details about what it does from a user's perspective. If it is similar to our Interlocking Towers mechanism, then you might also want to consider whether what you're doing could be implemented in those terms.

I also want to reiterate that we will be offering some form of compatibility mode during the transition. We're well aware that until the various scripts have been updated, there are going to be significant side effects. We're not expecting all of the problems to be solved tomorrow, or even next year.

We will be seeding builds which conduct experiments in this direction, and I want to be clear that nothing in these builds is set in stone. As with any change to the game systems, there will be significant breakage in the development builds which won't be present in the release builds. In this case, a lot of those breakages will be in the script reactions to the native changes, and will be resolved by the mechanisms that I've mentioned above and further mechanisms along the same lines that we develop based on your feedback and our own testing.

There will come a time when the scope of the changes are clearly defined, and we'll let you know when we feel that we've reached that point. Until that time, we welcome any feedback on breakages etc. and scripters should pay close attention to how changes affect their scripts, communicating any concerns to us. You shouldn't expend too much effort reworking your own scripts at this point however- you may find that a lot of the problems you're having will simply go away as the various systems come online.

hth,

chris

TRam__
January 28th, 2016, 04:39 AM
From user side it looks like this - http://www.youtube.com/watch?v=r2zJHQ-K5bo , http://www.youtube.com/watch?v=kK0-ocptUM4 .

Documentation (in Russian) - https://yadi.sk/d/yv0zmTveF5cSg, https://yadi.sk/d/MDi0IHn6GPNCb


* junctions - Will not remain loaded.
* signals - Will not remain loaded.
* triggers - Will not remain loaded.
* trackmarks - Will not remain loaded.
* commands - Will remain loaded.
* rules - Will remain loaded.
* libraries - Will remain loaded.
* scenery-with-track - Will not remain loaded, with the possible exception of Industries.
The last one, why not vice versia :) ? The industries are not so worth, especially with passenger-loading.

P.S. Just add a config tag for juctions, triggers and trackmarks, that adds their loading back. We add it into all needed objects, and everyone will be happy. You'll get sufficent improve among built-in routes, we will save all-route interaction .

WindWalkr
January 28th, 2016, 04:44 AM
WindWalkr, for exaple i need to check if a track is clear for a train, that is driven by AI and user is futher than 20 km distance.

You don't mention any context for this check, so I'll assume that we're talking about determining the state of the signals in front of the Train. This isn't a big problem for two reasons:

1. Signals don't generally change state unless something has changed within their area of influence. With a few minor exceptions, this means that unloaded signals will never change state, so we don't need to run any logic on them. There are various things which can change the signal's state (interlocking tower path assignment, train movement, neighbouring signal state change, junction change, rule override) and each of these will be handled in its own way.

2. An active train is constantly updating its awareness of nearby signals, and the movement of the trains are constantly updating the nearby signals because they are in the signals' area of influence. Because this process is continuous, everything important to the train stays relevant.

So we have everything that is necessary relevant, and any sudden changes which could affect this (such as a junction change) will start with a known-good state and propagate quickly to the active state.




For any interlocking system it is the first question. "Get all signals to sort them into station groups", or "get all juctions/tracks to attach the occupation state and occupation direction".

You will not be able to perform this kind of operation live, but that's typically fine because you only need to query this kind of thing during path setup. I'd suggest taking a look at how our Interlocking Towers UI works; it's not impervious to these changes and we'll need to make some updates, but by and large it avoids performing massive route-wide searching and sticks with local-area searching. As we update our own scripts, we'll be sure to provide details about what and why so that people who are working in similar areas can benefit from our work.

hth,

chris

WindWalkr
January 28th, 2016, 04:44 AM
Gotta run. Will talk more later. Thx.

chris

TRam__
January 28th, 2016, 06:16 AM
the state of the signals in front of the Train.Not only in front, but also behind the train. And that's for all moving trains all over the route. In sU signals i've taken 5 "handlers" (actually function calls) for signals:
1) front vehicle of train proceded the signal in the signal direction
2) last vehicle of train proceded the signal in the signal direction
3) front vehicle of train proceded the signal in the reverce direction
4) last vehicle of train proceded the signal in the reverce direction
5) the train left the signal area

Currently built-in algorithm show only the last event, so i needed to re-implement first 4 with tracksearches when the train enters the signal's area and continues motion (if stops, the checks skipped).

As for Interlocking towers, i can't see there per-junction path release, the direction of path occupation (it is worth to prevent two paths lead towards single curcit point), automatic generation of path, impossibility of "shunt path" (when all tracks are free, but the last one is occupied), automatic selection of clear tracks to prepare path. And mid-station block direction switching (yes, block system also has ability to change its direction !). For example z7-xPath and zxPath generate paths automatically, and not only in syrveyor (shunt paths are generated on-the-fly, and are limited in signle station area). And exists commands that sort the paths looking for clear track...


You will not be able to perform this kind of operation liveI am able to do them for ~5 years. The z7-xPath project was started in 2010, and approximatly in the winter of 2011 i've created first on-fly (in driver mode) path inside "local-area" limited with signals of other stations. That's why route builders have to setup the signals' attachment to stations and "signal type" for russian signals: for limitation of path searching area. It is tedious for casual users, but leads to wonderful results at complicated stations.

WindWalkr
January 28th, 2016, 06:57 PM
From user side it looks like this - http://www.youtube.com/watch?v=r2zJHQ-K5bo , http://www.youtube.com/watch?v=kK0-ocptUM4 .

Thanks, I'll look into this.



The last one, why not vice versia :) ? The industries are not so worth, especially with passenger-loading.

The industries need to continue processing while you're not there, and the results can be visible regardless of where you are on the route. The actual industry object isn't strictly necessary, only the processes and queues, but they're not really designed to be separated- we'll see.




Not only in front, but also behind the train.

Yes, of course. Anything that the train is actively monitoring.



i've created first on-fly (in driver mode) path inside "local-area" limited with signals of other stations

Note the keywords "local area". The kind of thing that we'll really have problem with is global queries. Local area work is a lot easier.


chris

pw3r
January 29th, 2016, 12:49 AM
As for Interlocking towers, i can't see there per-junction path release, the direction of path occupation (it is worth to prevent two paths lead towards single curcit point), automatic generation of path, impossibility of "shunt path" (when all tracks are free, but the last one is occupied), automatic selection of clear tracks to prepare path. And mid-station block direction switching (yes, block system also has ability to change its direction !). For example z7-xPath and zxPath generate paths automatically, and not only in syrveyor (shunt paths are generated on-the-fly, and are limited in signle station area). And exists commands that sort the paths looking for clear track...

I've created a new thread for any discussion related to the Interlocking Tower implementation, to avoid dragging this thread off-topic.

It can be found here: http://forums.auran.com/trainz/showthread.php?128299-Interlocking-Towers&p=1484065

Further information about any concerns/limitations of Interlocking Towers is always helpful so if you can reply to that thread it would be greatly appreciated.

Terry Palmer
Programmer
N3V Games

Dap
January 29th, 2016, 11:56 PM
Don't know if this is pertinent to the topic of this discussion, but in my CMTM software, I keep track of all the cars on the route that are being loaded /unloaded at industries. When a car has completed the load/unload cycle, it is moved from the "in process" soup to the 'available for pick-up" soup. One of the UI options is to display a list of cars available for pick-up. When this list is displayed, each car listed is a link, which when clicked upon changes the camera to look at that car. [ World.SetCamera(myvehicle,2); ] where myvehicle is identified by [ Router.GetGameObject(vehicleID); ]. This link list could include a car anywhere on the route. Will this function be affected by your proposed changes?

WindWalkr
January 30th, 2016, 01:14 AM
Don't know if this is pertinent to the topic of this discussion, but in my CMTM software, I keep track of all the cars on the route that are being loaded /unloaded at industries. When a car has completed the load/unload cycle, it is moved from the "in process" soup to the 'available for pick-up" soup. One of the UI options is to display a list of cars available for pick-up. When this list is displayed, each car listed is a link, which when clicked upon changes the camera to look at that car. [ World.SetCamera(myvehicle,2); ] where myvehicle is identified by [ Router.GetGameObject(vehicleID); ]. This link list could include a car anywhere on the route. Will this function be affected by your proposed changes?

Potentially.

There are a few possible areas of concern here, but there are also several mitigating factors that may already be in place, or which you can easily use to avoid problems.

1. As discussed above, it's unclear at this time how Vehicles will be affected by these changes. If the conclusion here ends up being that they're unaffected, then obviously the usage you describe here will be completely unaffected since you deal with Vehicles only. The rest of my comment here will assume that this isn't the case.

2. Router IDs are transient and not safe against load/save. New router IDs are assigned on each load. This either means that either you have an alternate permanent ID mechanism in use beyond the router IDs that you mention, or that you're already not safe against load/save. If the former, then whatever you're using should continue to work. If the latter, then your system is already broken and that's just going to become more obvious here. Middle ground: your code uses router IDs at runtime but maps them to names for load/save. If you are using this approach then you may need to change to using the permanent IDs everywhere rather than mapping them at load/save. Finally, it may be feasible for us to replace the router IDs for some objects with something that can survive load/save. I haven't looked into this, but it would solve the potential problem here, make life easier in general, and probably isn't prohibitive compared to the scope of this work overall.

3. A Router ID can only be mapped to a Vehicle if it's actually loaded. Attempting to map a Router ID to an unloaded vehicle will simply return null. This means that if you want to call SetCamera(), you're going to have to ensure that the Vehicle is loaded first. There will be helpers available for this kind of thing, but it will probably mean that you need to keep tabs on which Train the Vehicle belongs to (ie. so that you can tell the Train to load its Vehicles.)


The uncertainty in #1 is one of the earliest questions that we need to resolve, and isn't a discussion item in itself- it's simply a case of what the code can be made to support internally. We're still reviewing this (among many other questions) and this is something that I'll discuss further with the community once we have something definite to share. I don't expect that the answers will be more than a week or so away.

chris

WindWalkr
February 12th, 2016, 09:27 PM
We should have an early test build available soon (hopefully later today) so that we can start playing with this stuff. It's not remotely close to final- a lot of necessary compatibility work still has to be done, both to support internal systems and also in support of scripts- and we also need to provide new async script APIs as discussed above. Performance and stability will be a mixed bag- some areas which previously worked well are now horribly broken and will remain so until we have time to investigate all of the specific issues. Other areas which were horrendously slow are now working really well.

Case in point: on my fairly ancient computer, ECML now loads from CM in approximately 5 seconds.

chris

pcas1986
February 13th, 2016, 02:34 AM
So, will the test build provide any new/changed APIs or is it just a case of testing load times against SP1 routes?

I have a completely new project that I can use on this and other versions of T:ANE. How it fits with the aspirations of this initiative remains to be seen - especially by me. :D This is new territory for me.

WindWalkr
February 13th, 2016, 03:05 AM
So, will the test build provide any new/changed APIs or is it just a case of testing load times against SP1 routes?

No new APIs at this point. It's more a case of setting a compatibility benchmark (which will be pretty low) and then slowly moving back towards correct behaviour. Finding out what kind of things are broken, where our assumptions are overlooking something important, etc. We know of a lot of things that are broken and will need to be fixed, but everybody uses the sim a little differently, so I'd assume that there are things that we haven't noticed yet.

A lot of the problems encountered at this point will be solved by us and won't require anything from your end. As we identify specific problem areas that can't be solved internally, we'll add appropriate APIs and/or discuss mitigation strategies.

A "compatibility mode" switch is also present, but- as with the rest of the changes- it won't come as a surprise if important compatibility elements are currently being overlooked.

We don't expect any script testing or development to take place with the initial seed builds: loading/saving, navigation around the routes, and basic editing is probably about the limit of what we'll focus on for now.

chris

pcas1986
February 13th, 2016, 03:35 AM
...

We don't expect any script testing or development to take place with the initial seed builds: loading/saving, navigation around the routes, and basic editing is probably about the limit of what we'll focus on for now.

chris

OK......... Tks

TRam__
February 13th, 2016, 06:04 AM
Other areas which were horrendously slow are now working really well.And so, will an "always-load-script" tag be added to junctions/signals/triggers/markers/scenery or not?

WindWalkr
February 13th, 2016, 07:53 AM
And so, will an "always-load-script" tag be added to junctions/signals/triggers/markers/scenery or not?

Likely not; it's the antithesis of what we're trying to achieve here. We'll see where we end up, but that would be an absolute last resort if we find that important things simply can't be implemented in another way. (And if we were to go that way, it would be based on the item type and would not be a per-asset tag.)

chris

TRam__
February 13th, 2016, 08:10 AM
So, any ideas about nonlinear scenario operations occuring beyond "loaded area"?

Or you are just moving to RW&MSTS model of pre-defined train operations? With "dead area" outside the player's view?

P.S. Сould not imagine that copying RW ideas was started not from their implementation of cabin shadows, but from this. Loading times is neglible in comparation with from-cabin view (where player has to sit most time of scenario if he want image himself at "real" locomotive) .

WindWalkr
February 13th, 2016, 08:50 AM
As above, anything which occurs beyond the already-loaded scope will have to be prepared for asynchronous operation. This means caching data for which you require random access, and using new async APIs for everything else. Exactly what those APIs are, and how much caching the engine will provide, is something that we'll need to work out carefully as development progresses.

chris