PDA

View Full Version : .lm text files on scenery objects with LODs?



VinnyBarb
June 1st, 2015, 05:09 PM
Chris, is there a compelling reason why on can not use .lm text files without an animation mesh attached to a SCENERY asset mesh for a better LOD handling in T:ANE? As one needs to do with locomotives and bogies as these have animation meshes attached. I know, one can do this presently if a scenery mesh has an animation mesh attached to it but I was shot down by James Moody in the past when I once showed an animated scenery item with LODs in the forum.

I know, it is always the same "performance issue/penalty" doing it this way but is it really? Say, I make a very basic and simple animation mesh (as presently I can) in a xx.lm text equipped scenery item, hide this inside the asset and cull more or less animation and attachment cutoffs straightaway, just 1-2 meters away from the observer, so in theory they are from 3 meters onwards non existing and should not impose any extra load on performance. Am I right in this reasoning (not being a programmer myself)?

This way with a .lm text file for LODs on scenery items a content creator can increase the LODs meshes/levels and make better transition looking assets where one LOD, the LOD0 or even LOD1 changes to its next lower LOD and does not pop noticeably and ugly from one LOD level to the next and visa versa. No need for other lower LOD meshes to do this as they are usually too far away or too small to be noticed, only for LOD0 and possibly LOD1. Providing, notice, I say here providing, a content creator does the right thing and decreases/references the next lower LOD as soon as this is possible when moving away from the viewer. This way, it would not be an arbitrary distance of scenery assets where LOD changes as is now in your code. On some scenery models created with LODs presently one simply can not make the transition from one LOD to the next one without as said, that ugly popping into view of the next lower LOD (or visa versa) when moving to and fro in game.

I know, you will say, this will/might overburden the handling of such scenery objects in TANE, but as said, is it really? For example I tried some tests both in TS12 and in TANE when I had one my test routes some 30 different (not the same locos) LOD equipped locomotives running in view (on a marshaling yard with quite a few scenery objects there too) and I hardly seen, if any, performance degrading doing so. One the other hand, usually one has not more than 30 or so scenery objects (sometimes some of the same assets or even some other assets less than 500 polygons in size) in a given scene one can observe in game. Now LODs being mandatory on any asset having more than 500 "polygons", one needs to use LODs to get these into TANE but if these pop randomly with very noticeable ugly transition being an arbitrary/fixed distance presently, where this happens when not every next LOD can be reduced to a somewhat smoother looking transition. Isn't the visually aspect and look of TANE also important for this transition of LODs to be, as said, smoother, where the content creator can decide where this should happen in game (within reason of course) as some scenery models might be very elaborate and some are not. Plus one would not need an attached animation in the first place to be able to use an .lm text file for LODs if that was possible to do.

I hope this above makes sense and I would like to hear your opinion on this.

VinnyBarb

WindWalkr
June 1st, 2015, 08:43 PM
Chris, is there a compelling reason why on can not use .lm text files without an animation mesh attached to a SCENERY asset mesh for a better LOD handling in T:ANE?

In a single word: performance.

Everything we do in game development is a tradeoff between flexibility, performance, features, etc. LM deliberately trades performance overhead for better features. Specifically, it supports animation but incurs greater fixed overhead and loses support for mesh stitching.

If your asset requires animation, then you're already paying a performance penalty and you're already not eligible for stitching, so using LM format makes sense.

If your asset is a simple scenery object, then by switching to LM format, you're losing performance (potentially a lot of performance, depending on your exact usage) for no benefit.

Since animation carries an inevitable penalty, there's always going to be a question about whether a given object should be animated. Obviously the more animated objects in the scene, the more "alive" everything looks. On the face of it, this it what we're all about, trying to bring the world to life. However, we throw around a *lot* of objects in TANE. There's no way a current-gen GPU can keep up with animating each one individually. So instead of mass-applying animations to everything, we need to pick specific key objects which only occur once or a small number of times in the scene, and animate those.

I will add the caveat that there are techniques available that allow an entire family of objects to be animated simultaneously without paying most of the overhead. This could be used to animate grass blowing in the wind, for example. The techniques required here are very different to the animation that you're familiar with, and we do not currently expose anything that would allow you to do this. It's certainly something that we'd love to look into longer time, but it's a lot of work on both our side and for the content creators, so I'm not sure when we'll get to this.



Say, I make a very basic and simple animation mesh (as presently I can) in a xx.lm text equipped scenery item, hide this inside the asset and cull more or less animation and attachment cutoffs straightaway, just 1-2 meters away from the observer, so in theory they are from 3 meters onwards non existing and should not impose any extra load on performance. Am I right in this reasoning (not being a programmer myself)?

Yes and no. The attachment cutoffs are handled by the LM itself, so while you may be able to disappear the attachments and regain some of your performance that way, the LM still has to be processed by the CPU (checking whether it's still visible, checking whether it's changed enough that attachment points need to be shown or hidden) and potentially the GPU (assuming that there is geometry in the LM itself rather than just a zero-polygon mesh with attachment points.) And that's assuming that there's no animation in play, which adds a whole extra layer of work for both CPU and GPU (updating the timebase and skeleton, pushing that data to the GPU each frame, and then rendering it per-vertex.)


It's also worth observing that mesh-table LOD could theoretically be combined with LM to give the advantages of LM up close, and fall back to a simpler low-detail stitch-able variant once the mesh is some distance away. I say theoretically, because there are a number of compatibility features here which will probably get in your way, so it's likely that mesh stitching will not engage. This is again something that we could look at in the future, if we were willing to sacrifice some amount of compatibility.



.. some 30 different (not the same locos) LOD equipped locomotives running in view (on a marshaling yard with quite a few scenery objects there too) and I hardly seen, if any, performance degrading doing so.

You're off by many orders of magnitude. This is why we DO use LM for train cars- there are typically not very many of them on screen, and they're typically high-detail to start with so the overheads aren't making things that much worse. At the other end of the scale, we often have ten to one hundred thousand speedtrees in the scene. Static scenery falls somewhere in between, and exactly where it falls depends on the route. A speedtree-heavy route might get away with only a few thousand objects in the scene. A city scene, or a scene with scenery trees, may have many more scenery objects.

It's also fair to say "but there are only 30 of *my* object in the scene" and to some extent that's a valid argument. The place that this falls down is that every single content creator can use the same argument, and we end up with 20,000 poorly-performing objects in the scene. (It's very much like the "i'll just throw my litter out of the car window, what difference does one piece of plastic make?" argument..)


cheers,

chris

pcas1986
June 2nd, 2015, 12:08 AM
If I may a related question. What LOD scheme should attachments to a traincar use? I have a coupler that works with the mesh LOD structure (scenery style) but fails with the 500 poly error if it uses the LM.TXT. (We really give these techniques better names!)

T:ANE gripes if attachments do use LOD but I haven't seen any recommendations for methods for those attachments. I have a bell (U.S. style) to attach to a loco and I assume that will need to be the LM.TXT method.

WindWalkr
June 2nd, 2015, 01:34 AM
What LOD scheme should attachments to a traincar use?

This is very situational. You can't currently benefit from mesh stitching if you're attached to a traincar, so my generalised recommendations would be:

* Avoid attachments for low-detail (sub-500-polygon) meshes which can simply be built into the parent mesh.
* Try to use the attachment-point culling facilities in LM to get rid of physically-small-but-detailed meshes as quickly as possible.
* Your choice of LM or mesh-table LOD where the detail needs to be high enough that LOD is relevant.

hope that answers the question,

chris

pcas1986
June 2nd, 2015, 01:37 AM
...

hope that answers the question,

chris

Yes, it does. Thanks.

VinnyBarb
June 2nd, 2015, 04:09 AM
Chris, thank you for replying. I always understood about any animation attached to a scenery object and what perhaps this creates performance wise in game, unless of course such an animation is part of a real life copy of a content created scenery item like an animated windmill, airplane propeller, rotating radar dish, pumping jack pumping oil, turning wheel on a tractor etc.

Actually, I only want to talk here of only non animated scenery items like houses, warehouses, farms, railway stations, factories and the like. Not about trees, splines like bridges or tunnels and the like as these have somewhat different rules governing their LOD construction.

At the moment for scenery items larger than 500 polygons one can only use the current LOD reduction feature, where you, the coder, nominates in your code at what distance this LOD change can happen. With me so far? Now, if, lets say, one can create these same scenery items as I mentioned above with a LM text file type of LOD reduction, there should not be any difference as such, except the distance where the next lower LOD kicks in can be set/determined by the content creator. Surely a creator will not (must not) go overboard and nominate a somewhat ridiculous LOD change distance to satisfy his/her desire of a smooth LOD transition and should (must) stick to certain fair distance limits where these LOD changes happen.

In reality, it is most likely only LOD0 and LOD1 which are somehow "critical" for the distance where the LOD change might happen as any subsequent lesser LODs most likely will not be that important to see or to be noticed. So, in other words, either with the current LOD changing system, as this is now, compared to a (at the moment not feasible nor implementable) LM text file LOD system for these scenery items as outlined above, there should not be that much of an extra performance hit in TANE? Is it?

What are your thoughts on such a scenario if the LOD0 and LOD1 changing distances is reasonably configured in such a LM text file? As all other things should be more or less equal between these 2 different scenarios as outlined above. I also do not know at what viewing distance in game the LOD changes take place with the current LOD changing scheme. If these viewing distances at the moment are reasonable set in your code, perhaps a good content creator might even come very close to configure these same distances with a LM text file system and might even better that with the lesser LODs remaining, where this is not so noticeable.

One can even cull the rendering distance in such LM text file configured scenery item, I don't know if this is implemented in your code for the now in place LOD system or if indeed this is coded in game, where this render cull cutoff point is in the distance.

VinnyBarb

WindWalkr
June 2nd, 2015, 08:16 AM
At the moment for scenery items larger than 500 polygons one can only use the current LOD reduction feature, where you, the coder, nominates in your code at what distance this LOD change can happen.

To be completely fair, it's the end user who has final control over the LOD cut-offs. We just provide the minimum and maximum ranges and the relationships between each LOD.

This has been debated on the forums previously. It's not really a major sticking point for me. The current approach is simpler, but is not the "one true way" - it just offers a fairly decent solution. If someone wants to come up with an example asset (and is willing to take feedback on board and try to improve their model) then I'm more than willing to meet that half-way by offering advice, and if it turns out the the advice is not sufficient to reach our goals, then I'm happy to discuss config options which might rectify that.

We've done this twice before, and in each case it turned out that the model could be massively improved at the modelling level and thus didn't need us to make any changes, but I grant that two examples is not the same as "every mesh". That's part of the reason why I formed this group, so that you guys can find these kinds of examples.



In reality, it is most likely only LOD0 and LOD1 which are somehow "critical" for the distance where the LOD change might happen as any subsequent lesser LODs most likely will not be that important to see or to be noticed. So, in other words, either with the current LOD changing system, as this is now, compared to a (at the moment not feasible nor implementable) LM text file LOD system for these scenery items as outlined above, there should not be that much of an extra performance hit in TANE? Is it?

I don't really follow what you're saying here, but I hope that the following will answer you. Correct me if I'm misunderstanding:

* LM is naturally a heavy format, but that has little to do with the cost of selecting WHEN to change meshes.
* LM isn't suitable for regular scenery because of the associated costs.
* It would however be possible to give additional options in the mesh-LOD format.
* We would lose some options to optimise by doing this, but it's questionable how bad that is in practice. If we were careful, it would probably be ok.

chris

VinnyBarb
June 2nd, 2015, 06:24 PM
To be completely fair, it's the end user who has final control over the LOD cut-offs. We just provide the minimum and maximum ranges and the relationships between each LOD.

Thank you for replying. OK, HOW has the end user (meaning the content creator or do you mean the user of TANE have control?) what are these minimum and maximum ranges and are these ranges somewhere documented, to read up about where these are?


I don't really follow what you're saying here, but I hope that the following will answer you. Correct me if I'm misunderstanding:

* LM is naturally a heavy format, but that has little to do with the cost of selecting WHEN to change meshes.
* LM isn't suitable for regular scenery because of the associated costs.
* It would however be possible to give additional options in the mesh-LOD format.
* We would lose some options to optimise by doing this, but it's questionable how bad that is in practice. If we were careful, it would probably be ok.

chris

Quite simple really and I thought I explained this somewhat in the last post. Say a scenery object has 4 LODs, it is the LOD0 and LOD1 which is most critical and visually important when observing these in game and any radical "popping" (which happens frequently now) at any LOD change is there pronounced and very noticeable with these 2 LODs. LOD3 and LOD4 are much further back in the scenery and hence do not impact visually as much, if at all, if these 2 LODs change there with some "popping" when changing distance. Say one places a row of different houses and shops, gas station and what have you, along a street in the scenery and all of these change LODs at the same in game at the coded distance. How visually distracting would that be in game? Yet with an LM text file a creator can vary this distance change a bit and should make the LOD changes on different scenery objects on the same viewing level/distance, as is presently, more pleasant looking as these would not happen all at the same time. Isn't this what you as a programmer and I as a content creator should work towards the same goal, to make TANE a better looking and performing simulation?

Remember, a game/simulation is only as good as it visually is. Example, a game like Skyrim (as with quite a few other newer games) with the HD graphic improved pack, if visiting the city of Whitrun there or any of the other densely populated cities there, where in full view of the player one can see all the different exquisite and elaborate created houses, similar with market stalls, rotundas and parks, church(es), the fantastic looking castle there, with its different stone and block walls, many stairs all over and with animated flowing and running water, with little different animated waterfalls here and there, many, I say MANY different animated people like guards, traders, visitors and many different people living there all walking around and even some animals constantly walking about the landscape, ALL in view of the player. Some other places have exquisite and lovely looking animated windmills, different good looking castles, animated smitheries, light houses and ships, animated birds and animated dragons flying about and every now and then attacking realistically some township or scenery item with people and guards running from all over the place to fight and trying to kill at the same time such a dragon. Lots more included in the landscape, most of these can all be found in one scene, depending where the player is or is getting there, all viewed by the player.

Bethesta, its developer, has utilities available to get these 3DS Max meshes out of the game, modify these with one's own 3DS Max program and repack these into the game again. They call this "modding" the game. I examined most of the scenery item meshes in my 3DS Max and these are all somewhat very large polygon wise with their first LOD mesh. My reasoning is, if they can do wonderful work with so much going on in view of the player with all these heavy polygon intensive DIFFERENT objects interacting, many, I say MANY of these objects are animated and very good looking (meaning with very large polygon numbers in their first LOD mesh), why is N3V Games not even close like that visually with what they are doing and why does N3V Games place so many restrictions on LOD level changes? I know, not enough manpower etc. but one can still code a game efficiently.

If one were a cynic, one could perhaps reason, (perhaps rightfully so), is this where the problem with LOD meshes in TANE lies, one developer on one hand must be doing something right when LOD levels change in Skyrim despite all these heavy polygon meshes in game there, which does not bring the game to its knees and works wonderful there as it should. The other developer on the other hand might not be as astute to see how others are doing this presently and hence, content creators get penalized for this, where perhaps there might be not be as much needed as is now for this somewhat IMHO sudden "heavy handed" tightening of rules re content creating, especial with LODs.

I DO NOT mean this in an offensive way, it is my observations only when comparing 2 different games with heavy polygon sized meshes in each at a given time and how these 2 different games somehow perform differently. Skyrim (3 years old by now) installed is not much bigger than TANE in data size and would think, therefore would have as much or as little code data in its game code as probably TANE has. Why then the difference in the 2 games?


LM is naturally a heavy format, but that has little to do with the cost of selecting WHEN to change meshes.
LM isn't suitable for regular scenery because of the associated costs.

To me, these 2 quotes do not make sense if these are for NON animated scenery assets. Whether one uses the configuration text in the config.txt to bring different LOD meshes into view at (as you say above) where you provide the minimum and maximum ranges in game for this to happen and the other is where I could use a LM text file for the same (which presently is not possible). IMHO, a text entry is a text entry, be this in a config.txt for one or in a LM text file in the other and should not make much difference, if at all.

What changes for us content creators with a LM text file to configure LOD distances (which presently one can not use on non animated created scenery assets) is, now the content creator can somewhat dictate where this happens and where this looks reasonable in game. After all, there can/should not be a fixed distance of LOD changing (hence not a good solution) for every and ALL different created scenery objects with LOD. As scenery objects are indeed different in looks and in building these, hence different configurations distance wise will be needed for LOD changes. One simply can not apply a set rule for all non animated scenery objects. I would think a content creator will be wise enough not to go overboard with too far away LOD changes configured in any such LM text file (as said again, if this were possible at this point in time we are now).


LM isn't suitable for regular scenery because of the associated costs.

Can you please explain why LM text files might be not suitable for non animated scenery objects and what associated costs there might be. To me, apples still look like apples with either method, but then again, what do I know. Hence my asking.

VinnyBarb

WindWalkr
June 2nd, 2015, 07:14 PM
HOW has the end user (meaning the content creator or do you mean the user of TANE have control?) what are these minimum and maximum ranges and are these ranges somewhere documented, to read up about where these are?

I mean the user of T:ANE, through the performance sliders. It's not documented specifically because it can and will change. The recommendation for scenery content creators is to build with the detail set to the highest, but follow up with a sanity check of the other slider positions. Content should look its best in the "highest" position and should not attempt to look its best in other slider positions- but shouldn't fall apart completely, either :)



Say one places a row of different houses and shops, gas station and what have you, along a street in the scenery and all of these change LODs at the same in game at the coded distance. How visually distracting would that be in game?

Ideally, not at all. If we do our job right, LOD transitions are not particularly visible. We'll never hide this completely (at least, not without something like the fade that SpeedTree uses) but it should be subtle enough that you won't care about it unless you're looking for it.



..why is N3V Games not even close like that visually with what they are doing and why does N3V Games place so many restrictions on LOD level changes? I know, not enough manpower etc. but one can still code a game efficiently.

Our costs come from a few things:

* Our content is (on average) not very efficient. This isn't a dig at content creators, it's just the reality of using hobbiest-created content instead of paying people the big bucks. It's likely that we'll never get "professional-quality" efficiency across the board, though you do certainly see some great individual examples.

* Our scenes are built to mimic real life locations. In most games, the artists and programmers will attempt to build the best-looking scene they can within a given performance budget. Shortcuts are taken which don't detract from the "good looks" of the game, but which do deviate from reality in certain ways. A contrived example of this might be to place high walls around a busy street market- the market is high detail because the player can get up close, so to avoid killing the performance the walls block off visibility of the outside while you are in the market, and block off visibility of the market when you are outside. In real life, such walls might not be present, but since it's a made-up game environment, they don't look out of place. Another example is that a typical game may only have three types of tree repeated throughout an entire scene (or even through the entire game.) GPUs are very very good at pushing repeats of simple objects, so if this is your typical case, you can optimise it heavily.

* (Right at the moment in T:ANE) Inefficiencies on specific hardware variants. This is our problem and not yours, and is purely a matter of time/money. Unfortunately, back-end techniques that run beautifully on one set of hardware may struggle on another, due to differences in drivers or hardware tuning. We're aware of some specific cases where the engine performs poorly and are working on those. When those are resolved, we'll go looking for more and iterate- each pass through this will get us less gains as the low-hanging fruit are culled.

* Camera angle. If you're near to the ground (such as most first-person-shooters, most of the time), you can't see much. If we locked the player into the train cab, frame rate could be massively improved by making different assumptions about what they could see. Trainz has always been about flexibility, so we don't do this. Games like skyrim get their good looks by showing some really pretty stuff close to the camera, and forgetting about far-away detail for the most part. (Also, big teams and big budgets don't hurt..)

* Procedurals. In many games and tech demos, you'll see examples of massive forests and grass-covered plains. These can be rendered extremely quickly because rather than giving the content creator control over each clump of grass and each tree, they simply use "seed values" and a basic density map (or other similar techniques.) This is somewhere that we can take further in TANE, but it's also worth keeping in mind that we value flexibility over raw performance, so we will turn down some options here that other games may opt to take.






I DO NOT mean this in an offensive way, it is my observations only when comparing 2 different games with heavy polygon sized meshes in each at a given time and how these 2 different games somehow perform differently. Skyrim (3 years old by now) installed is not much bigger than TANE in data size and would think, therefore would have as much or as little code data in its game code as probably TANE has. Why then the difference in the 2 games?

It's not at all offensive. Skyrim had 100 people working on it. TANE had less than 20 (plus, it must be admitted, a lot of outside help. Thanks guys!) Our aim is to provide you with the technologies to make that kind of graphics possible, but we all know that it takes time for our content creators to learn how to get the most from each new addition to the tool set.

That said, as above, we're comparing apples with oranges. If you asked TANE to render what Skyrim typically renders, performance would likely be quite similar. We ask a lot more of our engine than you think.




To me, these 2 quotes do not make sense if these are for NON animated scenery assets. Whether one uses the configuration text in the config.txt to bring different LOD meshes into view at (as you say above) where you provide the minimum and maximum ranges in game for this to happen and the other is where I could use a LM text file for the same (which presently is not possible). IMHO, a text entry is a text entry, be this in a config.txt for one or in a LM text file in the other and should not make much difference, if at all.

You're looking at the very small part of the system visible on the surface and making broad assumptions about what goes on under the hood. That's a bit like saying that a ferrari has very similar controls to a toyota camry, so they should both get very similar performance and should be sold at the same price. In practice, one car is optimised for performance but is only accessible to specific people; the other is accessible to everyone but doesn't give the same level of performance. Neither car is "the wrong choice" but using the wrong car in the wrong situation isn't going to work well for you. Everyone would love to drive the ferrari all the time, but that's not really a practical option.


chris