PDA

View Full Version : Normals (Tangents?) issues on meshes in T:ANE



S301
June 1st, 2015, 07:34 AM
Hi All
I've noticed on a lot of content, including some built-in content, in T:ANE that there is a normals issue (It was mentioned to me by someone that it may be the tangents, rather than normals). It seems to be fairly random, even going through the middle of faces, causing strange specular effects and 'inverted' normals maps on the object.

An example from my current WIP model. You can see it quite clearly on the dome here: http://www.zecrail.com/files/tane/TANE%202015-05-31%2022-55-46-00.jpg

I recall seeing randomly inverted normals maps in TS12, however nothing quite to this degree.

Another example, which somewhat shows the inverted normals, can be seen here: http://www.zecrail.com/files/new-vr-steamer/TANE%202015-05-19%2022-38-11-13.jpg

Particularly across the rivets on the backhead, which become very flat where the normals are acting strangely.

Regards

WindWalkr
June 1st, 2015, 07:47 AM
Gui had a look into this last week. The initial results we got suggested two things:

1. The IM files in question have bad tangent data (or at least, the data is not in a format that we easily recognise.) This is probably due to a long-standing fault in the exporters, although we can't rule out other possibilities since it doesn't affect every case.
2. TS12 normal mapping makes a lot of simplifications for performance reasons (remembering that the hardware available when this stuff was written was very, very primitive) and as a result probably isn't affected by this issue.

Unfortunately we haven't come up with a solution as yet- this topic will definitely require more study. Assuming that the above two points turn out to be accurate, then the "obvious" short-term solution would be to have T:ANE ignore the tangent data in the mesh files, and build replacement (good) tangent data using the TS12 model as a workaround.

At this stage we've no reason to believe that this is an issue with the source models in Max, though we haven't pinpointed the location at which the bad data is entering the pipeline.

chris

WindWalkr
June 1st, 2015, 07:54 AM
Speaking of which, if there are any accomplished programmers out there who have access to the Max APIs and feel like tackling the max exporters, please do get in touch.. Our in-house efforts in this direction are sporadic at best, due to the amount of effort required to get up to speed on the API changes for what amounts to a relatively small update on an all-to-regular basis.

chris

S301
June 1st, 2015, 08:53 AM
Thanks Chris, hopefully a solution of some sort can be found, as a lot of content seems to be affected by this.

Zec

whitepass
June 1st, 2015, 09:00 AM
I have had something similar and it is do to the "set smooth" in Blender and only shows on complex curves, cylinders and spheres do not show it.

VinnyBarb
June 1st, 2015, 03:59 PM
I had a similar issue with 3DS Max as recently as yesterday when exporting to TANE 75947, where polygons on a submesh were reversed in TANE and showed transparent on the side they were texture mapped on in 3DS Max. The other side (the wrong side) was texture mapped in TANE. Yet in 3DS Max, when checking where the texture was on (ready to flip the polygons if ...), normals and even resetting XForm showed to be on the correct side of the submesh. I rebuild that same submesh from the ground up and the problem was gone. No smoothing on that submesh. Strange? Possibly related to this?

VinnyBarb

pcas1986
June 2nd, 2015, 12:21 AM
Speaking of which, if there are any accomplished programmers out there who have access to the Max APIs and feel like tackling the max exporters, please do get in touch.. Our in-house efforts in this direction are sporadic at best, due to the amount of effort required to get up to speed on the API changes for what amounts to a relatively small update on an all-to-regular basis.

chris

I don't use Max but I would be interested in the XML importer (TrainzMeshImporter.exe) if there were some value in it. For example, unused data fields in the IM and KIN data formats or, potentially, additional capability. Are the IM and KIN formats proprietary - i.e. unique to Trainz?

WindWalkr
June 2nd, 2015, 01:38 AM
I don't use Max but I would be interested in the XML importer (TrainzMeshImporter.exe) if there were some value in it. For example, unused data fields in the IM and KIN data formats or, potentially, additional capability. Are the IM and KIN formats proprietary - i.e. unique to Trainz?

Okay. I'll keep this in mind, thanks. Can't promise immediate action in this direction but I'd like to talk through what is possible and hopefully move in the direction that we can all agree has the most merit. I'm not really sure what you mean by "unused data fields"- pretty much anything you can do with these formats is already exposed to the user in some fashion.

The formats are proprietary but they're not secrets. They're also not a great match for how TANE handles things internally- there is some conversion penalty while loading, enough so that there's a consideration of moving away from these older formats eventually but not so much that we're in a hurry to push that through.

chris

pcas1986
June 2nd, 2015, 01:52 AM
... I'm not really sure what you mean by "unused data fields"- pretty much anything you can do with these formats is already exposed to the user in some fashion.


That answers the question.




The formats are proprietary but they're not secrets. They're also not a great match for how TANE handles things internally- there is some conversion penalty while loading, enough so that there's a consideration of moving away from these older formats eventually but not so much that we're in a hurry to push that through.

chris
I imagine that would be very difficult to manage unless CM, or whatever validates those files, can readily recognise a different data structure.

My interest is primarily in Blender but most of the recent advances in Blender have been for cycles rendering that, AFAIK, has no value for Trainz.

If you are willing to release the IM and KIN data structure formats, even with an NDA, I'd be willing to at least see if I can replicate what TrainzMeshImporter already does. If you don't want to at this point, or ever, then that's fine with me. I have many projects in mind for T:ANE anyway. :D

RPearson
June 2nd, 2015, 02:46 AM
Paul, the basic info on the .im and .kin file formats can be found in the old Jet documentation that Auran provided with the jet demo packages years ago. I'm sure it has been modified over the years but I was able to use it to read and write the .im files successfully in recent Trainz versions. I was only interested in making changes to the attachment point section which was straight forward and worked with TS12. Sort of like what PEV's utilities do but just working with the attachments section of the file. I don't know if N3V even offer the old or updated packages anymore. See what Chris has to say - I think I still have copies but probably can't redistribute them without permission.

I'm not so sure you'd really need them though as you would only be writing a program to provide input to N3V's xml importer. Blender's exporter was written in Python and writes out the model info/data as xml file. Blender has built-in Python language support that made it the language of choice for the exporter.

Of course if Chris was speaking of maintaining the max exporter that writes .im and .kin files directly that would be a slightly different effort.

Bob Pearson

pcas1986
June 2nd, 2015, 03:10 AM
Thanks Bob,
I'll look through my old Trainz files for that information.

I've never looked at the Max importers but I've assumed they call Max APIs to create the IM, KIN and texture.txt files directly. The Blender process is the export to XML, using Torsten's Python script, which, in turn, calls TrainzMeshImporter directly. I have some experience with the exporter and have modified it on occasion for my own purposes. The current Blender exporter is up to date and only needs changing if the XML format changes or some radical change occurs in the Blender internal format. There was such an occasion a couple of years back when they moved to ngon polys.

My interest is in TrainzMeshImporter and a revision of the XML format if there is new or additional information that T:ANE can use.

It seems a bit odd to me that a system can still be using the same data format after 10+ years. :)

Cheers

WindWalkr
June 2nd, 2015, 03:25 AM
It seems a bit odd to me that a system can still be using the same data format after 10+ years.

Things don't change much when you're talking about primitive data types, and the system was written to be fairly future-proof. That said, we haven't really changed the requirements much either. The IM files *have* changed a little over the years, to include more precached data, clarify issues with material sharing, add support for different animation techniques, etc. But the XML format was written after most of that anyway.

chris

andi06
June 3rd, 2015, 01:16 AM
Gui had a look into this last week. The initial results we got suggested two things:

1. The IM files in question have bad tangent data (or at least, the data is not in a format that we easily recognise.) This is probably due to a long-standing fault in the exporters, although we can't rule out other possibilities since it doesn't affect every case.
2. TS12 normal mapping makes a lot of simplifications for performance reasons (remembering that the hardware available when this stuff was written was very, very primitive) and as a result probably isn't affected by this issue.
I don't know if its the same issue or whether it helps but:

Every wingrail, checkrail and blade mesh that I've written exhibits incorrect lighting and issues with normals, similar issues are present in your own procedural-track assets. None of the other meshes are ever affected.

Paul Cass is creating procedural-tracks in Blender, if he is seeing the same thing that might at least rule out Max.

pcas1986
June 3rd, 2015, 03:16 AM
I don't know if its the same issue or whether it helps but:

Every wingrail, checkrail and blade mesh that I've written exhibits incorrect lighting and issues with normals, similar issues are present in your own procedural-track assets. None of the other meshes are ever affected.

Paul Cass is creating procedural-tracks in Blender, if he is seeing the same thing that might at least rule out Max.

I'm not seeing any real problems with normals in my test track but I haven't put much effort into the textures yet.

But I did try a modified version of Torsten's Blender normal map regression test asset and this is how it looks in T:ANE. It looks fine to me.

http://i1188.photobucket.com/albums/z410/casper131/Ashampoo_Snap_2015.06.03_18h09m53s_002__zpsv7hjdbg w.jpg

martinvk
June 3rd, 2015, 07:45 PM
I don't know if its the same issue or whether it helps but:

Every wingrail, checkrail and blade mesh that I've written exhibits incorrect lighting and issues with normals, similar issues are present in your own procedural-track assets. None of the other meshes are ever affected.

....I thought I was not mapping something right when those kept a shiny edge. I solved it by reducing the Specular level down to single digits. Also imported my old test sphere to validate the normals and it still looks OK.
580

ryanstrains
June 4th, 2015, 12:30 AM
http://hostthenpost.com/uploads/a0dbe923595da357d00fa1343ff83804.png

I have the same issue when bump maps are enabled, so I think it is related to the exporter issue that was supposed to be fixed for quite some time now.

norfolksouthern37
June 5th, 2015, 12:56 AM
This issue has been ongoing for a few years and is not only happening in TANE. I have had several different instances of talks with N3V and sent a couple of example assets to the trainzdev email only to have the replies dry up as as soon as I do. In some asset example as above try cutting a hole in that box - you will probably then notice the incorrect rendering of the normal map across a polygon edge. In the past this was countered by dividing the face again or until the 'wrinkle' was unnoticeable. In TANE the problem shows on different faces than in TS12 but nevertheless is there and in some cases a lot worse. Some models that did not display the problem in TS12 do so now in TANE.

pcas1986
June 5th, 2015, 01:31 AM
This problem also shows in one of my locos in TANE (76536 & 76401) but not in TS12. It occurs on the left side of the boiler but not on the right.

http://i1188.photobucket.com/albums/z410/casper131/Ashampoo_Snap_2015.06.05_15h27m45s_002__zpsxsisj64 2.jpg

http://i1188.photobucket.com/albums/z410/casper131/Ashampoo_Snap_2015.06.05_15h44m15s_003__zps0n3qfxt h.jpg

My models are made in Blender so if it is an exporter problem then it is common to both Max and the XML versions.


Note that the stretched texture that is evident along the top of the boiler is a mapping problem and my error.