Parallax mapping deals poorly with scenarios where the surface normal is changing rapidly and there is insufficient geometric detail to faithfully represent that change. (You'll also find that other parts of the rendering pipeline have issues in these conditions, but the parallax distortions are certainly an extreme case.) The "swimming" is an expected outcome in such a scenario, when viewed from certain camera angles. There are a variety of techniques which can be used to mitigate the outcome:
* Avoid such formations in the first place. This is the correct approach when you're making a model, but you're obviously more limited when it comes to a fixed-size terrain mesh. Certainly to some extent the terrain curvature is up to the artistic sensibilities of the route builder, but there are also going to be times where the desire to have sharp angles is going to be more important.
* Avoid using a parallax texture on the problematic terrain area. This doesn't prevent you from using parallax textures elsewhere in the scene, only directly on top of the problem geometry.
* Hide the problem area from the most problematic camera angles. Depending on the scenario, rocks, cliff splines, turffx, speedtrees, etc. can all be used to help hide the extreme edge cases.
* Live with it. This is the simple approach and is practical in many minor cases. Obviously this isn't a good idea for extreme cases which are really common and really visible, but for cases which only occur if you're standing in the middle of a steep gulley under a waterfall kilometres away from the track, or which only produce minor problems and when viewed with the camera pressed against the ground - it's quite a reasonable approach to take.
hth,
chris
