A DEM can be off by +20'/-20' in all the x-y-z locations (with river channels up on hillsides, and track in river channels). Tall trees and buildings throw the DEM creation off.
As a general statement this is not quite correct.
It all depends on the DEM source.
Orbital DEMs such as SRTM or ASTER are most likely subject to vertical error. And here indeed forests or built-up areas will influence the DEM. Horizontal error had been noticed in the first generation ASTER DEMs 10 years ago, not to be confused with the post-processed ASTER GDEM of today. Manually created DEMs, like Viewfinder-Panoramas, may also be slightly off in horizontal direction. Rivers flowing uphill is a different problem. The once popular 1:160,000 TIGER data may not be accurate enough to match the DEM. That's a map problem, not a DEM issue.
Terrestrial DEMs such as USGS NED will have much less vertical error than orbital DEMs. They will generally model the land and water surface of the earth, including earthworks, but excluding most civil engineering structures, and totally ignore vegetation. Higher resolution terrestrial DEMs may indicate railway cuttings and embankments, as Dermmy has already pointed out. Particularly NED 1/3 arc sec (and 1/9 arc sec where available) can be very detailed here. However, not all 1/3 arc sec NED DEMs already provide this accuracy.
Horizontal accuracy of a DEM depends on horizontal resolution. 1 arc sec, for instance, translates to 30 x 20 m for moderate latitudes. And that's the horizontal accuracy you can get from that DEM. There is a DEM point every 20 or 30 m. In-between points will be interpolated.
Higher resolution DEMs also require better than 1 m vertical resolution. Otherwise you will see a terracing effect. The DEM file format plays a role here, too. Only some file formats allow floating point notation. The remaining formats for NED DEMs, obtained via the National Map Viewer, are now floating point only, but older NED DEMs floating around may still have integer elevation.
Here is a typical example of 1/3 NED arc sec in Trainz Surveyor:
DEM source view as a shaded relief:
PS: Do not confuse "DEM" with "DEM-based Trainz route". Some Trainz users mix the two terms. The transfer mechanism from DEM to Trainz may introduce additional error or overly smooth the terrain, yielding a much more levelled landscape in Trainz compared to the one in the original DEM.