USGS "Historical' topo maps

[geophil]

Here is a paper on the history of the polyconic projection: Mark Monmonier - Practical and Emblematic Roles of the American Polyconic Projection.

From the early beginnings:

“In each of these sheets, it was intended to bring the
results of several parallels, so that the central meridian
alone should become a straight line, and all the other
meridians and parallels broken lines, nearest the curve,
to which they belong; the angular points of the trape-
zium being transferred to paper by their rectangular
ordinates, from the middle right angle, calculated from
the angle at the center of the projection, in the pro-
tracted axis of the earth” (HASSLER 1825, p. 407)

And still in use in the 1970s:

“Hydrographic survey data shall be plotted either on
the modified transverse Mercator or on a polyconic
projection. At the relatively large scales required for
hydrographic surveys, the two projections are essen-
tially equivalent and may be used interchangeably for
comparisons and transfer of hydrographic data” (UM-
BACH 1976, p. 1-6).

The paper also mentions field plots, where projection did not matter much at all, because of the large scale.

Simply put, the dis-
tortion associated with earth curvature can be far less
significant than the combined errors associated with
paper shrinkage, survey instrumentation, and manual
plotting.

It reminds me a lot of the history of the Cassini/Soldner projection officially in use in Germany between the 1880s and late 1920s. While there are significant differences - Cassini/Soldner was not used as a local projection, but as one with a relatively large number of regional grid systems with central reference points - bureaucratic inertia played a role here, too. For ease of use the Cassini/Soldner system of Berlin (federal state) survived even until 2010.

 

[steamboateng]

I've generated a 60 mile route into the very lumpy and bumpy Berkshire Hills of western Massachusetts; and used these georeferenced maps as an overlay. There appears to be little contradiction between the DEM and the maps. The only significant differences I find are generally along the courses of the Deerfield and Connecticut Rivers where flooding and washouts have altered the terrain over the 70 or so years since the maps were first made. Of course, the expansion of towns along the route have caused some discrepencies as well as small changes in the railroad right-of-way, over the years. All things considered, it seems that Grandpa and the USGS has defined Mother's form quite well.

 

[geophil]

I've generated a 60 mile route into the very lumpy and bumpy Berkshire Hills of western Massachusetts; and used these georeferenced maps as an overlay. There appears to be little contradiction between the DEM and the maps. The only significant differences I find are generally along the courses of the Deerfield and Connecticut Rivers where flooding and washouts have altered the terrain over the 70 or so years since the maps were first made.

Yes, the foundations of accurate cartography were laid many decades ago. For the 7.5' and 15' quadrangles the methods of production may have changed significantly, but they look very similar, comparing a 2010 with a 1950 edition (picking some uninhabited area where man-made topography hasn't changed).

It also appears that the map frame on a current quadrangle sheet covers exactly the same area (pixels/square millimeters) as its predecessors, with the only difference that the modern map is NAD83 and the older one NAD27, which explains the half-centimeter shifting of the map frame on the background. (Well, it should, since scale is identical.) There are differences, though. You can stitch all the UTM/NAD83 maps together seamlessly (within one UTM zone) and would still have a plane surface, but you would not be able to that with the historical maps and their polyconic projection. This is because the polyconic "zones" are not 6 degrees longitude as with UTM but only 7.5 minutes.

A few notes on the the current edition of the USGS topo maps:

I managed to get them georeferenced automatically last night, after some modification of my world file processing. There are two problems left. One is the ortho-image which comes for free with these maps. For Trainz ground textures I would want to switch it off. It's not possible with GDAL 1.9. GDAL 1.10, the next version, will support layer selection, but this new version is still in development. The other problem is that the modern maps don't show any railroads! These maps are actually vector maps, and data for the railroad layer was not available, when production started in 2009. Same for power lines and some other transportation features. There is a recent announcement that railroads will finally be added this spring.

So we probably better stick with the historical maps for a while, even when building a modern route.

 

[steamboateng]

I doubt there will be much difference in right-of way from 1990 maps; perhaps a few new/abandoned yards or facilities here and there or extended metro service. The intermodel boom of the 70's and 80's pretty much fixed the routes up to that point.
Intermodel was great for me; I sailed often to Bremerhaven with Sealand (CSX,Nedlloyd). Even got to freeze my butt off there one winter with Military Sealift Command. Good beer!

 

[geophil]

After more careful thinking I can imagine a way to open the GeoPDF in TransDEM itself, at least it will appear like that.

The method would still run the gdal_translate utility, but TransDEM can do that as well, no need for the user to execute this first step separately. And I wouldn't breach the license. GDAL with pdf support is linked against Poppler, the PDF access library. Poppler is GPL, making the whole GDAL build also GPL, unsuitable for TransDEM. But everything will be fine with GDAL/Poppler being called as an external utility.

gdal_translate will produce the jpeg image, the world file (projection coordinates for the image), and the GDAL auxiliary xml file, with the coordinate systems parameters, including the central meridian of the polyconic projection of the particular map sheet. TransDEM will then read and process these three files with its native capabilities.

The user would need to download and install a GDAL binary package and tell TransDEM where to find it, a one time step. No extra opening of a console window, no worries about command line syntax.

 

[steamboateng]

Exersized my Google finger a bit and looked up GDAL. I see it's an open source (that means free, to me) raster library for converting data into a variety of formats.
I think you're going to have to provide links to where we can download this exotic beast (for Windows).

 

[geophil]

Exersized my Google finger a bit and looked up GDAL. I see it's an open source (that means free, to me) raster library for converting data into a variety of formats.

That is correct. It is a very powerful collection and USGS uses it internally. "Free" has a bit of small print, though, but that is irrelevant for the end user.

I think you're going to have to provide links to where we can download this exotic beast (for Windows).

Here is the download page for Windows: http://www.gisinternals.com/sdk/

There are a many packages, versions and builds. To test it, the stable version 1.9.2 will do, but TransDEM will require 1.10. We only need the core components and, fortunately, there is an installer.

For 1.9.2 (stable):
release-1600-gdal-1-9-2-mapserver-6-2-0
Download: gdal-19-1600-core.msi

For 1.10 (beta):
release-1600-gdal-mapserver.zip
Download: gdal-110dev-1600-core.msi

 

[steamboateng]

Thanks for the references, Roland. I have downloaded both files you pointed to and have them safely tucked away.
I have no idea what to do with them!
Busy with other stuff now, anyhow; Blender and Gimp being the culprits. I am determined to get sumthin' up on the Dls before long. Also US Internal Revenue Service whispering in my ear to get Tax forms in if I want my $$ back! (I hate this time of year.......what a pia!)
I'll catch you later.
regards

 

[geophil]

Thanks for the references, Roland. I have downloaded both files you pointed to and have them safely tucked away.
I have no idea what to do with them!

Hopefully, the only thing you'll need to do is to install the 1.10 package and let TransDEM know where to find it.

The GDAL binary package consists of several utility programs which are run from the console. The most important one is gdal_translate.exe, which is able to extract both the image and the georeferencing info. The plan is that TransDEM will handle this for you.

 

[steamboateng]

I have TransDEM v.2.3.1.2. Will that do it or is a update required?
Got thru those tax forms..............Found out I can buy a new video card, now.........well actually later......
Still stumped on Blender textures............

 

[geophil]

The GeoPDF stuff is all new functionality, still in development, and will end up in TransDEM version 2.4. The release will also include ERDAS Imagine format for NED DEMs. (The usual free upgrade policy will apply.)

 

[steamboateng]

Will look for your latest release.
Thanks again for staying on top of this.

 

[geophil]

It's your fault anyway. :wave:

'cause it was you who raised the topic made me curious.

 

[steamboateng]

You will make a lot of TransDEM users Happy......er.

 

[KenRuof]

Hmm... sounds like a useful feature... I'll be waiting for the upgrade. Thanks Geophil for your fine work.

 

[geophil]

The GDAL link for importing GeoPDF is making progress.

gdalinfo.exe is used to retrieve image size and potential layers (modern maps only).

Layers and size are shown in this dialog:

The user can select/deselect layers and adjust size.

With the new parameters from this dialog, TransDEM will create the command line for gdal_translate.exe and run this utility as a sub-process (just as it did with gdalinfo.exe before):

"C:\Program Files (x86)\GDAL\gdal_translate.exe" -of PNG -co worldfile=yes 
"F:\Data\DigitalMapping\TransDEM\geopdf\OK_Oklahoma_City_20121211_TM_geo.pdf" 
"F:\Users\Roland\AppData\Local\TransDEM\temp\OK_Oklahoma_City_20121211_TM_geo.png" 
--config GDAL_PDF_DPI 131.867 --config GDAL_PDF_LAYERS_OFF "Map_Collar,Images"

This will produce the image file, the world file, and the GDAL .aux.xml file. TransDEM will then process the image file and its accompanying helpers.

This entire chain now works but will need a bit of fine tuning, like visualizing the gdal_translate progress report.

 

[steamboateng]

Thanks for th update, Roland.

 

[geophil]

Another lab report:

Steamboateng came up with an excellent idea yesterday.

One of the TransDEM features is the "Transparent Margins" function. It first makes the map collar transparent and will then reduce the map image to the smallest rectangle encompassing all non-transparent pixels. On a raster image, which is all pixels, we can't guess size and position of the map frame and would need sophisticated image processing to find it.

Therefore, the user has to adjust the mask manually to match the map frame. In the case of USGS GeoPDF topo maps, however, this can be automated.

The USGS GeoPDF files come with a nice meta data feature, the so-called "neatline", which tells us the border of the actual map frame in the form of a polygon.

The next version of TransDEM will extract the neatline from the GeoPDF meta data, eliminate any collinear points to make it a quadrangle and initialize the transparent margin mask with its values (shown in dark yellow):



At a closer look we find that the mask fits the map frame precisely (semi-transparent dark yellow overlay):



The final result, converted to UTM/WGS84, no longer has any white space around the net content. Transparent areas appear in black here.




Unfortunately this does not work quite as well with the modern maps. It looks okay at first glance:



But there is some white space between the neatline and the map frame for some unknown reason.



I'm afraid we have to live with this.

 

[Pencil42]

Very nice work!

Curtis

 

[steamboateng]

Geophil has once more played his magic and found a practical solution to improve his already excellent program.
This modification will allow USGS older' historical' maps to be almost totally 'automated' in the application of georeferencing and borderless DEM overlays.


Bravo, Maestro!