Holiday Special

Packt Publishing is offering a holiday special of $5 off all of their eBooks which include well over

1,000 technical titles.

"Learning Geospatial Analysis with Python" is here.

Below are all of the titles at the moment that have geospatial-related content.  More geospatial titles are on the way for 2104 including the little gem on the right :-)

1. Learning Geospatial Analysis with Python
2. Python Geospatial Development
3. Programming ArcGIS 10.1 Cookbook
4. Applying and Extending Oracle Spatial
5. Learning Bing Maps API
6. GeoServer Beginner’s Guide 
7. Google Maps JavaScript API Cookbook
8. Python Data Visualization Cookbook
9. OpenStreetMap
10. PostGIS Cookbook  
11. OpenLayers 2.10 Beginner's Guide
12. Learning QGIS 2.0
13. Learning QGIS - Second Edition
14. GeoServer Beginner’s Guide
15. Leaflet.js Essentials

GPX2SHP

Here's a nice Garmin GPX to shapefile converter using PyShp by Matt Rantala from his blog MapRantala.com:

http://www.maprantala.com/2014/05/16/garmin-gpx-to-shapefile-shp-conversion-gpx2shp-py/

Photo: FishStickTheatre.com

WKT EPSG Strings Made Easy

This quick post is a follow on to my posts on map projections and creating .prj files for shapefiles.

 The .prj file format, created by Esri originally but now recognized by most geospatial software, is just a text file with a WKT string for the projection information.  Most of projections can be referenced by a simple EPSG code which is just a number.  Usually you'll have this number handy and need to look up the full WKT string on SpatialReference.org.

Dr. David Forrest, a research scientist at the Virginia Institute of Marine Sciences, sent me a simple function to make fetching WKT projection strings easy:

def getPRJwkt(epsg):
   """
   Grab an WKT version of an EPSG code
   usage getPRJwkt(4326)

   This makes use of links like 
   http://spatialreference.org/ref/epsg/4326/prettywkt/
   """
   
   import urllib
   sr = "http://spatialreference.org"
   f=urllib.urlopen(sr + "/ref/epsg/{0}/prettywkt/".format(epsg))
   return (f.read())

So then getting a WKT projection string is as easy as:

>>> wkt = getPRJwkt(4326)
>>> print wkt
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.01745329251994328,
        AUTHORITY["EPSG","9122"]],
    AUTHORITY["EPSG","4326"]]

There's another, newer website called epsg.io that has some additional features beyond spatialreference.org.  The differences are in the FAQ on this page.

Errata

In Chapter 8 of my book, the shapefile doesn't have a .prj file which isn't a big deal for the example

but can be annoying if you try to open it in QGIS or ArcGIS.  There was also an unused numpy array in the example so I updated the whole package.  You can download the updated script and data here:
https://geospatialpython.googlecode.com/svn/NDVI-update.zip

This example is the very first example in the chapter.  The second example shows you how to classify the output data to create the image shown here.

"Learning Geospatial Analysis with Python" Cyber Monday Sale

The print edition of Learning Geospatial Analysis with Python qualifies for a 30% discount on Amazon.com until midnight tonight using discount code HOLIDAY30 at checkout.  This is for the print edition only which is black and white as are most programming books as opposed to the eBook which is full color.  Very soon I will make an image catalog available for download for people with the print edition.  This detail isn't critical as most of the illustrations are not color dependent with the exception of a 1 or 2 remote sensing examples where color is helpful.  The image catalog will give you the best of both worlds and with a 30% savings to boot!

 

Merging Shapefiles with PyShp and Dbfpy

A question on GIS StackExchange prompted me to write a quick script as an update to my previous merging shapefiles with pyshp. The SE question was the result of a typo but I thought this script was still useful based on other questions I've received regarding dbf files and pyshp.

post on

The shp and shx portions of pyshp are complete in that they can read and write all shapefiles allowed by the shapefile spec created by Esri.  The dbf format is included in the Esri shapefile spec by reference.  The dbf format is decades old and has several different versions.  Most people reference the generic XBase version of the spec.

Since the dbf format was already established, I grabbed the easiest code sample I could find written by Raymond Hettinger that is a reader and writer in pure Python.  This recipe is very useful and works 85% of the time with most shapefiles.  Many people run into issues though with dbf files produced by other software.

I have found that the pure-python, public-domain dbfpy module is far more robust than the simple dbf engine in pyshp.  So when people run into troubles with dbf files, I usually suggest using pyshp to read/write the shp and shx files and then dbfpy to handle the dbf file.  PyShp let's you work with each shapefile type independently to make this approach possible. 

The following code sample demonstrates merging all shapefiles in a directory into one shapefile called "merged".  Of course the shapefiles must all have the same geometry type, spatial reference system, and dbf schema.  The shp files are read by pyshp, and then the merged shp and shx file are created by pyshp.  Then the same is done for the dbf files separately by dbfpy.

import glob
import shapefile
from dbfpy import dbf
shp_files = glob.glob("*.shp")
w = shapefile.Writer()
# Loop through ONLY the shp files and copy their shapes
# to a writer object. We avoid opening the dbf files
# to prevent any field-parsing errors.
for f in shp_files:
    print "Shp: %s" % f
    shpf = open(f, "rb")
    r = shapefile.Reader(shp=shpf)
    w._shapes.extend(r.shapes())
    print "Num. shapes: %s" % len(w._shapes)
    shpf.close()
# Save only the shp and shx index file to the new
# merged shapefile.
w.saveShp("merged.shp")
w.saveShx("merged.shx")
# Now we come back with dbfpy and merge the dbf files
dbf_files = glob.glob("*.dbf")
# Use the first dbf file as a template
template = dbf_files.pop(0)
merged_dbf_name = "merged.dbf"
# Copy the entire template dbf file to the merged file
merged_dbf = open(merged_dbf_name, "wb")
temp = open(template, "rb")
merged_dbf.write(temp.read())
merged_dbf.close()
temp.close()
# Now read each record from teh remaining dbf files
# and use the contents to create a new record in
# the merged dbf file. 
db = dbf.Dbf(merged_dbf_name)
for f in dbf_files:
    print "Dbf: %s" % f
    dba = dbf.Dbf(f)
    for rec in dba:
        db_rec = db.newRecord()
        for k,v in rec.asDict().items():
            db_rec[k] = v
        db_rec.store()
db.close()

BOGO Offer to Celebrate 2000th Book Title from Packt

Packt Publishing is celebrating their 2000th book title, "Learning Dart" of all things, with a buy-one-

get-one offer on all 2000 books. You can access the limited-time offer on Packt's site here: bit.ly/1j26nPN

All 2000 books are available in print or multiple eBook flavors.  Don't forget to check your eReader email account to see if you got an eBook credit from the multi-million anti-trust settlement against the 5 major book publishers for price-fixing eBooks.

The following list of Amazon links covers all of the geospatial books in Packt's library which is the largest collection by any one publisher as far as I know. 

Note these links are from my Amazon store.  To take advantage of the Packt offer you must got to Packt's site.

I've also included a couple of titles that have significant geospatial chapters even if their title doesn't say it explicitly.

1. Learning Geospatial Analysis with Python
2. Python Geospatial Development
3. Programming ArcGIS 10.1 Cookbook
4. Applying and Extending Oracle Spatial
5. Learning Bing Maps API
6. GeoServer Beginner’s Guide 
7. Google Maps JavaScript API Cookbook
8. Python Data Visualization Cookbook
9. OpenStreetMap
10. PostGIS Cookbook  
11. OpenLayers 2.10 Beginner's Guide
12. Learning QGIS 2.0

If I missed any titles please post them in the comments.