Pages

Saturday, August 20, 2011

Create a Zipped Shapefile

Shapefiles consist of at least three files. So zipping up these files is a a common means of moving them around - especially in web applications. You can use PyShp and Python's zipfile module to create a zipped shapefile without ever saving the shapefile to disk (or the zip file for that matter).

Python's "zipfile" module allows you to write files straight from buffer objects including python's StringIO or cStringIO modules. For web applications where you will return the zipped shapefile as part of an http response, you can write the zip file itself to a file-like object without writing it to disk. In this post, the example writes the zip file to disk.

In Python, file-like objects provide a powerful way to re-route complex data structures from the disk to other targets such as a database, memory data structures, or serialized objects. In most other programming languages file-like objects are called "streams" and work in similar fashion. So this post also demonstrates writing shapefiles to file-like objects using a zip file as a target.

Normally when you save a shapefile you call the writer.save method which writes three files to disk. To use file-like objects you call separate save methods for each file: writer.saveShp, writer.saveShx, and writer.saveDbf.

import zipfile
import StringIO
import shapefile

# Set up buffers for saving
shp = StringIO.StringIO()
shx = StringIO.StringIO()
dbf = StringIO.StringIO()

# Make a point shapefile
w = shapefile.Writer(shapefile.POINT)
w.point(90.3, 30)
w.point(92, 40)
w.point(-122.4, 30)
w.point(-90, 35.1)
w.field('FIRST_FLD')
w.field('SECOND_FLD','C','40')
w.record('First','Point')
w.record('Second','Point')
w.record('Third','Point')
w.record('Fourth','Point')

# Save shapefile components to buffers
w.saveShp(shp)
w.saveShx(shx)
w.saveDbf(dbf)

# Save shapefile buffers to zip file 
# Note: zlib must be available for
# ZIP_DEFLATED to compress.  Otherwise
# just use ZIP_STORED.
z = zipfile.ZipFile("myshape.zip", "w", zipfile.ZIP_DEFLATED)
z.writestr("myshape.shp", shp.getvalue())
z.writestr("myshape.shx", shx.getvalue())
z.writestr("myshape.dbf", dbf.getvalue())
z.close()

If you've been using PyShp for awhile make sure you have the latest version. The file-like object save feature was uploaded to the PyShp subversion repository on Aug. 20, 2011 at revision 30.

You can download PyShp here.

You download the sample script above here.

5 comments:

  1. I've got an unstructured model grid with 500,000 triangles. Do I have to loop over all the triangles creating a record and shape for one, or is there a better way? I've got numpy arrays containing "lon", "lat", the triangle connectivity (nnodes,3), and data values at the nodes that I'd like to write as attributes (records).

    ReplyDelete
  2. Darn, I meant to say:
    lon(nodes),lat(nodes),data(nodes),triangle_connectivity(nelements,3)

    ReplyDelete
  3. Rich,

    Is this an ADCIRC grid per chance? What shapefile type are you trying to output?

    BTW - you can email me directly at jlawhead geospatialpython com

    - Joel

    ReplyDelete
  4. hi, i am vry new to GIS programming with python. I have the following problems,pls kindly advice

    the script goes as thus
    import osgeo.ogr
    shapefile = osgeo.ogr.Open("tl_2009_us_state.shp")
    numLayers = shapefile.GetLayerCount()
    Traceback (most recent call last):
    File "", line 1, in
    numLayers = datasource.GetLayerCount()
    AttributeError: 'NoneType' object has no attribute 'GetLayerCount'.

    my scripts also generate same error message whenever i try to extract features from shapefiles. I also have problem with 'importing shapefile module..eventhough the pyshp is in my 'site - package' folder i get the following

    import shapefile
    Traceback (most recent call last):
    File "", line 1, in
    import shapefile
    ImportError: No module named shapefile

    ReplyDelete
  5. Hi Joel,
    Please help on my problem as I'm very new to Python so couldn't find a way out.
    I want to extract a zip file containing shape files in memory and then find the shape of it. I am using/ prefer only arcpy module.
    Thanks in advance.

    ReplyDelete