Pages

Tuesday, February 28, 2012

Pyshp shapeRecords() Method

The shapefile.Reader.shapeRecords()
method lets you juggle both the
geometry and dbf attributes at the
same time.
The shapefile.Reader.shapeRecords() method is a simple convenience method which allows you to simultaneously loop through both the geometry and records of a shapefile.  Normally  you would loop through the shape records and then loop through the dbf records seperately.  But sometimes it's easier to have both sides of the shapefile equation accessible at the same time.  This ability is important sometimes because the link between geometry and attributes is implied by their order in the file and not explicit which can make referencing one side or the other a pain.  Warning: the current implementation pulls everything into memory at once which can be a problem for very large shapefiles. This weakness will be updated in future versions.

Here’s a simple usage example followed by a detailed explanation and a few other posts where I use this method without much explanation.

Let’s say you have a simple point-location address shapefile named “addr.shp” with the following structure:

GEOMETRY ADDRESS CITY STATE ZIP
[-89.522996, 34.363596] 7018 South 8th Oxford MS 38655
[-89.520695, 34.360863] 1199 South 11th Oxford MS 38655
[-89.520927, 34.362924] 8005 Fillmore Ave Oxford MS 38655

You could then use the shapeRecords method like this:

>>> import shapefile
>>> r = shapefile.Reader(“addr”)
>>> sr = r.shapeRecords()
>>> # get the first shaperecord
>>> sr_test = sr[0]
>>> # Look at the geometry of the shape
>>> sr_test.shape.points
[[-89.522996, 34.363596]]
>>> # Look at the attributes of the dbf record
>>> sr_test.record
[‘7018 South 8’,’Oxford’,’MS’,’38655’]
>>> # Now let’s iterate through all of them
>>> for sr in r.shapeRecords():
...    print “x: “, sr.points[0][0]
...    print “y: “, sr.points[0][1]
...    # Output just the address field
...    print “Address: “, sr.record[0]
x: -89.522996
y: 34.363596
Address: 7018 South 8th
x: -89.520695
y: 34.360863
Address: 1195 South 11th
x: -89.520927
y: 34.362924
Address: 805 Fillmore Ave

Here’s how it works.

The shapeRecords() method returns a list.

Each entry in that list is a _ShapeRecord object instance.

A _ShapeRecord object has two attributes: shape, record

_ShapeRecord.record contains a simple list of the attributes.

_ShapeRecord.shape contains a _Shape object instance.

A _Shape object has, at a minimum, two attributes: shapeType, points

If the _Shape instance contains a polygon a “parts” attribute will appear.  This attribute contains the index in the point list of the beginning of a “part”.  Parts let you store multiple shapes in a single record.

The shapeType attribute provides a number telling you if the shapefile is  a point, polygon, line, etc. file.  These constants are listed in the shapefile spec document as well as near the top of the source code.

The points is just a list containing lists of the point coordinates.  Two things to note:  If the geometry has multiple parts, such as multiple polygons, the points for all parts are just lumped together.  You must separate them by referencing the parts index list.  Some shape types allow for z and m values which may appear in addition to the x,y pairs.

This method is really just a clumsy convenience method that basically zips up the results of the shapes() and records() methods you are already using.

I have a few blog posts where I call this method as well:

http://geospatialpython.com/2011/02/changing-shapefiles-type.html

http://geospatialpython.com/2011/01/point-in-polygon.html

http://geospatialpython.com/2010/12/dot-density-maps-with-python-and-ogr.html  (in the comments)

8 comments:

  1. Awesome! I was hoping you'd add something like this. I just started using Pyshp in the past 3 weeks.

    ReplyDelete
  2. I created what appears to be a polylineM shapefile. It doesn't seem quite right in that none of the measurements seem to have gone over, but nonetheless, I can open with Esri and it thinks it's a polylineM. However, I can't seem to read the records using this method. So my problem is two-fold. I need a legitimate polylineM and I need to be able to read it. I'm sure I'm doing something wrong, but would appreciate any assistance.

    ReplyDelete
    Replies
    1. VeeAnn - I'm looking into this issue. It may be a bug.

      Delete
  3. Shouldn't the two print lines be:

    print “x: “, sr.shape.points[0][0]
    print “y: “, sr.shape.points[0][1]

    I wasn't able to get same output until I made these changes. But it is a usefull bit of code.

    ReplyDelete
  4. Joel... great job you have done here. Question.. when working with the Editor... is there any function to close the editing operations. I experience that the Editor "locks" the files, even after closing python. Any suggestions?

    ReplyDelete
  5. bluejp - Sorry for the slow reply. That is really weird. What platform are you on? When you call Editor.save() you are calling the Writer.save() method which explicitly closes all file handles.

    All the Editor class does is wrap the Reader and Writer classes so you don't have to explicitly create separate reader and writer objects to change a shapefile. But of course you can do that manually. You just create a reader and a writer and copy everything into the writer and make changes, then save over the old file. There are several posts where I do this.

    Also - the Editor is not as well tested so people continually find bugs but it's slowly getting better. The Reader and Writer classes are pretty solid and should work. I have examples of this technique in the following posts:
    http://geospatialpython.com/2011/02/merging-lots-of-shapefiles-quickly.html

    http://geospatialpython.com/2010/12/subsetting-shapefile-by-attributes.html

    I would try that and see if you get the same results. If you're using it in a web application it could be some web server behavior. But I really don't know. Let me know what you figure out and I'll see if I can help troubleshoot.

    ReplyDelete
  6. Hi, I dont know if my problem fix in this post (I´m new in the programmer world). Here is my problem:
    How can I write a program using python, that calculates some morphometric and drainage indexes using numeric attributes in the attribute table with ArcGIS?

    The data I have in my table are area and perimeter of the basin, and channel order of the watercourses (number and km of each order).

    For example, I need to calculate the Relationship of bifurcation with this formula: RB=Nu/Nu+1 (Nu - considered channel order; Nu+1 - channel order immediately ) above the considered one).

    So, I want to create a program that would be able to get the right value from the attribute table (in ArcGIS) and run with the formula. How can I do that?

    Thanks

    ReplyDelete
  7. Hello,

    My name is Kerry and I am writing on behalf of BlippApp.com (http://www.blippapp.com) . We recently noticed your website and thought that you'd be interested in a contest we are running for a Garmin Handheld GPS.

    Our new GPS sharing app, Blipp just went live on the Google Play store and I thought you and your readers would enjoy this app. Blipp is an exciting, interactive GPS location sharing app. Friends are able to share important events with one another using a Blipp on a map.

    ** Contest Details
    ------------------------------------------------------------
    To help promote Blipp we decided to run a contest. Simply post about Blipp on your website, Facebook or Twitter page and then shoot us an email back with the URL and then you will be entered to win! At the end of the contest (April 30th) we will randomly select 1 entry and that winner will receive a Garmin Handheld GPS (http://www.amazon.com/Garmin-eTrex-Worldwide-Handheld-Navigator/dp/B00542NV32/ref=sr_1_3?ie=UTF8&qid=1364226041&sr=8-3&keywords=handheld+GPS) .

    There are 4 simple ways to enter:
    1. Post a review on your website/blog and you will be entered to win 5 times.
    2. Post a link/mention on Facebook and you will be entered to win 2 times.
    3. Post a link/mention Blipp on Twitter and you will be entered to win 1 time.
    4. Like us on Facebook here www.facebook.com/blippapp and you will be entered to win 1 time.

    You can do all 4 for a total of 9 entries!

    Not only will you have a chance to win but you can also try an amazing location sharing app which is probably right up your alley (based on what we noticed on your website). Best of all it only takes a minute or two to enter. Please check out the app here:
    www.BlippApp.com (http://www.blippapp.com)

    Check out BlippApp.com (http://www.blippapp.com) today, and help spread the word... then simply shoot us back a quick email with the URL and you will be entered to win. The contest ends April 30th. Please reply to this email with any questions.

    Regards,

    Kerry
    BlippApp.com
    info@blippapp.com

    https://play.google.com/store/apps/details?id=com.blipp

    ReplyDelete