Pages

Wednesday, December 4, 2013

Python and Elevation Data: ASCII Grid Files

Editor's Note: This multi-part series is based on Chapter 7 of "Learning Geospatial Analysis with Python" and republished with permission from Packt Publishing. This series will cover using Python to work with elevation data.  


Introduction

Elevation data is one of the most fascinating types of geospatial data. It represents many different types of
Image courtesy of the
US National Park Service
data sources and formats. Elevation data can display properties of both vector and raster data resulting in unique data products. Elevation data can serve the following purposes:
  • Terrain visualization
  • Land cover classification
  • Hydrology modelling
  • Transportation routing
  • Feature Extraction
You can't perform all of these options with both raster and vector data but because elevation data is three dimensional, containing x, y, and z coordinates, you can often get more out of these data than any other type.

In this chapter, we're going to learn to read and write elevation data in both raster and vector point formats. We'll also create some derivative products. The topics we'll cover are:
  • ASCII Grid elevation data files
  • Shaded-relief images
  • Elevation contours
  • Gridding LIDAR data
  • Creating a 3D mesh
Editor's Note:  This post will cover ASCII Grid elevation data files.  Each bullet above will be a separate post in this series. Instructions for installing GDAL and its Python bindings can be found here: http://goo.gl/WVPd1n.  On most linux systems you can run: apt-get install python-gdal. In my book I provide detailed instructions and links for Windows/Mac/Linux.  On all platforms PIL and Numpy can be installed using Python SetupTools through easy_install or pip.

For most of this chapter we'll use ASCII Grid files or ASCIIGRID . These files are a type of raster data usually associated with elevation data. This grid format stores data as text in equally sized square rows and columns with a simple header. Each cell in a row/column stores a single numeric value, which can represent some feature of terrain, such as elevation, slope, or flow direction. The simplicity makes it an easy-to-use, platform independent raster format. This format is described in the ASCII GRIDS section in Chapter 2, Geospatial Data.

Throughout the book we've relied on GDAL and to some extent PIL to read and write geospatial raster data including the gdalnumeric module to load raster data into NumPy arrays. But ASCIIGRID allows us to read and write rasters using only Python or even NumPy.
Tip: As a reminder, some elevation data sets use image formats to store elevation data. Most image formats only support 8-bit values ranging between 0-255; however, some formats, including TIFF, can store larger values. Geospatial software can typically display these data sets; however, traditional image software and libraries usually do not. For simplicity in this chapter, we'll stick to the ASCIIGRID format for data, which is both human and machine readable, as well as being widely supported.

Reading grids

NumPy has the ability to read the ASCIIGRID format directly using its loadtxt() method designed to read arrays from text files. The first six lines consist of the header, which are not part of the array. The following lines are a sample of a grid header:

ncols        250
nrows        250
xllcorner    277750.0
yllcorner    6122250.0
cellsize     1.0
NODATA_value      -9999

Line 1 contains the number of columns in the grid, which is synonymous with the x axis. Line 2 represents the y axis described as a number of rows. Line 3 represents the x coordinate of the lower left corner, which is the minimum x value. Line 4 is the corresponding minimum y value in the lower left corner of the grid. Line 5 is the cell size or resolution of the raster. Because the cells are square, only one size value is needed, as opposed to the separate x and y resolution values in most geospatial rasters. The fifth line is no data value, which is a number assigned to any cell for which a value is not provided. Geospatial software ignores these cells for calculations and often allows special display settings for it, such as coloring them black. The value -9999 is a common no data placeholder value used in the industry, which is easy to detect in software. In some examples, we'll use the number zero; however, zero can often also be a valid data value.

The numpy.loadtxt() method includes an argument called skiprows , which allows you to specify a number of lines in the file to be skipped before reading array values. To try this technique out you can download a sample grid file called myGrid.asc at the following URL:

https://geospatialpython.googlecode.com/files/myGrid.asc

So for myGrid.asc we would use the following code:

myArray  = numpy.loadtxt("myGrid.asc", skiprows=6)

This line results in the variable myArray containing a numpy array derived from the ASCIIGRID file myGrid.asc. The ASC file name extension is used by the ASCIIGRID format. This code works great but there's one problem. NumPy allows us to skip the header but not keep it. And we need to keep it to have a spatial reference for our data for processing, as well as for saving this grid or creating a new one.

To solve this problem we'll use Python's built-in linecache module to grab the header. We could open the file, loop through the lines, store each one in a variable, and then close the file. But linecache reduces the solution to a single line. The following line reads the first line in the file into a variable called line 1:

import linecache
line1 = linecache.getline("myGrid.asc", 1)

In the examples in this chapter we'll use this technique to create a simple header processor that can parse these headers into python variables in just a few lines.

Writing grids

Writing grids in Numpy is just as easy as reading them. We use the corresponding numpy.savetxt() function to save a grid to a text file. The only catch is, we must build and add the six lines of header information before we dump the array to the file. This process is slightly different depending on if you are using NumPy versions before 1.7 or after. In either case, you build the header as a string first. If you are using NumPy 1.7 or later, the savetext() method has an optional argument called header, which lets you specify a string as an argument. You can quickly check your NumPy version from the command line using the following command:

python -c "import numpy;print numpy.__version__" 
1.6.1

The backwards compatible method is to open a file, write the header then dump the array. Here is a sample of the Version 1.7 approach to save an array called myArray to an ASCIIGRID file called myGrid.asc:


header = "ncols     %s\n" % myArray.shape[1]
header += "nrows    %s\n" % myArray.shape[0]
header += "xllcorner 277750.0\n"
header += "yllcorner 6122250.0\n"
header += "cellsize 1.0\n"
header += "NODATA_value -9999\n"

numpy.savetxt("myGrid.asc", myArray, \
header=header, fmt="%1.2f")

We make use of python format strings, which allow you to put placeholders in a string to format python objects to be inserted. The %s format variable turns whatever object you reference into a string. In this case we are referencing the number of columns and rows in the array. In NumPy, an array has both a size and shape property. The size property returns an integer for the number of values in the array. The shape property returns a tuple with the number of rows and columns, respectively. So, in the preceding example, we use the shape property tuple to add the row and column counts to the header of our ASCII Grid. Notice we also add a trailing newline character for each line (\n). There is no reason to change the x and y values, cell size, or nodata value unless we altered them in the script. The savetxt() method also has a fmt argument, which allows you to use Python format strings to specify how the array values are written. In this case the %1.2f value specifies floats with at least one number and no more than two decimal places.

The backwards compatible version for NumPy, before 1.6, builds the header string in the same way but creates the file handle first:

import numpy
f = open("myGrid.asc", "w")
f.write(header)
numpy.savetxt(f, myArray, fmt="%1.2f")
f.close()

In the examples in this chapter, we'll introduce Python with an approach for writing files, which provides more graceful file management by ensuring files are closed properly. If any exceptions are thrown, the file is still closed cleanly:

with open("myGrid.asc", "w") as f:
    f.write(header)
    numpy.savetxt(f, myArray, fmt="%1.2f")

As you'll see in the upcoming examples, this ability to produce valid geospatial data files using only NumPy is quite powerful. In the next couple of examples we'll be using an ASCIIGRID Digital Elevation Model (DEM) of a mountainous area near Vancouver, British Columbia in Canada. You can download this sample as a ZIP file at the following URL:

https://geospatialpython.googlecode.com/files/dem.zip

The following image is the raw DEM colorized using QGIS with a color ramp that makes lower elevation values dark blue and higher elevation values bright red:

While we can conceptually understand the data this way, it is not an intuitive way to visualize the data. Let's see if we can do better.  In the next post we'll create a shaded relief image using this data.

52 comments:

  1. Thanks - this worked great for asc files. Any suggestions for modifications for using the DEM format used by the national elevation dataset?

    ReplyDelete
  2. This is great, you are good, i like your post and i still waiting our next post
    Bandar Ceme

    ReplyDelete
  3. Thanks, guys you always provide useful information through your articles
    Agen Domino

    ReplyDelete
  4. Really a nice content i am going to bookmark your site thanks
    Domino QQ

    ReplyDelete
  5. Your blog information is nice and friendly. This post is very helpful for me. Thank for post.
    Judi Ceme

    ReplyDelete
  6. Property rates in Hyderabad @ http://www.sahiba.co.in/rate.html

    ReplyDelete
  7. property rates in Chennai @ http://www.sagari.co.in/rate.html

    ReplyDelete
  8. Rates of property Mumbai @ http://www.saesha.co.in/rate.html

    ReplyDelete
  9. Property rates in Nashik @ http://www.sadhna.co.in/rate.html

    ReplyDelete
  10. Apartment rates in Mumbai @ http://www.sachita.co.in/rate.html

    ReplyDelete
  11. Rates of plots in Nashik @ http://www.sabita.co.in/rate.html

    ReplyDelete
  12. Property price in Ludhiana @ http://www.saanjh.co.in/rate.html

    ReplyDelete
  13. Latest property rates Noida @ http://www.saachisingh.co.in/rate.html

    ReplyDelete
  14. property rates in Ludhiana @ http://www.ruhijosh.co.in/rate.html

    ReplyDelete
  15. Rates of property in Noida @ http://www.roshnikhanna.co.in/rate.html

    ReplyDelete
  16. https://jualobatdewasa.net/category/obat-pembesar-penis/http://obatperbesarpenis.com/

    ReplyDelete
  17. https://jualobatdewasa.net/category/obat-pembesar-penis/

    ReplyDelete
  18. Thank you for your post, I look for such article along time. myself very happy to read it because it can give me more insight,thanks.
    dewa poker

    ReplyDelete
  19. Thanks you for sharing your thoughts.I truly appreciate your post thank you once again.
    bandarq
    bandarq
    taruhan bola
    bandar bola
    bandar togel
    agen togel
    prediksi togel

    ReplyDelete
  20. Sinidomino.net

    temen-temen mau memilih menjadi pemain atau bandar?

    Di Sinidomino teman-teman bisa memilih menjadi pemain atau bandar ?
    ada 7 jenis permainan yang bisa temen-temen mainkan hanya dengan1 user id

    Temen-temen juga bisa menikmati permainan ini melalui android
    dan dimanapun atau kapanpun temen-temen bisa menikmati permainan ini
    seru kan?

    Hanya dengan minimal deposit 20.000 dan penarikan dana 20.000
    transaksi akan diproses dengan cepat

    dapatkan kemenangan puluhan juta rupiah bersama ribuan player lainnya di permainan ini
    dengan kemudahan bertransaksi yang didukung oleh perbankan lokal

    -BCA
    -BNI
    -MANDIRI
    -BRI

    Daftar sekarang , klik disini ya

    -www.kartusini.com
    -http://goo.gl/hz1eC3

    Dengan bantuan costumer service yang ramah dan profesional
    untuk info lebih lamjut silakan hubungi :
    -PIN BBM : D61E3506
    -LINE : +6285598249684
    -www.sinidomino.ccom (official website)
    -WHATAPP : +6288598249684

    Semoga beruntung ya
    Salam hoki sinidomino.net

    ReplyDelete
  21. Good article, I've seen many articles today, but only this article is of interest to me, thanks poker online

    ReplyDelete

  22. I like your all post. You have done really good work. Thank you for the information you provide, it helped me a lot.
    I hope to have many more entries or so from you.
    Poker online uang asli

    Judi poker terpercaya

    promo bonus new member

    poker jackpot terbesar

    situs poker online

    poker online terpercaya

    judi online

    domino online

    judi poker

    jackpot terbesar

    ReplyDelete

  23. Kami adalah agen judi online yang terpercaya di indonesia.Kami menyediakan beberapa macam permainan
    untuk yang hobi bermain Judi online Poker
    dan pasang angka di Bandar togel
    Untuk yang gemar Taruhan bola
    kami juga menyediakan website nya.

    ReplyDelete
  24. Are you looking to cash into a checking account, the total amount usually cannot be immediately accessed; there may be a waiting period for the check to clear? We are the company http://www.unitedcheckcashing.com providing you the best services about http://www.unitedcheckcashing.com within a short period. In USA you may find us everywhere, every city and 24/7. We really love to serve you the best things with 100% satisfactions. check cashing

    ReplyDelete
  25. This is very educating post, thanks for sharing with us.
    Tips Menang AduQ
    Tips Bermain AduQ

    ReplyDelete
  26. https://situs-ikutqq.blogspot.com/
    https://ikutqqblog.wordpress.com/
    https://ikutqq.wixsite.com/agenbandarq
    https://ikutqq.wixsite.com/bandarq
    https://ikutqq.yolasite.com/
    http://ikutqq.weebly.com/
    http://agenbandarq.zohosites.com/
    http://agenbandarqonline.webs.com/
    http://agenbandarq.bravesites.com/
    http://ikutqq-blog.logdown.com/

    ReplyDelete
  27. https://daftarbandarqterbaik.blogspot.com/
    https://bandarq99.wordpress.com/
    https://bandarqq.wixsite.com/boyaqq
    https://bandarqq.wixsite.com/bandarq
    http://daftarbandarq.weebly.com/
    http://daftarbandarq.slidi.net/
    http://bandar-qq.ml/
    http://bandar-qq.ga/
    http://bandar-qq.cf/
    http://bandar-qq.gq/

    ReplyDelete
  28. Yuk Gabung Di Bolacasino88.com
    Dapatkan Promo Deposit 100% Sportsbook
    Minimal Deposit Promo 100% Sportsbook Rp 300.000
    Maksimal Bonus Sampai Rp 500.000
    Dapatkan Juga Promo Menarik Lain nya !!!

    Untuk Informasi Lebih Lanjut Silahkan Hubungi CS Kami Di :

    - No Tlp ( +855962671826 )
    - BBM ( 2BF2F87E )
    - Yahoo ( cs_bolacasino88 )
    - WhatsApp ( +855962671826 )

    Baca Juga Prediksi Togel Tanggal 21 October 2017 :

    http://prediksitoto.online/bocoran-togel-sgp-21-okt-2017/
    http://prediksitoto.online/prediksi-jitu-hk-21-okt-2017/
    http://prediksitoto.online/prediksi-jitu-sydney-21-okt-2017/
    http://prediksitoto.online/prediksi-top-milan-21-oktober-2017/
    http://prediksitoto.online/prediksi-top-pattaya-21-okt-2017/
    http://prediksitoto.online/prediksi-top-genting-21-oktober-2017/
    http://prediksitoto.online/prediksi-top-magnum-21-okt-2017/

    ReplyDelete
  29. interesting articles and this is my first first reading a very interesting article thanks for this article that fits with the theme news.
    dewa poker

    ReplyDelete
  30. I am actually amazed with your initiatives as well as actually pleased to see this post.
    Affordable Gurgaon Escorts Roshanikhanna
    Female Escorts Services in Gurgaon

    ReplyDelete

  31. Daftar dan buktikan pelayanan yang super cepat dari kami
    Hanya dengan 10 ribu rupiah saja kalian bisa memainkan semua game yang ada di website kami
    Untuk info dan keluhan silahkan hubungi Customer Service kami yang siap melayani
    selama 24 jam full tanpa henti

    JUDI POKER ONLINE UANG ASLI

    POKER ONLINE TERPERCAYA

    BANDAR JUDI TOGEL ONLINE

    TARUHAN JUDI BOLA

    FILM BOKEP

    ReplyDelete
  32. Mari Boskuu...
    hanya Butuh 1 ID dan Anda Bisa Bermain 6 Permainan Di Sini.
    jadi Tunggu Apa lagi,, Ayo Bergabung Dan MenangKan Jutaan Rupiah.

    Jangan Ragu Dengan kami Di sini Kami MenjaminKan Pemainan Tanpa Robot.
    Ataupun Admin Kami Bermain Bersama anda.
    DI sini kami Fairplay, 100% Player VS player.
    Poker Aman Dan Terpercaya Hanya Bersama Gadispoker.
    Kami Juga Menyediakan Bermacam Promo untuk Para Member Setia Kami.

    -Bonus Deposit Setiap hari Untuk Para member.
    -Bonus Refferal 10% Seumur Hidup.
    -Bonus TO/Turn Over Up TO 0,3% - 0,5%
    - Hadiah Jackpot Puluhan Jutaan Rupiah
    Kalian Juga Akan Di layani dengan CS Kami Yang Ramah dan Sangat propesional.
    Selama 24jam Nonstop,
    JAdi Jangan Berpikir panjang lagi mari Bergabung bersama Gadispoker dan Di tunggu Kemenangannya,
    Untuk Info Lebih lanjut Silakan hubungi CS (custumer Service).
    -Live chat ( gadispoker )
    -Whatsapp : +855966624192
    -Skype : Gadispoker
    -Yahoo : gadispokercs
    -BBM : D8C893A4
    -Line : gadispoker-cs
    Hubungi Segera Dan Jangan Lewatkan Kemenanganmu..!
    agen judi poker online terpercaya di indonesia

    bandar judi poker online yang paling aman

    taruhan judi poker online terpercaya

    poker online

    judi poker

    dewa poker

    raja poker

    gadispoker

    judi poker online terbesar

    judi poker online yang aman dan terpercaya

    bandar poker online indonesia

    judi dewa poker terbaik indonesia

    agen poker online indonesia

    poker online betting

    poker online terbaik

    ReplyDelete
  33. Articles that are very interesting and I really like your thoughts in article writing is very extraordinary.
    raja poker

    ReplyDelete
  34. We are the company http://www.unitedcheckcashing.com providing you the best services about unitedcheckcashing.com within a short period. In USA you may find us everywhere, every city and 24/7. We actually love to oblige you the best things with 100% agreements and faster ever.
    check cashing

    ReplyDelete
  35. your article is really good, visit also my article in: dewa poker

    ReplyDelete
  36. Here are several of the websites we advocate for our visitors
    AgenQQ
    AduQ
    Agen BandarQ

    ReplyDelete
  37. Really nice post in this site..
    Domino99



    Amazing posting. I like it and keep doing goods.
    Agen Domino99



    So impresive posting like this.. Thanks sharing.
    Agen Poker

    ReplyDelete