[Matplotlib-users] plotting a contour map from CSV file

[Tim Michelsen]

Dear Matplotlib-Users,
I am tryring to create a contour plot over a basemap.

My main problem is creating the array for the Z values as a basis for the
plt.contour command from a CSV file where latitude, longitude and value are
stored column-wise:

lat; lon; value
50; 10; 6
...

The data represents a regular spaced grid with a datapoint each 0.25 degrees.

I tried various possibilities but didn't have success:

1) following simpletest.py from the basemap examples:
X, Y = meshgrid(data[:,1], data[:,0])

Z = data[:,2]

m.contourf(x,y, Z)

=> Error: Z must be a 2D array
-> How do I get Z to be a 2D array?

2) using the griddata package
Here I was nearly without orientation how to call griddata correctly.


3) Using the python bindings of ogr
Any examples on this one?

[Jeff Whitaker]

Post by Tim Michelsen
Dear Matplotlib-Users,
I am tryring to create a contour plot over a basemap.
My main problem is creating the array for the Z values as a basis for the
plt.contour command from a CSV file where latitude, longitude and value are
lat; lon; value
50; 10; 6
...
The data represents a regular spaced grid with a datapoint each 0.25 degrees.
X, Y = meshgrid(data[:,1], data[:,0])
Z = data[:,2]

Timmie: Try:

X, Y = meshgrid(data[:,1], data[:,0])
Z = data[:,2]
nlons = X.shape[1]; nlats = X.shape[0]
Z = Z.reshape(nlats,nlons)

Post by Tim Michelsen
m.contourf(x,y, Z)
=> Error: Z must be a 2D array
-> How do I get Z to be a 2D array?
2) using the griddata package
Here I was nearly without orientation how to call griddata correctly.

You don't need to use griddata since you have regularly gridded data.

Post by Tim Michelsen
3) Using the python bindings of ogr
Any examples on this one?

Again, no need. A simple reshape will get you the 2d lat/lon array you
need.

[Jeff Whitaker]

Post by Tim Michelsen

Post by Tim Michelsen
Dear Matplotlib-Users,
I am tryring to create a contour plot over a basemap.
My main problem is creating the array for the Z values as a basis for the
plt.contour command from a CSV file where latitude, longitude and value are
lat; lon; value
50; 10; 6
...
The data represents a regular spaced grid with a datapoint each 0.25 degrees.
X, Y = meshgrid(data[:,1], data[:,0])
Z = data[:,2]

X, Y = meshgrid(data[:,1], data[:,0])
Z = data[:,2]
nlons = X.shape[1]; nlats = X.shape[0]
Z = Z.reshape(nlats,nlons)

Timmie: Sorry, but upon further reflection I don't think this will
work. You'll need to know the number of
lats and the number of lons on the grid beforehand. Then you should be
able to do

X = X.reshape(nlats,nlons)
Y = Y.reshape(nlats,nlons)
Z = Z.reshape(nlats,nlons)

after reading the data in.

(skip the meshgrid call, that's only useful when X is a vector with length nlons and Y is a vector with length nlats).

If you still have problems, send us a full example.

-Jeff

Post by Tim Michelsen

Post by Tim Michelsen
m.contourf(x,y, Z)
=> Error: Z must be a 2D array
-> How do I get Z to be a 2D array?
2) using the griddata package
Here I was nearly without orientation how to call griddata correctly.

You don't need to use griddata since you have regularly gridded data.

Post by Tim Michelsen
3) Using the python bindings of ogr
Any examples on this one?

Again, no need. A simple reshape will get you the 2d lat/lon array you
need.

[Tim Michelsen]

Hello Jeff,

Post by Tim Michelsen

- Points stored in the above descripbed format (lat, lon, value)?

This one I solved using a m.scatter() function

Post by Tim Michelsen

- Interpolate a grid of data points by using different interpolation
methods like inverse distance wheighting, natural neighbor

interpolation, etc. to get a contour map?

Post by Tim Michelsen

For interpolation of irregular, randomly distributed data points see
http://www.scipy.org/Cookbook/Matplotlib/

Gridding_irregularly_spaced_data.

Post by Tim Michelsen

However, if there is some structure to the data grid then it's probably
better not to use these approaches.

The problem is that although regular spaced the grid is still to large to
countour to a nice map. I will play a bit more with contour and other
interpolation functions.

Post by Tim Michelsen
2) using the griddata package
Here I was nearly without orientation how to call griddata correctly.

I tried again.

Here is what I got:
x = data[:,1]
y = data[:,0]
z = data[:,2]
X, Y = mlab.meshgrid(x, y)
X, Z = mlab.meshgrid(x, y)
# zi = griddata(x,y,z,xi,yi,**kwargs)
Z = grid.griddata(x,y,z, X, Y)
plt.contour(X,Y, Z)

=> ValueError: output grid defined by xi,yi must be monotone increasing

The coordinates are stored in a way that first longitude (x) increases and
then the latitude (y) increases.
10 6.0 4
10 6.25 3
10 6.50 2
10 6.75 1
10 6.0 6
11 6.25 7
11 6.50 6
11 6.75 9
12 6.0 4

What how do I need to arrange my data to get it monotone increasing for
griddata?

Thanks for your help. One settled I will send you another example for the
examples package.

Kind regards,
Timmie

[Jeff Whitaker]

Post by Tim Michelsen
Hello Jeff,

Post by Tim Michelsen

- Points stored in the above descripbed format (lat, lon, value)?

This one I solved using a m.scatter() function

Post by Tim Michelsen

- Interpolate a grid of data points by using different interpolation
methods like inverse distance wheighting, natural neighbor

interpolation, etc. to get a contour map?

Post by Tim Michelsen

For interpolation of irregular, randomly distributed data points see
http://www.scipy.org/Cookbook/Matplotlib/

Gridding_irregularly_spaced_data.

Post by Tim Michelsen

However, if there is some structure to the data grid then it's probably
better not to use these approaches.

The problem is that although regular spaced the grid is still to large to
countour to a nice map. I will play a bit more with contour and other
interpolation functions.

Post by Tim Michelsen
2) using the griddata package
Here I was nearly without orientation how to call griddata correctly.

I tried again.
x = data[:,1]
y = data[:,0]
z = data[:,2]
X, Y = mlab.meshgrid(x, y)
X, Z = mlab.meshgrid(x, y)
# zi = griddata(x,y,z,xi,yi,**kwargs)
Z = grid.griddata(x,y,z, X, Y)
plt.contour(X,Y, Z)
=> ValueError: output grid defined by xi,yi must be monotone increasing
The coordinates are stored in a way that first longitude (x) increases and
then the latitude (y) increases.
10 6.0 4
10 6.25 3
10 6.50 2
10 6.75 1
10 6.0 6
11 6.25 7
11 6.50 6
11 6.75 9
12 6.0 4
What how do I need to arrange my data to get it monotone increasing for
griddata?
Thanks for your help. One settled I will send you another example for the
examples package.
Kind regards,
Timmie

Timmie: You shouldn't use griddata. You have a regular lat/lon grid,
so it's just a matter of loading the data into the proper 2-d array.
Please send a self-contained script (and post the data somewhere) and
then we can help you.

-Jeff

[Tim Michelsen]

Hello,
thanks.
I checked again from contour_demo.py of the basemap distribution.

There lats, lons are uniquely monoton increasing from 0-360 and from -90 to 90.
In my case data is written row-by-row:
* increasing from lowest latitude western most longitude to easternmost
longitude and then increasing each rows in the same manner to the northermost
latitude (see below).

So, as you said, it's a question of re-aranging the data. that it fits the to
the way m.contour expects the 2-D array.
Also, since the grid is still coarse, I would need to apply some smoothing
afterwards. What do you recommend for that?

I don't know how I can do this easily by hand. May you give me some guidance
here, please?

But I may just convert it to a shape file using GIS then load it with the
shapefile interface you wrote.
What would you see as most convenient way?
If I produce maps with a GIS but want to use matplotlib for the map plotting,
what would be the preferred export format? Any gdal format?

Many thanks in advance,
Timmie

### data example

Latitude Longitude value
45 7 7.65251434
45 7.25 6.841345477
45 7.5 3.923153289
45 7.75 3.644313708
45 8 3.550977951
45 8.25 3.352525137
45 8.5 3.080082094
45 8.75 2.971992657
45 9 2.998723785
45 9.25 3.080082094
45 9.5 3.185687405
45 9.75 3.102075854
45 10 3.185687405
45 10.25 3.213960325
45 10.5 3.32326373
45 10.75 3.465643983
45 11 3.612980369
45 11.25 3.644313708
45 11.5 3.701277511
45 11.75 3.923153289
45 12 3.797848342
45 12.25 3.612980369
45 12.5 3.435577844
45 12.75 3.294210812
45 13 3.26536503
45.25 7 6.485050223
45.25 7.25 6.343081631
45.25 7.5 3.856783573
45.25 7.75 3.405725407
45.25 8 3.550977951
45.25 8.25 3.294210812
45.25 8.5 3.294210812
45.25 8.75 3.185687405
45.25 9 3.15761656
45.25 9.25 3.213960325
45.25 9.5 3.15761656
45.25 9.75 3.32326373
45.25 10 3.405725407
45.25 10.25 3.495925216
45.25 10.5 3.465643983
45.25 10.75 3.550977951
45.25 11 3.465643983
45.25 11.25 3.765429652
45.25 11.5 3.95669157
45.25 11.75 3.797848342
45.25 12 3.923153289
45.25 12.25 3.733239867
45.25 12.5 3.550977951
45.25 12.75 3.520306012
45.25 13 3.376085288
45.5 7 6.383367092
45.5 7.25 6.383367092
45.5 7.5 6.009422688
45.5 7.75 4.679469855
45.5 8 3.435577844
45.5 8.25 3.435577844
45.5 8.5 3.236725042
45.5 8.75 3.236725042
45.5 9 3.185687405
45.5 9.25 3.102075854
45.5 9.5 3.102075854
45.5 9.75 3.185687405
45.5 10 3.352525137
45.5 10.25 3.405725407
45.5 10.5 3.376085288
45.5 10.75 3.612980369
45.5 11 3.520306012
45.5 11.25 3.352525137
45.5 11.5 3.823949103
45.5 11.75 3.856783573
45.5 12 3.856783573
45.5 12.25 3.765429652
45.5 12.5 3.669541114
45.5 12.75 3.550977951
45.5 13 3.435577844
45.75 7 5.309043916
45.75 7.25 6.057519881
45.75 7.5 5.030958443
45.75 7.75 4.836570243
45.75 8 4.836570243
45.75 8.25 2.724965001
45.75 8.5 2.607751091
45.75 8.75 3.26536503
45.75 9 2.898163214
45.75 9.25 2.872155245
45.75 9.5 1.893252754
45.75 9.75 2.043669061
45.75 10 1.75488883
45.75 10.25 2.004264146
45.75 10.5 2.971992657
45.75 10.75 1.804949998
45.75 11 2.846334614
45.75 11.25 5.519419657
45.75 11.5 2.517818813
45.75 11.75 3.733239867
45.75 12 3.376085288
45.75 12.25 3.550977951
45.75 12.5 3.612980369
45.75 12.75 3.520306012
45.75 13 3.495925216
46 7 5.06399168
46 7.25 4.949174095
46 7.5 5.266087828
46 7.75 5.352298328
46 8 4.757472437
46 8.25 2.800325674
46 8.5 3.612980369
46 8.75 3.185687405
46 9 2.323282473
46 9.25 1.671485743
46 9.5 3.856783573
46 9.75 4.572079662
46 10 4.679469855
46 10.25 4.679469855
46 10.5 5.309043916
46 10.75 3.294210812
46 11 3.405725407
46 11.25 3.669541114
46 11.5 3.495925216
46 11.75 4.255093726
46 12 3.495925216
46 12.25 3.185687405
46 12.5 3.213960325
46 12.75 3.550977951
46 13 3.520306012
46.25 7 1.969297411
46.25 7.25 4.908706364
46.25 7.5 3.052767233
46.25 7.75 3.765429652
46.25 8 3.95669157
46.25 8.25 5.06399168
46.25 8.5 5.266087828
46.25 8.75 3.669541114
46.25 9 3.185687405
46.25 9.25 3.797848342
46.25 9.5 3.352525137
46.25 9.75 5.439709782
46.25 10 5.69098301
46.25 10.25 4.949174095
46.25 10.5 5.736883145
46.25 10.75 5.105542055
46.25 11 4.255093726
46.25 11.25 3.701277511
46.25 11.5 4.255093726
46.25 11.75 4.572079662
46.25 12 3.98369323
46.25 12.25 4.148941623
46.25 12.5 3.129746478
46.25 12.75 3.236725042
46.25 13 3.550977951
46.5 7 2.872155245
46.5 7.25 3.701277511
46.5 7.5 3.15761656
46.5 7.75 3.765429652
46.5 8 5.18951259
46.5 8.25 6.105948261
46.5 8.5 5.266087828
46.5 8.75 5.69098301
46.5 9 6.009422688
46.5 9.25 5.147381739
46.5 9.5 5.829636932
46.5 9.75 5.654489904
46.5 10 6.243327668
46.5 10.25 5.395852976
46.5 10.5 5.736883145
46.5 10.75 6.057519881
46.5 11 5.147381739
46.5 11.25 3.520306012
46.5 11.5 3.856783573
46.5 11.75 4.148941623
46.5 12 4.71833512
46.5 12.25 4.71833512
46.5 12.5 3.701277511
46.5 12.75 3.889851131
46.5 13 3.32326373
46.75 7 1.859766825
46.75 7.25 2.198852355
46.75 7.5 2.345277833
46.75 7.75 2.517818813
46.75 8 3.856783573
46.75 8.25 3.856783573
46.75 8.5 5.06399168
46.75 8.75 4.184077131
46.75 9 5.829636932
46.75 9.25 3.644313708
46.75 9.5 3.765429652
46.75 9.75 5.309043916
46.75 10 6.009422688
46.75 10.25 5.147381739
46.75 10.5 5.609155594
46.75 10.75 5.783100444
46.75 11 5.147381739
46.75 11.25 3.581868928
46.75 11.5 4.908706364
46.75 11.75 3.465643983
46.75 12 3.465643983
46.75 12.25 4.148941623
46.75 12.5 3.98369323
46.75 12.75 3.581868928
46.75 13 3.644313708

[Jeff Whitaker]

Post by Tim Michelsen
Hello,
thanks.
I checked again from contour_demo.py of the basemap distribution.
There lats, lons are uniquely monoton increasing from 0-360 and from -90 to 90.
* increasing from lowest latitude western most longitude to easternmost
longitude and then increasing each rows in the same manner to the northermost
latitude (see below).
So, as you said, it's a question of re-aranging the data. that it fits the to
the way m.contour expects the 2-D array.
Also, since the grid is still coarse, I would need to apply some smoothing
afterwards. What do you recommend for that?

Timme: Here's one way to do it

from matplotlib.mlab import load
import matplotlib.pyplot as plt
import numpy as np
data = load("data.txt")
# need to know nlons and nlats beforehand!
nlons = 8; nlats = 25
X = data[0::nlats,0]
Y = data[0:nlats,1]
# data is in nlons,nlats order in file, need to transpose
Z = data[:,2].reshape(nlons,nlats).transpose()
X,Y = np.meshgrid(X,Y)
CS = plt.contourf(X,Y,Z,20)
plt.show()

I don't have any recommendations for smoothing - why don't you plot the
raw data first and see if you really need it?

Post by Tim Michelsen
I don't know how I can do this easily by hand. May you give me some guidance
here, please?
But I may just convert it to a shape file using GIS then load it with the
shapefile interface you wrote.
What would you see as most convenient way?
If I produce maps with a GIS but want to use matplotlib for the map plotting,
what would be the preferred export format? Any gdal format?

I prefer netCDF format for gridded data (basemap contains a function for
reading netCDF files - NetCDFFile).

-Jeff

Post by Tim Michelsen
Many thanks in advance,
Timmie
### data example
Latitude Longitude value
45 7 7.65251434
45 7.25 6.841345477
45 7.5 3.923153289
45 7.75 3.644313708
45 8 3.550977951
45 8.25 3.352525137
45 8.5 3.080082094
45 8.75 2.971992657
45 9 2.998723785
45 9.25 3.080082094
45 9.5 3.185687405
45 9.75 3.102075854
45 10 3.185687405
45 10.25 3.213960325
45 10.5 3.32326373
45 10.75 3.465643983
45 11 3.612980369
45 11.25 3.644313708
45 11.5 3.701277511
45 11.75 3.923153289
45 12 3.797848342
45 12.25 3.612980369
45 12.5 3.435577844
45 12.75 3.294210812
45 13 3.26536503
45.25 7 6.485050223
45.25 7.25 6.343081631
45.25 7.5 3.856783573
45.25 7.75 3.405725407
45.25 8 3.550977951
45.25 8.25 3.294210812
45.25 8.5 3.294210812
45.25 8.75 3.185687405
45.25 9 3.15761656
45.25 9.25 3.213960325
45.25 9.5 3.15761656
45.25 9.75 3.32326373
45.25 10 3.405725407
45.25 10.25 3.495925216
45.25 10.5 3.465643983
45.25 10.75 3.550977951
45.25 11 3.465643983
45.25 11.25 3.765429652
45.25 11.5 3.95669157
45.25 11.75 3.797848342
45.25 12 3.923153289
45.25 12.25 3.733239867
45.25 12.5 3.550977951
45.25 12.75 3.520306012
45.25 13 3.376085288
45.5 7 6.383367092
45.5 7.25 6.383367092
45.5 7.5 6.009422688
45.5 7.75 4.679469855
45.5 8 3.435577844
45.5 8.25 3.435577844
45.5 8.5 3.236725042
45.5 8.75 3.236725042
45.5 9 3.185687405
45.5 9.25 3.102075854
45.5 9.5 3.102075854
45.5 9.75 3.185687405
45.5 10 3.352525137
45.5 10.25 3.405725407
45.5 10.5 3.376085288
45.5 10.75 3.612980369
45.5 11 3.520306012
45.5 11.25 3.352525137
45.5 11.5 3.823949103
45.5 11.75 3.856783573
45.5 12 3.856783573
45.5 12.25 3.765429652
45.5 12.5 3.669541114
45.5 12.75 3.550977951
45.5 13 3.435577844
45.75 7 5.309043916
45.75 7.25 6.057519881
45.75 7.5 5.030958443
45.75 7.75 4.836570243
45.75 8 4.836570243
45.75 8.25 2.724965001
45.75 8.5 2.607751091
45.75 8.75 3.26536503
45.75 9 2.898163214
45.75 9.25 2.872155245
45.75 9.5 1.893252754
45.75 9.75 2.043669061
45.75 10 1.75488883
45.75 10.25 2.004264146
45.75 10.5 2.971992657
45.75 10.75 1.804949998
45.75 11 2.846334614
45.75 11.25 5.519419657
45.75 11.5 2.517818813
45.75 11.75 3.733239867
45.75 12 3.376085288
45.75 12.25 3.550977951
45.75 12.5 3.612980369
45.75 12.75 3.520306012
45.75 13 3.495925216
46 7 5.06399168
46 7.25 4.949174095
46 7.5 5.266087828
46 7.75 5.352298328
46 8 4.757472437
46 8.25 2.800325674
46 8.5 3.612980369
46 8.75 3.185687405
46 9 2.323282473
46 9.25 1.671485743
46 9.5 3.856783573
46 9.75 4.572079662
46 10 4.679469855
46 10.25 4.679469855
46 10.5 5.309043916
46 10.75 3.294210812
46 11 3.405725407
46 11.25 3.669541114
46 11.5 3.495925216
46 11.75 4.255093726
46 12 3.495925216
46 12.25 3.185687405
46 12.5 3.213960325
46 12.75 3.550977951
46 13 3.520306012
46.25 7 1.969297411
46.25 7.25 4.908706364
46.25 7.5 3.052767233
46.25 7.75 3.765429652
46.25 8 3.95669157
46.25 8.25 5.06399168
46.25 8.5 5.266087828
46.25 8.75 3.669541114
46.25 9 3.185687405
46.25 9.25 3.797848342
46.25 9.5 3.352525137
46.25 9.75 5.439709782
46.25 10 5.69098301
46.25 10.25 4.949174095
46.25 10.5 5.736883145
46.25 10.75 5.105542055
46.25 11 4.255093726
46.25 11.25 3.701277511
46.25 11.5 4.255093726
46.25 11.75 4.572079662
46.25 12 3.98369323
46.25 12.25 4.148941623
46.25 12.5 3.129746478
46.25 12.75 3.236725042
46.25 13 3.550977951
46.5 7 2.872155245
46.5 7.25 3.701277511
46.5 7.5 3.15761656
46.5 7.75 3.765429652
46.5 8 5.18951259
46.5 8.25 6.105948261
46.5 8.5 5.266087828
46.5 8.75 5.69098301
46.5 9 6.009422688
46.5 9.25 5.147381739
46.5 9.5 5.829636932
46.5 9.75 5.654489904
46.5 10 6.243327668
46.5 10.25 5.395852976
46.5 10.5 5.736883145
46.5 10.75 6.057519881
46.5 11 5.147381739
46.5 11.25 3.520306012
46.5 11.5 3.856783573
46.5 11.75 4.148941623
46.5 12 4.71833512
46.5 12.25 4.71833512
46.5 12.5 3.701277511
46.5 12.75 3.889851131
46.5 13 3.32326373
46.75 7 1.859766825
46.75 7.25 2.198852355
46.75 7.5 2.345277833
46.75 7.75 2.517818813
46.75 8 3.856783573
46.75 8.25 3.856783573
46.75 8.5 5.06399168
46.75 8.75 4.184077131
46.75 9 5.829636932
46.75 9.25 3.644313708
46.75 9.5 3.765429652
46.75 9.75 5.309043916
46.75 10 6.009422688
46.75 10.25 5.147381739
46.75 10.5 5.609155594
46.75 10.75 5.783100444
46.75 11 5.147381739
46.75 11.25 3.581868928
46.75 11.5 4.908706364
46.75 11.75 3.465643983
46.75 12 3.465643983
46.75 12.25 4.148941623
46.75 12.5 3.98369323
46.75 12.75 3.581868928
46.75 13 3.644313708
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
Matplotlib-users mailing list
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

[Tim Michelsen]

Hello Jeff,

Post by Jeff Whitaker
Timme: Here's one way to do it

many thanks so far. I still have to inspect and improve my script. But
at least your code lead me to some contourd surface.

I will come back and tell if it worked. Unfortunately I cannot disclose
the data nor the results because of copyright issues.

So this mail is just to thank you for your responsiveness to my questions.

Timmie

[Tim Michelsen]

Hello Jeff,
I just wanna give feedback on what got me going here:

#### data preparation
### data is loaded from a CSV file
###
lats = y # data[:,0]
## lon => x
lons = x # data[:,1]
## values => z
values = z #data[:,2]
###
lat_uniq = list(set(lats.tolist()))
nlats = len(lat_uniq)
lon_uniq = list(set(lons.tolist()))
nlons = len(lon_uniq)
color_map = plt.cm.spectral
print lats.shape, nlats, nlons
yre = lats.reshape(nlats,nlons)
xre = lons.reshape(nlats,nlons)
zre = values.reshape(nlats,nlons)
#### later in the defined map
CT = m.contourf(xre, yre, zre, cmap=color_map)

Of course, this can be simplified. But, really, in the end the solution
was just simple and I hadn't thought of it...

If you want I can try to create a short example for the distribution.

Thanks for your help again,
Timmie