This morning before church, I wrote some code to build a contour plot of schools in New Jersey. The “hot” reddish regions have the best schools and the cooler regions don’t.
Tech Stuff: How it’s made
This wasn’t that easy. Since I had all the code from my previous post, it should have been very straightforward to make another contour map. However, to get this map to close, I had four tech challenges to overcome: the GreatSchools API omitted schools when searching from geo-coördinates, a number of locations didn’t have any schools, and I had to mix python and ruby code in a way that exchanged values.
Fixing GreatSchools Omitted Schools
One of the biggest challenges came from the Great Schools API. While the GS nearby API provides schools within a specified radius of a geographic position, I noticed that schools weren’t showing up within the radius specified. Fortunately, I was familiar with Google’s Directions API, which provides a reverse geocode feature that provides a town name for a given spot. This was far from straightforward since google provides multiple layers of data for each coordinate. On top of that, town names can be locality, postal_town, administrative_area_level_2, administrative_area_level_1. This necessitated the following code from Geoff Boeing:
def parse_city(geocode_data):
if (not geocode_data is None) and ('address_components' in geocode_data):
for component in geocode_data['address_components']:
if 'locality' in component['types']:
return component['long_name']
elif 'postal_town' in component['types']:
return component['long_name']
elif 'administrative_area_level_2' in component['types']:
return component['long_name']
elif 'administrative_area_level_1' in component['types']:
return component['long_name']
return None
Fixing locations with no schools
Additionally, many points were not associated with a school. While bodies of water naturally had no school, the Great Schools API failed to report any school for many locations. Since these data didn’t exist, I populated the matrix with NumPy null values or “NaNs”. These showed up as blank regions on the map and wrecked havoc with the contour lines. To fix this, I interpolated in two dimensions using NumPy’s ma.masked_invalid feature followed by SciPy’s interpolate.griddata capability with cubic interpolation. (Wow, Python has a lot of math available on demand.)
Mixed Python and Ruby
The final challenge was connecting to the Great Schools API. I could connect with Python’s default tools, and parse the result with lxml, but that meant building each request. Fortunately, there was a ruby wrapper ready for use, but I had to call it from Python. Python’s subprocess with check_output did the trick, but due to the fickle nature of the API, I had to add a bunch of exception handling.
I’m hoping to make a map like this for northern Virginia soon.
https://gist.github.com/tbbooher/27092a4aa3d6f013e0e07a558c21f5fb
Leave a Reply