Python Web services, JSON, and ISS Oh My!

11/14/2017

In this post I will talk about how to handle JSON data from an external API utilizing python. Making calls to web services is made simple with python, with just a few lines of code you can track the International Space Station’s (ISS) position and time, realtime with a sleek graphical user interface. The following is a link to the project files download, https://codeclubprojects.org/en-GB/python/iss/.

The Turtle module is an object oriented graphics tool that draws to a canvas or screen. Turtle’s methods derived include forward(), backwards(), left() and right() like telling a turtle in what direction to draw. Turtle will draw over a NASA curated 2D map of Earth, so you should place the ‘map.jpg’ file in your project directory.

So one of the first things we need to do is instantiate a turtle screen with the following command.

# turtle provides a simple graphical interface to display data
# we need a screen to plot our space station position
import turtle
screen= turtle.Screen()

The image size is 720w by 360h so our turtle screen size should fit the image size.


# the image size is 720w x 360h
screen.setup(720,360)
# set coordinates to map longitude and latitude
screen.setworldcoordinates(-180,-90,180,90)
# set background picture to NASA world map, centered at 0
screen.bgpic('map.jpg')

 

iss

To represent the ISS on the 2D map let’s choose an image, it doesn’t have to be the following icon but it’s a nice icon so Houston we have liftoff!

# adds turtle object with name iss to list of objects
screen.register_shape('iss.png')
iss= turtle.Turtle()
iss.shape('iss.png')

 

Our location object will tell turtle to write the ISS png file to the screen at a specific position given the latitude and longitude of the ISS. Instantiate a Turtle() to create an object with the following code.

 

# location object for turtle to plot
location= turtle.Turtle()

# used later to write text
style=('Arial',6,'bold')
location.color('yellow')

Now, before we can tell our turtle to draw the ISS overhead-time we need the actual latitude and longitude coordinates of the passing ISS. A quick google search gives us the coordinates to store in a dictionary.

# Cape Canaveral ---> 28.392218, -80.607713
# Central Park, NYC ---> 40.782865, -73.965355
# create python dictionary to iterate and plot time of overhead location
coords={}
coords['nasa_fl']=(28.523397, -80.681874)
coords['centralp']=(40.782865, -73.965355)

To call the api we first need the url, ‘http://api.open-notify.org/astros.json,’ this will tell the api to give us the data we need to extrapolate the ISS data.

import urllib.request
import json
url='http://api.open-notify.org/astros.json'
response=urllib.request.urlopen(url)
result=json.loads(response.read())
print(result['people'])

Then to make the call to the url use urllib.request to access the url, querying for each given location. The data is then stored as a result,  loaded in json format. Json stands for JavaScript Object Notation and is used to conveniently organize data.

Screenshot (76)

The lines above are the contents of the json data, data is accessed similar to a python dictionary utilizing keys and indices.

import time

# setup loop to iterate and plot when the iss will be at the plotted location.
for k,v in coords.items():
 pass_url= 'http://api.open-notify.org/iss-pass.json'
 pass_url= pass_url+'?lat='+str(v[0])+'&lon='+str(v[1])
 pass_response= urllib.request.urlopen(pass_url)
 pass_result= json.loads(pass_response.read())
 over=pass_result['response'][1]['risetime']
# write turtle at new location coords
 location.penup()
 location.color('yellow')
 location.goto(v[1],v[0])
 location.write(time.ctime(over), font=style)
 location.pendown()

The above code block makes a call to the api, loads the json data, parses the overhead pass time (when the iss will be over the specified position) and then plots the time at the given location.

Screenshot (77)

# init current loc off iss coord
# make call to api
loc_url= 'http://api.open-notify.org/iss-now.json'
loc_response=urllib.request.urlopen(loc_url)
loc_result=json.loads(loc_response.read())</pre>
# the coords are pcked into jso, iss_position key
location= loc_result['iss_position']
lat= float(location['latitude'])
lon= float(location['longitude'])
<pre># set up while loop to plot moving iss
while(1):
# iss loc updates approx 3 sec
 time.sleep(1.5)

# update call to webservice to get new coords
 loc_url= 'http://api.open-notify.org/iss-now.json'
 loc_response=urllib.request.urlopen(loc_url)
 loc_result=json.loads(loc_response.read())
 location= loc_result['iss_position']
 lat= float(location['latitude'])
 lon= float(location['longitude'])
# write turtle at new location coords
 iss.setheading(90.0)
 iss.penup()
 iss.goto(lon,lat)
 iss.pendown()

 

The above code block makes a call to the api, loads the json data, parses the overhead position at the current geographic coordinates and plots the iss icon. The while loop is infinite to constantly track the iss.

Screenshot (75)

If you like these blog posts or want to comment and or share something do so below and follow py-guy!

Solar Radiation Prediction

07-21-2017

Sci-kit learn is a fantastic set of tools for machine learning in python. It is built on numpy, scipy, and matplotlib introduced in the first py-guy post and makes data analysis and visualization simple and intuitive. sci-kit learn provides classification, regression, clustering, dimensionality reduction, model selection, and preprocessing algorithms making data analysis in python accessible to everyone. We will cover an example of linear regression in this weeks post exploring Solar Radiation data from a NASA hackathon.

First after importing packages let’s read in the SolarPrediction.csv data set. The link to the data set is commented in the code block.


 

Taking a first look at the data set, specifically, UNIXTime and Date, note it is not formatted to a particular type so we will look at this later.

headshape.png

 

df.shape
df.describe()

Calling the describe method on the data frame returns some descriptive statistics on the data set and tells us there might be a relationship between radiation, humidity and or temperature.

descr

So let’s look at a correlation plot to get a better feel for any possible relationships.

truthmat= df.corr()
sns.heatmap(truthmat, vmax=.8, square=True)

matrix

There is a strong relationship between radiation and temperature (unsurprisingly or surprisingly) so let’s choose two features with some ambiguity. Pressure and Temperature will do fine, we will use seaborn, a statistical visualization library based on matplotlib to explore the relationship between the two features.

p = sns.jointplot(x="Pressure", y="Temperature", data=df)
pp.subplots_adjust(top=.9)
p.fig.suptitle('Temperature vs. Pressure')

 

temp_press.png

There is a clear positive trend albeit noisy because of the low pressure gradient. Lets do some quick feature engineering to get a better look at the trend.

 

#Convert time to_datetime
df['Time_conv'] = pd.to_datetime(df['Time'], format='%H:%M:%S')

#Add column 'hour'
df['hour'] = pd.to_datetime(df['Time_conv'], format='%H:%M:%S').dt.hour

#Add column 'month'
df['month'] = pd.to_datetime(df['UNIXTime'].astype(int), unit='s').dt.month

#Add column 'year'
df['year'] = pd.to_datetime(df['UNIXTime'].astype(int), unit='s').dt.year

#Duration of Day
df['total_time'] = pd.to_datetime(df['TimeSunSet'], format='%H:%M:%S').dt.hour - pd.to_datetime(df['TimeSunRise'], format='%H:%M:%S').dt.hour
df.head()

First we will convert to date time to manipulate later then add hour, month and year columns for a granular scope. Much Better!

screen-shot-2017-07-21-at-8-05-13-pm.png

With sklearn linear regression we can train python to model the data and then test the model for its accuracy. We will drop temperature column from the dependent variables  because that is what we want to learn.

 

y = df['Temperature']
X = df.drop(['Temperature', 'Data', 'Time', 'TimeSunRise', 'TimeSunSet','Time_conv',], axis=1)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)
from sklearn.linear_model import LinearRegression
lm = LinearRegression()
lm.fit(X_train,y_train)

Now let’s predict the temperature given the features.

 

X.head()
predictions = lm.predict( X_test)
pp.scatter(y_test,predictions)
pp.xlabel('Temperature Test')
pp.ylabel('Predicted Temperature')

linreg.png

MSE and RMSE values tell us the there is significance and the model performed well and as you can see there is a positive upward trend centered around the mean.

print(metrics.mean_squared_error(y_test, predictions))
print(np.sqrt(metrics.mean_squared_error(y_test, predictions)))

Screen Shot 2017-07-21 at 8.16.00 PM

If you like these blog posts or want to comment and or share something do so below and follow py-guy!

Note: I referenced kaggler Sarah VCH’s notebook in making todays blog post, specifically the feature engineering code in the fifth code block. If you want to see her notebook I’ve listed the link below.

https://www.kaggle.com/sarahvch/investigating-solar-radiation