Firebase IoT Tutorial

I recently wrote a blog post on writing a temperature logger using Arduino + Python + Orchestrate.io database service. Several readers wrote back to use an alternative database if possible. I have decided to rewire that tutorial but this time with Firebase database. This would also act as a introductory tutorial on Firebase.

What we will build?

We are going to build a Temperature Logger IoT Project that comprises of the following:

  • Arduino Uno
  • A temperature sensor (LM35)
  • A Python application that can read the data from the Arduino Uno
  • Post that Temperature Data into a Cloud Database (Firebase)

A high level diagram of the process is shown below:

templogger

The Hardware Setup

I used the following:

  • Arduino Uno microcontroller
  • LM35 Temperature Sensor
  • Eventually we will have the Raspberry Pi that interfaces with the Arduino to read and transmit off the values but to validate things for now, the Uno was powered via a laptop/desktop with Python installed on it. The communication between the Uno and the PC is via serial port communication.

Arduino Uno + Temperature Sensor Setup

Here is how the LM35 sensor is connected to the Arduino Uno board.

Temperature_Sensor_bb

Ofcourse, we used a breadboard to connect all this together but I am simplifying the diagram here so that you know what is connected to which pin. The LM35 has 3 pins. The first one goes to the 5V power pin on Arduino, the 3rd one is the GND and the middle pin is the VOUT where it emits out the values that we need to capture. We connect this to the Analog Pin (A0) on the Arduino. We can then write our Arduino code to read that value, as is shown next.

Arduino Code

The Arduino Code is straight forward as given below:


float temp;
int tempPin = 0;

void setup()
{
 Serial.begin(9600);
}

void loop()
{
 temp = analogRead(tempPin);
 temp = temp * 0.48828125;
 Serial.print(temp);
 delay(10000);
}

You will notice in the loop that every 10 seconds, we are printing out the temperature value that was read from the Analog Pin (#0)

If you run the Serial Port Monitor that comes with the Arduino IDE and if the Arduino is powered up and connected as per the diagram shown, then you will find the Temperature value being printed on the Serial Monitor as given below:

orcpy2

Once this happens, we know that the Arduino setup is looking good and all we need to do now is to write a client program on the PC that interfaces with this Arduino, read the values via the Serial port and then push them to Firebase.

Introduction to Firebase

Firebase is a real-time database in the cloud that provides an API to store and sync data in real-time. Granted that we do not need the real-time features over here but the idea of this post is to demonstrate that you could use Firebase as a solution to storing your data in the cloud. It could act as a great aggregator of data from various devices and possibly for different use-cases even sync to other clients where you are monitoring stuff from.

Getting started with Firebase is free and there is a generous free quota that you can use today. One other added benefit of investing your time in Firebase is that it is now part of Google and we should expect some cool integration in the days to come within the realm of other services in Google Cloud Platform.

I have gone ahead and created a free account . The free account gives the following to you:

  • 50 Max Connections
  • 5GB data transfer
  • 100 MB Storage space

This should suffice for small projects. But if there you have a plan for a much larger number of devices, check out the other plans available at the Pricing page.

Creating our Firebase Application

Once you have signed up for Firebase and logged in with your account, you need to create an application. An application gives you a unique name space for your data. In our case, you can think of the application as Temperature Logger where I want to log the Temperature being collected from various weather stations (each setup of the Arduino + Temperature Sensor would be a single station).

So I have gone ahead and given the following information in the “Create New App” form as given below: (Of course use your own name and url as needed). Notice that the APP URL will need to be unique. since you are getting a namespace for your data within the overall firebaseIO.com space.

iotfb1

Clicking on CREATE NEW APP should create the application for you and you should see your Dashboard with a single app listed as given below:

iotfb2

That is all you need to do to setup an application. We are not going to delve into “Set up Hosting” since we do not want to host any application i.e. web pages, etc over here. We simply want our Firebase App to act as a data logger for the temperature readings that are being collected across various locations.

Click on Manage App now and you should be led to your unique APP URL i.e. https://your-app-name.firebaseio.com. In my case, I named my app romin-iot-templogger and hence the url is https://romin-iot-templogger.firebaseio.com

Creating a Firebase Data Model

When you visit the Manage Your App page for your application, you should find a screen that looks something like this:

iotfb3

You will notice that the Data option is selected and you are right in assuming that eventually we should be able to see our data over here.

Now, obviously we can just start dumping our data into Firebase but it helps to plan out how we want our data to be aggregated. I suggest the following links to the Firebase documentation : Understanding Data & Structuring Data.

Let us go back to what I want to do in the application. I am setting up various weather stations with their own unique names and each Weather Station is going to collect the Temperature + Date + Time (per reading).

So, what I want to do is collect these readings under each Location. My tree structure looks something like this:

Root

|—- Location 1

|—- Temperature Record 1 (Temperature Reading, Date, Time)
|—- Temperature Record 2 (Temperature Reading, Date, Time)
|—- …..
|—- Temperature Record N (Temperature Reading, Date, Time)
|—- Location 2
……
|—- Location N

Sounds ok?

It should be easy to now imagine how the Firebase REST API would function? First up, we will need 3 pieces of information:

  1. App URL : In my case it is https://romin-iot-templogger.firebaseio.com
  2. Data to post i.e. a JSON object that encapsulates the temperature reading, date and time. For e.g. {“date”:some_date,”time”:some_time,value=temperature_value}
  3. The path into which I should log this data. Remember that I wanted to each temperature record under its respective location. Hence I will need the location name and then the full path to this branch would be the APP_URL + “/” + “Temperature_Location_Name”.

Keep in mind that Firebase supports various Client Libraries, so you need not use the raw REST API to perform your client side operations. I usually prefer to understand the REST API since it helps to understand a little bit better on what is going on . Check out the documentation for a list of client side libraries that are available.

Python Code

Now, let us move on to the Python code that interfaces over Serial port to read the temperature values from the Arduino setup and posts that data into Firebase.

The steps are simple:

1) We initialize the serial port communication via which we will be reading the Temperature values that the Arduino unit will be emitting every 10 seconds. You need to figure out which Serial Port on your machine is interfaced to the Arduino.

2) Every 10 seconds, the code will read the value from the Serial Port. We can obviously build in more validations in the code, but this is good for now to demonstrate how all the pieces come together.

3) There are a few Python libraries available for Firebase but since they are not on the official list and hence I did not use them. I went with the requests package in Python and that serves the purpose well when dealing with a REST API.

Each Temperature record is being sent to Firebase as a JSON object with the following attributes:

  • Temperature in Centigrade
  • Date of the recording
  • Time of the recording

The URL to which we are posting is as explained above i.e.

APP_URL + “/” + “Temperature_Location_Name” 

and to that we are appending the “temperature.json” to indicate that it is a temperature reading.

Take a look at the Python client program below:

import serial
import time
import requests
import json

firebase_url = 'https://romin-iot-templogger.firebaseio.com'

#Connect to Serial Port for communication
ser = serial.Serial('COM15', 9600, timeout=0)

#Setup a loop to send Temperature values at fixed intervals
#in seconds
fixed_interval = 10

while 1:
 try:
 #temperature value obtained from Arduino + LM35 Temp Sensor
 temperature_c = ser.readline()
 #current time and date
 time_hhmmss = time.strftime('%H:%M:%S')
 date_mmddyyyy = time.strftime('%d/%m/%Y')

 #current location name
 temperature_location = 'Mumbai-Kandivali';

 print temperature_c + ',' + time_hhmmss + ',' + date_mmddyyyy + ',' + temperature_location

 #insert record

 data = {'date':date_mmddyyyy,'time':time_hhmmss,'value':temperature_c}

 result = requests.post(firebase_url + '/' + temperature_location + '/temperature.json', data=json.dumps(data))

 #insert record
 print 'Record inserted. Result Code = ' + str(result.status_code) + ',' + result.text
 time.sleep(fixed_interval)
 except IOError:
 print('Error! Something went wrong.')
 time.sleep(fixed_interval)

Checking our data

Assuming that all the hardware is wired up and you have your Python code ready, executing the code is straightforward. A sample run of the Python code is shown below:

27.55,16:50:38,09/03/2015,Mumbai-Kandivali
Record inserted. Result Code = 200,{'name':'-JjyFyyJDgm9Y82oOZM9'}

The final step was to validate if our data was being transmitted successfully and into Firebase. Let us visit the Dashboard for the IoT App in Firebase and you should see something similar to the screen shown below:

The Dashboard shown when a single record has been inserted:

iotfb4

Notice that Firebase automatically created the Location Name (Mumbai-Kandivali) and the node for the temperature record under it and finally the attributes (date, time, value) for each temperature reading.

Once we insert a few more records, the Dashboard should start looking similar to this:

iotfb5

This validates the end to end working of the project.

Viewing Data

You can check out your data via the Dashboard -> Data option. Alternatively, you can also navigate it via the standard tree / folder navigational style.

So, if we hit the following URL:

https://romin-iot-templogger.firebaseio.com/<LocationName&gt; then we will be lead to the specific data records only for that location.

For e.g. https://romin-iot-templogger.firebaseio.com/Mumbai-Kandivali is one of the locations and if we hit that in the browser, we see the following:

iotfb6

You want the JSON data, all you need to do is append the .json at the end.

So, if we hit https://romin-iot-templogger.firebaseio.com/Mumbai-Kandivali.json in the browser, you will get the data as shown below:

iotfb7

Securing our Firebase Data/API

You would noticed that our application is public and the API is open for both reading and writing. This is fine for demo purposes but not something that you want to keep if your data is sensitive and the clients need to be authenticated in some form or the other.

Ideally, we would also like to see fine grained permissions when it comes to security i.e. read and write operations should have their own user defined rules.

Firebase tackles that quite well and provides you with an intuitive way to take care of this.

But first up, let us understand what allows the read and write access to the API and why is it open by default. Firebase by default allows open read and write access to your database. To understand and see that, simply go to the Dashboard for your application and then click on Security & Rules as shown below:

iotfb8

You will find that the rules for both read and write are shown and they are currently set to true. This is Firebase’s way of keeping both read and write operations open. You are now free to remove the .write if you want and that way , it becomes a read only API and so on.

Not just that, but we can further add our own script like authentication to not just read and write but even at a fine grained level for each collection of data that we have. I suggest that you look up the documentation on security for this.

Our job for now is to do the following:

  1. Let our API be available for reading to all. No security is required over there. This means that we will keep the .read part as is.
  2. For write operations however, we would like to use a simple API Token mechanism. What this means is that your clients will be provided unique tokens and you can pass that token into each API call (write operations) and Firebase will validate those tokens for you and allow the write operations only if the token is valid.

But where do we get these tokens from? Who will generate it for us ? All this is taken care of by Firebase itself. It supports authentication via Secure JSON Web Tokens. These tokens can be generated by your Dashboard and you can use these tokens as is. Alternatively, if you want to generate them then they provide client libraries for that too (but that is beyond the scope of this article).

For now, go to your Application Dashboard and then click on Secrets as shown below:

iotfb9

This will show a list of Tokens that you already have. In fact you can generate more if you want via the Add New button.

iotfb10

It is this very token that you will need to pass from your client application while making a call to Firebase APIs that write data.

So the steps are as follows:

1. Generate a Token as shown above and note down the value. We will be modifying our Python application to use the Token later.

2. Modify the Security and Rules of this application to have the following block which defines the .write rule:


{
'rules': {
'.read': true,
'$name' : {
'.write': 'auth != null && !data.exists()'
}
}
}

What we have done in the above script block is to leave the read access open to all but changed the write condition to ensure that the auth variable is not null and that data exists. Keep in mind that Firebase will validate the authorization and the auth variable will be null if it is not successful.

Remember to click on “Save Rules” button to enforce the rules. 

If we try to post the temperature data now with the write rule in place, you will find that it gives the following output in the Python application:


27.35,18:59:05,09/03/2015,Mumbai-Kandivali
Record inserted. Result Code = 401,{
'error' : 'Permission denied'
}

Now, onto the last part of the puzzle i.e. how do we pass on the authentication token in the client code.

3. We need to modify our Python client a bit such that the authentication token is passed along with the HTTP POST Request. Firebase expects that you pass an additional parameter auth and put the token value in it. So all it means is something like ?auth=<token_value> in the URL.

You only need to change the line where we were creating the POST URL to include the additional ?auth=<token_value> as shown below. The auth_token is a variable in your Python application and it should contain one of the Token values that you have in the Secrets section of your application.


requests.post(firebase_url + '/' + temperature_location + '/temperature.json' + '?auth=' + auth_token, data=json.dumps(data))

Firebase supports a lot more than just token based security. It has a custom login, email based login plus integration with other popular login services like Google, Github, etc.

Summary

This blog post covered several things. First up, setting up a simple Arduino & temperature sensor to collect temperature readings. The next step was an introduction to Firebase, the real-time database available in the cloud and how we could use that as our database for the temperature logger system.

I look forward to your feedback. Till then, Happy Firebasing!

Note: If you are not interested in the Arduino part and simply want to work with Firebase, then I suggest that you could take the Python code and modify it by providing your own values or data records that you wish to persist in Firebase. Simply remove out the serial port piece of the code in the Python and substitute it with data and you are good to go.

 

Advertisements

5 thoughts on “Firebase IoT Tutorial

  1. Great post! Followed step by step and eventually get my data on the firebase. Just a question, when I look at the values (pressure reading from a barometric board) date and time look as yours while value is something like “1020.80\r\n”. I guess this are end of line char from arduino serial. How do i get rid of these, or, they could affect some further data processing like creating chart?

    1. You are right. I had put in a println() after printing the temperature value. This was done so that viewing it in the Serial Monitor is easy in the Arduino IDE.

      But, what you should do is simply delete that line of code. I have updated the code now and the extra println that was causing the ‘\r\n’ is now removed.

      Thanks for pointing it out.

    1. I am returning an error : Record inserted. Result Code = 404,not found, I am passing in gps data. (do I need to add the tables to the db side first?)I also end up getting a utf-8 decode error that ends up shutting the script down (probably due to the code im running on arduino.. either way good stuff.

      1. Thanks for the comments. Glad you liked the tutorial.

        Ideally, it should not be a 404. I do not think that you need to create the db side first since Firebase is able to cleverly map it out. What you need to be careful about is:
        1) Your base URL is correct i.e. https://.firebaseIO.com.

        I believe there is some mistake there which is resulting in a 404.

        The UTF-8 decode error could be more tricky to solve at times, Are you sending in some hard coded text copy – pasted from somewhere? How are you generating the data to be sent to Arduino?

        If its working for you now – that is great. Do share your points – will help me learn better. Thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s