Internet of things (IoT) with Cortana, Raspberry Pi and a weather sensor

BlogLeadImage_loTCortanaIntelligence.png
In this post Leila Etaati explores the internet of things to see how we can collect information from different devices, sensors and applications, track data changes via software and even react to those changes. Then we can use machine learning to further analyse the data.

Imagine a greenhouse full of fabulous plants and flowers. We need to track the temperature and humidity of the greenhouse as the flowers are sensitive to these measures. A sudden change in greenhouse temperature and humidity can harmful. So we would like to monitor the humidity and temperature level live and from different locations. We also need to find out if there’s a sudden change in the temperature. For this, an anomaly detection process via machine learning can be helpful.

Greenhouse measuring the temperature and Humidity

We are going to use Microsoft Cortana intelligence, Raspberry Pi 3, and Spark fun weather station sensors to help the owner of greenhouse with live data they can access and monitor from anywhere.

But before I get into that, some introductory information about IoT, Cortana intelligence, Raspberry Pi 3, and the weather station. Then I will describe how we can use these components together to get live data from sensors, upload it to the cloud (to be accessible for every one), and receive some intelligence from the data (detecting anomalies using machine learning).

Internet of things (IoT)

The internet of things, or IoT, is about the internetworking of physical devices: “connecting devices over the internet, and letting them talk to us, applications, and each other” (Kobie, 2015). Domestic applications of IoT (eg smart meters, security sensors) are the foundation for smart homes, and when scaled up drive smart cities and manufacturing automation.

Cortana Intelligence Suite

Cortana is Microsoft’s fully managed big data and advanced analytics suite. It includes four main components: Information Management, Big Data Stores, Machine Learning and Analytics, and Dashboard and Visualization.

Cortana Intelligence Suite

Information Management: where data is collected from different resources.

Big data stores include SQL Database, Azure SQL Data Warehouse and Azure Data.

Machine Learning and Analytics includes machine learning and big data modules.

Dashboard and Visualization different ways to understand, present and visualise data.

Related post on our tech blog: Cortana Analytics Suite: first look 

Raspberry Pi 3

Raspberry Pi 3

The Raspberry Pi is a credit card-sized single-board computer developed in the United Kingdom by the Raspberry Pi Foundation to promote the teaching of basic computer science in schools and developing countries. The Raspberry Pi hardware has evolved through several versions that feature variations in memory capacity and peripheral-device support. It can have the performance of a computer. Following are the specifications of the Raspberry Pi 3 used in my experiment.

Raspberry Pi 3 specs

To install Windows, we need to use the SD memory card.

Raspberry Pi SD card

SparkFun Weather Shield (Sensor)

The Weather Shield comes as a stand-alone board. It is easy to use and it can measure humidity, temperature, wind speed, pressure and so forth.

Sparkfun weather shield (sensor)

The experiment

We want to create a project that gets data from SparkFun sensors and loads it into the cloud so anyone anywhere can see changes in weather, using a reporting tool like Power BI.

First set up the Raspberry Pi 3 and Spark fun.

The first step is to connect the Sparkfun and Raspberry Pi 3. Raspberry Pi 3 has a place we can connect the sensors to, and I just followed the provided instructions.

conecting the Raspberry Pi to the Weather station

Install Windows IoT on Raspberry Pi 3

After the hardware is set up, now we should set up the operating system on the Raspberry Pi 3 and then connect it to the SparkFun sensor. (Read detailed guidelines for installing Windows on Raspberry Pi 3).  

First we need to identify the hardware that we are going to use, in our case Raspberry Pi 3 has been used to get the data from weather sensor. Then we should select the installation media: SD memory card. For installation there are two different OS available: Windows 10 IoT Core and Windows 10 IoT core insider preview.

 select hardware

select installation media
select OS version

Connect the Raspberry Pi 3 to Spark Fun sensor

Then we need to program the Raspberry Pi 3 to read data from the SparkFun sensor. To do that we need to set up a Windows universal application in Visual Studio that can remotely run the code on Raspberry Pi 3.

Windows universal application

 

I won’t get into the technical process here, but there are full instructions here

Now our Raspberry Pi 3 is connected to the sensor and can fetch humidity, pressure, temperature and altitude from the environment.

weather data

Sending the Data from Raspberry Pi 3 to Cloud (Cortana Intelligence Suite)

Now we have the weather data, we want to send it to the cloud (Cortana Intelligence Suite).

We need these components of Cortana Intelligence to send the weather data to Power BI, to show change and allow Azure ML to extract insights. 

Cortana components needed

First we get the weather information in JSON format from the Raspberry Pi 3 (which has already fetched it from the sensor). This data is then sent to Azure Service Bus.

the main process to get data from sensor to show on Power BI and Azure ML

1. Azure Service bus works as a connector between all external components and Cortana Intelligence Components. First, an Azure Service bus should be created. Then an Azure Event Hub should be installed upon it .

Azure Event Hub gets the data from different devices out of Cortana Intelligence. Azure Event Hub can only get the data from devices but can’t send the information to them.

 

Azure service bus

 

Now we want to send the collected data to Power BI or Azure ML. To do that, we have to use an internal bus in Cortana Intelligence named “Azure Stream Analytics”.

2. Azure Stream Analytics is an internal service bus that can transform data inside the Cortana Intelligence suite. Azure Stream analytics has three main components: Input, Output and Query. It can get data from Event Hub, Azure Storage Blob, and Azure IoT and put the output data into Azure SQL Database, Azure Blob, Power BI, Azure Table, Azure Data Lake, Azure Document and so forth.

input-query-output

However, for this transformation a query is needed. The query is so similar to SQL scripts it’s called Analytics SQL! As can be seen in the picture below, we select all information from “weatherStation” which is our Event Hub and then insert into “PBI” – the Power BI report and then into SQLDB for storing the data in a SQL database. 

weather IoT

3. Power BI

Azure Stream Analytics automatically creates a Power BI dataset in the Power BI website. We can now create a report there:

Power BI

The live data will be sent from the SparkFun sensor to Raspberry Pi 3, then to Event Hub and now can be seen live on the Power BI website dashboard as below:

live data in the Power BI website dashboard

We can use the power query Q&A to see the average temperature for the last 60 seconds.

power BI Query Q&A

4. Azure ML

We can also send live data about the temperature into Azure ML for further analysis. To do this in Azure Stream analytics (discussed above), we can identify a function.

Azure Stream Analytics: set up function for calling Azure ML API

new function

With this function we can call the web service already created in Azure ML. We just need to create a new Function in Azure Stream Analytics and then call the function from Azure Stream Analytics Query.

The Azure Stream Analytics will call the Azure ML API (Web service) and the result can be shown in Power BI or any other service.

 

Applications of IoT

Tracking data from different sources and monitoring live data from different devices (sensors, cameras, electronic devices and so forth) and from different locations is a new strategy for many companies, made possible by IoT. In this post we have shown a simple example of this, how we can get weather data from a weather sensor using Raspberry Pi 3 and then using the Cortana intelligence suite to monitor and even get some insight out of it. There are many possible applications for this kind of process, in industries such as farming, manufacturing and navigation systems.  

Video of the experiment 

This video shows the display of live data from a weather sensor, connected to a Raspberry Pi, on a Power BI dashboard. The Windows application on our laptop runs the code remotely on Raspberry Pi 3, which gets the data from the weather sensor. Then, the gathered data is sent to the cloud (Azure Event hub). Then weather information can be seen on the Power BI dashboard. It shows live weather information for the last 60 seconds. When we put the sensor onto a mug of hot water we can see the changes to temperature on the Power BI dashboard. 

Leila Etaati (1).jpg

Leila Etaati is a senior Microsoft BI and analytics consultant and data scientist. She recently completed a PhD in information technologies (tourism recommendation engines) from the University of Auckland.

Follow Leila on Twitter: @leila_etaati