IoT MQTT Infrastructure

Since the beginning of my home automation project I made several tests to choose the right platform and right controller. I've done test with various sensor technologies etc. etc.
Most of the articels can be found in the Arduino category.

One decision was, that my webhost should be the main datastore for all data measured. With this decision, I've started to build a data collection and distribution infrastructure. The complete setup consists of two major components:

These components are detailed further down the page.

The home infrastructure

The main purpose of the home infrastructure is the collection and distribution of sensor data. Sensor data can be things like outside temperature, luminosity level or the flow temperature of the main heating.

Here you find a brief overview of the home infrastructure:

Folie6

Sensors

I've implemented a hierachical sensor schema.

  • It consists of sensor nodes with a unique id (NID).
  • Each sensor can have any number of sensors, each identified by an id (SID) unique to this specific sensor.

The following picture tries to visualize this hierachie:

Folie10

This addressing scheme is called the "Sensor domain".

A single sensor node can a have any number of sensors. A sensor can be an administrative one like the power supply voltage of the sensor board or a environmental one like the outside temperature.

Fat sensors

 At the beginning, the arduino controlling the circulation pump for the hot water posts its data to the PHP-Script running on the webhost.

Heizung

This configuration is still in use and consists of:

nRF sensors

As the NordicRF data transceivers are currently available for less than $2 on ebay, I decided to attach some more sensors via these nRF transceivers to my home automation project These transceiver operate in the 2.4GHz ISM band - same as i.e. WLAN.

 

Folie9

For the controller part I've used the RF Duinode boards.

Each sensor transmits on a dedicated PIPE-ID on the nRF network. Each nRF node may host several sensors. Each sensor is defined by a unique sensor ID.

This address scheme is called the "nRF Domain".

RD Duinode

For the remote nodes, I use the RF Duinode boards which offer a neat way to build a low cost and low power sensor node. Informations how to integrate these Arduino-clones into the Arduino IDE can be found here.

The remote nodes usually transmits their data AES encrypted to the Raspberry Pi.

Currently I'm running the RF Duinode boards to measure the

  • temperatures of the floor heating
  • temperature in various rooms of our house
  • luminosity

Meteohub

A special sensor is my METEOHUB server. This server is connected to my weather station. For details on this configuration please check this article.

The METEOHUB server collects and stores all collected weather sensor data. To transfer this data in parallel to the MQTT infrastructure, a C++ application is running on the RaspberryPI which permanently reads the collected sensor data from the METEOHUB server and pushes the data to the message broker.

Message handling

But now the problem occurs, how to send data from these low cost sensors via ISM band to the webhost? The initial solution was to add a bridge between the nRF network and the webhost. As host for this bridge I selected a cheap RaspberryPI Model B to handle the data transfer between the nRF network and the webhost. The nRF transceiver is attached via a small interface board to the GPIO connector on the RaspberryPI. (PS: If someone is interested - I have one spare board to offer).

With this solution I was bound to the connection Sensor(ISM) - nRF-Receiver - WebHost. To get more flexibility I've decided to add a component between the nRF-Receiver and the component that talks to the webhost.

This component is a message broker. Details for my implementation can be found here.

Raspberry PI

On the RaspberryPI there are running various software components:

In the next chapters, these components are explained a little bit more in detail.

The RaspberryPI is mounted inside a sturdy case.

DSCF4123

The nRF module (I've used one with on-board power amplifier) is mounted on a daughterboard. This board can also carry a realtime clock and has same free solder area, on which I placed the 1-wire interface for the temperature sensor on the power supply module.

MQTT Broker

As a message broker implementation I'm using the MQTT broker mosquitto.

In my scenario the MQTT broker is used as a data collector and distributor.

  • All sensor data is posted to defined topics on the MQTT broker. Currently only the nRFBridge posts data to the MQTT broker.
  • All sensor data users subsribe to these topics. Currently only the WebBridge subscribes to data topics, to forward sensor data to the web host.

Topics

The topic handling is staight forward. All sensors post their data to

dl2sba.de/sensorData/NID/SID

Where NID is the sensors node id and SID is the sensor id at the sensor node.

nRFBridge

The nRF bridge is a C++ application running on the RaspberryPI.

The nRFBridge receives data from the distributed nRF RF Duinode boards. In a configuration file at the nRFBridge, there is a mapping between the nRF PIPE-ID in the nRF domain and the NID in the sensor domain.

The sensor ids map 1:1 from the nRF domain to the sensor domain!

Web Bridge

The nRF bridge is a C++ application running on the RaspberryPI. This applicatipon subscribes to the topic dl2sba.de/sensorData/+/+ to receive all relevant data.

Whenever new sensor data is received, this application post the received data to the web host.

As web host pages, as well as every sensor are password protected, the WebBridge hosts a password file to provide the correct passwords to the web host.

The internet infrastructure

 Web Host

Currently the data is received by a PHP-Script which inserts the data into a connected MYSQL database.

PHP