Analyzing IoT Data Using InfluxDB, Python, and Modbus

 

image credit researchgate

IoT Data Sources for Industrial and Smart Applications

IoT devices generate real-time data from various sensors. Below are some key IoT data sources and example use cases, focusing on an Arduino-based warehouse monitoring system with temperature and humidity sensors.

---

1. IoT Data Sources

1.1. Industrial and Smart Warehouse Sensors

• Temperature & Humidity Sensors (e.g., DHT11, DHT22, BME280) – Monitor warehouse climate.
• CO2 and Air Quality Sensors (e.g., MQ135) – Ensure air quality for workers and storage conditions.
• Light Sensors (LDR) – Adjust warehouse lighting automatically.
• Vibration Sensors – Detect abnormal equipment movements or seismic activity.
• RFID & Barcode Scanners – Track inventory movement.
• Weight Sensors (Load Cells) – Monitor stock levels in real-time.
• Motion Sensors (PIR) – Detect unauthorized movement in restricted areas.

---

2. IoT Warehouse Setup with Arduino & DHT11 (Temperature & Humidity)

Components Required

• Arduino Uno
• DHT11/DHT22 Sensor
• ESP8266 Wi-Fi Module (For sending data to InfluxDB)
• InfluxDB + Python for data storage & visualization

---

3. Arduino Code to Read Temperature & Humidity

This Arduino code reads temperature and humidity from a DHT11 sensor and sends it over Wi-Fi (ESP8266) to an InfluxDB server.

#include <DHT.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>

#define DHTPIN 2         // DHT11 sensor connected to pin D2
#define DHTTYPE DHT11    // DHT11 Sensor Type
DHT dht(DHTPIN, DHTTYPE);

// Wi-Fi Credentials
const char* ssid = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";
const char* server = "http://YOUR_INFLUXDB_SERVER/write?db=iotdb"; // InfluxDB URL

void setup() {
    Serial.begin(115200);
    dht.begin();
    WiFi.begin(ssid, password);
    
    // Connect to Wi-Fi
    while (WiFi.status() != WL_CONNECTED) {
        delay(1000);
        Serial.println("Connecting to Wi-Fi...");
    }
    Serial.println("Connected to Wi-Fi");
}

void loop() {
    float temperature = dht.readTemperature();
    float humidity = dht.readHumidity();

    if (isnan(temperature) || isnan(humidity)) {
        Serial.println("Failed to read from DHT sensor!");
        return;
    }

    Serial.print("Temperature: ");
    Serial.print(temperature);
    Serial.print(" °C | Humidity: ");
    Serial.print(humidity);
    Serial.println(" %");

    // Send Data to InfluxDB
    if (WiFi.status() == WL_CONNECTED) {
        WiFiClient client;
        HTTPClient http;

        String postData = "temperature_sensor temperature=" + String(temperature) + ",humidity=" + String(humidity);
        http.begin(client, server);
        http.addHeader("Content-Type", "application/x-www-form-urlencoded");
        int httpResponseCode = http.POST(postData);
        
        Serial.print("InfluxDB Response: ");
        Serial.println(httpResponseCode);
        
        http.end();
    }

    delay(5000);  // Wait for 5 seconds before the next reading
}

---

4. Query & Analyze IoT Data in Python

Once data is sent to InfluxDB, we can retrieve and analyze it using Python.

from influxdb import InfluxDBClient
import pandas as pd
import matplotlib.pyplot as plt

# Connect to InfluxDB
client = InfluxDBClient(host='YOUR_INFLUXDB_SERVER', port=8086, database='iotdb')

# Query Data
query = 'SELECT * FROM "temperature_sensor" ORDER BY time DESC LIMIT 100'
result = client.query(query)
data = list(result.get_points())

# Convert to Pandas DataFrame
df = pd.DataFrame(data)
df['time'] = pd.to_datetime(df['time'])

# Plot Data
plt.figure(figsize=(10, 5))
plt.plot(df['time'], df['temperature'], label="Temperature (°C)", marker='o')
plt.plot(df['time'], df['humidity'], label="Humidity (%)", marker='s')
plt.xlabel('Time')
plt.ylabel('Values')
plt.title('Warehouse Temperature & Humidity')
plt.legend()
plt.xticks(rotation=45)
plt.grid()
plt.show()

---

5. Real-World Use Cases of IoT Data

Industry	IoT Data Sources	Use Case
Smart Warehouse	Temperature, Humidity, RFID, Load Cells	Inventory tracking, Climate monitoring
Agriculture	Soil Moisture, pH, Light Sensors	Smart irrigation, Crop health monitoring
Manufacturing	Vibration, Pressure, Proximity Sensors	Predictive maintenance, Equipment health
Energy	Smart Meters, Power Sensors	Energy efficiency, Real-time consumption
Healthcare	Heart Rate, ECG, Oxygen Sensors	Patient monitoring, Remote healthcare
Transportation	GPS, Fuel Sensors, Tire Pressure	Fleet tracking, Fuel optimization

---

Conclusion

• IoT data comes from various sensors used in warehouses, industries, smart homes, and healthcare.
• Arduino with DHT11 can be used for warehouse temperature & humidity monitoring.
• ESP8266 sends data to InfluxDB, and Python processes and visualizes it.
• This approach can be extended with AI-based predictive analytics for anomaly detection. 🚀

Analyzing IoT Data Using InfluxDB, Python, and Modbus

Overview

IoT (Internet of Things) devices generate vast amounts of time-series data. InfluxDB is an excellent choice for storing and analyzing time-series data, while Python provides a powerful ecosystem for data retrieval and processing. Additionally, Modbus is a communication protocol commonly used for industrial automation.

This guide explains:

• Collecting IoT data using Modbus (for industrial sensors)
• Storing the data in InfluxDB
• Querying and analyzing the data using Python
• Visualizing the results

---

Step 1: Install Required Tools

Ensure you have the following installed:

• InfluxDB (for time-series data storage)
• Python (for scripting and data analysis)
• Modbus Libraries (for reading sensor data)

Install dependencies:

pip install influxdb pymodbus pandas matplotlib

---

Step 2: Set Up InfluxDB

1. Install InfluxDB

Download and install InfluxDB from https://portal.influxdata.com/downloads/.
For Docker:

docker run -p 8086:8086 -v influxdb:/var/lib/influxdb -e INFLUXDB_DB=iotdb influxdb

2. Create a Database

Start the InfluxDB shell:

influx

Create a database:

CREATE DATABASE iotdb;

---

Step 3: Collect Data from IoT Sensors Using Modbus

1. Read Sensor Data Using Python (Modbus TCP)

Modbus-enabled IoT sensors (like temperature, humidity, or pressure sensors) can be read using pymodbus.

Python Script to Read Modbus Data

from pymodbus.client.sync import ModbusTcpClient

# Connect to Modbus device (replace IP & port)
client = ModbusTcpClient('192.168.1.100', port=502)

# Read holding register (assuming sensor data is at register 100)
result = client.read_holding_registers(100, 1)
temperature = result.registers[0]  # Read first register

print(f"Temperature: {temperature} °C")

client.close()

Modify the IP address and register address based on your sensor.

---

Step 4: Store Data in InfluxDB

Once we fetch data from the Modbus device, we need to store it in InfluxDB.

1. Write Data to InfluxDB

from influxdb import InfluxDBClient
import time

# Connect to InfluxDB
influx_client = InfluxDBClient(host='localhost', port=8086, database='iotdb')

# Example sensor data (replace with Modbus read)
temperature = 25.4  

# Format data for InfluxDB
json_body = [
    {
        "measurement": "temperature_sensor",
        "tags": {
            "location": "factory1"
        },
        "fields": {
            "temperature": temperature
        }
    }
]

# Write data to InfluxDB
influx_client.write_points(json_body)
print("Data written to InfluxDB.")

---

Step 5: Query and Analyze IoT Data

Now that data is stored in InfluxDB, we can retrieve and analyze it using Python.

1. Query Data from InfluxDB

# Query temperature data
query = 'SELECT * FROM "temperature_sensor" ORDER BY time DESC LIMIT 10'
result = influx_client.query(query)

# Display data
for point in result.get_points():
    print(f"Time: {point['time']}, Temperature: {point['temperature']} °C")

---

Step 6: Visualize Data Using Matplotlib

We can visualize the sensor data to identify trends.

import pandas as pd
import matplotlib.pyplot as plt

# Query data
query = 'SELECT * FROM "temperature_sensor" ORDER BY time DESC LIMIT 100'
result = influx_client.query(query)

# Convert to Pandas DataFrame
data = list(result.get_points())
df = pd.DataFrame(data)

# Convert time column to datetime
df['time'] = pd.to_datetime(df['time'])

# Plot data
plt.figure(figsize=(10, 5))
plt.plot(df['time'], df['temperature'], marker='o', linestyle='-')
plt.xlabel('Time')
plt.ylabel('Temperature (°C)')
plt.title('IoT Sensor Temperature Data')
plt.xticks(rotation=45)
plt.grid()
plt.show()

---

Conclusion

• Modbus reads sensor data from IoT devices.
• InfluxDB stores time-series data efficiently.
• Python retrieves, processes, and visualizes the data.

This setup can be extended for real-time monitoring, alerts, and machine learning models for predictive analytics. 🚀

You can get the code and instructions downloaded directly from https://github.com/dhirajpatra/iot_data_analysis_with_influxdb

Most of the sensors and types of equipment I have purchased from the different stores including robu.in

Thank you

Comparison MongoDB and InfluxDB

                                             Photo by Acharaporn Kamornboonyarush

Let's compare MongoDB and InfluxDB by providing a simple example of how you can use both databases with Python for storing and retrieving time-series data. We'll use Python's official client libraries for both databases. This example will cover data insertion and retrieval operations.

MongoDB Example:

First, make sure you have the `pymongo` library installed. You can install it using pip:

```bash

pip install pymongo

```

Here's a simple Python example for using MongoDB to store and retrieve time-series data:

```python

from pymongo import MongoClient

from datetime import datetime

# Connect to MongoDB

client = MongoClient("mongodb://localhost:27017/")

db = client["timeseries_db"]

collection = db["timeseries_data"]

# Insert a time-series data point

data_point = {

    "timestamp": datetime.now(),

    "value": 42.0,

}

collection.insert_one(data_point)

# Retrieve data for a given time range

start_time = datetime(2023, 1, 1)

end_time = datetime(2023, 1, 2)

query = {"timestamp": {"$gte": start_time, "$lt": end_time}}

result = collection.find(query)

for doc in result:

    print(doc)

```

InfluxDB Example:

Make sure you have the `influxdb` library installed. You can install it using pip:

```bash

pip install influxdb

```

Here's a Python example for using InfluxDB to store and retrieve time-series data:

```python

from influxdb import InfluxDBClient

from datetime import datetime

# Connect to InfluxDB

client = InfluxDBClient(host="localhost", port=8086, database="timeseries_db")

# Insert a time-series data point

data_point = {

    "measurement": "time_series_measurement",

    "time": datetime.utcnow(),

    "fields": {"value": 42.0},

}

client.write_points([data_point])

# Query data for a given time range

query = f'SELECT "value" FROM "time_series_measurement" WHERE time > \'{start_time.isoformat()}\' AND time < \'{end_time.isoformat()}\''

result = client.query(query)

for point in result.get_points():

    print(point)

```

Comparison:

1. Data Model:

   - MongoDB: Uses a document-based data model.

   - InfluxDB: Specialized for time-series data with timestamp-based data model.

2. Query Language:

   - MongoDB: Uses a flexible query language, similar to SQL.

   - InfluxDB: Uses InfluxQL, designed for querying time-series data.

3. Write Performance:

   - MongoDB: Good for general purposes, but may not be as efficient as InfluxDB for high-frequency writes.

   - InfluxDB: Optimized for high write performance in time-series data scenarios.

4. Data Retention and Downsampling:

   - MongoDB: Requires manual management.

   - InfluxDB: Offers built-in retention policies and continuous queries for data downsampling.

5. Ecosystem:

   - MongoDB: Offers a wide range of use cases, suitable for various applications.

   - InfluxDB: Part of the TICK Stack (Telegraf, InfluxDB, Chronograf, Kapacitor), designed for time-series data monitoring and analysis.

In conclusion, while both MongoDB and InfluxDB can store time-series data, InfluxDB is purpose-built for this use case and provides better performance and features tailored for time-series data storage and analysis. Your choice should depend on your specific requirements and use cases. If you primarily deal with time-series data, InfluxDB is a strong candidate.