Set up and test IoT MQTT Broker with Raspberry Pi, PyPortal Titano and machinechat's JEDI One

ScottRaeker · June 29, 2021, 2:57pm

Description

This project sets up an IoT MQTT broker on a Raspberry Pi 4 using machinechat’s JEDI One IoT data management software. JEDI One includes a data collector that can be configured as an MQTT broker to which external client devices can publish and(or) subscribe to topics on the JEDI One. To test the MQTT broker, an Adafruit PyPortal Titano is set up as a client device that both subscribes and publishes to the MQTT broker on the Raspberry Pi over WiFi.

Background

MQTT (Message Queuing Telemetry Transport) is an open OASIS and ISO standard which defines a lightweight, publish-subscribe network protocol that transports messages between devices. It includes two types of network entities: a message broker and a number of clients. machinechat’s JEDI One IoT platform includes a data collector that can be configured as an MQTT message broker. External client devices can be configured to publish data or subscribe to a topic on the JEDI One (topics include all data coming into JEDI One from any source, not just MQTT). JEDI One requires publish messages to be in JSON payload format and provides subscribe messages in JSON payload format.

Hardware

RASPBERRY PI 4B/4GB
Raspberry PI 4 Model B with 4GB SDRAM
Adafruit PyPortal Titano
PyPortal IoT Kit - Arduino IDE, Circuit Python Starter Kit

Software

JEDI One
JEDI One is a ready-to-use IoT data management software solution. Capabilities include: collect data from sensors, devices and machines; build intuitive real-time and historical data and system view dashboards; create rules to monitor and respond to data conditions automatically; receive alert notifications by email and SMS.
CircuitPython
CircuitPython is Adafruit’s branch of MicroPython designed to simplify experimentation and education on low-cost microcontrollers. No compiler, linker or IDE is required.

Implementation

For this project, the JEDI One application had previously been installed on a Raspberry Pi with the the HTTP data collector set up and receiving external sensor data (for instructions and details see Machinechat with ESP32 and TE Connectivity MS8607 sensor). The JEDI One MQTT data collector is then set up. For testing the system, a PyPortal Titano is configured to both subscribe and publish to topics on the MQTT broker. CircuitPython is used to implement the application code on the PyPortal.

Set up the JEDI One MQTT broker

1 - If machinechat JEDI One is not already installed on the Raspberry Pi see below:

Get Raspberry Pi version of JEDI One DK-JEDIONE-RP
Install on Pi, see Raspberry Pi - Installing JEDI One as a Service

2 - Configure the MQTT broker

In the JEDI One “Data Collector” tab, select “Add Collector” and configure. Name the “Data Collector” and select “MQTT Broker” for “Collector Type”. “Listen IP” in the MQTT collector setup screenshot is the JEDI One Raspberry Pi IP address and “Listen Port” is 1883. (note: below shows example unencrypted configuration but TLS encryption can be configured as shown in machinechat Product Guide - How to Generate TLS Certificates and Keys )

Set up the PyPortal CircuitPython MQTT client test application

The Pyportal test application has three parts:

MQTT client that subscribes to existing sensor data that is being collected on the JEDI One

image947×302 26 KB
MQTT client that publishes sensor data to the JEDI One
Display application that prints the subscribed sensor data to the PyPortal Titano display

image2820×2034 625 KB

1 - Set up CircuitPython on the PyPortal Titano. See link https://learn.adafruit.com/adafruit-pyportal-titano/circuitpython
(note: this project used CircuitPython 6.3.0)

2 - Install libraries needed for application:

Verify the following libraries are installed in CIRCUITPY (D:\lib). Libraries can be downloaded from Libraries . (note: this project used adafruit-circuitpython-bundle-6.x-mpy-20210618.zip)

import neopixel
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_esp32spi import adafruit_esp32spi_wifimanager
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
import adafruit_minimqtt.adafruit_minimqtt as MQTT
from adafruit_pyportal import PyPortal
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text import label

3 - Code walkthrough (filename: mqtt_jedi_display.py)

Initial setup code:


import time
import board
import busio
from digitalio import DigitalInOut
import neopixel
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_esp32spi import adafruit_esp32spi_wifimanager
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
import adafruit_minimqtt.adafruit_minimqtt as MQTT
import json
import sys
from adafruit_pyportal import PyPortal
import displayio
import terminalio
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text import label

unique_id = "ppt10:52:1C:88:B6:F8" # set unique id based on pyportal MAC
latest_msg = "test"                # set initial message

display = board.DISPLAY
main_group = displayio.Group(max_size=10)
MEDIUM_FONT = bitmap_font.load_font("fonts/Arial-16.bdf")
BIG_FONT = bitmap_font.load_font("fonts/Arial-Bold-24.bdf")

### WiFi ###

# Get wifi details and more from a secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

# If you have an externally connected ESP32:
# esp32_cs = DigitalInOut(board.D9)
# esp32_ready = DigitalInOut(board.D10)
# esp32_reset = DigitalInOut(board.D5)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)
"""Use below for Most Boards"""
status_light = neopixel.NeoPixel(
    board.NEOPIXEL, 1, brightness=0.2
)  # Uncomment for Most Boards

wifi = adafruit_esp32spi_wifimanager.ESPSPI_WiFiManager(esp, secrets, status_light)

Set up MQTT publish and subscribe feeds, print to PyPortal Titano display

### Feeds ###

# Setup a feed named 'photocell' for publishing to a feed
photocell_feed = "datacache/photocell"  # modify feed for JEDI One MQTT broker


# Setup a feed named 'pump' for subscribing to changes
pump_feed = "datacache/T960981B2D"     # map feed to JEDI One MQTT broker data stream


### Code ###

# Define callback methods which are called when events occur
# pylint: disable=unused-argument, redefined-outer-name
def connected(client, userdata, flags, rc):
    # This function will be called when the client is connected
    # successfully to the broker.
    print("Connected to JEDI One MQTT broker ! Listening for topic changes on %s" % pump_feed)
    # Subscribe to all changes on the pump_feed.
    client.subscribe(pump_feed)


def disconnected(client, userdata, rc):
    # This method is called when the client is disconnected
    print("Disconnected from JEDI One!")


def message(client, topic, message):
    # This method is called when a topic the client is subscribed to
    # has a new message.
    global latest_msg
    print("New message on topic {0}: {1}".format(topic, message))
    latest_msg = message
    parsed = json.loads(latest_msg)
    data1 = "Humidity " + str(parsed["data1"])
    print(data1)
    print(parsed["data2"])
    print(parsed["data3"])
    print(parsed["timestamp"])
    print(latest_msg)


# Connect to WiFi
print("Connecting to WiFi...")
wifi.connect()
print("Connected!")


# Initialize MQTT interface with the esp interface
MQTT.set_socket(socket, esp)


# Set up a MiniMQTT Client with JEDI One MQTT broker
#  set client_id to unique id for connecting to mqtt broker
mqtt_client = MQTT.MQTT(
    broker="192.168.1.7",
    port = 1883,
    client_id = unique_id,
)

# Setup the callback methods above
mqtt_client.on_connect = connected
mqtt_client.on_disconnect = disconnected
mqtt_client.on_message = message

# Connect the client to the MQTT broker.
print("Connecting to JEDI One MQTT broker...")
mqtt_client.connect()

# print header message on display
text_area1 = label.Label(BIG_FONT, text="Pump House Monitor", max_glyphs=40)
text_area1.x = 10
text_area1.y = 10
main_group.append(text_area1)
display.show(main_group)


data1 = "1234567"  #set default display message
text_area = label.Label(MEDIUM_FONT, text=data1, max_glyphs=40)
text_area.x = 10
text_area.y = 100
main_group.append(text_area)
display.show(main_group)

text_area2 = label.Label(MEDIUM_FONT, text="abcde", max_glyphs=40)
text_area2.x = 10
text_area2.y = 200
main_group.append(text_area2)
display.show(main_group)

# photocell is a simulated sensor used by the pyportal to test publishing to JEDI One MQTT broker
photocell_val = 0  # set initial photocell value

# map photocell data dictionary object in prep to be compatible with JEDI One mqtt broker
photocell_dict = {}
photocell_dict["deviceType"] = "photocell"
photocell_dict["value"] = photocell_val
print(photocell_dict)

while True:
    # Poll the message queue
    mqtt_client.loop()
    if latest_msg != "test":  # check to see if new mqtt subscribe message
        print("received mqtt subscribe message")
        parsed = json.loads(latest_msg)
        data1 = "Humidity " + str(parsed["data1"]) + " %"
        data2 = "Temperature " + str(parsed["data2"]) + " F"
        print(data1)
        text_area.text = data1   # update humidity value for display
        text_area2.text = data2  # update temperature value for display
        latest_msg = "test"  # reset latest message

    photocell_dict["value"] = photocell_val       # update to latest photocell value
    photocell_json = json.dumps(photocell_dict)   # convert photocell data to json format for JEDI One mqtt broker
    print(photocell_json)
    # Send a new message
    print("Sending photocell json string: " + photocell_json)
    mqtt_client.publish(photocell_feed, photocell_json)
    print("Sent!")
    photocell_val += 1
    if photocell_val > 100: # reset counter for simulated sensor data
        photocell_val = 0
    print(latest_msg)
    print(data1)
    time.sleep(11)

Latest source code for the MQTT client test application is on github at below link:

github.com

eewiki/machinechat/blob/master/PyPortalMQTT_Client/mqtt_jedi_display.py

# file name mqtt_jedi_display.py
# application: use pyportal titano to subscribe/publish to JEDI One mqtt broker and display data on pyportal titano display
# SR initial6/24/2021
# modified 6/25 to add additional mqtt fields to subscribe serial message print
# update 7/1 add in code for display, add json send data from pyportal
# update 7/21 add in code to add second MQTT subscribe for outdoor sensor and display
# below code updated from original Adafruit minimqtt code example
# SPDX-FileCopyrightText: 2021 ladyada for Adafruit Industries
# SPDX-License-Identifier: MIT

import time
import board
import busio
from digitalio import DigitalInOut
import neopixel
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_esp32spi import adafruit_esp32spi_wifimanager
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
import adafruit_minimqtt.adafruit_minimqtt as MQTT
import json

This file has been truncated. show original

Conclusion

The combination of machinechat’s JEDI One data management software and a Raspberry Pi results in a standalone, low cost, easy to use IoT MQTT broker platform. Client devices can easily be configured to subscribe and/or publish to topics on the MQTT broker platform without the need of third party cloud services or an internet connection.

References

Adafruit - PyPortal Titano
machinechat Product Guide - Built In MQTT Broker - Data Collector
machinechat Product Guide - MQTT Broker - Subscribing to a topic
HIVEMQ - MQTT Essentials
Machinechat JEDI One MQTT Broker Subscribe Example using Arduino / ESP8266
Getting Started with machinechat’s JEDI One IoT Platform

Topic	Replies	Views
Machinechat JEDI One MQTT Broker Subscribe Example using Arduino / ESP8266 machinechat	1937	April 21, 2021
用樹莓派、PyPortal Titano 和 machinechat JEDI One 設定和測試 MQTT 伺服器產品技術提示 raspberry-pi , machinechat , adafruit	115	May 22, 2024
라즈베리 파이, PyPortal Titano 및 Machinechat의 JEDI One으로 IoT MQTT 브로커 설치 및 테스트 하기 기술 조언 machinechat , tech-tips , wireless-and-iot	181	January 10, 2024
Set up Wio Terminal as MQTT Display for machinechat JEDI One Sensor Data machinechat	2251	September 9, 2021
Raspberry Pi、PyPortal Titano、およびmachinechatのJEDI OneによるIoT MQTTブローカのセットアップとテスト電子部品ペディア　～基礎、応用、実践～	275	June 27, 2023