Overview
Wearable devices, such as fitness trackers and smart watches, often employ optical sensors to measure the wearer’s vital signs. Through a technique known as photoplethysmography (PPG), these sensors generate a signal which corresponds to changes in blood flow and volume within the subject’s skin tissue. This PPG signal must undergo varying levels of processing in order to obtain a reasonable estimate of vital signs such as heart rate, respiration rate, blood oxygen saturation, and even blood pressure. As heart rate is the simplest of these to ascertain, the project presented below demonstrates a straightforward method of deriving this value using an optical sensor.
While the utilized method is indeed straightforward, it does come with a trade-off. The filtering operation it employs is very computationally expensive, requiring nearly one hundred multiply instructions for every incoming data sample. Implementing this algorithm would prove challenging on most microcontrollers (MCUs), likely demanding that further trade-offs be made (e.g., lowering the sampling rate). The MCX N94 MCU from NXP, however, has an integrated DSP accelerator which allows this algorithm to execute quickly and affords greater flexibility to the embedded designer.
NXP’s solid offering of evaluation hardware, SDKs, and configuration tools greatly expedited the development of this project. The remainder of this article provides details regarding the chosen hardware components and how the firmware brings them together to implement a vital signs detection application.
Hardware
Figure 1 shows the project in action. Clearly, it is composed of three elements:
- the MCU development board (FRDM-MCXN947)
- the optical sensor daughter board (Heart Rate 10 Click)
- the LCD daughter board (LCD-PAR-S035)
Figure 1: A user obtaining heart rate information using the demo project by placing their finger over the optical sensor.
FRDM-MCXN947
At the base of this project is the FRDM-MCXN947 development board, which features the MCX N94x series MCU. The FRDM platform is an inexpensive evaluation option prioritizing I/O access with numerous headers and providing basic prototyping features such as push buttons, LEDs, and external flash memory. The FRDM-MCXN947 even includes a CAN connector, Ethernet connector, temperature sensor, and touch pad. Along with supporting the standard Arduino, mikroBUS, and Pmod form factors; camera and LCD headers are included as well. The SmartDMA/Camera header is compatible with camera modules such as the CAMERA-OV7670 and the FlexIO/LCD header is compatible with displays such as the LCD-PAR-S035.
This project utilizes both the mikroBUS and LCD headers to interface with the user. The Heart Rate 10 Click board is plugged into the mikroBUS socket to collect PPG data from the user’s finger and the LCD-PAR-S035 display is used to present a custom graphical user interface (GUI) in which the post-processed PPG data is shown as a live graph.
Optical Sensor
The fundamental operating principal of PPG sensors is a simple one: light is emitted from an LED into the skin tissue where a portion of it is absorbed by the circulating blood. The intensity of the light exiting the tissue (either through reflection or transmission) is measured by a photodetector, allowing a change in its intensity to be observed over time. This change corresponds to the changes in blood flow at different stages of the cardiac cycle.
Different wavelengths of light may be used depending on the application (e.g., which vital signs are being measured, the location of the photodetector relative to the light source, the activity level of the subject). The longer the wavelength of the light, the deeper into the tissue it will penetrate. For instance, light with a wavelength of 470 nm (i.e., blue light) will only reach as far as the epidermis where the capillaries reside. Wavelengths of 570 nm (i.e., green light) can go deeper and reach the dermis layer, which contains the arterioles. To reach the arteries found in the hypodermis, wavelengths of 640 nm to 940 nm (i.e., red light and IR light) are required. Many PPG sensors utilize multiple LEDs which are quickly switched on and off in sequence to obtain multiple channels of data. Figure 2 provides a simplified illustration of this, where four LEDs are employed by a reflective-type sensor to collect data from different depths of a human fingertip.
Figure 2: The core operating principal of reflective-type optical sensors which utilize multiple wavelengths of light.
The MAX86916 optical sensor from Analog Devices is designed to target wearable and mobile devices. Integrated within its 3.5mm x 7.0mm x 1.5mm package are four LEDs (IR, red, green, and blue); a photodiode; and low-noise circuitry for ambient light cancellation, crosstalk cancellation, and some digital filtering. Maximum flexibility is granted by allowing the host to configure which LEDs are utilized, the order in which they are sequenced, the intensity of each LED, as well as their pulse width. The internal ADC has 19-bit resolution and can be programmed for a data rate up to 3200 samples per second (sps). Data is stored in a 32-sample deep FIFO until the host processor is ready to retrieve it over the I2C bus.
MikroElectronika offers the Heart Rate 10 Click expansion board as a means to easily interface with the MAX86916 sensor. Aside from the sensor itself; it includes a 1.8V LDO, a voltage-level translator for the I2C bus, and a pull-up resistor for the interrupt signal. This board is connected to the FRDM-MCXN947 board via the mikroBUS socket to establish a connection to the MCU.
MCX N947
NXP’s MCX N series MCUs target applications that require both high computational performance and low power consumption. Available features such as dual Arm Cortex-M33 cores, co-processors, accelerators, and intelligent peripherals facilitate expedited, autonomous, and parallel operations. Thereby, latency and power consumption can be greatly reduced. The MCX N94x is the top-of-the-line in this series, including all of the aforementioned features as summarized by it’s block diagram in Figure 3.
Of particular interest to this project is one of the blocks in the Accelerator section: the PowerQuad DSP co-processor and accelerator. This peripheral is intended to assist the Cortex-M33(s) with common digital signal processing (DSP) tasks. As shown in it’s block diagram in Figure 4; it contains several computation engines capable of FIR filtering (i.e., convolution), FFT/IFFT operations, matrix operations (e.g., multiplication, inversion, dot-product, scaling), dual biquad IIR filtering, trigonometric functions, transcendental functions, and CORDIC (i.e. Volder’s algorithm) for inverse trigonometric functions.
Each of the PowerQuad’s engines makes use of some or all of the four floating-point multiply accumulator (MAC) blocks, enabling computations to be performed in parallel. For instance, the FIR filtering engine can run four MAC operations in parallel which markedly speeds up the underlying convolution operations. Faster convolution implies faster filtering, which in turn means a) the MCU can spend more time in a low-power state or b) the signal processing chain can become more capable.
Power consumption is of little concern for this simple project, so we go with option (b) and employ the PowerQuad to filter each of the four PPG data channels with a large, 99-tap FIR high-pass filter (HPF). In doing so, the DC offset and baseline wander will be eliminated and the resulting signal will be centered about zero. While this effect could be achieved using less computationally expensive filtering methods, it would result in significant deformation of the PPG waveform. Although this is not necessarily an issue for simple heart rate detection, we do desire that the PPG data charted on the LCD display accurately expose the features related to cardiac activity.
Note: While adding more taps can indeed improve filter performance, it does present another disadvantage aside from increasing the computation time. The group delay of an FIR filter is half the filter order (\frac{N-1}{2}), meaning that as the number of filter coefficients grows, so too does the delay introduced in the output signal. Thus, the 99-tap filter we employ delays the PPG signal by 49 samples. While this is acceptable for our project, it is up to the application designer to determine how much delay is tolerable in their particular system.
Firmware
The MCUXpresso project for this application can be found in it’s GitHub repository. To find the high-level application code implementing the data collection, data processing, and GUI control; navigate to the PpgTask() function in the source/lvgl_guider.c file.
PPG Data Acquisition and Processing
Before collecting data from the MAX86916 optical sensor, it first must be configured by modifying its internal registers over the I2C bus. Doing so is straightforward and said registers are well documented in the device’s datasheet. For this application the device is configured to illuminate each of the four LEDs in sequence and measure the reflected light intensity for each, thereby producing four channels of PPG data. The sampling rate is set to 800 sps and the “sample average” setting is set to 16. This way, the sensor will internally average every 16 samples and write the result to the FIFO for the MCU to read, yielding an effective sampling rate of 50 sps. Once 17 of these samples are written to the FIFO, an interrupt signal will be sent to the MCU on the interrupt pin. The proximity function of the sensor is enabled as well.
With the proximity function enabled, the IR LED will serve dual purpose as a proximity detector to determine whether or not an object is near the sensor. Only when an object is close enough to the sensor will it enter data acquisition mode. In this context, the user placing their finger over the sensor will trigger the LED sequencing operation and sample data accumulation in the FIFO. The MCU will be periodically signaled via the interrupt pin to read data from the FIFO before it becomes full. Should the object move away from the sensor, it will automatically exit data acquisition mode to reduce power consumption.
Once the MCU has read a window of sample data from the sensor’s FIFO, it is high-pass filtered with the help of the PowerQuad DSP co-processor as described in the previous section. The results of doing so are shown in the figure below. Notice that the original waveform (in black) has a high DC offset and the baseline of the signal wanders over time. By attenuating these low frequency components and allowing the high-frequency components to remain unaffected, we are left with the zero-centered waveform shown in red. Notice also that the shape of the filtered PPG signal has not been visibly distorted.
Once enough samples are collected and filtered, they can be used to derive an estimate of the user’s heart rate. While there are many sophisticated methods and algorithms that may be employed for this purpose, this project utilizes a very straightforward approach for the sake of simplicity. It works by detecting and time stamping negative zero-crossings as a means of indexing individual heart beats. The heart rate estimate can then easily be derived by averaging the time spans between subsequent heart beats and converting this “samples per beat” metric to one of “beats per minute”.
Graphical User Interface
The LCD daughter board is used to display a simple GUI which presents the most recent filtered PPG data as well as the estimated heart rate value to the user. It was designed using NXP’s GUI Guider tool, which provides a drag-and-drop editor for placing and configuring widgets such as charts, progress bars, labels, and images. The screenshot in Figure 6 shows which widgets were utilized for this project and how they were laid out. Once the GUI design is complete, the tool can be used to generate C code to be implemented in an MCUXpresso project.
The tool uses the open-source LVGL graphics library to implement the GUI elements and the application code uses its high-level interface to control the GUI’s behavior and update the data to be displayed. This includes updating the chart elements with the latest PPG data, updating the heart rate label with the latest derived estimate, and triggering various animations to provide smooth transitions between the different application states. In fact, coding the animations has been made simpler thanks to the GUI Guider’s generated ui_animation() function. With it, each animation in this application can be implemented with only a single line of code.
As previously mentioned, data acquisition is triggered in this application once the user places their finger on the optical sensor. As initial samples are collected by the MCU, a progress bar will proportionately be filled until the group delay of the filter has been surpassed. Once completely filled, the progress bar fades out and the labels/image fade in. Then, as long as the user continues to make contact with the sensor, the four charts will continuously be updated with filtered data samples, the heart rate value will updated every time a negative zero-crossing is detected, and the heart image in the upper right-hand corner will periodically “beat” with a zoom in/out animation.
Once the user removes their finger and the optical sensor exits data acquisition mode, the GUI updates and animations are frozen. Replacing the finger causes the GUI to reset and start from the beginning.
Conclusion
This simple project barely scratches the surface of what is possible with NXP’s MCX N94x series of MCUs. The eIQ Neutron NPU in particular opens the door for many advanced edge AI applications by dramatically reducing the inference time of embedded neural networks. For instance, a CNN or RNN model could be utilized in place of the simple heart rate estimation algorithm implemented above. Such a deep learning approach has the potential to provide much more accurate and robust heart rate estimates given a very noisy PPG signal.
Of course, the list of potential applications extends well beyond the domain of healthcare. This MCU series is an excellent solution for motor control and robotics challenges in the industrial space as well as smart home products such as surveillance cameras and smart speakers. Whatever your application, I hope you find this project a useful resource on your getting started journey. If you have any questions, I encourage you to hit the Reply button and let me know.