Sound → Preprocessing → STFT → Image → TinyML Model → Ripeness Prediction
Durian’s fruit maturity can be inferred from its corresponding vibration frequency response. Therefore, the STFT algorithm is applied to extract this feature. Figure 2 illustrates the signal pipeline.
Firstly, a segment of one second raw acoustic signal is sampled at the rate of 8kHz.
Then normalization is applied to eliminate the differences in signal strength. Next, STFT is implemented to obtain the signal’s corresponding amplitude spectrum.
Finally, since the MAX78002’s CNN accelerator only supports square-shaped input data, the STFT spectrum is zero-padded and folded before fed into the TinyDurianNet for inference.
In this solution, STFT spectrums are converted into RGB images to be used as datasets.
The initial dataset has 105 samples (35 for approaching maturity, 35 for fully mature, and 35 for immature). The data was augmented by randomly sliding the STFT spectrum in the window to analog the behavior of real slide window sampling, as shown in Figure 3. In this case, the dataset is updated to 1155 samples: 20% of the samples are used as a test set and 80% of the samples as a train set.
Applicable Part Numbers
| DigiKey Part Numbers | Manufacturer Part Number |
|---|---|
| MAX9867ETJ+TTR-ND,MAX9867ETJ+TCT-ND,MAX9867ETJ+TDKR-ND | MAX9867ETJ+T |
| MAX9867EWV+TTR-ND,MAX9867EWV+TCT-ND | MAX9867EWV+T |
| MAX9867ETJ±ND | MAX9867ETJ+ |
| 505-MAX9867ETJ+G3U-ND | MAX9867ETJ+G3U |
| MAX9867ETJ+TG3U-ND | MAX9867ETJ+TG3U |
| 505-MAX9867EWV+G47-ND | MAX9867EWV+G47 |
| 505-MAX78002GXE±ND | MAX78002GXE+ |
| 3318-PG1302XA-NA0-ND | PG1302XA-NA0 |
| 3318-PG2405CA-NA0-ND | PG2405CA-NA0 |
| 757-NHD-2.4-240320AF-CSXP-ND | NHD-2.4-240320AF-CSXP |
| NHD-2.4-240320CF-CSXN#-F-ND | NHD-2.4-240320CF-CSXN#-F |
| 1548-NHD-2.4-240320CF-BSXV-F-ND | NHD-2.4-240320CF-BSXV-F |
| 757-NHD-2.4-240320CF-BSXV-FT-ND | NHD-2.4-240320CF-BSXV-FT |
| NHD-2.8-240320AF-CSXP-F-ND | NHD-2.8-240320AF-CSXP-F |
| 1597-104990802-ND | 1.05E+08 |
| 757-NHD-2.4-240320AF-CTXP-ND | NHD-2.4-240320AF-CTXP |
| 5491-AFY240320B0-2.4INTH-ND | AFY240320B0-2.4INTH |
| 3387-ATM0200B3-ND | ATM0200B3 |
| NHD-2.4-240320CF-CSXN#-FT-ND | NHD-2.4-240320CF-CSXN#-FT |
| 5491-AFY240320A1-2.8INTH-ND | AFY240320A1-2.8INTH |
| 757-NHD-2.4-240320AF-CTXP-T-ND | NHD-2.4-240320AF-CTXP-T |
| 757-NHD-2.4-240320AF-CSXP-T-ND | NHD-2.4-240320AF-CSXP-T |
| 4863-WF0200BSYAJDNN0-ND | WF0200BSYAJDNN0 |
| 3390-GLT035240320WH1-ND | GLT035240320WH1 |
| 3086-MDT0280A1IH-MULTI-ND | MDT0280A1IH-MULTI |
| 1528-4421-ND | 4421 |
| 1597-104990803-ND | 1.05E+08 |
| 5491-AFL240320A0-2.0INTM-ANO-ND | AFL240320A0-2.0INTM-ANO |
| 5491-AFL240320A0-2.4N12NTM-ANO-ND | AFL240320A0-2.4N12NTM-ANO |
| 5491-AFL240320A0-2.8N12NTM-ANO-ND | AFL240320A0-2.8N12NTM-ANO |
| 757-NHD-2.4-240320AF-CSXP-CTP-ND | NHD-2.4-240320AF-CSXP-CTP |
| 757-NHD-2.8-CSXP-BREAKOUT-ND | NHD-2.8-CSXP-BREAKOUT |
| 5491-AFR240240A0-1.5INTN-ND | AFR240240A0-1.5INTN |
| 5491-AFL240240A0-1.3INTM-ANO-ND | AFL240240A0-1.3INTM-ANO |
| 5491-AFR240320A0-2.0INTM-ND | AFR240320A0-2.0INTM |
| 5491-AFR240240A0R-1.3INTM-ND | AFR240240A0R-1.3INTM |
| 757-NHD-2.4-240320AF-CTXP-CTP-ND | NHD-2.4-240320AF-CTXP-CTP |
| 5491-AFY240320A1-2.4INTH-C1-ND | AFY240320A1-2.4INTH-C1 |
| 5491-AFY240320A1-2.8INTH-C1-ND | AFY240320A1-2.8INTH-C1 |
| NHD-2.8-240320AF-CSXP-FT-ND | NHD-2.8-240320AF-CSXP-FT |
| 277-1132710-ND | 1132710 |
| 3387-ATM0240B32-ND | ATM0240B32 |
| 5491-AFY240320A1-2.8INTH-R-ND | AFY240320A1-2.8INTH-R |
| 757-NHD-1.5-240240AF-CSXP-ND | NHD-1.5-240240AF-CSXP |
| 757-NHD-1.5-240240AF-CSXP-T-ND | NHD-1.5-240240AF-CSXP-T |
| 5491-AFY240320A2-2.4INTH-R-ND | AFY240320A2-2.4INTH-R |
| 757-NHD-2.4-CTXP-BREAKOUT-ND | NHD-2.4-CTXP-BREAKOUT |
| 757-NHD-2.4-CSXP-BREAKOUT-ND | NHD-2.4-CSXP-BREAKOUT |
| 757-NHD-2.4R-CTXP-BREAKOUT-ND | NHD-2.4R-CTXP-BREAKOUT |
| 757-NHD-2.4R-CSXP-BREAKOUT-ND | NHD-2.4R-CSXP-BREAKOUT |
| 757-NHD-2.8R-CSXP-BREAKOUT-ND | NHD-2.8R-CSXP-BREAKOUT |
| 757-NHD-2.4C-CTXP-BREAKOUT-ND | NHD-2.4C-CTXP-BREAKOUT |
| 757-NHD-2.4C-CSXP-BREAKOUT-ND | NHD-2.4C-CSXP-BREAKOUT |
| 757-NHD-2.8C-CSXP-BREAKOUT-ND | NHD-2.8C-CSXP-BREAKOUT |
| 2544-AFL240135A0-1.14INTM-ANO-ND | AFL240135A0-1.14INTM-ANO |
| 4863-WF0200BTYAJDNN0-ND | WF0200BTYAJDNN0 |
| 3390-GLT024240320TR2-ND | GLT024240320TR2 |
| 3190-DLC-T2.0ASC#1-ND | DLC-T2.0ASC#1 |
| 4863-WF24MTLAJDNN0-ND | WF24MTLAJDNN0 |
| 3749-ST0154A1W-RSLW-C-ND | ST0154A1W-RSLW-C |
| 2544-AFY240320A0-2.4INTH-R-ND | AFY240320A0-2.4INTH-R |
| 3086-MDT0200FIH-MULTI-ND | MDT0200FIH-MULTI |
| 2632-E15RA-FW255-N-ND | E15RA-FW255-N |
| 2632-E24GA-I-CW850-RTB-ND | E24GA-I-CW850-R |
| 2632-E24GA-I-CW1000-N-ND | E24GA-I-CW1000-N |
| 2632-E28RA-I-CW480-N-ND | E28RA-I-CW480-N |
| 2632-E24RA1-FW420-N-ND | E24RA1-FW420-N |
| 3190-DLC0283BCM04CB-1-ND | DLC0283BCM04CB-1 |
| 3749-ST0280A3WY-RSLW-F-ND | ST0280A3WY-RSLW-F |
| 4863-WF0200BTYAJDNG0-ND | WF0200BTYAJDNG0 |
| 2544-AFY240320A0-2.4INTH-C-ND | AFY240320A0-2.4INTH-C |
| 2632-E24RA-I-CW500-N-ND | E24RA-I-CW500-N |
| 2632-E32RA-CW380-N-ND | E32RA-CW380-N |
| 2632-E32RA-CW350-R-ND | E32RA-CW350-R |
| 3190-DLC0140AZP03SF-1-ND | DLC0140AZP03SF-1 |
| 2632-E28RA-FW480-N-ND | E28RA-FW480-N |
| 2632-E28RA-I-CW600-C-ND | E28RA-I-CW600-C |
| 3749-ST0430E2WCY-RSLW-F-ND | ST0430E2WCY-RSLW-F |
| 2632-E32RA-CW370-C-ND | E32RA-CW370-C |
| 2632-E28RA-FW580-C-ND | E28RA-FW580-C |
| 3749-ST0144S4W-RSLW-F-ND | ST0144S4W-RSLW-F |
| 3318-PG28006A-ND | PG28006A |
| 2632-E24RB-I-CW1150-N-ND | E24RB-I-CW1150-N |
| 3749-ST0280A3W-RSLW-F-ND | ST0280A3W-RSLW-F |
| 2632-E24RE-I-CW400-C-ND | E24RE-I-CW400-C |
| 5491-AFL240240A0-1.54INTM-ANO-ND | AFL240240A0-1.54INTM-ANO |
| 5491-AFR240320A0-1.77INTM-ND | AFR240320A0-1.77INTM |
| NHD-2.8-240320AF-CSXP-FCTP-ND | NHD-2.8-240320AF-CSXP-FCTP |
| 1597-104990542-ND | 1.05E+08 |
| 2470-ATM0280B3-ND | ATM0280B3 |
| 1471-MIKROE-6587-ND | MIKROE-6587 |
| 1471-MIKROE-6588-ND | MIKROE-6588 |
| 2114-4DLCD-24-240-320-RT-NM-165-M16/S-ND | 4DLCD-24-240-320-RT-NM-165-M16/S |
| 2114-4DLCD-24-240-320-NT-CLB-170-M16/S-ND | 4DLCD-24-240-320-NT-CLB-170-M16/S |
| 2114-4DLCD-28-240-320-NT-NM-175-M8/M16-ND | 4DLCD-28-240-320-NT-NM-175-M8/M16 |
| 2114-4DLCD-28-240-320-RT-NM-140-M8/M16-ND | 4DLCD-28-240-320-RT-NM-140-M8/M16 |
| 2114-4DLCD-28-240-320-RT-NM-170-M16/S-ND | 4DLCD-28-240-320-RT-NM-170-M16/S |
| 2114-4DLCD-24-320-240-CT-NM-170-M16/S-ND | 4DLCD-24-320-240-CT-NM-170-M16/S |
| 2114-4DLCD-24-240-320-CT-CLB-165-M16/S-ND | 4DLCD-24-240-320-CT-CLB-165-M16/S |
| 2114-4DLCD-28-320-240-CT-NM-170-M8/M16-ND | 4DLCD-28-320-240-CT-NM-170-M8/M16 |
| 2114-4DLCD-28-240-320-CT-CLB-157-M8/M16-ND | 4DLCD-28-240-320-CT-CLB-157-M8/M16 |
| 5491-AFY240320A1-2.8INTH-C-ND | AFY240320A1-2.8INTH-C |
| 2544-AGU240320A00-1.77INSM-ND | AGU240320A00-1.77INSM |
| 5491-AFY240320A1-2.4INTH-C-ND | AFY240320A1-2.4INTH-C |
| 5491-AFR240240A0-1.3INTM-ND | AFR240240A0-1.3INTM |
| 3749-ST0103A2W-WSNLW-F-ND | ST0103A2W-WSNLW-F |
| 2544-AFY240320A0-2.8INTH-C-ND | AFY240320A0-2.8INTH-C |
| 2114-4DLCD-25-240-240-NT-NM-300-M16/S-ND | 4DLCD-25-240-240-NT-NM-300-M16/S |
| 2114-4DLCD-25-240-240-CT-NM-255-M16/S-ND | 4DLCD-25-240-240-CT-NM-255-M16/S |
| 2114-4DLCD-25-240-240-CT-CLB-300-M16/S-ND | 4DLCD-25-240-240-CT-CLB-300-M16/S |
| 2544-AFY240320A0-2.8INTH-ND | AFY240320A0-2.8INTH |
| 2544-AFY240320A0-2.8INTH-R-ND | AFY240320A0-2.8INTH-R |
| 2632-E28RA-I-CW384-R-ND | E28RA-I-CW384-R |
| 5491-AFY240320A1-2.4INTH-R-ND | AFY240320A1-2.4INTH-R |
| 5491-AFY240320A1-2.4INTH-ND | AFY240320A1-2.4INTH |
| 3749-ST0280H4W-RSMLW-F-ND | ST0280H4W-RSMLW-F |
| 3749-ST0200H2W-RSLW-F-ND | ST0200H2W-RSLW-F |
| 4863-WF0200BSYAJDNG0-ND | WF0200BSYAJDNG0 |
| 6233-GF-M36XGF20383-01-ND | GF-M36XGF20383-01 |
| 2632-E24BA-CW1000-N-ND | E24BA-CW1000-N |