FALL DETECTION
Fall Detection Prototype
We will be able to detect and track users with vision sensors and microphones and using Computer Vision with Machine Learning algorithms to study a patterns. Aim being to develop the system that can sync with mobile apps to capture and share information among families, caregivers, volunteers, hospitals etc.
AI & Machine Learning is to be used with the camera technology to study patterns of behaviour in home or a defined space and algorithms will study patterns and build data sets which will help understand anomalies in behaviour like a skid, fall in the room or if a person doesn’t appear in view for a period of time in that area etc, then an immediate alert is sent out.
Prototype
We use ‘MediaPipe’ which offers an open-source cross-platform with a customizable ML solution for live and streaming media. This gives better performance and works better in small CPU devices. For the Prototype we have used am OpenCV a real-time optimized Computer Vision library for getting the user video and developing the data sets. The fall model runs successfully on Raspberry Pi 3 or a Jetson Nano Board (specs below) and detects if there is an anomaly and if a person falls.
Audio Identification
In addition to this, we added a Deep Learning Audio Classification that differentiates a person's falling sound from other sounds. We first preprocess the data to extract the audio signal's relevant features using Mel frequency cepstral coefficients (MFCC) and then pass those important features through the Tensorflow Keras neural network for the audio classification.
Neural Network at work
We are developing a Convolutional Neural Network (ConvNet/CNN) using Pytorch that can identify when a person falls. PyTorch is an open-source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing. We used the Nvidia Jetson Nano is a small, powerful computer designed to power entry-level edge AI applications and devices. Which runs on a quad-core ARM Cortex-A57 64-bit @ 1.42 GHz, 4/2GB Ram, 128 CUDA core GPU, based on the Maxwell architecture.