Dataset Description

The project involves building a Flutter app that can classify audio using a deep learning model based on Long Short-Term Memory (LSTM) and deploying the app on Amazon Web Services (AWS).

The app will allow users to upload the audio file from their devices, and then the audio will be processed using the LSTM model to classify it into one of several predefined categories. The LSTM model will be trained on a dataset of audio recordings that have been labeled with their corresponding categories.

Once the app is developed, it will be deployed on AWS so that it can be accessed by users from anywhere. This involves setting up a virtual machine on AWS, installing all the necessary dependencies and libraries, and then deploying the app to the virtual machine.

By analyzing sound properties, ML/DL models can learn to distinguish between different types of sounds and classify them accurately.

Music classification, Speech recognition, Emotion recognition, Environmental sound recognition, Medical diagnosis, Security and surveillance, Quality control, Automotive safety, etc.

The UrbanSound8K dataset is a dataset for sound classification which contains 8732 labeled sound (WAV format) excerpts (<=4s) of urban sounds from 10 classes, including:

Air Conditioner, Car Horn, Children Playing, Dog bark, Drilling, Engine Idling, Gun Shot, Jackhammer, Siren, Street Music

Each sound excerpt is labeled with a class, and a corresponding text file provides additional metadata such as the file name, fold number, and salience.

The dataset is linked below:

Load the data: The first step is to load the audio data into the notebook. You can use Python libraries like librosa or PyAudio to load the audio data.

Plot the waveform: The waveform represents the amplitude of the sound wave over time. You can use the Matplotlib library to plot the waveform.

Mel Frequency Cepstral Coefficients (MFCCs) are widely used in audio signal processing, particularly for speech recognition and music information retrieval. MFCCs are a type of feature extraction method used to represent the spectral envelope of an audio signal.

An audio signal can be represented as a time-domain waveform, but it can also be represented in the frequency domain using a Fast Fourier Transform (FFT). The resulting frequency domain representation of the audio signal is known as the power spectral density (PSD) of the signal, which shows how much energy is present at each frequency in the signal.

To extract the features from all the audio files, we will define a function. It will take the filename and load it using librosa. This will return two variables — Audio data and Sample rate. Then we will compute MFCC for the audio data and find the mean of the transpose of the array.

The number of classes is 10, which is our output shape (number of classes).

Compile the Model: Specify the loss function, optimizer, and metrics for evaluating the model during training and testing.

We will train the model and save the model in HDF5 format. We will train the model for 200 epochs or more and batch size as 32. We’ll use callback, which is a checkpoint to know how much time it took to train over data.

After Evaluating our Model, we got an accuracy of 96% on the training data and 92% on test data.

This is a Flask API for audio classification using a trained LSTM model. The API receives a WAV audio file via a POST request and processes it to predict the class label of the audio using the loaded LSTM model.

The API defines a function func() that extracts MFCC features from the audio file using Librosa, scales the features, reshapes the data to the required input shape of the LSTM model, predicts the class label of the audio using the loaded model and returns the predicted class label.

The Flask API is defined with a route ‘/predict’ which accepts the POST request containing the audio file.

We will be deploying the Flask API and the LSTM model on AWS EC2 instance.

Open the terminal at the required folder.

We will be creating a flutter app as the front end to our Audio Classification Model in Android Studio.

The first page will be a firebase phone verification page. On entering a phone number, it will redirect it to the OTP page where the user receives an OTP.

On verifying the OTP, it will be redirected to the next page which will be our home page of the application.

In this page, an audio file can be picked by pressing the button ‘pick file’. After picking the file, there is an option of playing the selected audio.

On pressing the ‘predict’ button, the audio will be identified through the IPv4 address of our instance in which the flask API is running.

A reference audio of the predicted class for the identified audio can be played.

Machine Learning and Deep Learning have not yet been fully utilized in the domain of sound. Audio Processing has its challenges but libraries like Librosa and Tensorflow make it easier.

In this project, one can understand how to deal with audio data, use MFCC for feature extraction and pre-processing of audio samples and build an LSTM Model to classify audio in different classes.

Our project, Audio Classification using RNN LSTM has classified different audios of the UrbanSound 8K dataset in their respective category with optimum accuracy and has been successfully integrated with Flask API and deployed on AWS.

For the complete code, refer the Github repository below

Related posts:

Music is an integral part of our lives, and with advancements in technology, creating music has become easier than ever before. AI-powered music generation tools have emerged as a promising area in…

Is it worth it to eat outside everyday, for the sake of sense gratification, whereas we dream of getting in shape and looking good every other day? Is it worth it to fight for right/wrong, proving…

Computer science is a field of study that encompasses the understanding of computers and computational systems. It involves the design, implementation, and application of algorithms, data structures…