You'll need to find a YouTube to MP3 converter if you would like to try on a song of your choice. Beware of popups and advertisements on these sites!
If you'd like, you can just use the demo using one of my favorite songs of all time!
Drag & drop some files here, or click to select files
CURRENT MOOD
sad
0:00 / 0:00
I used the Python library, Librosa, to extract features from the raw audio files such as MFCC's, spectrograms, chromagrams, and tempo.
The dataset I trained on only had 5 second clips of instrumental audio so I break each song into segments and classify based off of 5 second intervals to make sure my predictions are using a similar environment that the model was trained with.
I used a Random Forest and achieved ~95% accuracy on my test data with an F1 score of ~0.95. My model is hosted reliably on a Flask with AWS EC2 deployed using Gunicorn and Nginx.
I've always been passionate learning new skills and since music is a huge part of my life, I decided to learn both machine learning and DSP over the summer and use this as a sort of capstone project.
Since I'm also well versed in full stack web development, I decided to build out a quick Next.js app with a custom Audio Player and file uploader.
(Spotify please consider me for your internship 🙏)
There are known limitations with my model such as loudness and drums corresponding to aggresivness even if the drums are backing happy instrumentals.
I believe that the biggest improvement to my model can only be made by training with better data. Unfortunately, good data is difficult to come by since not many datasets provide mood and raw audio files. I've already tried a few other datasets but upon data exploration, I decided they weren't fit for training.
Once I find a better dataset, I'll attempt to train a more powerful model using Convolutional Neural Networks. Feel free to contact me with recommendations for datasets. Any help would be greatly appreciated!