Too-distracted-didn’t-listen-enough (Toddle)

Google x Breakthrough AI project

Toddle is an AI agent that takes notes for the learner from the lecture videos. This project utilized AWS and ML tools for speech-to-text conversion and summarization of the text. The project was done by Minjung, Sahana, Audite, Kamile from the Breakthrough AI program and advised by the company adivsors, Shanika and Josh, from Google.

AWS SDK

AWS Transcribe SDK was used to convert audio lecture files (mp3/mp4) to text files. The audio file was first uploaded to AWS S3 bucket, and its uri link was used for using transcribe_file function.

Gensim

Gensim is a Python library for topic modelling, document indexing and similarity retrieval with large corpora. Its keywords and summerizer function were used to extract keywords and summerize the transcription. More information can be found here.

Why this project?

Since the pandemic, virtual class settings have become more prominent which has led to more virtual learning obstacles. This project is a stepping stone to create a more productive and efficient learning environment for students and will lead them to great academic success. This project can also help students who use English as their second language for taking notes during the lectures, as well as create a more accessible environment for those who have ADHD and other disabilities.

Presentation Slide

The slide can be viewed here.

Team repository

The original repository can be found here.