Wav2vec2 for dementia screening based on spontaneous speech
wav2vec2 for dementia screeningDementia is hard to diagnose. And there is no known cure, maybe because it too late by the time the diagnosis is confirmed. It is understood that the development of dementia commences 10 to 15 years before the symptoms first appear.  Also, language, especially spontaneous speech, is a promising indicator/biomarker for diagnosing dementia and other cognitive disorders.2. LanguageThe model will be trained on English audio.3. ModelAs far as I know, wav2vec2 is the best candidate.4. DatasetsThere is dementia classification from dementia bank. However, it is a binary classification dataset for dementia and no dementia.To predict dementia 10 to 15 before the onslaught of symptoms, we would need longitudinal data on individuals who develop dementia. The only such available datasets I know is the Framingham heart study, and it is text only.I have been building a list of public figures diagnosed with dementia and scrapping videos of youtube into different categories; after symptoms, two years before symptoms, five years before symptoms etc. Over the next week, I will build a streamlit app to extract 8 to 10 secs of audio files of the person of interest from the video.Maybe we can use data from dementia bank or other sources for no dementia class??Possible links to publicly available datasets include:Dementia BankGoogle sheetwith list of public figures with dementia and YouTube urls5. Training scriptsTODO6. ChallengesThe dataset is too small, too noisy?No dataset on time for no dementia7. Desired project outcomeA proof of concept streamlit app that this works?8. ReadsThe following links can be helpful to better understand the project andwhat has previously been done.IBM effortsusing Farmingham data for dementia prediction
2021-06-29T15:45:29Z
Really like this project! Hope more people will be interested
2021-06-29T18:28:09Z
I am hoping for the same
2021-06-29T20:29:18Z
Actually I am interested to this project, unfortunately I am already in other projects. Maybe next time
2021-06-30T07:07:04Z
Hi! Are you still looking for people to work with? I’m very interested in doing this project
2021-06-30T08:06:36Z
Yup, we are still open for people
2021-06-30T10:06:07Z
Cool can I join?
2021-06-30T12:19:54Z
Awesome two people is enough - let’s create the project!Think this is a really cool & interesting project and we should be able to demo it well! Here I think it’ll actually make most sense to fine-tune a pretrained Wav2Vec2 model, maybe this one:facebook/wav2vec2-base-100k-voxpopuli · Hugging Face?There is currently no official fine-tuning script for Wav2Vec2, but it should be relatively easy to adapt the pretraining script:https://github.com/huggingface/transformers/pull/12271or the official one in PyTorch.Let’s create the project - very excited about this one
2021-07-01T02:55:00Z
@shreyasgitecool project, let me know if I can give a hand. Is there a channel on discord for this project?
2021-07-01T10:05:17Z
Great, added you to the official project@mattbui
2021-07-01T10:43:06Z
I would love to be a part of this project. I actually used wav2vec2 embeddings for participating in a Alzheimer dementia challenge. The wav2vec2 embeddings didn’t work so well for the task but this might be because there wasn’t enough data. I’m more then happy to try this out again.
2021-07-01T10:55:55Z
I made a discord channelDiscordDiscord - A New Way to Chat with Friends & CommunitiesDiscord is the easiest way to communicate over voice, video, and text.  Chat, hang out, and stay close with your friends and communities.
2021-07-01T12:04:44Z
This is awesome@birgermoelland@mattbui
2021-07-01T12:18:27Z
Thanks Partick for setting the vector:)
2021-07-08T02:13:13Z
Hey@shreyasgiteI would love to join and contribute to the project.If it is late can I follow the group’s progress. Really want to get going with a small project in the areaHow is the group coordinating?Thanks
2021-07-08T15:01:11Z
@asharma85You can join the channel onDiscord
2021-07-08T15:27:27Z
It keeps showing me this when I try to join on discordScreen Shot 2021-07-08 at 11.26.37 AM2056×1168 119 KB
2021-07-08T18:47:36Z
also@shreyasgite, make sure you right click on your server icon → invite and copy the link provided. You cannot access a discord server via the link in your browser.
2021-07-08T19:34:45Z
@asharma85Flax-HuggingFace-Community-Week
2021-08-05T20:55:50Z
Hi@shreyasgite, if this project is still on-going I am also quite interested in it. The invite link has expired though.