INS NYC 2024 Program

Poster	Poster Session 04 Program Schedule 02/15/2024 12:00 pm - 01:15 pm Room: Majestic Complex (Posters 61-120) Poster Session 04: Neuroimaging \| Neurostimulation/Neuromodulation \| Teleneuropsychology/Technology Final Abstract #82 Speech Analysis in Older Adults for Neuropsychological Status Prediction Jingjing Nie, Washington State Uniersity, Pullman, United States Diane Joyce Cook, Washington State University, Pullman, United States Maureen Schmitter-Edgecombe, Washington State University, Pullman, United States Category: Cognitive Neuroscience Keyword 1: speech Keyword 2: ecological validity Keyword 3: neuropsychological assessment Objective: Ecologically-valid assessment of cognitive health paves the way for early detection, treatment, and prevention of cognitive decline. Additionally, researchers need a way to quantitatively measure the outcome of interventions in real-world environments. This study aims to investigate the predictive relationship between audio and text features with neurocognitive measures by integrating the measured outcomes from audio journals collected daily via a smartwatch app. We hypothesized that 1) both text and audio features would provide indicators of cognitive health and 2) machine learning techniques would benefit from a combination of features in predicting clinical measures. Participants and Methods: Participants were 20 community-dwelling older adults age 50+ (M age=72.68, SD=7.23; M education=15.89, SD=2.31; 63.2% female; 100% white) recruited for an ongoing 16-month study designed to automate assessment of functional health from in-home sensor data. Participants were healthy adults and individuals with subjective cognitive complaints, mild cognitive impairment, and mild dementia. At baseline, participants completed the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). At baseline and every 4 months, participants engaged in one week of data collection where they were prompted 4x/day to provide audio descriptions of their day using provided Apple watches. Each recording was no longer than 2 minutes. From the recordings, we extracted audio features including pitch, intensity, jitter, mel frequency cepstral coefficients (MFCCs). We also converted speech to text and extracted corresponding text features, including text structure, word choice, sentence complexity, and sentiment markers. A random forest regression model was used to predict RBANS scores from audio features, text features, or a combination of the two. Performance of the random forest regression was compared with linear regression as a baseline method. Results: Data were screened to remove audio not containing detected speech. After screening, one participant without an available RBANS score was excluded from the dataset. Consequently, the study's dataset comprised 1059 audio journal recordings, collected from 19 participants. A random forest model predicted RBANS scores from audio features with mean absolute error (MAE) of 8.617, outperforming linear regression (MAE=11.162, p<.001). Random forest prediction from text features alone yielded MAE=8.098, outperforming linear regression (MAE=11.645, p<.001). Combining audio and text features yielded MAE=8.075, significantly outperforming linear regression (MAE=12.585, p<.001) and prediction from audio alone (p<.001), though improvement over text alone was not significant. Conclusions: Both text and audio features predicted the RBANS scores, though combining the information sources provided the best results. The evident contrast between text and audio features highlights the potential inherent within audio components for capturing differences in neurological conditions among older adults. Further exploration will require larger sample sizes, more diverse participants, and more frequent clinical measures. Additional text and audio features may also be defined that will enhance the accuracy and sensitivity of neurological health evaluations and dimensionality reduction methods may reduce overfit. This study reveals the potential for audio journals to support automated cognitive health assessment in informal settings.

Poster

02/15/2024
12:00 pm - 01:15 pm
Room: Majestic Complex (Posters 61-120)

Poster Session 04: Neuroimaging | Neurostimulation/Neuromodulation | Teleneuropsychology/Technology

Final Abstract #82

Speech Analysis in Older Adults for Neuropsychological Status Prediction

Jingjing Nie, Washington State Uniersity, Pullman, United States
Diane Joyce Cook, Washington State University, Pullman, United States
Maureen Schmitter-Edgecombe, Washington State University, Pullman, United States

Category: Cognitive Neuroscience

Keyword 1: speech
Keyword 2: ecological validity
Keyword 3: neuropsychological assessment

Objective:

Ecologically-valid assessment of cognitive health paves the way for early detection, treatment, and prevention of cognitive decline. Additionally, researchers need a way to quantitatively measure the outcome of interventions in real-world environments. This study aims to investigate the predictive relationship between audio and text features with neurocognitive measures by integrating the measured outcomes from audio journals collected daily via a smartwatch app. We hypothesized that 1) both text and audio features would provide indicators of cognitive health and 2) machine learning techniques would benefit from a combination of features in predicting clinical measures.

Participants and Methods:

Participants were 20 community-dwelling older adults age 50+ (M age=72.68, SD=7.23; M education=15.89, SD=2.31; 63.2% female; 100% white) recruited for an ongoing 16-month study designed to automate assessment of functional health from in-home sensor data. Participants were healthy adults and individuals with subjective cognitive complaints, mild cognitive impairment, and mild dementia. At baseline, participants completed the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). At baseline and every 4 months, participants engaged in one week of data collection where they were prompted 4x/day to provide audio descriptions of their day using provided Apple watches. Each recording was no longer than 2 minutes. From the recordings, we extracted audio features including pitch, intensity, jitter, mel frequency cepstral coefficients (MFCCs). We also converted speech to text and extracted corresponding text features, including text structure, word choice, sentence complexity, and sentiment markers. A random forest regression model was used to predict RBANS scores from audio features, text features, or a combination of the two. Performance of the random forest regression was compared with linear regression as a baseline method.

Results:

Data were screened to remove audio not containing detected speech. After screening, one participant without an available RBANS score was excluded from the dataset. Consequently, the study's dataset comprised 1059 audio journal recordings, collected from 19 participants. A random forest model predicted RBANS scores from audio features with mean absolute error (MAE) of 8.617, outperforming linear regression (MAE=11.162, p<.001). Random forest prediction from text features alone yielded MAE=8.098, outperforming linear regression (MAE=11.645, p<.001). Combining audio and text features yielded MAE=8.075, significantly outperforming linear regression (MAE=12.585, p<.001) and prediction from audio alone (p<.001), though improvement over text alone was not significant.

Conclusions:

Both text and audio features predicted the RBANS scores, though combining the information sources provided the best results. The evident contrast between text and audio features highlights the potential inherent within audio components for capturing differences in neurological conditions among older adults. Further exploration will require larger sample sizes, more diverse participants, and more frequent clinical measures. Additional text and audio features may also be defined that will enhance the accuracy and sensitivity of neurological health evaluations and dimensionality reduction methods may reduce overfit. This study reveals the potential for audio journals to support automated cognitive health assessment in informal settings.