In comparison to a baseline generation composed of frequent English words, our method generated vocabulary with a 4.6 gain in mean average precision, regardless of the level of contextual information in the input photographs, and 6.9 for photographs in which contextual information was extracted correctly. Performance was calculated with information retrieval concepts on the relevance of vocabulary generated for communicating a corpus of 9730 narrative phrases about events depicted in 1946 photographs. We contribute a novel method for generating context-related vocabulary from photographs of personally relevant events aimed at supporting people with language impairments in retelling their past experiences.
Research on the application of NLP in symbol-based Augmentative and Alternative Communication (AAC) tools for improving social interaction support is scarce. Automated Generation of Storytelling Vocabulary from Photographs for use in AAC, ACL Meeting 2021. Mauricio Fontana de Vargas, Karyn Moffat. In such way, the user has the opportunity to find and use appropriate vocabulary to talk about those past events and activities in a timely manner, without needing to navigate an extensive list of words often found in traditional AAC devices The application generates a situation-specific vocabulary from photos of events and activities and display it as pictograms that produce speech audio when selected. Our Augmentative and Alternative Communication (AAC) mobile application Click AAC aims to help people with aphasia to retell personally relevant events and activities, consequently supporting their social interactions. It is estimated that 120,000 people in Canada and two million in the USA are living with aphasia-a communication disorder most often caused by a stroke that limit their ability to being understood and to participate independently in society.
0 kommentar(er)
