More than a billion people live with disability and there is a need to explore how AI technologies can affect this very diverse group. AI research could be a force for good for disabled people as long as they are not marginalised. A roadmap, including ethical issues and the exploration of the gaps in innovative digital accessibility, has yet to be developed. The creation of a network of experts and resources for AI and inclusion could help to address the “unmet need of assistive products crucial to achieve the Sustainable Development Goals, to provide Universal Health Coverage, and to implement the UN Convention on the Rights of Persons with Disabilities" (WHO 2018).
Explaining the science
Current supporting AI technologies might be better described as 'augmented' or 'assistive' intelligence technologies, supporting what can be achieved by humans rather than acting as a replacement. Machine learning requires clear and narrowly defined tasks, so as a start to a project on AI and inclusion it has been interesting to explore the way a list of machine learning applications can be adapted from integrate.ai's ‘Responsible AI in Consumer Enterprise’.
For example, for people unable to communicate easily through speech or writing, AI could enhance 'Augmented and Alternative Communication' (AAC) services, devices and applications in the following ways:
- Recommendation systems: Compare user actions to other AAC user actions and recommend suitable symbols or language possibilities based on attributes and activities
- Audience segmentation: Separate AAC users into groups that look/act like one another in a way that is relevant to the user application improvements needed
- Personalisation: Modify the experience of an application, symbol set etc, that best suits the user 'at a scale too large for human teams to execute'
- Chatbots: Help users answer questions, resolve problems, or identify the right solution without requiring human intervention and allow for the redirection of human resources to higher-value interactions that require judgment
- Risk assessments: Modify options offered on a device or application when there is a predicted risk or likelihood of failure arising that could mean lack of use or attention loss etc
- Anomaly detection: Identify a shift in user behaviour that could signal an opportunity to enhance options on the device and application or prevent likely behaviour that could result in an undesirable occurrence
- Support prediction: Enhance communication speeds through the use of natural language generation
- Data products: Use algorithms to identify useful insights about user behaviour that can be shared with others to enhance product development
Applications include the development of assistive technologies using AI, such as description of images for blind people, speech recognition, captioning for hearing impaired people, sign language recognition and creation for deaf people, multilingual text-to-speech options for reading text for those with cognitive disabilities including dyslexia, care ‘robots’ for elderly people and mobility guides for visually impaired people.
Wider integration and harmonisation of multicultural and multilingual symbol access across domains is needed for communication, literacy skills and support for understanding those who may have cognitive disabilities and/or complex speech and language difficulties (e.g. dementia, cerebral palsy and autism). Symbols can work on speech generating devices, communication boards, in books and on the web with the potential to improve translation between symbols and text to aid understanding.
Decision support can help the identification, selection, use and personalisation of assistive technologies and identifying accessible travel route planning for disabled people.
Identifying what is unique about disability as an ethical issue compared to other minority or protected groups is key and there remains questions as to how AI affects human rights for disabled people. Strategies are needed to prevent barriers to equal access and inclusion where innovative technologies are being developed.
January - July 2020
- The last seven months have been a challenge with many changes in the way we have all been working. Managing data on laptops and evaluating results has not always been easy, but with the support of the Web and Internet Science Research Group (https://www.wais.ecs.soton.ac.uk/) at the University of Southampton we have continued to work on the use of machine learning and natural language processing to support Web Accessibility and the concept linking of symbols used for Augmentative and Alternative Communication (AAC) to provide improved searching and text to symbol translation.
- A 4th year computer science student Group Design Project has supported our intention to improve some automated web accessibility
checks on our Web2Access review system (https://www.emptech.info/wp/2020/04/28/image-recognition-to-check-image-description-accuracy-on-web-pages/). The project has resulted in a way of making sure alternative text used to describe images on web pages is accurate.
- Despite the COVID-19 pandemic our dissemination continues (https://www.emptech.info/wp/2020/05/15/covid-19-ai-and-our-
- We have had several papers accepted for ICCHP 2020 that will be delivered remotely, as was the one we submitted for a WebSci 2020 workshop. Three presentations were accepted for ISAAC 2020, but this conference has been moved to 2021, so who knows if we will get to Mexico.
- In June we were granted a Winston Churchill Memorial Trust COVID Action Fund award (https://www.emptech.info/wp/2020/06/03/winston-churchill-memorial-trust-covid-19-action-fund-support-symbol-charts/) to build charts with symbols to support communication in these difficult times for those with cognitive impairment or speech and language difficulties in hospitals and care homes.
- Using AI technologies and Concept Net to link freely available multilingual AAC symbol sets has produced results (https://www.emptech.info/wp/2020/06/30/symbols-and-concept-linking). We have been able to show that the process developed can make it easier for people to search for symbols to use on communication charts and for information. We hope this will also help to offer more options for text to symbol translation being developed by the Web Content Accessibility Group (WCAG) Personalisation task force.
- Over the last few months we have been concentrating on projects related to automated web accessibility checks and the automatic linking and categorisation of open licensed
and freely available augmentative and alternative communication symbol sets for those with complex communication needs.
- We presented these projects at a workshop at The Alan Turing Institute on 'AI and Inclusion Challenge' on 22 November 2019 and our slides can be found on our blog
- Work has been ongoing and it is hoped that the results will be shared by the end of March 2020.
- Regional Forum for Europe on “Accessible Europe: ICTs 4 ALL" took place in Malta, from 4 to 6 December 2019. See the background paper provided on “Artificial Intelligence and Information Communication Technology Accessibility” and the related video.
- In preparation for the completion of our roadmap we have started to list the gaps or perhaps we should call them challenges to aspects of AI and Inclusion. It is a longer list than expected and still has to be debated, but there is a need to
find strategies that promote equality and equity.
- Aiming to tackle some of the challenges facing those with disabilities in the areas of our interest we have applied for a Microsoft AI for Accessibility Azure grant and to the ESRC as part of a UKRI-JST Call on Artificial Intelligence and Society for funding to work with
Japanese colleagues who have a FutureGym programme for children with special needs.
- Meanwhile we have also been working on an ITU background paper on AI and ICT Accessibility that will be part of a session we are leading on at the
Accessible European Forum in Malta in December.
- Exploring ways in which text has been analysed, simplified and works with speech synthesis to support disabled students and Assistive Technology users has been an interesting part of our research and has led to yet more innovative apps
supported by Microsoft Azure. But during our special thematic session on AI and Inclusion at the AAATE 2019 conference on Global Challenges in Assistive Technology we learnt from other researchers about the dilemmas that still face those working in the area of AI and Inclusion.
- Exploring Blockchain and Digital Accessibility. We noticed that has been an increase in
the amount of discussions around Blockchain but found very little about the way it could support inclusion. However, there was an article by Nathanial Biddle that touched accessibility and upholding standards.
- During the month we attended meetings with UNICEF about AI and Innovation relating to Augmentative and Alternative
Communication and Assistive Technologies. We also began a background report about AI and ICT Accessibility for ITU and were involved with the European Disability Forum's 'Plug and Pray' report that discussed the impact that AI could have on those with disabilities.
- AI and Inclusion versus AI and Assistance. We have been exploring the differences between these two ideas and how many more assistive technology apps are appearing using machine learning and natural language processing.
- On the inclusion side during the last few weeks we have looked at AI and digital accessibility, augmentative and alternative forms of communication and the issues of linking up AAC symbol sets for different cultures and languages and finally a discussion about the use of AI in education for management and assessments for an ED-ICT network symposium in June.
- Microsoft hosted an evening about AI and Accessibility where the company showcased many of the ways they have been using image and speech recognition, text support and other apps to provide assistive technologies for inclusion.
- Later in the month a chance discussion during an Easy Reading project tele-meeting. Professor Clayton Lewis mentioned his white paper on the “Implications of Developments in Machine Learning for People with Cognitive Disabilities”. This paper mentions yet more helpful strategies whilst encouraging more research initiatives into NLP benchmarking and discussions around direct brain interfaces.
- Joining UNICEF at 'The Role of Assistive Technology in Inclusive Education for Children with Disabilities' exhibition in Geneva on 7-8 March 2019 provided the team with an occasion to see AI being used in several different types of assistive technology from communication apps to virtual reality. Meeting developers to discuss how they were making the most of deep learning, signal processing and customisable speech recognition technologies highlighted how 'AI for Good' can accelerate progress towards the UN's Sustainable Development Goals.
- Visits to the University of Edinburgh and Dr Maria Wolters and Professor Annalu Waller at Dundee University were undertaken to discuss future links around AI and inclusion.
- Submission of a Special Thematic Session on ‘AI and Inclusion’ was accepted by the AAATE 2019 conference committee with speakers from Germany, UK and Ireland offering abstracts.
- Professor Mike Wald, E.A. Draffan and the W3C UK and Ireland office welcomed members of the Cognitive and Learning Disabilities Accessibility Task Force to the University of Southampton for a discussion about the gap analysis being undertaken to improve the Web Content Accessibility Guidelines with particular attention to future technologies that could support this work.