Accessible Human-Robot Interaction and Collaboration

To maintain human dignity and autonomy, it is important that the person or people interacting with cyber-physical health and assistive robotics technologies can provide guidance and instructions in mutually supportive task. Ensuring safe, effective, accessible and even enjoyable interaction is particularly crucial for people with different levels of physical or sensory impairments. For example, if a robot is being used by a disabled person to eat a meal, the person might want to direct the robot through their gaze or verbally guide a physically assistive robot doing the physical task as to what food items to pick up and how, such as dipping a bread-roll in the soup before offering it to them or how much gravy to mix in with the mashed potato. Where elements of the task might be challenging for a robot to decide for itself, or relate to personal preferences, determining effective modalities and strategies for collaborative Human-Robot Interaction (HRI) is particularly important. At the same time the system needs to be able to learn preferences and skills which can make future interactions more efficient.

There are several research areas within HRI which can be explored for activities such as dressing, mobility, medication management, physiotherapy and rehabilitation. 

Research Topics

Accessible interfaces for people with sensory, cognitive and physical impairments using a range of sensory modalities: 

This ranges from brain-computer interaction, gaze tracking to haptic feedback to make interaction with technologies easier, cost-effective and efficient.  

Typical research themes include:

• Analysing Human-Computer/Robot Interaction, Human-Factors, Usability and User Experience in different contexts

• Designing and Evaluating Accessible Multi-Modal (Speech, Haptic, Tactile, Gesture) Interfaces

• Developing and Testing Augmented Alternative Interaction devices for communication and dialogue

Collaborative Human-Robot Interaction: 

When considering variable autonomy (including adaptive control) we need to determine who should have initiative in different interactions scenarios and contexts. Typical research themes include:

• Developing optimal machine learning strategies for realising shared control and cooperation

• Learning from observations of human-human interaction and collaboration, and capturing and translating knowledge t to develop intelligent control algorithms

• AI techniques and intelligent sensing to improve situational awareness, recognition of human intent and behaviour for accessible HRI

Decision-support systems with easy-to-use dashboards and tools for healthcare professionals: 

Designing and evaluating interfaces for healthcare professionals to support safe, effective remote monitoring and management of people with long-term conditions or for example, those who have been discharged early from hospital and require remote clinical monitoring, is crucial for successful integration of intelligent assistive technologies into care pathways.  Typical research themes:

• Accessible and Actionable information visualisation

• AI and ML techniques for data analysis, modelling and classification 

• Augmented and Mixed Reality Interfaces

Related Research Projects

• TARICS:  Trustworthy Accessible Robots for Inclusive Cultural experiences

TARICS aims to create an interactive cultural experience for museum visits by provisioning a social robot to make the experience more accessible and inclusive for people with learning disabilities and/or autism. The proposed learning and interaction techniques will be deployed on Lindsey, a robot tour guide operating autonomously in The Collection Museum since 2018.

Contact: Ayse Kucukyilmaz

• Open All Senses - Accessible Multimodal Telepresence Robots in Social Spaces

Open All Senses will explore people’s feelings of safety and comfort when controlling and interacting with telepresence robots and will provide opportunities for transforming their design and utility.

Project Lead: Horia Maior

Related Past Research Projects with CHART team member involvement

• FLOURISH - Empowerment through trusted secure mobility

FLOURISH, Led by Atkins,  was a multi-sector collaboration that helped to advance the successful implementation of Connected and Autonomous Vehicles (CAVs) in the UK. While at UWE Praminda Caleb-Solly worked on designing and evaluating accessible interfaces for people with aging-related impairments who could be early adopters of autonomous vehicles. 

• I-DRESS - Assistive Interactive Robotic System for Support in Dressing

The main objective of the I-DRESS project , which was based at UWE, was to develop a system that would provide proactive assistance with dressing to disabled users or users such as high-risk health-care workers, whose physical contact with the garments must be limited to avoid contamination. The robotic system consisted of two highly dexterous robotic arms, sensors for multi-modal human-robot interaction and safety features.

The system comprised three major components: (a) intelligent algorithms for user and garment recognition, specifically designed for close and physical human-robot interaction, (b) cognitive functions based on the multi-modal user input, environment modelling and safety, allowing the robot to decide when and how to assist the user, and (c) advanced user interface that facilitates intuitive and safe physical and cognitive interaction for support in dressing.  

Praminda Caleb-Solly led the work package on Human Factors and Interface Design when she worked at UWE, and is continuing aspects of the research on safety in close-proximate human-robot interaction. 

Related Research Publications

• Stephenson, A.C., Eimontaite, I., Caleb-Solly, P., Morgan, P.L., Khatun, T., Davis, J. and Alford, C., 2020. Effects of an Unexpected and Expected Event on Older Adults’ Autonomic Arousal and Eye Fixations During Autonomous Driving. Frontiers in psychology, 11, p.571961. https://www.frontiersin.org/articles/10.3389/fpsyg.2020.571961/full

• Stephenson, A., Eimontaite, I., Caleb-Solly, P. and Alford, C., 2020, July. The impact of a biological driver state monitoring system on visual attention during partially automated driving. In International Conference on Applied Human Factors and Ergonomics (pp. 193-200). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-030-50943-9_25

• Voinescu, A., Morgan, P.L., Alford, C. and Caleb-Solly, P., 2020. The utility of psychological measures in evaluating perceived usability of automated vehicle interfaces–A study with older adults. Transportation research part F: traffic psychology and behaviour, 72, pp.244-263. https://www.sciencedirect.com/science/article/abs/pii/S1369847820304174

• van Maris, A., Sutherland, A., Mazel, A., Dogramadzi, S., Zook, N., Studley, M., Winfield, A. and Caleb-Solly, P., 2020, March. The impact of affective verbal expressions in social robots. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 508-510). https://dl.acm.org/doi/abs/10.1145/3371382.3378358 

• He, J., van Maris, A. and Caleb-Solly, P., 2020, March. Investigating the effectiveness of different interaction modalities for spatial human-robot interaction. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 239-241). https://dl.acm.org/doi/abs/10.1145/3371382.3378273

• van Maris, A., Dogramadzi, S., Zook, N., Studley, M., Winfield, A. and Caleb-Solly, P., 2020, March. Speech related accessibility issues in social robots. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 505-507). https://dl.acm.org/doi/abs/10.1145/3371382.3378379 

• Eimontaite, I., Voinescu, A., Alford, C., Caleb-Solly, P. and Morgan, P., 2019, July. The impact of different human-machine interface feedback modalities on older participants’ user experience of CAVs in a simulator environment. In International Conference on Applied Human Factors and Ergonomics (pp. 120-132). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-030-20503-4_11 

• Bellamy, D.D., Caleb-Solly, P. (2019). Collaborative HRI and Machine Learning for Constructing Personalised Physical Exercise Databases. In: Althoefer, K., Konstantinova, J., Zhang, K. (eds) Towards Autonomous Robotic Systems. TAROS 2019. Lecture Notes in Computer Science(), vol 11649. Springer, Cham.  https://link.springer.com/chapter/10.1007/978-3-030-23807-0_18 

• Morgan, P.L., Voinescu, A., Alford, C. and Caleb-Solly, P., 2018, July. Exploring the usability of a connected autonomous vehicle human machine interface designed for older adults. In International Conference on Applied Human Factors and Ergonomics (pp. 591-603). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-319-93885-1_54 

• Voinescu, A., Morgan, P.L., Alford, C. and Caleb-Solly, P., 2018, July. Investigating older adults’ preferences for functions within a human-machine interface designed for fully autonomous vehicles. In International Conference on Human Aspects of IT for the Aged Population (pp. 445-462). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-319-92037-5_32 

• Chance, G., Jevtić, A., Caleb-Solly, P., Alenya, G., Torras, C. and Dogramadzi, S., 2018. “elbows out”—predictive tracking of partially occluded pose for robot-assisted dressing. IEEE Robotics and Automation Letters, 3(4), pp.3598-3605. https://ieeexplore.ieee.org/abstract/document/8410029

• Caleb-Solly, P., Dogramadzi, S., Huijnen, C.A. and Heuvel, H.V.D., 2018. Exploiting ability for human adaptation to facilitate improved human-robot interaction and acceptance. The Information Society, 34(3), pp.153-165. https://www.tandfonline.com/doi/abs/10.1080/01972243.2018.1444255 

• Jevtić, A., Valle, A.F., Alenyà, G., Chance, G., Caleb-Solly, P., Dogramadzi, S. and Torras, C., 2018. Personalized robot assistant for support in dressing. IEEE transactions on cognitive and developmental systems, 11(3), pp.363-374. https://ieeexplore.ieee.org/abstract/document/8320312 

• Chance, G., Caleb-Solly, P., Jevtić, A. and Dogramadzi, S., 2017, August. What's “up”?—Resolving interaction ambiguity through non-visual cues for a robotic dressing assistant. In 2017 26th IEEE international symposium on robot and human interactive communication (RO-MAN) (pp. 284-291). IEEE. https://ieeexplore.ieee.org/abstract/document/8172315 

• Morgan, P.L., Voinescu, A., Williams, C., Caleb-Solly, P., Alford, C., Shergold, I., Parkhurst, G. and Pipe, A., 2017, July. An emerging framework to inform effective design of human-machine interfaces for older adults using connected autonomous vehicles. In International Conference on Applied Human Factors and Ergonomics (pp. 325-334). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-319-60441-1_33

• Chance, G., Jevtić, A., Caleb-Solly, P. and Dogramadzi, S., 2017. A quantitative analysis of dressing dynamics for robotic dressing assistance. Frontiers in Robotics and AI, 4, p.13. https://www.frontiersin.org/articles/10.3389/frobt.2017.00013/full