Safety and Benchmarking

Robotics and autonomous systems (RAS) are emerging as disruptive technologies with the potential to provide personalised support in a range of care-related tasks, including the provision of physical and social assistance and physiotherapy. However, in order to ensure real-world deployment and commercialisation, application focused research into safe Human-Robot Interaction, Hazard Analysis and Risk Assessment in a range of dynamic environments and scenarios of use is needed.

The importance of this research is particularly significant, given the vulnerability of the end-users interacting with these systems – giving rise to a range of very complex issues and concerns. It is necessary to carefully and deeply consider the safety of physically assistive robots at an operational and functional level, as well as from human factors and clinical efficacy perspectives.

Our research objectives in this area include:

1.      Evaluating and benchmarking assistive robots which are already deployed as well as those being currently developed.

2.      Reviewing existing methods and approaches for safety assurance and benchmarking for these systems and developing new ones

3.      Understanding requirements for real-world use of assistive robots by people with different accessibility needs

4.      Identifying critical barriers to assurance and regulation

5.      Analysing the adequacy of existing benchmarks and safety standards for assistive robots and identifying gaps in the standards using a set of real-world use cases.

Research Topics

• Frameworks for safe collaboration in physical assistance applications

• Risk assessment and hazard analysis when bringing collaborative robotics to real-world environments (homes, care facilities and hospitals)

• Understanding stakeholders (end-users, carers, relatives, therapists, clinicians) requirements in safe human-robot interaction

• Developing regulatory compliance procedures for safety in assistive robotics

• Validation of human-robot interaction systems for assistive robots

• Developing robot capabilities for self-assessment of operational safety and risk management

• Estimation of probabilities and models of unintended behaviors by both robots and humans

• Approaches for modelling and assessing safety of assistive robots

Related Research Projects

• METRICS (Metrological Evaluation and Testing of Robots in International CompetitionS)

The METRICS project is organising challenge-led robotics competitions in the four priority areas identified in the topic ICT-09-2019-2020: Healthcare, Inspection and Maintenance, Agri-Food, and Agile Production. METRICS is designed to organise competitions as reproducible and objective evaluation campaigns and aims to structure in a sustainable way the European robotics and Artificial Intelligence (AI) community around the four priority areas. In order to assess the reliability of the different competing robots in a rigorous and unbiased way, METRICS is developing an evaluation framework based on metrological principles. For each competition, METRICS is organising field evaluation campaigns (in physical environments) and cascade evaluation campaigns (on datasets) to engage with the AI community. 

CHART is part of the Healthcare Robotics Technologies - Metrified (HEART-MET) work stream https://metricsproject.eu/healthcare/ 

Project lead: Praminda Caleb-Solly

• HEAP Human Guided Learning and Benchmarking of Robotic Heap Sorting

HEAP is a research project funded by Chist-Era that investigates Robot Manipulation Algorithms for Robotic Heap Sorting. This project will provide scientific advancements for benchmarking, object recognition, manipulation and human-robot interaction. We focus on sorting a complex, unstructured heap of unknown objects –resembling nuclear waste consisting of a set of broken deformed bodies– as an instance of an extremely complex manipulation task. The consortium aims at building an end-to-end benchmarking framework, which includes rigorous scientific methodology and experimental tools for application in realistic scenarios. 

Project lead: Ayse Kucukyilmaz

Related Past Research Projects with CHART team member involvement

• Assuring Autonomy International Programme Assistive Robotics in Healthcare Demonstrator

The Assistive Robotics in Healthcare demonstrator project was investigating and evaluating the safety, regulatory and requirements of close proximate human-robot interaction (HRI) in unstructured domestic environments. It was utilising the CHIRON robotic system co-developed by Prof Caleb-Solly while she was working at the Bristol Robotics Laboratory and Designability. 

Project lead: Praminda Caleb-Solly

Related Publications

• Camilleri, A., Dogramadzi, S. and Caleb-Solly, P., 2022. A Study on the Effects of Cognitive Overloading and Distractions on Human Movement During Robot-Assisted Dressing. Frontiers in robotics and AI, 9, p.815871. https://www.frontiersin.org/articles/10.3389/frobt.2022.815871/full 

• Camilleri, A., Dogramadzi, S. and Caleb-Solly, P., 2022, March. Learning from Carers to inform the Design of Safe Physically Assistive Robots-Insights from a Focus Group Study. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (pp. 703-707). https://dl.acm.org/doi/abs/10.5555/3523760.3523859 

• Delgado Bellamy, D., Chance, G., Caleb-Solly, P. and Dogramadzi, S., 2021. Safety Assessment Review of a Dressing Assistance Robot. Frontiers in Robotics and AI, 8, p.667316. https://www.frontiersin.org/articles/10.3389/frobt.2021.667316/full 

• Caleb-Solly, P., Harper, C. and Dogramadzi, S., 2021, March. Standards and regulations for physically assistive robots. In 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR) (pp. 259-263). IEEE. https://ieeexplore.ieee.org/abstract/document/9419508 

• Harper, C. and Caleb-Solly, P., 2021. Towards an Ontological Framework for Environmental Survey Hazard Analysis of Autonomous Systems. SafeAI@ AAAI.  http://ceur-ws.org/Vol-2808/Paper_41.pdf 

• Laparidou, D., Curtis, F., Akanuwe, J., Goher, K., Niroshan Siriwardena, A., & Kucukyilmaz, A. (2021). Patient, carer, and staff perceptions of robotics in motor rehabilitation: a systematic review and qualitative meta-synthesis. Journal of neuroengineering and rehabilitation, 18(1), 1-24. https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-021-00976-3