Through a prestigious three-year NASA grant, a collaborative team of researchers across six different universities will work to understand the vital role of trust in autonomous systems. Dr. Sandip Ray serves as the lead for the University of Florida team, working in partnership with Dr. Swarup Bhunia.
The overall project is led by Dr. Siddhartha Bhattacharyya and Dr. Meredith Carroll at Florida Institute of Technology, and also includes researchers from Santa Fe College, The Pennsylvania State University, North Carolina A&T State University, and Stanford University.
The project, “Trustworthy Resilient Autonomous Agents for Safe City Transportation in the Evolving New Decade” (TRANSCEND), is funded through the NASA University Leadership Initiative (ULI) program. TRANSCEND aims to develop a framework to determine the safety and trustworthiness of machine-learning-enabled autonomous aviation systems.
“Safety is a critical thing we don’t really understand yet, and we need more people to work on it. Not many people realize there are safety issues in AI. We need to figure out what are the safety issues, how to make systems safer, and then teach others how to do it.”
– Dr. Sandip Ray
Autonomous systems are on the rise, and AI-based software is expected to be increasingly used in airplanes, drones and avionics. In order to ensure the safety of these systems, human operators need to be able to trust and rely on systems to perform as they expect and trust that the software will alert them when it needs a human to take over.
The research group will work to bridge the trust gap by integrating methods, data, explainability, and practices from different domains in order to develop the TRANSCEND framework. To test the framework, they’ll use a range of tools such as simulators to demonstrate trustworthiness in practice. This and other data will help them develop a toolkit of guidelines for development processes, recommendations and suggested safety measures for engineers to reference when designing “trustworthy,” learning-enabled autonomous systems.
“Safety for aerospace systems is paramount – there is a growing need to explore a principled automated approach to design and verification of safe systems, potentially harnessing the power of AI, which this project will investigate,” said Dr. Bhunia.
The involvement of Santa Fe College in Gainesville adds an important workforce development component to this project. UF is at the forefront of the effort by introducing hands-on AI and safety training for Santa Fe College instructors. This will integrate and promote the concept of AI safety in community college courses at Santa Fe and throughout Florida, helping to establish a future workforce well-trained in autonomous systems concepts and challenges.