The framework improves accuracy of detection while running faster on embedded hardware

A research team led by a Johns Hopkins civil and systems engineer has developed a new system that significantly improves how autonomous vehicles and other automated systems detect objects without sacrificing the speed and efficiency required for real-time decision making. Current systems often require significant processing time and computational power, slowing them down when they need to analyze data quickly. The team’s results appear in IEEE Transactions on Pattern Analysis and Machine Learning, a flag-ship AI and Machine Learning journal in IEEE proceedings.

“In autonomous driving, accuracy and efficiency are often seen as competing goals. Big models are usually accurate but slow, while small models are typically fast but fragile,” said team leader Hao (Frank) Yang, an assistant professor of civil and systems engineering at the Whiting School of Engineering and a member of the Johns Hopkins Institute for Assured Autonomy. “Our idea was to build a system that adapts its intelligence depending on the driving scene, ensuring that vehicles respond both accurately, timely and safely.”

Instead of relying on a single source of data and an individual computing model, the team created a system called Edge-based Mixture-of-Experts Optimal Sensing (EMOS) that uses data from both cameras and light-detection and ranging sensors to assess surroundings. Mixture-of-experts is an innovative machine learning approach that fuse several neural networks into a specialized ensemble system, where each expert learns to handle different types of inputs at different scenarios.

The study revealed that EMOS improves the accuracy of detection by more than 3% while running more than twice as fast on embedded hardware, without compromising the safety of autonomous vehicles.

“By dynamically choosing the right expert for the right situation, our framework achieves what existing object detection models cannot,” the authors write in their study.

They explain that EMOS can allow for the accelerated deployment of safer, more efficient self-driving systems. In the future, the research team plans to extend this work so that connected vehicles can continually share data and adapt to variables in real time.

The study team included Linshen Liu and Pu Wang, both first-year doctoral students in civil and systems engineering.