Q. Meng, E. Flores, C. Quintero-Peña, P. Qian, Z. Kingston, S. K. Hamlin, V. Unhelkar, and L. E. Kavraki, “Look as You Leap: Planning Simultaneous Motion and Perception for High-DoF Robots,” IEEE Transactions on Robotics, Aug. 2026.
Most common tasks for robots in dynamic spaces require that the environment is regularly and actively perceived. The perception task considered in this work can represent a broad range of robot perception objectives, including object detection, human activity recognition, and human face detection. For example, a service robot may need to continuously detect and localize a target object during manipulation, while an assistive robot in a healthcare setting may need to maintain reliable perception of a human face or activity for interaction and safety monitoring. These tasks impose perception constraints on the robot motion. However, solving motion and perception tasks simultaneously is challenging, as these objectives often impose conflicting requirements. Furthermore, while robots must react quickly to changes in the environment, directly evaluating the quality of perception (e.g., object detection confidence) is often expensive or infeasible at runtime. This problem is especially important in human-centered environments, such as homes and hospitals, where effective perception is essential for safe and reliable operation. In this work, we address the challenge of solving motion planning problems for high-degree-of-freedom (DoF) robots from a start to a goal configuration with continuous perception constraints under both static and dynamic environments. Our solution is a GPU-parallelized perception-score-guided probabilistic roadmap planner with a neural surrogate model (PS-PRM). Unlike existing active perception-, visibility-aware or learning-based planners, our work jointly considers perception tasks and constraints when searching for a solution to the motion planning problem. Our method uses a neural surrogate model to approximate perception scores, incorporates them into a roadmap-based solution, and leverages GPU parallelism to enable efficient online replanning in dynamic settings. We demonstrate that our planner, evaluated on high-DoF robots, outperforms RL- and trajectory-optimization-based baseline methods in both static and dynamic environments in both simulation and real-robot experiments.
Publisher: http://dx.doi.org/10.1109/TRO.2026.3731487
PDF preprint: http://kavrakilab.org/publications/meng_psprm_2026.pdf