A Framework for Manipulation Planning and Execution under Uncertainty in Partially-Known Environments

There is a pressing need to make today’s robots capable, robust, and efficient during real-world operation. This proposal focuses on near-future scenarios that require complex long-horizon reasoning with non-trivial constraints on the robot’s motion. Examples include a robot operating in a warehouse with a partly automated storage and retrieval system or a robot running experiments in an automated laboratory. Several methods in robotics address the challenges of the above scenarios with explicit and carefully crafted planning domains that model how the robot interacts with the environment. Planning domains provide an abstraction over the world that is essential for Task and Motion Planning (TAMP) methods that plan over long horizons, that is, compute executable complex plans that require many steps or have non-monotonic properties, such as rearranging objects on shelves or fetching reactants for an experiment. This proposal will develop interpretable TAMP methods with the capability to deal with increasing uncertainty in the environment, while not sacrificing their strengths and providing a structured framework that allows for a meaningful connection with emerging model-free approaches. The project makes fundamental contributions to the core robotics problem of effective and efficient long-horizon planning under uncertainty by bringing together ideas from motion planning and control under uncertainty, optimization theory, probability and statistics, inverse reinforcement learning, imitation learning, high-dimensional search, factor graphs, probabilistic inference, classical AI planning, and formal methods.

This work has been supported by grant NSF CCF 2336612.

Related Publications

1. Q. Meng, E. Flores, C. Quintero-Peña, P. Qian, Z. Kingston, S. K. Hamlin, V. Unhelkar, and L. E. Kavraki, “Look as You Leap: Planning Simultaneous Motion and Perception for High-DoF Robots,” IEEE Transactions on Robotics, Aug. 2026.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @article{meng_psprm_2026,
     author = {Meng, Qingxi and Flores, Emiliano and Quintero-Pe{\~n}a, Carlos and Qian, Peizhu and Kingston, Zachary and Hamlin, Shannan K. and Unhelkar, Vaibhav and Kavraki, Lydia E.},
     journal = {IEEE Transactions on Robotics},
     title = {Look as You Leap: Planning Simultaneous Motion and Perception for High-DoF Robots},
     year = {2026},
     month = aug,
     keywords = {Motion and path planning;constrained motion planning;nonholonomic motion
         planning;reactive and sensor-based planning;perception-aware planning;GPU-accelerated
         planning},
     abstract = {Most common tasks for robots in dynamic spaces require that the environment is
         regularly and actively perceived. The perception task considered in this work can represent
         a broad range of robot perception objectives, including object detection, human activity
         recognition, and human face detection. For example, a service robot may need to continuously
         detect and localize a target object during manipulation, while an assistive robot in a
         healthcare setting may need to maintain reliable perception of a human face or activity for
         interaction and safety monitoring. These tasks impose perception constraints on the robot
         motion. However, solving motion and perception tasks simultaneously is challenging, as these
         objectives often impose conflicting requirements. Furthermore, while robots must react
         quickly to changes in the environment, directly evaluating the quality of perception (e.g.,
         object detection confidence) is often expensive or infeasible at runtime. This problem is
         especially important in human-centered environments, such as homes and hospitals, where
         effective perception is essential for safe and reliable operation. In this work, we address
         the challenge of solving motion planning problems for high-degree-of-freedom (DoF) robots
         from a start to a goal configuration with continuous perception constraints under both
         static and dynamic environments. Our solution is a GPU-parallelized perception-score-guided
         probabilistic roadmap planner with a neural surrogate model (PS-PRM). Unlike existing active
         perception-, visibility-aware or learning-based planners, our work jointly considers
         perception tasks and constraints when searching for a solution to the motion planning
         problem. Our method uses a neural surrogate model to approximate perception scores,
         incorporates them into a roadmap-based solution, and leverages GPU parallelism to enable
         efficient online replanning in dynamic settings. We demonstrate that our planner, evaluated
         on high-DoF robots, outperforms RL- and trajectory-optimization-based baseline methods in
         both static and dynamic environments in both simulation and real-robot experiments.},
     doi = {10.1109/TRO.2026.3731487}
   }
   

   Abstract

   ×

   Most common tasks for robots in dynamic spaces require that the environment is regularly and actively perceived. The perception task considered in this work can represent a broad range of robot perception objectives, including object detection, human activity recognition, and human face detection. For example, a service robot may need to continuously detect and localize a target object during manipulation, while an assistive robot in a healthcare setting may need to maintain reliable perception of a human face or activity for interaction and safety monitoring. These tasks impose perception constraints on the robot motion. However, solving motion and perception tasks simultaneously is challenging, as these objectives often impose conflicting requirements. Furthermore, while robots must react quickly to changes in the environment, directly evaluating the quality of perception (e.g., object detection confidence) is often expensive or infeasible at runtime. This problem is especially important in human-centered environments, such as homes and hospitals, where effective perception is essential for safe and reliable operation. In this work, we address the challenge of solving motion planning problems for high-degree-of-freedom (DoF) robots from a start to a goal configuration with continuous perception constraints under both static and dynamic environments. Our solution is a GPU-parallelized perception-score-guided probabilistic roadmap planner with a neural surrogate model (PS-PRM). Unlike existing active perception-, visibility-aware or learning-based planners, our work jointly considers perception tasks and constraints when searching for a solution to the motion planning problem. Our method uses a neural surrogate model to approximate perception scores, incorporates them into a roadmap-based solution, and leverages GPU parallelism to enable efficient online replanning in dynamic settings. We demonstrate that our planner, evaluated on high-DoF robots, outperforms RL- and trajectory-optimization-based baseline methods in both static and dynamic environments in both simulation and real-robot experiments.
   Details
2. Y. Liang, E. Kim, J. A. Knoll, W. Thomason, Z. Kingston, L. E. Kavraki, and H. Kurniawati, “Think Fast and Far: Long-Horizon Online POMDP Planning via Rapid State Sampling,” June 2026.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @article{liang_2026,
     title = {Think Fast and Far: Long-Horizon Online POMDP Planning via Rapid State Sampling},
     author = {Liang, Yuanchu and Kim, Edward and Knoll, J. Arden and Thomason, Wil and Kingston, Zachary and Kavraki, Lydia E. and Kurniawati, Hanna},
     year = {2026},
     month = jun,
     eprint = {2606.04355},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2606.04355}
   }
   

   Details
3. W. Guo, T. Tyrovouzis, and L. E. Kavraki, “Python Bindings for a Large C++ Robotics Library: The Case of OMPL,” in IEEE International Conference on Robotics and Automation (ICRA), 2026.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @inproceedings{guo_python_ompl_2026,
     author = {Guo, Weihang and Tyrovouzis, Theodoros and Kavraki, Lydia E.},
     title = {Python Bindings for a Large C++ Robotics Library: The Case of OMPL},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     year = {2026},
     month = mar,
     abstract = {Python bindings are a critical bridge between high-performance C++ libraries and the
         flexibility of Python, enabling rapid prototyping, reproducible experiments, and integration
         with simulation and learning frameworks in robotics research. Yet, generating bindings for
         large codebases is a tedious process that creates a heavy burden for a small group of
         maintainers. In this work, we investigate the use of Large Language Models (LLMs) to assist
         in generating nanobind wrappers, with human experts kept in the loop. Our workflow mirrors
         the structure of the C++ codebase, scaffolds empty wrapper files, and employs LLMs to fill
         in binding definitions. Experts then review and refine the generated code to ensure
         correctness, compatibility, and performance. Through a case study on a large C++ motion
         planning library, we document common failure modes, including mismanaging shared pointers,
         overloads, and trampolines, and show how in-context examples and careful prompt design
         improve reliability. Experiments demonstrate that the resulting bindings achieve runtime
         performance comparable to legacy solutions. Beyond this case study, our results provide
         general lessons for applying LLMs to binding generation in large-scale C++ projects.}
   }
   

   Abstract

   ×

   Python bindings are a critical bridge between high-performance C++ libraries and the flexibility of Python, enabling rapid prototyping, reproducible experiments, and integration with simulation and learning frameworks in robotics research. Yet, generating bindings for large codebases is a tedious process that creates a heavy burden for a small group of maintainers. In this work, we investigate the use of Large Language Models (LLMs) to assist in generating nanobind wrappers, with human experts kept in the loop. Our workflow mirrors the structure of the C++ codebase, scaffolds empty wrapper files, and employs LLMs to fill in binding definitions. Experts then review and refine the generated code to ensure correctness, compatibility, and performance. Through a case study on a large C++ motion planning library, we document common failure modes, including mismanaging shared pointers, overloads, and trampolines, and show how in-context examples and careful prompt design improve reliability. Experiments demonstrate that the resulting bindings achieve runtime performance comparable to legacy solutions. Beyond this case study, our results provide general lessons for applying LLMs to binding generation in large-scale C++ projects.
   Details
4. Q. Meng, E. Flores, T. Duong, V. Unhelkar, and L. E. Kavraki, “Sampling-Based Motion Planning With Scene Graphs Under Perception Constraints,” IEEE Robotics and Automation Letters, vol. 11, no. 5, pp. 5478–5485, Mar. 2026.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @article{meng_2026,
     author = {Meng, Qingxi and Flores, Emiliano and Duong, Thai and Unhelkar, Vaibhav and Kavraki, Lydia E.},
     journal = {IEEE Robotics and Automation Letters},
     title = {Sampling-Based Motion Planning With Scene Graphs Under Perception Constraints},
     year = {2026},
     month = mar,
     volume = {11},
     number = {5},
     pages = {5478-5485},
     keywords = {Robots;Costs;Planning;Monitoring;Collision avoidance;Trajectory;Robot vision
         systems;Kinematics;Cameras;Semantics;Constrained motion planning;motion and path
         planning;vision-based navigation},
     doi = {10.1109/LRA.2026.3674013}
   }
   

   Details
5. N. Hart, N. Dunkelberger, E. Holum, L. E. Kavraki, E. Zemler, and S. Azimi, “The iMETRO Dynamic Simulation: An Open-Source Simulator for Intravehicular Space Robotics Research,” in NASA Technical Reports Server, 2026. Accepted for publication at the International Conference of Robotics and Automation.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @inproceedings{hart_2026,
     author = {Hart, Nikki and Dunkelberger, Nathan and Holum, Erik and Kavraki, Lydia E. and Zemler, Emma and Azimi, Shaun},
     booktitle = {NASA Technical Reports Server},
     title = {The iMETRO Dynamic Simulation: An Open-Source Simulator for Intravehicular Space
         Robotics Research},
     year = {2026},
     month = feb,
     note = {Accepted for publication at the International Conference of Robotics and Automation.},
     url = {https://ntrs.nasa.gov/citations/20260001395}
   }
   

   Details
6. W. Guo, Z. Kingston, K. Hang, and L. E. Kavraki, “Efficient Multi-Robot Motion Planning for Manifold-Constrained Manipulators by Randomized Scheduling and Informed Path Generation,” IEEE Robotics and Automation Letters, vol. 11, no. 4, pp. 4385–4392, Feb. 2026.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @article{weihang-manifoldpath2026,
     author = {Guo, Weihang and Kingston, Zachary and Hang, Kaiyu and Kavraki, Lydia E.},
     journal = {IEEE Robotics and Automation Letters},
     title = {Efficient Multi-Robot Motion Planning for Manifold-Constrained Manipulators by
         Randomized Scheduling and Informed Path Generation},
     year = {2026},
     month = feb,
     volume = {11},
     number = {4},
     pages = {4385-4392},
     keywords = {Robot kinematics;Collision avoidance;Schedules;Manipulators;Manifolds;System
         recovery;Multi-robot systems;End effectors;Trajectory;Timing;Multi-robot systems;constrained
         motion planning;motion and path planning;collision avoidance},
     doi = {10.1109/LRA.2026.3662639}
   }
   

   Details
7. Y. Liang, E. Kim, W. Thomason, Z. Kingston, H. Kurniawati, and L. E. Kavraki, “Scaling Long-Horizon Online POMDP Planning via Rapid State Space Sampling,” in Robotics Research, 2025. Appeared at International Symposium of Robotics Research (ISRR) 2024.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @inproceedings{liang2024-scaling,
     title = {Scaling Long-Horizon Online POMDP Planning via Rapid State Space Sampling},
     author = {Liang, Yuanchu and Kim, Edward and Thomason, Wil and Kingston, Zachary and Kurniawati, Hanna and Kavraki, Lydia E.},
     booktitle = {Robotics Research},
     note = {Appeared at International Symposium of Robotics Research (ISRR) 2024.},
     year = {2025},
     month = nov,
     abstract = {Partially Observable Markov Decision Processes (POMDPs) are a general and principled
         framework for motion planning under uncertainty. Despite tremendous improvement in the
         scalability of POMDP solvers, long-horizon POMDPs (e.g., steps) remain difficult to solve.
         This paper proposes a new approximate online POMDP solver, called Reference-Based Online
         POMDP Planning via Rapid State Space Sampling (ROP-RaS3). ROP-RaS3 uses novel extremely fast
         sampling-based motion planning techniques to sample the state space and generate a diverse
         set of macro actions online which are then used to bias belief-space sampling and infer
         high-quality policies without requiring exhaustive enumeration of the action space -- a
         fundamental constraint for modern online POMDP solvers. ROP-RaS3 is evaluated on various
         long-horizon POMDPs, including on a problem with a planning horizon of more than 100 steps
         and a problem with a 15-dimensional state space that requires more than 20 look ahead steps.
         In all of these problems, ROP-RaS3 substantially outperforms other state-of-the-art methods
         by up to multiple folds.}
   }
   

   Abstract

   ×

   Partially Observable Markov Decision Processes (POMDPs) are a general and principled framework for motion planning under uncertainty. Despite tremendous improvement in the scalability of POMDP solvers, long-horizon POMDPs (e.g., steps) remain difficult to solve. This paper proposes a new approximate online POMDP solver, called Reference-Based Online POMDP Planning via Rapid State Space Sampling (ROP-RaS3). ROP-RaS3 uses novel extremely fast sampling-based motion planning techniques to sample the state space and generate a diverse set of macro actions online which are then used to bias belief-space sampling and infer high-quality policies without requiring exhaustive enumeration of the action space – a fundamental constraint for modern online POMDP solvers. ROP-RaS3 is evaluated on various long-horizon POMDPs, including on a problem with a planning horizon of more than 100 steps and a problem with a 15-dimensional state space that requires more than 20 look ahead steps. In all of these problems, ROP-RaS3 substantially outperforms other state-of-the-art methods by up to multiple folds.
   Details
8. K. Ren, G. Wang, A. S. Morgan, L. E. Kavraki, and K. Hang, “Object-Centric Kinodynamic Planning for Nonprehensile Robot Rearrangement Manipulation,” IEEE Transactions on Robotics, vol. 41, pp. 5761–5780, Sept. 2025.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @article{ren-kinodynamic-planning2025,
     author = {Ren, Kejia and Wang, Gaotian and Morgan, Andrew S. and Kavraki, Lydia E. and Hang, Kaiyu},
     journal = {IEEE Transactions on Robotics},
     title = {Object-Centric Kinodynamic Planning for Nonprehensile Robot Rearrangement
         Manipulation},
     year = {2025},
     month = sep,
     volume = {41},
     number = {},
     pages = {5761-5780},
     abstract = {Nonprehensile actions, such as pushing, are crucial for addressing multiobject
         rearrangement problems. Many traditional methods generate robot-centric actions, which
         differ from intuitive human strategies and are typically inefficient. To this end, we adopt
         an object-centric planning paradigm and propose a unified framework for addressing a range
         of large-scale, physics-intensive nonprehensile rearrangement problems challenged by
         modeling inaccuracies and real-world uncertainties. By assuming that each object can
         actively move without being driven by robot interactions, our planner first computes desired
         object motions, which are then realized through robot actions generated online via a
         closed-loop pushing strategy. Through extensive experiments and in comparison with
         state-of-the-art baselines in both simulation and on a physical robot, we show that our
         object-centric planning framework can generate more intuitive and task-effective robot
         actions with significantly improved efficiency. In addition, we propose a benchmarking
         protocol to standardize and facilitate future research in nonprehensile rearrangement.},
     keywords = {Robots;Planning;Uncertainty;Physics;Training;Sorting;Real-time systems;Shape;Robot
         motion;Clutter;Multi-object rearrangement;nonprehensile manipulation;object-centric
         planning},
     doi = {10.1109/TRO.2025.3613532},
     url = {https://ieeexplore.ieee.org/abstract/document/11176447},
     issn = {1941-0468}
   }
   

   Abstract

   ×

   Nonprehensile actions, such as pushing, are crucial for addressing multiobject rearrangement problems. Many traditional methods generate robot-centric actions, which differ from intuitive human strategies and are typically inefficient. To this end, we adopt an object-centric planning paradigm and propose a unified framework for addressing a range of large-scale, physics-intensive nonprehensile rearrangement problems challenged by modeling inaccuracies and real-world uncertainties. By assuming that each object can actively move without being driven by robot interactions, our planner first computes desired object motions, which are then realized through robot actions generated online via a closed-loop pushing strategy. Through extensive experiments and in comparison with state-of-the-art baselines in both simulation and on a physical robot, we show that our object-centric planning framework can generate more intuitive and task-effective robot actions with significantly improved efficiency. In addition, we propose a benchmarking protocol to standardize and facilitate future research in nonprehensile rearrangement.
   Details
9. W. Guo, Z. Kingston, and L. E. Kavraki, “CaStL: Constraints as Specifications through LLM Translation for Long-Horizon Task and Motion Planning,” in IEEE International Conference on Robotics and Automation (ICRA), 2025, pp. 11957–11964.
   
   pdf publisher bibtex abstract details

   BibTeX

   ×

   @inproceedings{guo2025-castl,
     author = {Guo, Weihang and Kingston, Zachary and Kavraki, Lydia E.},
     title = {CaStL: Constraints as Specifications through LLM Translation for Long-Horizon Task and
         Motion Planning},
     year = {2025},
     month = may,
     doi = {10.1109/ICRA55743.2025.11127555},
     url = {https://ieeexplore.ieee.org/abstract/document/11127555},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     abstract = {Large Language Models (LLMs) have demonstrated remarkable ability in long-horizon
         Task and Motion Planning (TAMP) by translating clear and straightforward natural language
         problems into formal specifications such as the Planning Domain Definition Language (PDDL).
         However, real-world problems are often ambiguous and involve many complex constraints. In
         this paper, we introduce Constraints as Specifications through LLMs (CaStL), a framework
         that identifies constraints such as goal conditions, action ordering, and action blocking
         from natural language in multiple stages. CaStL translates these constraints into PDDL and
         Python scripts, which are then solved using an custom PDDL solver. Tested across three PDDL
         domains, CaStL significantly improves constraint handling and planning success rates from
         natural language specification in complex scenarios.},
     pages = {11957--11964}
   }
   

   Abstract

   ×

   Large Language Models (LLMs) have demonstrated remarkable ability in long-horizon Task and Motion Planning (TAMP) by translating clear and straightforward natural language problems into formal specifications such as the Planning Domain Definition Language (PDDL). However, real-world problems are often ambiguous and involve many complex constraints. In this paper, we introduce Constraints as Specifications through LLMs (CaStL), a framework that identifies constraints such as goal conditions, action ordering, and action blocking from natural language in multiple stages. CaStL translates these constraints into PDDL and Python scripts, which are then solved using an custom PDDL solver. Tested across three PDDL domains, CaStL significantly improves constraint handling and planning success rates from natural language specification in complex scenarios.
   Details
10. T. S. Wilson, W. Thomason, Z. Kingston, L. E. Kavraki, and J. D. Gammell, “Nearest-Neighbourless Asymptotically Optimal Motion Planning with Fully Connected Informed Trees (FCIT*),” in IEEE International Conference on Robotics and Automation (ICRA), 2025, pp. 14140–14146.
    
    pdf publisher bibtex abstract details

    BibTeX

    ×

    @inproceedings{wilson2025-fcit,
      author = {Wilson, Tyler S. and Thomason, Wil and Kingston, Zachary and Kavraki, Lydia E. and Gammell, Jonathan D.},
      title = {Nearest-Neighbourless Asymptotically Optimal Motion Planning with Fully Connected
          Informed Trees (FCIT*)},
      year = {2025},
      month = mar,
      booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
      abstract = {Improving the performance of motion planning algorithms for high-degree-of-freedom
          robots usually requires reducing the cost or frequency of computationally expensive
          operations. Traditionally, and especially for asymptotically optimal sampling-based motion
          planners, the most expensive operations are local motion validation and querying the nearest
          neighbours of a configuration. Recent advances have significantly reduced the cost of motion
          validation by using single instruction/multiple data (SIMD) parallelism to improve solution
          times for satisficing motion planning problems. These advances have not yet been applied to
          asymptotically optimal motion planning. This paper presents Fully Connected Informed Trees
          (FCIT*), the first fully connected, informed, anytime almost-surely asymptotically optimal
          (ASAO) algorithm. FCIT* exploits the radically reduced cost of edge evaluation via SIMD
          parallelism to build and search fully connected graphs. This removes the need for
          nearest-neighbours structures, which are a dominant cost for many sampling-based motion
          planners, and allows it to find initial solutions faster than state-of-the-art ASAO (VAMP,
          OMPL) and satisficing (OMPL) algorithms on the MotionBenchMaker dataset while converging
          towards optimal plans in an anytime manner.},
      pages = {14140--14146}
    }
    

    Abstract

    ×

    Improving the performance of motion planning algorithms for high-degree-of-freedom robots usually requires reducing the cost or frequency of computationally expensive operations. Traditionally, and especially for asymptotically optimal sampling-based motion planners, the most expensive operations are local motion validation and querying the nearest neighbours of a configuration. Recent advances have significantly reduced the cost of motion validation by using single instruction/multiple data (SIMD) parallelism to improve solution times for satisficing motion planning problems. These advances have not yet been applied to asymptotically optimal motion planning. This paper presents Fully Connected Informed Trees (FCIT*), the first fully connected, informed, anytime almost-surely asymptotically optimal (ASAO) algorithm. FCIT* exploits the radically reduced cost of edge evaluation via SIMD parallelism to build and search fully connected graphs. This removes the need for nearest-neighbours structures, which are a dominant cost for many sampling-based motion planners, and allows it to find initial solutions faster than state-of-the-art ASAO (VAMP, OMPL) and satisficing (OMPL) algorithms on the MotionBenchMaker dataset while converging towards optimal plans in an anytime manner.
    Details
11. T. Pan, C. K. Verginis, and L. E. Kavraki, “Robust and Safe Task-Driven Planning and Navigation for Heterogeneous Multi-Robot Teams with Uncertain Dynamics,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024, pp. 3482–3489.
    
    pdf publisher bibtex abstract details

    BibTeX

    ×

    @inproceedings{pan2024-iros,
      author = {Pan, Tianyang and Verginis, Christos K. and Kavraki, Lydia E.},
      title = {Robust and Safe Task-Driven Planning and Navigation for Heterogeneous Multi-Robot Teams
          with Uncertain Dynamics},
      booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
      year = {2024},
      month = oct,
      doi = {10.1109/IROS58592.2024.10802695},
      pages = {3482 - 3489},
      abstract = {Task and motion planning (TAMP) can enhance intelligent multi-robot coordination.
          TAMP becomes significantly more complicated in obstacle-cluttered environments and in the
          presence of robot dynamic uncertainties. We propose a control framework that solves the
          motion-planning problem for multi-robot teams with uncertain dynamics, addressing a key
          component of the TAMP pipeline. The principal part of the proposed algorithm constitutes a
          decentralized feedback control policy for tracking of reference paths taken by the robots
          while avoiding collision and adapting in real time to the underlying dynamic uncertainties.
          The proposed framework further leverages sampling-based motion planners to free the robots
          from local-minimum configurations. Extensive exper- imental results in complex, realistic
          environments illustrate the superior efficiency of the proposed approach, in terms of
          planning time and number of encountered local minima, with respect to state-of-the-art
          baseline methods.}
    }
    

    Abstract

    ×

    Task and motion planning (TAMP) can enhance intelligent multi-robot coordination. TAMP becomes significantly more complicated in obstacle-cluttered environments and in the presence of robot dynamic uncertainties. We propose a control framework that solves the motion-planning problem for multi-robot teams with uncertain dynamics, addressing a key component of the TAMP pipeline. The principal part of the proposed algorithm constitutes a decentralized feedback control policy for tracking of reference paths taken by the robots while avoiding collision and adapting in real time to the underlying dynamic uncertainties. The proposed framework further leverages sampling-based motion planners to free the robots from local-minimum configurations. Extensive exper- imental results in complex, realistic environments illustrate the superior efficiency of the proposed approach, in terms of planning time and number of encountered local minima, with respect to state-of-the-art baseline methods.
    Details
12. K. Elimelech, J. Motes, M. Morales, N. M. Amato, M. Y. Vardi, and L. E. Kavraki, “Encoding Reusable Multi-Robot Planning Strategies as Abstract Hypergraphs,” in 40th Anniversary of the IEEE International Conference on Robotics and Automation, 2024.
    
    pdf publisher bibtex abstract details

    BibTeX

    ×

    @inproceedings{elimelech2024-hypergraphs,
      title = {Encoding Reusable Multi-Robot Planning Strategies as Abstract Hypergraphs},
      author = {Elimelech, Khen and Motes, James and Morales, Marco and Amato, Nancy M. and Vardi, Moshe Y. and Kavraki, Lydia E.},
      booktitle = {40th Anniversary of the IEEE International Conference on Robotics and Automation},
      year = {2024},
      month = sep,
      abstract = {Multi-Robot Task Planning (MR-TP) is the search for a discrete-action plan a team of
          robots should take to complete a task. The complexity of such problems scales exponentially
          with the number of robots and task complexity, making them challenging for online solution.
          To accelerate MR-TP over a system's lifetime, this work looks at combining two recent
          advances: (i) Decomposable State Space Hypergraph (DaSH), a novel hypergraph-based framework
          to efficiently model and solve MR-TP problems; and (ii) learning-by-abstraction, a technique
          that enables automatic extraction of generalizable planning strategies from individual
          planning experiences for later reuse. Specifically, we wish to extend this strategy-learning
          technique, originally designed for single-robot planning, to benefit multi-robot planning
          using hypergraph-based MR-TP.}
    }
    

    Abstract

    ×

    Multi-Robot Task Planning (MR-TP) is the search for a discrete-action plan a team of robots should take to complete a task. The complexity of such problems scales exponentially with the number of robots and task complexity, making them challenging for online solution. To accelerate MR-TP over a system’s lifetime, this work looks at combining two recent advances: (i) Decomposable State Space Hypergraph (DaSH), a novel hypergraph-based framework to efficiently model and solve MR-TP problems; and (ii) learning-by-abstraction, a technique that enables automatic extraction of generalizable planning strategies from individual planning experiences for later reuse. Specifically, we wish to extend this strategy-learning technique, originally designed for single-robot planning, to benefit multi-robot planning using hypergraph-based MR-TP.
    Details
13. C. W. Ramsey, Z. Kingston, W. Thomason, and L. E. Kavraki, “Collision-Affording Point Trees: SIMD-Amenable Nearest Neighbors for Fast Collision Checking,” in Robotics: Science and Systems, 2024.
    
    pdf publisher bibtex abstract details

    BibTeX

    ×

    @inproceedings{ramsey2024-capt,
      author = {Ramsey, Clayton W. and Kingston, Zachary and Thomason, Wil and Kavraki, Lydia E.},
      booktitle = {Robotics: Science and Systems},
      title = {Collision-Affording Point Trees: SIMD-Amenable Nearest Neighbors for Fast Collision
          Checking},
      year = {2024},
      month = jul,
      doi = {10.15607/RSS.2024.XX.038},
      url = {https://roboticsconference.org/2024/program/papers/38/},
      abstract = {Motion planning against sensor data is often a critical bottleneck in real-time
          robot control. For sampling-based motion planners, which are effective for high-dimensional
          systems such as manipulators, the most time-intensive component is collision checking. We
          present a novel spatial data structure, the collision-affording point tree (CAPT): an exact
          representation of point clouds that accelerates collision-checking queries between robots
          and point clouds by an order of magnitude, with an average query time of less than 10
          nanoseconds on 3D scenes comprising thousands of points. With the CAPT, sampling-based
          planners can generate valid, high-quality paths in under a millisecond, with total
          end-to-end computation time faster than 60 FPS, on a single thread of a consumer-grade CPU.
          We also present a point cloud filtering algorithm, based on space-filling curves, which
          reduces the number of points in a point cloud while preserving structure. Our approach
          enables robots to plan at real-time speeds in sensed environments, opening up potential uses
          of planning for high-dimensional systems in dynamic, changing, and unmodeled environments.}
    }
    

    Abstract

    ×

    Motion planning against sensor data is often a critical bottleneck in real-time robot control. For sampling-based motion planners, which are effective for high-dimensional systems such as manipulators, the most time-intensive component is collision checking. We present a novel spatial data structure, the collision-affording point tree (CAPT): an exact representation of point clouds that accelerates collision-checking queries between robots and point clouds by an order of magnitude, with an average query time of less than 10 nanoseconds on 3D scenes comprising thousands of points. With the CAPT, sampling-based planners can generate valid, high-quality paths in under a millisecond, with total end-to-end computation time faster than 60 FPS, on a single thread of a consumer-grade CPU. We also present a point cloud filtering algorithm, based on space-filling curves, which reduces the number of points in a point cloud while preserving structure. Our approach enables robots to plan at real-time speeds in sensed environments, opening up potential uses of planning for high-dimensional systems in dynamic, changing, and unmodeled environments.
    Details
14. T. Pan, R. Shome, and L. E. Kavraki, “Task and Motion Planning for Execution in the Real,” IEEE Transactions on Robotics, pp. 1–16, June 2024. Honorable Mention, 2024 IEEE Transactions on Robotics (T-RO) King-Sun Fu Memorial Best Paper Award.
    
    pdf publisher bibtex abstract details

    BibTeX

    ×

    @article{pan2024-tamper,
      author = {Pan, Tianyang and Shome, Rahul and Kavraki, Lydia E.},
      journal = {IEEE Transactions on Robotics},
      title = {Task and Motion Planning for Execution in the Real},
      year = {2024},
      month = jun,
      volume = {},
      number = {},
      pages = {1-16},
      doi = {10.1109/TRO.2024.3418550},
      note = {Honorable Mention, 2024 IEEE Transactions on Robotics (T-RO) King-Sun Fu Memorial Best
          Paper Award.},
      abstract = {Task and motion planning represents a powerful set of hybrid planning methods that
          combine reasoning over discrete task domains and continuous motion generation. Traditional
          reasoning necessitates task domain models and enough information to ground actions to motion
          planning queries. Gaps in this knowledge often arise from sources like occlusion or
          imprecise modeling. This work generates task and motion plans that include actions cannot be
          fully grounded at planning time. During execution, such an action is handled by a provided
          human-designed or learned closed-loop behavior. Execution combines offline planned motions
          and online behaviors till reaching the task goal. Failures of behaviors are fed back as
          constraints to find new plans. Forty real-robot trials and motivating demonstrations are
          performed to evaluate the proposed framework and compare against state-of-the-art. Results
          show faster execution time, less number of actions, and more success in problems where
          diverse gaps arise. The experiment data is shared for researchers to simulate these
          settings. The work shows promise in expanding the applicable class of realistic partially
          grounded problems that robots can address.}
    }
    

    Abstract

    ×

    Task and motion planning represents a powerful set of hybrid planning methods that combine reasoning over discrete task domains and continuous motion generation. Traditional reasoning necessitates task domain models and enough information to ground actions to motion planning queries. Gaps in this knowledge often arise from sources like occlusion or imprecise modeling. This work generates task and motion plans that include actions cannot be fully grounded at planning time. During execution, such an action is handled by a provided human-designed or learned closed-loop behavior. Execution combines offline planned motions and online behaviors till reaching the task goal. Failures of behaviors are fed back as constraints to find new plans. Forty real-robot trials and motivating demonstrations are performed to evaluate the proposed framework and compare against state-of-the-art. Results show faster execution time, less number of actions, and more success in problems where diverse gaps arise. The experiment data is shared for researchers to simulate these settings. The work shows promise in expanding the applicable class of realistic partially grounded problems that robots can address.
    Details