RoboKudo is a perception framework targeted for robot manipulation tasks. It is a multi-expert approach to analyze unstructured sensor data and annotate particular regions of the data given the expertise of specific computer vision algorithms.
Interactive Actions and/or Examples
Description
RoboKudo, an open-source perception framework for mobile manipulation systems, allows to flexibly generate and execute task-specific perception processes that combine multiple vision methods. The framework is based on a novel concept that we call Unstructured Information Management (UIM) on Behavior Trees (BTs), short UIMoBT, which is a mechanism to analyze unstructured data with non-linear process flows. The technical realization is done with a datastructure called Perception Pipeline Tree (PPT), which is an extension of Behavior Trees with a focus on robot perception. RoboKudo is developed to be included in a perception-action loop of a robot system. The system can state perception tasks to RoboKudo via a query-answering interface. The interface translates a perception task query such as “find a milk box in the fridge” into a specialized perception process, represented as a PPT, which contains a combination of suitable computer vision methods to fulfill the given task.
Overview video
Our overview video presents the key ideas of the RoboKudo framework and highlights demonstrations and experiments implemented in RoboKudo. The video contains sound for narration.
Publications
- P. Mania; F. K. Kenfack; M. Neumann; M. Beetz, “Imagination-enabled robot perception.” 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021.
- P. Mania; S. Stelter; G. Kazhoyan; M. Beetz, “An Open and Flexible Robot Perception Framework for Mobile Manipulation Tasks” 2024 IEEE International Conference on Robotics and Automation(ICRA). IEEE, 2024. Submitted for review.