Kognitive Systeme

CS::APEX

Sebastian Buck, Richard Hanten

CS::APEX (Algorithm Prototyper and EXperimentor for Cognitive Systems) is a framework based on synchronous dataflow and event-based message passing that aims to speed up prototyping of new robotic algorithms using visual programming aspects. The framework provides different features to allow users to prototype new algorithms more quickly. Calculations are represented by a nodes in a directed graph.

Manipulation of this execution graph is possible through a simple graphical user interface that allows spawning and deleting nodes, adding and removing edges and arranging those elements visually. To speed up the prototyping process, other features like undo/redo mechanisms and profiling are implemented as well. Finally there exists an easy to use parameter system that generates UI controls for each block and allows parameter tuning and optimization.

The framework itself does not provide any predefined computation nodes and does not depend on specific message definitions or node types. These details are instead implemented in distinct plugin libraries that extend the functionality of the whole system. This way maximal independence can be achieved, while new modules can easily be added to introduce nodes and message types.

Video 1 shows an overview of the features of cs::APEX.

Features Video

Video 1: Exemplary workflows and features of cs::APEX

Externer Inhalt

Hier wäre eigentlich ein Video zu sehen. Damit Sie diesen Inhalt (Quelle: www.xyz.de) sehen können, klicken Sie bitte auf "Akzeptieren". Wir möchten Sie darauf hinweisen, dass durch die Anzeige dieses Videos Daten an Dritte übertragen oder Cookies gespeichert werden könnten.

Weitere Informationen finden Sie in unserer Datenschutzerklärung.

Downloads

CS::APEX is open-source with a BSD license.

Software page

Features

Modularity

  • Nodes in the execution graph are grouped into modules, which are mostly independent. Any subset of needed modules can be used. Other modules can be omitted.
  • Nodes are completely encapsulated and communicate only via message passing.
  • Separation of concerns: Different areas of perception and cognition are developed in isolation, yet modules can interact via the computation graph.

Extensibility

  • The system is based on plug-ins. By creating a custom package and exporting a plug-in library, the user can easily add new types of computation nodes and messages.
  • Nodes are only dependent on messages and not on other nodes. This way code can easily be reused.
  • Adding a node only requires the implementation of two to three functions in a new class.
  • Nodes are implemented in C++11 and can therefore be linked against any other library.

Rapid Prototyping

  • A graphical user interface allows the user to do a variety of different tasks without the need for programming and compilation.
  • Nodes can be profiled, which can reveal bottlenecks in the processing pipeline.
  • The effects of parameter changes on the behaviour of the system can easily be experimented with graphically.
  • Many common tasks can be achieved without programming or compiling.

Supported Libraries

CS::APEX is targeted toward use in robotics. We fully support the ROS framework and use catkin as the build system for the framework. Supported ROS functionality includes:

  • Importing messages from ROS topics with support for time stamp based lookup
  • Direct import of Direct import of ROS bag files and automatic wrapping of ROS messages for direct compatibility
  • Optional: Conversion between ROS and APEX message
  • ROS parameter server and roslaunch compatibility

This framework is mainly used to model perception tasks, which consist if perceiving data from the environment and performing computations with them. We provide plug-ins that make many image processing operations from OpenCV available. Supported OpenCV functionality includes:

  • Messages for images based on cv::Mat
  • Many nodes that wrap existing cv functions
  • Wrappers around more abstract cv constructs, for example FeatureExtractors
  • Conversion of many message types to cv::Mat

Besides image processing, another very important type of information is 3D point clouds. We also provide support for point cloud processing via the PCL:

  • Wrapping nodes for many commonly used PCL functions
  • Point cloud visualization and conversion to images
  • Conversion between point cloud generation and raw data
  • Support for many point types, among others pcl::PointXYZ, pcl::PointXYZI and pcl::PointXYZRGB

Since parameters are modelled as first-class objects, we can change their values through the dataflow. This enables parameter optimization by changing parameters and observing the effects. We provide a package to connect cs::APEX with the Java-based EvA2 in order to optimize parameters using differential evolution:

  • Communication over TCP/IP via jcppsocket
  • Selection of parameters to optimize via the GUI, no configuration file editing
  • Minimal programming effort, only a node to calculate the fitness of the current parameters is necessary

Publications

[1] Sebastian Buck and Andreas Zell. Cs::apex: A framework for algorithm prototyping and experimentation with robotic systems. Journal of Intelligent & Robotic Systems, Apr 2018. [ DOI | details | link ]
[2] Sebastian Buck, Richard Hanten, Karsten Bohlmann, and Andreas Zell. Multi-sensor payload detection and acquisition for truck-trailer agvs. In Robotics and Automation (ICRA), 2017 IEEE International Conference on, Singapore, 2017. [ details ]
[3] Sebastian Buck, Richard Hanten, Karsten Bohlmann, and Andreas Zell. Generic 3d obstacle detection for agvs using time-of-flight cameras. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pages 4119 - 4124, Daejeon, Korea, October 2016. [ DOI | details ]
[4] Sebastian Buck, Richard Hanten, C. Robert Pech, and Andreas Zell. Synchronous dataflow and visual programming for prototyping robotic algorithms. In Intelligent Autonomous Systems (IAS), The 14th International Conference on, pages 911-923, Shanghai, CN, July 2016. [ DOI | details | pdf ]
[5] Sebastian Buck, Richard Hanten, Goran Huskić, Gerald Rauscher, Alina Kloss, Jan Leininger, Eugen Ruff, Felix Widmaier, and Andreas Zell. Conclusions from an object-delivery robotic competition: Sick robot day 2014. In Advanced Robotics (ICAR), The 17th International Conference on, pages 137-143, Istanbul, TR, July 2015. [ DOI | details | link | pdf ]