Robust Real-time 3D Person Detection for Indoor and Outdoor Applications

Richard Hanten, Sebastian Buck, Philipp Kuhlmann, Sebastian Otte

Person detection is one of the most crucial tasks in mobile robotics involving human interaction. The transportation of supply containers in populated areas or a mobile robot companion for runners [2] are only two of many application scenarios which require the ability to perceive human beings.

The person detection approach we present [1] is successfully used in different projects at our research chair. Our approach is implemented with a pipeline of different computation steps which are shown above. The pipeline is implemented using a synchronous data flow graph. This graph was configured and is executed by the CS::APEX framework, the Cognitive Science Algorithm Prototyper and Experimentor [4].

We base our approach on a generic 3D obstacle detection approach [3] which can be applied even if there is no flat floor in front of the robot.

Furthermore, the approach is able to deal with noisy sensor data and can be applied to different types of 3D sensors, including RGB-D cameras, time-of-flight cameras and stereo cameras.

After 3D data analysis, our approach uses visual feature extraction and classification to determine whether found obstacles are persons or not. Different state-of-the-art feature types and machine learning approaches can be used with our pipeline. Currently we support histograms of oriented gradients (HOG) and aggregate channel features (ACF). At the moment, random decision forests, support vector machines and different neural network implementations are the algorithms, which can be used for classification. The image below gives an illustration of the different computation steps, beginning with the input point cloud, the obstacle detection, voxel clustering and the final regions of interest for feature extraction.

All components of the person detection approach can run on a single cpu core with no necessity for GPU computation or parallelization.


Externer Inhalt

Hier wäre eigentlich ein Video zu sehen. Damit Sie diesen Inhalt (Quelle: sehen können, klicken Sie bitte auf "Akzeptieren". Wir möchten Sie darauf hinweisen, dass durch die Anzeige dieses Videos Daten an Dritte übertragen oder Cookies gespeichert werden könnten.

Weitere Informationen finden Sie in unserer Datenschutzerklärung.

Getting the software

More information about CS::APEX can be found at Generally our software is available via To set up the person detection pipeline in CS::APEX, please use the configurations supplied alongside the multi-sensor datasets.


To test our person detection on different sensors, we recorded a multi-sensor dataset with different sensors simultaneously. The multi-sensor datasets can be found here...


[1] Richard Hanten, Philipp Kuhlmann, Sebastian Otte, and Andreas Zell. Robust real-time 3d person detection for indoor and outdoor applications. In IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, May 2018. (Accepted for publication). [ details ]
[2] Goran Huskić, Sebastian Buck, and Andreas Zell. Path following control of skid-steered wheeled mobile robots at higher speeds on different terrain types. In IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017. [ details ]
[3] Sebastian Buck, Richard Hanten, Karsten Bohlmann, and Andreas Zell. Generic 3d obstacle detection for agvs using time-of-flight cameras. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pages 4119 - 4124, Daejeon, Korea, October 2016. [ DOI | details ]
[4] Sebastian Buck, Richard Hanten, C. Robert Pech, and Andreas Zell. Synchronous dataflow and visual programming for prototyping robotic algorithms. In Intelligent Autonomous Systems (IAS), The 14th International Conference on, pages 911-923, Shanghai, CN, July 2016. [ DOI | details | pdf ]