Cognitive Systems Table Tennis Robot

Jonas Tebbe, Yapeng Gao

The chair is equipped with a KUKA Agilus Robot with 6 degrees of freedom. Motivated by the KUKA comercial ( we are teaching the robot to play table tennis.

A high-speed vision system was set up to identify a table tennis ball in the scene and infer its 3D position. To find the ball in an image we use three filters which take the balls movement, color, and shape into account. Its 3D position is estimated by triangulation using the balls pixel coordinates in both camera frames. To speedup the image processing, we restrict to a small region of interest if a ball is already found in the previous frame. This is schematically shown in above figure. Based on the position information, trajectory state is estimated by an extended Kalman filter and predicted into the future using a standard airodynamic force model.

The spin influences the trajectory of the ball via the Magnus force. Thus, it is beneficial to consider spin information in order to improve the trajecory prediction. A high speed monocular vision system tracks the spin of the flying table tennis ball. It uses the brand logo imprinted on the ball. The brand vectors are fitted into a plane using RANSAC. The rotation axis corresponds to the normal vector of this plane. The angular speed is calculated by using the angle between two adjacent vectors divided by the elapsed time.

The robot is controlled by the KR-C4 controller using KRL programming language. We can establish a connection to the controller PC via the network using the KUKA EthernetKRL package. We use approximated motions which allow for the trajectory of the robot to change before it arrives to the initial target. Thus we are able to send an initial guess of the hitting point ahead of time and then send corrections with more precise information of the hitting point as we get to see more of the actual trajectory of the ball.


[1] Jonas Tebbe, Lukas Klamt, Yapeng Gao and Andreas Zell. Spin Detection in Robotic Table Tennis. In IEEE International Conference on Robotics and Automation (ICRA), Paris, France, May 2020. (Accepted for publication).
[2] Yapeng Gao, Jonas Tebbe, Julian Krismer, and Andreas Zell. Markerless Racket Pose Detection and Stroke Classification based on Stereo Vision for Table Tennis Robots. In IEEE International Conference on Robotic Computing (IRC), Naples, Italy, February 2019.
[3] Jonas Tebbe, Yapeng Gao, Marc Sastre-Rienitz, and Andreas Zell. A Table Tennis Robot System using an industrial KUKA Robot Arm. In Pattern Recognition. GCPR 2018.