ROS 2 and EGM-driven sketching pipeline for ABB collaborative robots

Abstract

This paper presents an open-source, end-to-end pipeline that automates robot sketching on an ABB CRB 15000-5 “GoFa” cobot from user-supplied images. The software integrates ROS 2 with ABB’s Externally Guided Motion (EGM) interface and features a graphical Python application for image acquisition, edge processing (via classical Canny edge detection with OpenCV, local AI models, or cloud APIs), vectorization, trajectory generation, and real-time execution preview. We provide an extensive empirical evaluation comparing Deep Reinforcement Learning (DQN, PPO, SAC) against a deterministic geometric approach. Results demonstrate that while discrete DQN achieves a high in-distribution similarity (Jaccard index of 0.601), it degrades significantly out-of-distribution. In contrast, the geometric method generalizes reliably across unseen targets and reaches a 149-times reduction in total sketch processing time (21.8 ms versus 3248 ms). The key contributions are an open-source ROS 2 driver interfacing EGM, a user-friendly GUI workflow, and a comparative benchmark of RL versus geometric trajectory generation for robot sketching.

Publication
In 2026 International Conference on Robotic Systems and Applications (ICRSA)
Accepted for IEEE ICRSA 2026 (Huizhou, China, October 2026). The accepted manuscript is available as a PDF on this page; the DOI and official proceedings link will be added when they become available.
Qiang Li
Qiang Li
Professor and header of AG

My research interests include underwater robot, collaborative robots, humanoid robots.