Comparison of Segmentation and Detection in the First Robotics Competition

Abstract

Advancements in computer vision have transformed robotic perception, with segmentation models offering unprecedented precision over traditional object detection methods. This paper explores the application of Meta AI’s Segment Anything Model (SAM) in the context of the 2024 FIRST Robotics Competition (FRC) game, CRESCENDO. SAM delivers high-resolution, pixel-level object masks with minimal input, operating in a zero-shot manner that eliminates the need for extensive retraining. In robotics domains characterized by clutter and occlusion—such as competitive FRC gameplay—segmentation models like SAM and RISE have proven highly effective, especially when labeled data is scarce or conditions evolve over time. We compare SAM’s segmentation capabilities with conventional detection models, highlighting its advantages in spatial awareness, contextual understanding, and engineering simplicity. Real-world FRC examples demonstrate how segmentation-based systems enhance localization, alignment, and obstacle avoidance. We propose a vision framework that fuses SAM's segmentation with sensor-based data to improve reliability and strategic autonomy, presenting SAM as a scalable and adaptable vision solution in dynamic robotics environments.