Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels
Computer Vision and Pattern Recognition
Summary
The authors created two new datasets featuring images and videos of tomato plants grown in settings similar to real farms, captured using a robot. One dataset (BUTom21) contains still pictures with manual labels identifying the ripeness of tomatoes, while the other (BUTom-ST21) is video-based and uses AI to help label the tomatoes automatically. Both datasets include detailed pixel-level information about how ripe each fruit is. The goal is to help researchers develop better ways to monitor tomato plants and measure fruit ripeness in natural growing conditions, especially using counting and timing information from the videos.
Authors
Michael Halstead, Esra Guclu, Mohamed Farag, Enrico Pallotta, Christian Hund, Ribana Roscher, Maren Bennewitz, Juergen Gall, Cyrill Stachniss, Chris McCool
Abstract
In this manuscript we release two datasets for visual sensing of tomato plants grown in commercial-like settings and acquired using a robot. The first is BUTom21 which consists of still images and manual annotations. The second is BUTom-ST21 which consists of video-based data and semi-automated annotations through AI-based methods, referred to as pseudo-labels. In both cases, we provide pixel-level labels for the ripeness of the fruit. The aim is to provide the research community a challenging set of real-world imagery to explore methods to sense and estimate the state of tomato plants and their fruit, which is an important horticultural crop. Importantly, the spatial-temporal dataset provides individual fruit count and ripeness information enabling researchers to push the boundaries of field-based phenotyping.