Summary
Growing lettuce in controlled environments needs smart computer programs to decide how to care for the plants, but these programs require many pictures of lettuce labeled with growth details. The authors created LettuceVisSim, a tool that simulates how lettuce grows and produces series of images showing this growth, complete with labels. Their tool combines a growth model, a layout method to place leaves realistically, and image generation using a game engine. Tests show it accurately mimics real growth patterns and creates images fast enough for training artificial intelligence. This makes it possible to train vision-based AI to manage lettuce farming without needing tons of real photos.
What this means in practice
- •For indoor farming engineers: Train AI systems to manage lettuce growth by using simulated time-series images of plants under varying environmental conditions.
- •For computer vision developers: Develop and test vision algorithms for plant growth monitoring with accurately labeled synthetic images without expensive real-world data collection.
Authors
Ziye Zhu, Bert van 't Ooster, Congcong Sun, Eldert van Henten, Sjoerd Boersma
Abstract
Vision-based reinforcement learning holds strong potential for decision-making in controlled environment agriculture (CEA). However, its development is hindered by the scarcity of labelled crop images. To address this gap, LettuceVisSim, a lettuce growth simulator that generates labelled time series of crop images, was developed and validated. The simulator contains a process-based model (PBM) for shoot dry weight dynamics, a canopy layout algorithm for deriving canopy layout representations from shoot dry weight, and a Unity rendering engine for image generation. Five findings support the simulator. First, the PBM reproduced shoot dry weight under dynamic plant-density management with $\mathrm{R}^{2}=0.84$. Second, a piecewise cubic regression mapped shoot dry weight to potential projected area with $\mathrm{R}^{2}=0.94$. Third, the canopy layout representation was validated using 12 experimental datasets each having different dynamic environmental and spacing conditions. It reproduced the ground coverage ratio dynamics observed in measured images, achieving $\mathrm{R}^{2}=0.84$ when driven by measured shoot dry weight and $\mathrm{R}^{2}=0.40$ (0.76 excluding one outlier) when driven by PBM-simulated values. Fourth, the Unity rendering engine converted canopy layout representations into RGB and segmentation images at less than 10~ms. Fifth, a demonstration showed that a lighting-control policy can be learned and applied by observing only crop images that were generated with LettuceVisSim, providing a proof of concept of vision-based reinforcement learning in CEA using LettuceVisSim.