Uploading large volumes of data and training machine learning models requires a lot of processing power, which can be costly. As a growing startup, Taranis did not want to invest millions of pounds on infrastructure, but still needed to handle large volumes of data. “Each of our drone flights collects around 10,000 images, and each image is between 10MB and 20MB,” says Eli. “We looked for a way of getting those images into our system as quickly as possible, as well as improving our machine learning training performance.”

To do that, Taranis migrated to GCP. “The Google data centers in South America, Australia, and Europe offer really fast connectivity, and can handle the large volumes of drone images we upload. We have a throughput of around 30TB in total,” says Eli. “For processing the images once they are uploaded, we use V100 GPUs on Compute Engine. We have the capacity to easily scale from 1,000 to 4,000 V100s, and the system scales automatically when new images arrive. Information is deduced from the images, which is then uploaded to a Cloud SQL database before being served to our customers. We also have an image processing pipeline on Kubernetes Engine for our satellite images, in addition to using Cloud Functions and Cloud Pub/Sub.”

Flexibility and scalability are key to the Taranis image serving pipeline. “Agriculture is a seasonal business, so we have certain months of peak activity followed by quiet months, and we also have peaks throughout the day,” says Eli. “During quiet times, we can scale back all our high-level Compute Engine GPU resources automatically so we don’t have to prepare our system in advance.”

Preventing Crop Loss with AI

“For our machine learning model training pipeline, we use TensorFlow,” Eli explains. “Choosing TensorFlow has helped us develop our models rapidly, as there is a lot of support available through the open source community. To develop our models, we use tens of millions of photographs that we have collected over the past year and a half, which have been analyzed and tagged. Each photo might have up to a thousand items of interest, such as insect damage or leaf discoloration, so the data volumes are really significant. In total, we have processed around 100 million distinct features in around 700,000 images.”

The insights provided through Taranis’s dashboards give farmers the information they need to intervene early and prevent crop loss. “We enable farmers to target problems with concrete solutions, like adding fertilizer in a particular area that is low in nutrients. We are also looking to integrate our platform with autonomous farm machinery so a farmer can deploy the solution in a few clicks.”