At Gamaya we develop solutions to improve agricultural practices with remote sensing data and advanced AI technology, delivering actionable results to large sugarcane and soybean growers. As a top-10 Swiss start-up, we are leading the way towards disruptive developments and smarter solutions. Here is how we have improved our orthomosaic analysis pipeline for faster and more cost-effective turnaround.

Aerial image processing and analysis workflow

Gamaya garners images of ag fields from two main sources- satellite and drones. In the case of drone-based analytics, users fly drones over their fields and then upload the gathered images onto the Gamaya platform. Algorithms and advanced AI automatically stitch them into one large map, which is called an orthomosaic.

‍

Once the orthomosaic is generated, advanced AI tech and machine learning-based models are applied to identify various issues in the fields (e.g. weeds, planting gaps, and more). This actionable and accurate information allows users to address the issues quickly and in an area-specific location, ultimately improving their yields and reducing their costs.

Satellite data is also used for field analysis. The primary difference is in spatial resolution. Drones are able to provide images with a pixel size in centimeters, whilst satellites are able to provide images in 3-10 meters of pixel size. This makes satellite images have a much lower level of detail, but unlike drones, they are able to survey a much larger field area.

The common approach when analyzing drone-based orthomosaics is to take the full field area and process it as a whole. This is what Gamaya was doing until 2020. However, the main problem with this approach is scalability. Whilst this approach worked well for smaller fields (5 to 20 ha), it required very powerful machines and took a lot of time for fields any larger than that. We faced this issue in late 2019, where processing a 75 ha field required upwards of 12 hours of processing time, and computers with more than 100 GB of memory. The sizes of fields in agriculture range anywhere between 1 ha to upwards of 1000 ha, so from a long-term perspective this approach was unsustainable.

Shifting to a smarter analytics pipeline

Similar to the MapReduce approach used frequently in the field of Computer Science, we decided to shift and totally change our approach when processing orthomosaic data by spliting large fields into smaller regions, run the analytics algorithms per region, and then combine the results into one single data layer.

As with all big changes, there were lots of concerns and unknowns. However, particularly from a longer-term perspective, we did see it as the right approach, as it allowed us to process the data of much bigger parcels in a much better, faster and easier way than prior.

It did require us to rebuild our processing pipeline from scratch. We moved away from having huge machines processing huge amounts of data from big fields to an architecture where many small jobs run simultaneously on much smaller machines.

‍

It also required us to change the way we train and run our AI analytics. Rather than training and testing them to be able to cope with fields varying in size and shape, we established a standard, fixed-size square region that all models use.

We also took this as an opportunity to completely revise our programming tools and our cloud provider. Some examples include: 

• Switching to Google Cloud for computing and data storage (learn more about our work with Google in this article).
• All our backend code is now in Go Programming language (previously in Java and NodeJS).
• We switched our database from SQL to MongoDB
• Frontend is now developed with the help of VueJS rather than Angular
• We now have an architecture system similar to microservices ones, instead of the previous monolithic application.

We continued to run and support the prior system for a few months after the change, but increasingly large parcel sizes and the rapid growth of Gamaya showed us that this new architecture is the best way forward. This marked the definitive end of the old system.

As of today, this architectural change has allowed us to:

• Analyze and process data from fields of 500 ha in under 30 minutes (beforehand, it took 12h for 75 ha, with processing time increasing with field size)
• Use small computers with 4-8GB of memory (beforehand, a 75 ha field would require 100 GB of memory)
• Reduce the cost of running analytics by 10x
• Significantly speed up the iteration cycle for our data science team. Beforehand, they sometimes had to wait a day to get results of their new model on a field, and now they are able to receive results in 30 min. They can also run 50 experiments a day instead of just one. 
• Significantly simplify our code base. Recently, when a new software engineer joined our team, we were delighted to see that he was able to start coding in our system within his first week as everything was very clear and well structured.

Gamaya is now applying a very similar approach and restructuring to our satellite analytics pipeline and expects to see very similar successes in the near future.

‍In a fast-paced tech environment, continuous innovation is key

This processing pipeline shift has been a tremendous technical leap forward for Gamaya. Yet we must also attribute this success to our open-minded and ingenious team, who are helping Gamaya offer disruptive digital agronomy solutions to revolutionize the methods and lives of farmers around the world- ultimately helping feed our rapidly growing world population and making it more sustainable for the future.

‍

Get faster sugarcane field intelligence with CaneFit

‍

Publication date: August 2021