What's the Most Innovative Thing You've Done?

I was recently asked “what is the most innovative thing you’ve ever done?”. My one line answer is “Built a robot to help solve beehive colony collapse”. 

To thoughtfully answer, first I asked myself “what’s innovation, really?” From a customer’s perspective, innovation is the nifty way a product does a valuable job. Customers experience innovation through the outcomes actually realized by products. Everything else in product development is research, learning, or marketing - including that less useful robot, which evolved into an actual product that is useful to customers!

The most innovative thing I have done recently is create Eyesonhives, a smart camera which enables a customer to see, track and understand the health of a beehive. Eyesonhives eliminates the problem of beekeepers not knowing why and when a colony is in trouble or collapses, by monitoring bee hive activity with data and video over time.

Situation:

Bees pollinate one in three bites of food we eat, around $15B in US agricultural output yearly. Since 2006, bee colonies have been dying off at unsustainable rates. In 2018 40% of bee colonies in the US perished, and beekeepers continue to struggle to raise replacement colonies in time for the next pollination season. Beekeepers often lack visibility into what happened between a seemingly healthy colony at inspection, and a ‘dead out’ several weeks later.  

Tasks:

The problem Eyesonhives aims to solve is how to give beekeepers and researchers a clear and timely understanding of what’s going on with a beehive. 

The tasks identified to achieve this outcome include:

• Capture useful data from the beehive, which actually indicates hive health
• Enable easy interpretation of the data for researcher, grower and beekeeper users
• Facilitate remote access to beehive data, often from agricultural settings 

Actions: 

To get started I pitched the importance of the problem, and a basic solution approach, first to my friend Scott Ross and then at a Startup Weekend. The solution approach was fun in itself. 

• Build a low-cost mobile robot to drive between beehives
• Design a computer vision algorithm to measure bee flight activity
• Report the data up to a webserver accessible through a webapp.   

With a motivated team brought together, we iteratively tackled the tasks.

To capture useful data relating to bee health, I talked with experienced beekeepers, gained my own experience beekeeping and started with a simple intuitive metric - “how many bees were seen flying in a sample time period?” A ‘strong hive’ has lots of bees.  

• I set up ‘bucket bot’ - a webcam attached to a laptop in a home-depot bucket, pointed at a beehive. This made it possible to get data faster than working on weather proofing a mobile rover. It was ultimately quite challenging to develop a simple enough algorithm to track bee activity, and the computer vision algorithm I came up with was novel enough to apply for, and be granted a patent by the USPTO!

To enable easy interpretation of the data for researchers, growers and beekeepers, my teammate Jon Simpson created a Ruby on Rails webapp we call the Eyesonhives Analytics Platform.  

• Jon leveraged Highcharts to graph the bee activity metric, and display the actual video behind the bee activity algorithm. This makes understanding the data much easier. It’s easy for a researcher, grower or beekeeper to validate the data by playing the video back. The video is like standing in front of the beehive!

To facilitate Remote Access, we recognized that commercial beehives are often in agricultural settings, and backyard beehives may have spotty home WiFi connections. It would not be ideal to require a direct connection to the devices, and the devices should work reliably outdoors, ideally for years!

• We created an architecture which enables the devices to reach a remote server and post up their data. In the interests of speed and cost, I built a basic home server and data center in my garage, and Jon implemented a framework on the device side to upload data to the Rails App via a web API called from Python scripts.  
• I ported my Matlab code to run in Python/OpenCV on an embedded ARM processor (Raspberry Pi) and designed a prototype enclosure.  
• My teammate Nicholas Cunningham improved the mechanical design, and iterated on the 3D printed internal parts. Nick created a much easier product to manufacture, which works reliably in the field to capture that useful data. 
• As we put devices into the field, I improved the device side software with a local sqlite database which would store processed video data. This enabled a ‘store and forward’ mode if a link to the remote server was not available. I also created a remote link to each of the devices so that over the air updates and remote troubleshooting would be easy.

So what was the innovation?  

All of the above. The point is the team and I were motivated to innovate in these areas so that our customers would be able to achieve their goals! By themselves, none of the actions and implementation are ‘the innovation’, instead the system that solves the customers problem is the innovation.

Results

The first phase of the project achieved the initial results we were aiming for.

• We enabled beekeepers to understand the health of a hive, including using the data to proactively intervene and rescue beehives in trouble.
• We gathered an 8TB dataset of bee videos and developed a baseline of bee behaviors with over 26M datapoints in our primary PostgreSQL database.
• We tested product-market fit for backyard beekeepers, commercial beekeepers, and researcher market segments, and built relationships with customers from leading academic institutions such as UC Davis and La Trobe University.

Recommended next steps

The 3 key ideas for Eyesonhives are useful data, remote access and easy interpretation. This is true now, and it will be true 10 years from now. Our product strategy for Eyesonhives is checked against these guiding principles for the product. In the immediate term, the product roadmap is to:

1. Enhance remote access by evaluating migration from the data center built in my garage, to a highly available service such as AWS or Google Cloud Platform.
2. Continue to improve the UX of the web app, publish articles, and how-to content to enhance Easy Interpretation.
3. Continue to invest in Computer Vision and Machine Learning algorithms to mine the video dataset for Useful Data to present to users.

Thank you for reading! I hope you enjoyed an introduction to the Eyesonhives project. I’m truly grateful and honored for the experience and the opportunity to work with the team, and mentors supporting us, that this project cultivated! 

If you’d like to learn more, check out my TEDx talk on Eyesonhives “The Heartbeat of a Beehive”, and the website https://info.eyesonhives.com/