SEMI-SUPERVISED AUDIO REPRESENTATION LEARN-ING FOR MODELING BEEHIVE STRENGTHS

Abstract

Honey bees are critical to our ecosystem and food security as a pollinator, contributing 35% of our global agriculture yield (Klein et al., 2007). In spite of their importance, beekeeping is exclusively dependent on human labor and experiencederived heuristics, while requiring frequent human checkups to ensure the colony is healthy, which can disrupt the colony. Increasingly, pollinator populations are declining due to threats from climate change, pests, environmental toxicity, making their management even more critical than ever before in order to ensure sustained global food security. To start addressing this pressing challenge, we developed an integrated hardware sensing system for beehive monitoring through audio and environment measurements, and a hierarchical semi-supervised deep learning model, composed of an audio modeling module and a predictor, to model the strength of beehives. The model is trained jointly on audio reconstruction and prediction losses based on human inspections, in order to model both low-level audio features and circadian temporal dynamics. We show that this model performs well despite limited labels, and can learn an audio embedding that is useful for characterizing different sound profiles of beehives. This is the first instance to our knowledge of applying audio-based deep learning to model beehives and population size in an observational setting across a large number of hives.

1. INTRODUCTION

Pollinators are one of the most fundamental parts of crop production worldwide (Klein et al., 2007) . Without honey bee pollinators, there would be a substantial decrease in both the diversity and yield of our crops, which includes most common produce (van der Sluijs & Vaage, 2016) . As a model organism, bees are also often studied through controlled behavioral experiments, as they exhibit complex responses to many environmental factors, many of which are yet to be fully understood. A colony of bees coordinate its efforts to maintain the overall health, with different types of bees tasked for various purposes. One of the signature modality of characterizing bee behavior is through the buzzing frequencies emitted through the vibration of the wings, which can correlate with various properties of the surroundings, including temperature, potentially allowing for a descriptive 'image' of the hive in terms of strength (Howard et al., 2013; Ruttner, 1988) . However, despite what is known about honey bees behavior and their importance in agriculture and natural diversity, there remains a substantial gap between controlled academic studies and the field practices carried out (López-Uribe & Simone-Finstrom, 2019). In particular, beekeepers use their long-tenured experience to derive heuristics for maintaining colonies, which necessitates frequent visual inspections of each frame of every box, many of which making up a single hive. During each inspection, beekeepers visually examine each frame and note any deformities, changes in colony size, amount of stored food, and amount of brood maintained by the bees. This process is labor intensive, limiting the number of hives that can be managed effectively. As growing risk factors make human inspection more difficult at scale, computational methods are needed in tracking changing hive dynamics on a faster timescale and allowing for scalable management. With modern sensing hardware that can record data for months and scalable modeling with state-of-the-art tools in machine learning, we can potentially start tackling some of challenges facing the management of our pollinators, a key player in ensuring food security for the future.

