Classification of Acoustic Data with Transformer Model

Abstract

Bees are essential to global ecosystems, particularly for pollinating crops, yet in recent years their populations have faced significant decline. One critical aspect of bee colony health is the ability to detect negative in-hive events such as a queen leaving the hive. Traditionally, beekeepers rely on manual inspections to assess hive conditions, a labor-intensive and time-consuming process. However, recent advances in machine learning offer new approaches to automating this task. Since 2016, there have been attempts to classify bee sounds using machine learning, employing the power of different machine learning methods, including deep learning architectures.
In this research, we explore the use of acoustic labeled data for in-hive event classification, focusing specifically on detecting when a queen leaves the hive. We utilize 12-hour recordings from different locations, with the data preprocessed and transformed to be suitable for input into a transformer-based neural network. Our goal is to demonstrate that transformer models yield superior results in this task compared to previous approaches. The study is organized into several key sections: we first highlight the ecological importance of bees, followed by a literature review on the state of bee sound classification research. We then delve into the data preparation process, model design, and present our findings. Our results underscore the potential of transformer models in automating hive monitoring, offering a scalable solution for beekeepers to protect and preserve bee populations.