At the forefront of digital technology, Google has developed an innovative bioacoustic model—HeAR, which can detect diseases by analyzing sounds. This technology undoubtedly opens up new horizons for remote medical diagnosis.
The capabilities of the HeAR model stem from its training process, which utilized 313 million audio clips from YouTube, including approximately 100 million cough sounds. Through this data, HeAR has learned to identify disease signals from coughs, speech, and breathing sounds.
Video courtesy of the official channel, translation: Xiaohu
The applications of the HeAR model are impressive, capable of detecting a range of diseases including tuberculosis, COVID-19, and chronic obstructive pulmonary disease (COPD). This capability has been validated in 13 health acoustic event detection tasks, 14 cough inference tasks, and 6 lung function inference tasks, outperforming existing models in many tasks.
In the cough inference tasks, HeAR achieved the best performance in 10 tasks, especially in detecting COVID-19 and tuberculosis. In lung function inference tasks, HeAR also excelled, particularly in critical indicators such as forced expiratory volume (FEV1) and forced vital capacity (FVC).
India's Salcit Technologies has already applied the HeAR model to its product Swaasa®, which assesses lung health by analyzing cough sounds, showing great potential in early detection of tuberculosis.
Salcit Technologies is exploring how to leverage HeAR to expand the capabilities of its bioacoustic AI models. Swaasa® is researching and enhancing early tuberculosis detection based on cough sounds, which not only aids in improving the accuracy of disease diagnosis but also makes medical services more accessible.