All research

Physiological signal processing · 2022

QUCoughScope: COVID-19 screening from cough and breath sounds

A web and mobile application that records cough and breath sounds and screens for COVID-19 with stacked deep learning models, for people with and without symptoms.

Qatar University · Qatar National Research Fund UREP28-144-3-046

Two parallel pipelines, one for coughing and one for breathing, each for people with symptoms, without symptoms and healthy people. Sounds are converted to spectrograms, split into five folds and augmented, passed to three CNN base learners, and combined by a logistic regression meta-learner that outputs COVID-19 or healthy.
Figure 1 from Rahman et al. (2022). The cough and breath pipelines: spectrogram conversion, five-fold splitting and augmentation, three CNN base learners and a logistic regression meta-learner.View full-size figureSource articleCC BY 4.0

Approach

Can cough and breath recordings made at home screen for COVID-19 in people with and without symptoms?

Many people with COVID-19 have no symptoms and are unlikely to be tested, so they can spread the virus without knowing it, while RT-PCR testing is invasive and depends on clinical resources. Earlier studies suggested that infection changes how people cough and breathe. QUCoughScope combined a public cough and breath dataset with recordings crowdsourced through its own application, trained stacked deep learning models on sound spectrograms, and deployed the best models behind an application that returns a screening result to the user.

  1. Collect cough and breath recordings
  2. Classify sound spectrograms
  3. Return a screening result in the app

Combining public and crowdsourced recordings

Diagnostics · 2022

The University of Cambridge COVID-19 Sounds dataset, shared on request, provided cough and breath recordings from 582 healthy people and 141 people with COVID-19. The team added recordings collected anonymously through the QUCoughScope application from 245 healthy people and 96 with COVID-19.

Participants reported their symptoms, so each group was divided into people with a cough and people without symptoms. This gave four experiments: cough or breath sounds, for people with or without symptoms.

Each recording was converted into a spectrogram with the short-time Fourier transform. The spectrogram images were augmented and evaluated with five-fold cross-validation, using 80% for training and 20% for testing in each fold.

Participants across both datasets
1,064
People with COVID-19
237
Experiments by sound and symptom group
4
Participants by dataset and group. From Table 1 of the paper.
Healthy, with coughHealthy, no symptomsCOVID-19, with coughCOVID-19, no symptoms
University of Cambridge2643185487
QUCoughScope application322131878

Stacking CNNs on sound spectrograms

Model development

Eight pretrained CNNs, including ResNet, DenseNet201, Inception-v3, MobileNetV2 and EfficientNet-B0 and B7, were trained on the spectrograms. For each experiment the three best became base learners, and a logistic regression meta-learner combined their predictions.

The stacked model outperformed every single CNN. Cough sounds separated people with COVID-19 from healthy people more reliably than breath sounds: accuracy reached 96.5% for people with symptoms and 98.85% for those without, compared with 91.03% and 80.01% from breath.

Accuracy from cough, no symptoms
98.85%
Accuracy from cough, with symptoms
96.5%
CNNs compared as base learners
8
Results of the stacked model, averaged over five folds (%). From the paper.
AccuracySensitivitySpecificity
Cough, with symptoms96.596.4295.47
Cough, no symptoms98.8597.0199.6
Breath, with symptoms91.0388.991.5
Breath, no symptoms80.0172.0482.67

Screening through a web and mobile application

Deployed prototype

The application, built with Flutter for the web, Android and iOS, asks for basic demographic details and symptoms, then records cough and breath sounds through the device microphone. Recordings are sent over HTTPS to a backend running on Google cloud computing, converted into spectrograms and classified by the pipeline that matches the reported symptoms.

The screening result is returned to the user and stored in a cloud database. The same application collected the new recordings used in the study. As a next step, the team planned a check so that only cough and breath sounds would be accepted.

Diagram of the application. A smartphone running QUCoughScope records cough and breath sounds, shown as an audio waveform. Audio and symptoms travel over the internet to a data storage and AI server, which sends a screening result back to the phone.
Figure 7 from Rahman et al. (2022). The QUCoughScope application: recordings and symptoms go to the AI server, which returns a screening result.View full-size figureSource articleCC BY 4.0

My contribution

I contributed to the investigation and methodology, including the technical analysis and validation of the audio classification pipeline, and to drafting the manuscript.

Study scope

Recordings came from volunteers who chose to take part, and COVID-19 status and symptoms were reported by participants. The crowdsourced group with symptoms was small, and performance was measured with five-fold cross-validation rather than on a separate prospective cohort. The application is intended as pre-screening to prioritise testing, not as a diagnostic test.