BirdNET Lite

README.md

@@ -1,2 +1,88 @@
 # BirdNET-Lite
 TFLite version of BirdNET. Bird sound recognition for more than 6,000 species worldwide.
+
+Center for Conservation Bioacoustics, Cornell Lab of Ornithology, Cornell University
+
+Go to https://birdnet.cornell.edu to learn more about the project.
+
+Want to use BirdNET to analyze a large dataset? Don't hesitate to contact us: ccb-birdnet@cornell.edu
+
+# Setup (Ubuntu 18.04)
+
+TFLite for x86 platforms comes with the standard Tensorflow package. If you are on a different platform, you need to install a dedicated version of TFLite (e.g., a pre-compiled version for Raspberry Pi).
+
+We need to setup TF2.3+ for BirdNET. First, we install Python 3 and pip:
+
+```
+sudo apt-get update
+sudo apt-get install python3-dev python3-pip
+sudo pip3 install --upgrade pip
+```
+
+Then, we can install Tensorflow with:
+
+```
+sudo pip3 install tensorflow
+```
+
+TFLite on x86 platform currently only supports CPUs. 
+
+Note: Make sure to set `CUDA_VISIBLE_DEVICES=""` in your environment variables. Or set `os.environ['CUDA_VISIBLE_DEVICES'] = ''` at the top of your Python script.
+
+In this example, we use Librosa to open audio files. Install Librosa with:
+
+```
+sudo pip3 install librosa
+sudo apt-get install ffmpeg
+```
+
+You can use any other audio lib if you like, or pass raw audio signals to the model.
+
+Note: BirdNET expects 3-second chunks of raw audio data, sampled at 48 kHz.
+
+# Usage
+
+You can run BirdNET via the command line. You can add a few parameters that affect the output.
+
+The input parameters include:
+
+```
+--i, Path to input file.
+--o, Path to output file. Defaults to result.csv.
+--lat, Recording location latitude. Set -1 to ignore.
+--lon, Recording location longitude. Set -1 to ignore.
+--week, Week of the year when the recording was made. Values in [1, 48] (4 weeks per month). Set -1 to ignore.
+--overlap, Overlap in seconds between extracted spectrograms. Values in [0.0, 2.9]. Defaults tp 0.0.
+--sensitivity, Detection sensitivity; Higher values result in higher sensitivity. Values in [0.5, 1.5]. Defaults to 1.0.
+--min_conf, Minimum confidence threshold. Values in [0.01, 0.99]. Defaults to 0.1.
+--custom_list, Path to text file containing a list of species. Not used if not provided.
+```
+
+Note: A custom species list needs to contain one species label per line. Take a look at the `model/label.txt` for the correct species label. Only labels from this text file are valid. You can find an example of a valid custom list in the 'example' folder.
+
+Here are two example commands to run this BirdNET version:
+
+```
+
+python3 analyze.py --i 'example/XC558716 - Soundscape.mp3' --lat 35.4244 --lon -120.7463 --week 18
+
+python3 analyze.py --i 'example/XC563936 - Soundscape.mp3' --lat 47.6766 --lon -122.294 --week 11 --overlap 1.5 --min_conf 0.25 --sensitivity 1.25 --custom_list 'example/custom_species_list.txt'
+
+```
+
+Note: Please make sure to provide lat, lon, and week. BirdNET will work without these values, but the results might be less reliable.
+
+The results of the anlysis will be stored in a result file in CSV format. All confidence values are raw prediction scores and should be post-processed to eliminate occasional false-positive results.
+
+# Contact us
+
+Please don't hesitate to contact us if you have any issues with the code or if you have any other remarks or questions.
+
+Our e-mail address: ccb-birdnet@cornell.edu
+
+We are always open for a collaboration with you.
+
+# Funding
+
+This project is supported by Jake Holshuh (Cornell class of ’69). The Arthur Vining Davis Foundations also kindly support our efforts.
+

analyze.py

@@ -0,0 +1,228 @@
+
+import os
+os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
+os.environ['CUDA_VISIBLE_DEVICES'] = ''
+
+try:
+    import tflite_runtime.interpreter as tflite
+except:
+    from tensorflow import lite as tflite
+
+import argparse
+import operator
+import librosa
+import numpy as np
+import math
+import time
+
+def loadModel():
+
+    global INPUT_LAYER_INDEX
+    global OUTPUT_LAYER_INDEX
+    global MDATA_INPUT_INDEX
+    global CLASSES
+
+    print('LOADING TF LITE MODEL...', end=' ')
+
+    # Load TFLite model and allocate tensors.
+    interpreter = tflite.Interpreter(model_path='model/BirdNET_6K_GLOBAL_MODEL.tflite')
+    interpreter.allocate_tensors()
+
+    # Get input and output tensors.
+    input_details = interpreter.get_input_details()
+    output_details = interpreter.get_output_details()
+
+    # Get input tensor index
+    INPUT_LAYER_INDEX = input_details[0]['index']
+    MDATA_INPUT_INDEX = input_details[1]['index']
+    OUTPUT_LAYER_INDEX = output_details[0]['index']
+
+    # Load labels
+    CLASSES = []
+    with open('model/labels.txt', 'r') as lfile:
+        for line in lfile.readlines():
+            CLASSES.append(line.replace('\n', ''))
+
+    print('DONE!')
+
+    return interpreter
+
+def loadCustomSpeciesList(path):
+
+    slist = []
+    if os.path.isfile(path):
+        with open(path, 'r') as csfile:
+            for line in csfile.readlines():
+                slist.append(line.replace('\r', '').replace('\n', ''))
+
+    return slist
+
+def splitSignal(sig, rate, overlap, seconds=3.0, minlen=1.5):
+
+    # Split signal with overlap
+    sig_splits = []
+    for i in range(0, len(sig), int((seconds - overlap) * rate)):
+        split = sig[i:i + int(seconds * rate)]
+
+        # End of signal?
+        if len(split) < int(minlen * rate):
+            break
+        
+        # Signal chunk too short? Fill with zeros.
+        if len(split) < int(rate * seconds):
+            temp = np.zeros((int(rate * seconds)))
+            temp[:len(split)] = split
+            split = temp
+        
+        sig_splits.append(split)
+
+    return sig_splits
+
+def readAudioData(path, overlap, sample_rate=48000):
+
+    print('READING AUDIO DATA...', end=' ', flush=True)
+
+    # Open file with librosa (uses ffmpeg or libav)
+    sig, rate = librosa.load(path, sr=sample_rate, mono=True, res_type='kaiser_fast')
+
+    # Split audio into 3-second chunks
+    chunks = splitSignal(sig, rate, overlap)
+
+    print('DONE! READ', str(len(chunks)), 'CHUNKS.')
+
+    return chunks
+
+def convertMetadata(m):
+
+    # Convert week to cosine
+    if m[2] >= 1 and m[2] <= 48:
+        m[2] = math.cos(math.radians(m[2] * 7.5)) + 1 
+    else:
+        m[2] = -1
+
+    # Add binary mask
+    mask = np.ones((3,))
+    if m[0] == -1 or m[1] == -1:
+        mask = np.zeros((3,))
+    if m[2] == -1:
+        mask[2] = 0.0
+
+    return np.concatenate([m, mask])
+
+def custom_sigmoid(x, sensitivity=1.0):
+    return 1 / (1.0 + np.exp(-sensitivity * x))
+
+def predict(sample, interpreter, sensitivity):
+
+    # Make a prediction
+    interpreter.set_tensor(INPUT_LAYER_INDEX, np.array(sample[0], dtype='float32'))
+    interpreter.set_tensor(MDATA_INPUT_INDEX, np.array(sample[1], dtype='float32'))
+    interpreter.invoke()
+    prediction = interpreter.get_tensor(OUTPUT_LAYER_INDEX)[0]
+
+    # Apply custom sigmoid
+    p_sigmoid = custom_sigmoid(prediction, sensitivity)
+
+    # Get label and scores for pooled predictions
+    p_labels = dict(zip(CLASSES, p_sigmoid))
+
+    # Sort by score
+    p_sorted = sorted(p_labels.items(), key=operator.itemgetter(1), reverse=True)
+
+    # Remove species that are on blacklist
+    for i in range(min(10, len(p_sorted))):
+        if p_sorted[i][0] in ['Human_Human', 'Non-bird_Non-bird', 'Noise_Noise']:
+            p_sorted[i] = (p_sorted[i][0], 0.0)
+
+    # Only return first the top ten results
+    return p_sorted[:10]
+
+def analyzeAudioData(chunks, lat, lon, week, sensitivity, overlap, interpreter):
+
+    detections = {}
+    start = time.time()
+    print('ANALYZING AUDIO...', end=' ', flush=True)
+
+    # Convert and prepare metadata
+    mdata = convertMetadata(np.array([lat, lon, week]))
+    mdata = np.expand_dims(mdata, 0)
+
+    # Parse every chunk
+    pred_start = 0.0
+    for c in chunks:
+
+        # Prepare as input signal
+        sig = np.expand_dims(c, 0)
+
+        # Make prediction
+        p = predict([sig, mdata], interpreter, sensitivity)
+
+        # Save result and timestamp
+        pred_end = pred_start + 3.0
+        detections[str(pred_start) + ';' + str(pred_end)] = p
+        pred_start = pred_end - overlap
+
+    print('DONE! Time', int((time.time() - start) * 10) / 10.0, 'SECONDS')
+
+    return detections
+
+def writeResultsToFile(detections, min_conf, path):
+
+    print('WRITING RESULTS TO', path, '...', end=' ')
+    rcnt = 0
+    with open(path, 'w') as rfile:
+        rfile.write('Start (s);End (s);Scientific name;Common name;Confidence\n')
+        for d in detections:
+            for entry in detections[d]:
+                if entry[1] >= min_conf and (entry[0] in WHITE_LIST or len(WHITE_LIST) == 0):
+                    rfile.write(d + ';' + entry[0].replace('_', ';') + ';' + str(entry[1]) + '\n')
+                    rcnt += 1
+    print('DONE! WROTE', rcnt, 'RESULTS.')
+
+def main():
+
+    global WHITE_LIST
+
+    # Parse passed arguments
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--i', help='Path to input file.')
+    parser.add_argument('--o', default='result.csv', help='Path to output file. Defaults to result.csv.')
+    parser.add_argument('--lat', type=float, default=-1, help='Recording location latitude. Set -1 to ignore.')
+    parser.add_argument('--lon', type=float, default=-1, help='Recording location longitude. Set -1 to ignore.')
+    parser.add_argument('--week', type=int, default=-1, help='Week of the year when the recording was made. Values in [1, 48] (4 weeks per month). Set -1 to ignore.')
+    parser.add_argument('--overlap', type=float, default=0.0, help='Overlap in seconds between extracted spectrograms. Values in [0.0, 2.9]. Defaults tp 0.0.')
+    parser.add_argument('--sensitivity', type=float, default=1.0, help='Detection sensitivity; Higher values result in higher sensitivity. Values in [0.5, 1.5]. Defaults to 1.0.')
+    parser.add_argument('--min_conf', type=float, default=0.1, help='Minimum confidence threshold. Values in [0.01, 0.99]. Defaults to 0.1.')   
+    parser.add_argument('--custom_list', default='', help='Path to text file containing a list of species. Not used if not provided.')
+
+    args = parser.parse_args()
+
+    # Load model
+    interpreter = loadModel()
+
+    # Load custom species list
+    if not args.custom_list == '':
+        WHITE_LIST = loadCustomSpeciesList(args.custom_list)
+    else:
+        WHITE_LIST = []
+
+
+    # Read audio data
+    audioData = readAudioData(args.i, args.overlap)
+
+    # Process audio data and get detections
+    week = max(1, min(args.week, 48))
+    sensitivity = max(0.5, min(1.0 - (args.sensitivity - 1.0), 1.5))
+    detections = analyzeAudioData(audioData, args.lat, args.lon, week, sensitivity, args.overlap, interpreter)
+
+    # Write detections to output file
+    min_conf = max(0.01, min(args.min_conf, 0.99))
+    writeResultsToFile(detections, min_conf, args.o)
+
+if __name__ == '__main__':
+
+    main()
+
+    # Example calls
+    # python3 analyze.py --i 'example/XC558716 - Soundscape.mp3' --lat 35.4244 --lon -120.7463 --week 18
+    # python3 analyze.py --i 'example/XC563936 - Soundscape.mp3' --lat 47.6766 --lon -122.294 --week 11 --overlap 1.5 --min_conf 0.25 --sensitivity 1.25 --custom_list 'example/custom_species_list.txt'

example/custom_species_list.txt

@@ -0,0 +1,3 @@
+Poecile atricapillus_Black-capped Chickadee
+Junco hyemalis_Dark-eyed Junco
+Pipilo maculatus_Spotted Towhee

result.csv

@@ -0,0 +1,21 @@
+Start (s);End (s);Scientific name;Common name;Confidence
+15.0;18.0;Junco hyemalis;Dark-eyed Junco;0.63864714
+19.5;22.5;Poecile atricapillus;Black-capped Chickadee;0.28910682
+21.0;24.0;Junco hyemalis;Dark-eyed Junco;0.2813258
+21.0;24.0;Pipilo maculatus;Spotted Towhee;0.26970756
+22.5;25.5;Junco hyemalis;Dark-eyed Junco;0.36897075
+25.5;28.5;Poecile atricapillus;Black-capped Chickadee;0.36192033
+28.5;31.5;Junco hyemalis;Dark-eyed Junco;0.54385275
+34.5;37.5;Junco hyemalis;Dark-eyed Junco;0.6392924
+43.5;46.5;Junco hyemalis;Dark-eyed Junco;0.395477
+45.0;48.0;Junco hyemalis;Dark-eyed Junco;0.35142577
+49.5;52.5;Junco hyemalis;Dark-eyed Junco;0.36010146
+51.0;54.0;Junco hyemalis;Dark-eyed Junco;0.3235896
+54.0;57.0;Junco hyemalis;Dark-eyed Junco;0.61683154
+55.5;58.5;Junco hyemalis;Dark-eyed Junco;0.30236518
+61.5;64.5;Junco hyemalis;Dark-eyed Junco;0.5145074
+63.0;66.0;Junco hyemalis;Dark-eyed Junco;0.55427563
+66.0;69.0;Poecile atricapillus;Black-capped Chickadee;0.7129054
+67.5;70.5;Junco hyemalis;Dark-eyed Junco;0.4047741
+69.0;72.0;Junco hyemalis;Dark-eyed Junco;0.70841223
+70.5;73.5;Junco hyemalis;Dark-eyed Junco;0.25039598