Training Music Spectrogram Captures On GCP With Tensorflow, Keras & Librosa Via Panotti

A music classifier does not hear a guitar riff or recognize a drum groove the way a person does. It learns patterns in a numerical representation of sound. A spectrogram makes those patterns accessible: time runs along one axis, frequency along the other, and each value describes energy in a particular time-frequency region.

Panotti brings audio preparation and convolutional neural-network classification into one workflow, using Librosa for audio analysis and Keras with TensorFlow for learning. Google Compute Engine supplies a machine that can handle longer preprocessing jobs and GPU-backed training without tying up your laptop.

This tutorial walks through a practical, 2019-era approach: organize labeled music, prepare consistent spectrogram features, train on GCP, and evaluate predictions without fooling yourself with data leakage.

This guide reflects a 2019 workflow; tools, dependencies, and cloud interfaces may have changed since then.

What you are actually training

You train a model on spectrograms, not the spectrograms themselves. For this example, the task is single-label music classification: every source recording belongs to one class, such as acoustic or electronic. Those labels are illustrative; choose categories your recordings can support consistently.

The stack has four responsibilities:

Librosa loads audio and computes features such as mel spectrograms.
Panotti provides an audio-classification workflow, including preparation and model-training code.
Keras and TensorFlow define and optimize the neural network.
GCP Compute Engine supplies CPU, memory, storage, and optionally a GPU.

A spectrogram need not be a PNG. Numeric feature arrays retain the actual values without introducing plot borders, labels, compression, or arbitrary color maps. Use visualization for inspection, and the feature representation expected by your training pipeline for learning.

1. Define labels and split the recordings

Start with audio you have permission to process. WAV, MP3, and M4A can all be source formats, but compressed formats depend on available decoders. The extension alone does not guarantee successful loading.

For your staging dataset, use an explicit split structure:

music/
  train/
    acoustic/
    electronic/
  validation/
    acoustic/
    electronic/
  test/
    acoustic/
    electronic/

Adapt this structure to the input conventions of your checked-out Panotti revision. Keep a separate manifest listing each file, label, source recording, and split; directory names alone are not enough to audit a growing dataset.

Split before cutting clips

If you divide a song into ten-second windows and then randomly split those windows, nearly identical material can appear in training and validation. The resulting score measures familiarity with recordings rather than generalization.

Assign complete recordings to splits first. Where relevant, group by artist, album, recording session, or original source as well. Duplicate encodings of the same song must stay together.

Listen to samples from every class. Ambiguous labels, silence, spoken introductions, and inconsistent recording quality often cause more trouble than the model architecture. Keep the test set untouched until model choices are settled.

2. Create a GCP training machine

For a modest experiment, prepare features on a CPU machine and add GPU training only when it saves meaningful time. Spectrogram extraction is largely a CPU and storage workload; a GPU will not automatically accelerate Librosa preprocessing.

In Compute Engine:

Select a region and zone with the required accelerator availability and quota.
Create an Ubuntu-based VM or use a suitable Deep Learning VM image.
Allocate persistent disk space for original audio, decoded files, features, and checkpoints.
Attach a GPU if needed, using an image with compatible NVIDIA drivers, CUDA, and TensorFlow.
Give the VM narrowly scoped access to the Cloud Storage bucket holding your data.

A 2019-era GPU such as a Tesla T4 can be appropriate where available, but availability and quota are separate requirements. Confirm both before planning a run.

After connecting over SSH, inspect the accelerator:

nvidia-smi

For TensorFlow 1.x, check visibility from the actual Python environment used for training:

python -c "import tensorflow as tf; print(tf.__version__); print(tf.test.is_gpu_available())"

If the driver sees a GPU but TensorFlow does not, investigate the TensorFlow package and CUDA/cuDNN compatibility before starting a long job.

Keep the VM in the same region as its data when practical. Stop it when finished, remembering that retained disks and stored objects can still incur charges.

3. Install a reproducible Panotti environment

Panotti is a source-based project, so its checked-out revision matters. Do not combine an older checkout with whatever TensorFlow, Keras, NumPy, and Librosa happen to be newest.

A typical setup begins with system audio tools and an isolated Python environment:

sudo apt-get update
sudo apt-get install -y git ffmpeg python3-venv
python3 -m venv ~/venvs/panotti
source ~/venvs/panotti/bin/activate

git clone https://github.com/drscotthawley/panotti.git
cd panotti
git rev-parse HEAD

Choose a revision available in 2019, then follow that revision's installation instructions and dependency requirements. Panotti revisions using standalone keras should not be casually rewritten to use tf.keras; imports, serialization, and backend behavior can differ.

Record the working environment:

python -m pip freeze > environment.freeze.txt
ffmpeg -version > ffmpeg-version.txt

Store these alongside the Git commit and preprocessing configuration. Install a GPU-enabled TensorFlow build compatible with the image if one is not already present; installing CPU and GPU distributions indiscriminately can create confusing environments.

Verify Panotti's commands locally

Panotti's commonly used preparation and training entry points include preprocess_data.py and train_network.py. Inspect the checked-out documentation and available arguments:

python preprocess_data.py --help
python train_network.py --help

Use that revision's documented input directories, output locations, and options. Do not assume it accepts your staging layout unchanged or preserves a preassigned split. Check its splitting behavior explicitly; if necessary, prepare partitions separately or adapt its loader so recording-level boundaries remain intact.

4. Standardize decoding and spectrogram extraction

Copy your dataset to the VM's persistent disk rather than repeatedly fetching individual audio files during training:

gsutil -m rsync -r gs://YOUR_BUCKET/music ./music

Test decoding before processing the full collection. FFmpeg can normalize problematic inputs into a predictable intermediate format:

ffmpeg -i input.m4a -ac 1 -ar 22050 output.wav

This example produces mono audio at 22,050 Hz. It is a design choice, not a universal requirement. Stereo information or higher-frequency content may matter for your task.

Understand the feature settings

The following Librosa example illustrates the numeric representation behind the workflow. It is not a replacement for Panotti's on-disk format; when using Panotti's preprocessing, inspect and retain its actual settings.

import librosa
import numpy as np

SR = 22050
SECONDS = 10
N_MELS = 128

def extract_feature(path):
    y, _ = librosa.load(path, sr=SR, mono=True)
    if len(y) == 0:
        raise ValueError("Empty audio: " + path)

    target = SR * SECONDS
    y = y[:target]
    if len(y) < target:
        y = np.pad(y, (0, target - len(y)), mode="constant")

    mel = librosa.feature.melspectrogram(
        y=y, sr=SR, n_fft=2048, hop_length=512,
        n_mels=N_MELS, power=2.0
    )
    log_mel = librosa.power_to_db(mel, ref=np.max)
    return log_mel.astype("float32")[..., np.newaxis]

The output has frequency, time, and channel dimensions. Fixed-duration input makes batches consistent. The FFT window controls the time-frequency tradeoff; hop length controls the spacing between frames; mel bands summarize frequency content on a perceptually motivated scale.

This example takes only the beginning of each recording. For real music datasets, consider multiple windows covering the track, while preserving source identity for splitting and evaluation. Cap or balance windows per recording so long tracks do not dominate.

Relative decibel conversion reduces sensitivity to absolute level, but it does not eliminate recording differences. Inspect features for silence, clipping, unexpected shapes, and nonfinite values before training.

5. Train with Panotti and inspect the baseline

Use Panotti's training entry point with the prepared feature location and configuration supported by your revision. First run a small smoke test: load a few examples, complete an epoch, save a model, and reload it.

Verify these essentials before the full run:

Input dimensions and channel ordering match the model.
Class indices match the dataset labels.
Validation data is separate from training data.
Normalization statistics come from training data only.
Checkpoints are written to persistent storage.

What the Keras training stage looks like

For orientation, this standalone Keras baseline accepts the channels-last arrays illustrated above. It is an alternative diagnostic model, not a claim about Panotti's built-in architecture:

from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import GlobalAveragePooling2D, Dense, Dropout
from keras.callbacks import EarlyStopping, ModelCheckpoint

model = Sequential([
    Conv2D(32, (3, 3), activation="relu",
           input_shape=X_train.shape[1:]),
    MaxPooling2D((2, 2)),
    Conv2D(64, (3, 3), activation="relu"),
    MaxPooling2D((2, 2)),
    GlobalAveragePooling2D(),
    Dropout(0.3),
    Dense(num_classes, activation="softmax")
])
model.compile(optimizer="adam",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(
    X_train, y_train,
    validation_data=(X_val, y_val),
    epochs=40, batch_size=32,
    callbacks=[
        EarlyStopping(monitor="val_loss", patience=5),
        ModelCheckpoint("best_model.h5", monitor="val_loss",
                        save_best_only=True)
    ]
)

Here, feature arrays must already be loaded and consistently normalized, labels must be integer class indices, and num_classes must match the label mapping. This in-memory example suits small datasets; use batching from disk when features exceed available RAM.

Start with a modest batch size. GPU out-of-memory errors usually call for smaller batches or inputs, not repeated retries. Rising training accuracy with worsening validation loss suggests overfitting, leakage-free distribution differences, or label problems, not an automatic need for more layers.

6. Evaluate recordings and preserve the pipeline

Reload the best validation checkpoint before evaluating the untouched test set. Report a confusion matrix and per-class precision and recall alongside overall accuracy, especially when class sizes differ.

If prediction operates on windows, decide how to combine them into a track prediction. Averaging class probabilities is a simple baseline. Choose that rule using validation data, then report track-level results rather than treating correlated windows as independent recordings.

Save the model together with:

Class names in index order.
Sample rate, duration, FFT, hop, and mel settings.
Normalization parameters learned from training data.
Dataset manifest, split assignments, and source identifiers.
Panotti revision and dependency versions.

Back up checkpoints and metadata to Cloud Storage before deleting the VM. Finally, test inference on a fresh file through the same decoder, feature extraction, normalization, and prediction steps. A model file alone is not a reproducible music classifier.

Takeaway

Panotti, Librosa, Keras, and TensorFlow provide a useful route from labeled audio to a spectrogram-based classifier, while GCP supplies scalable compute. The most important work is keeping that route consistent: split by recording before windowing, verify the exact Panotti revision, inspect numeric features, and preserve preprocessing alongside the trained model.