AutoMix — Audio Transition Generator

AutoMix is a Streamlit-based application for generating audio transitions(inpainting mask gaps) for the purpose of continuations between musics. The app provides two generation engines you can choose from:

• AudioLDM Diffusion-based audio model suitable for transitions, inpainting, and text-guided generation.

• MusicGen (Audiocraft) Transformer-based model that has been modified to support bidirectional generation, allowing the model to consider both left and right context when generating transitions.

🚀 1. Environment Setup

AutoMix uses both AudioLDM and Audiocraft, installed separately.

1.1.0 Install AudioLDM Environment

Use this as reference: https://github.com/haoheliu/AudioLDM

conda create -n audioldm python=3.8
conda activate audioldm

pip3 install git+https://github.com/haoheliu/AudioLDM.git

git clone https://github.com/haoheliu/AudioLDM
cd AudioLDM

1.2.0 Install Audiocraft (MusicGen) Environment

⚠️ Important: Create a separate conda env so it does not conflict with AudioLDM.
Use this as reference: https://github.com/facebookresearch/audiocraft

conda create -n audioldm python=3.9
conda activate automix

python -m pip install 'torch==2.1.0'

# You might need the following before trying to install the packages
python -m pip install setuptools wheel

# Install MusicGen / Audiocraft
python -m pip install -U audiocraft  # stable release

# Install ffmpeg for audio I/O
conda install "ffmpeg<5" -c conda-forge

1.2.1 Patch MusicGen Transformer (Required for Bidirectional Generation)

The default MusicGen transformer only supports forward generation. To generate transitions that blend both previous and upcoming audio (bidirectional context), AutoMix requires a modified transformer.py download here

Locate Audiocraft instalation path and replace the transformer file, example path:

/home/user/miniconda3/envs/audioldm/lib/python3.9/site-packages/audiocraft/modules/transformer.py

⚠️ Ensure you replace the file in the automix environment, not the audioldm environment.

🧠 2. Checkpoints

Download the finetuned model weights here

Place them inside checkpoint folder:

checkpoint/
    ├── audioldm-s-full-V1.ckpt
    └── others...

🎵 3. Dataset Structure

AutoMix expects:

dataset/
│
├── chunks/   # 10-second WAV files, 32 kHz
│
└── music/    # Full-length tracks

🎛️ 4. Run the Streamlit App

pip install streamlit
streamlit run app.py

✨ Credits

• AudioLDM — https://github.com/haoheliu/AudioLDM
• MusicGen / Audiocraft — https://github.com/facebookresearch/audiocraft