METEOR: Melody-aware Texture-controllable
Symbolic Orchestral Music Generation via Transformer VAE
Dinh-Viet-Toan Le, Yi-Hsuan Yang
Abstract.
Re-orchestration is the process of adapting a music piece for a different set of instruments. By altering the
original instrumentation, the orchestrator often modifies the musical texture while preserving a recognizable
melodic line and ensures that each part is playable within the technical and expressive capabilities of the chosen
instruments.
In this work, we propose METEOR, a model for generating Melody-aware Texture-controllable
re-Orchestration with a Transformer-based variational auto-encoder (VAE). This model performs symbolic
instrumental and textural music style transfers with a focus on melodic fidelity and controllability. We allow
bar- and track-level controllability of the accompaniment with various textural attributes while keeping a
homophonic texture. With both subjective and objective evaluations, we show that our model outperforms style
transfer models on a re-orchestration task in terms of generation quality and controllability. Moreover, it can be
adapted for a lead sheet orchestration task as a zero-shot learning model, achieving performance comparable to a
model specifically trained for this task.
Audio samples are synthesized from the generated MIDI files using the Musescore default soundfonts.
