Finetuning Strategies for Querying Sounds by Vocal Imitation
A technical report describes a winning submission to the AES AIMLA 2025 Challenge on querying sound effects by vocal imitation. It investigates two fine-tuning strategies: contrastive learning with a frozen, pretrained CED encoder, and joint contrastive-triplet learning with semi-hard negatives using a MobileNetV3 encoder. The report was updated for posterity to include details released after the challenge.
The evidence is a technical report on fine-tuning strategies for querying sounds by vocal imitation, submitted to the AES AIMLA 2025 Challenge. The authors investigate two complementary approaches: contrastive learning with a frozen, pretrained CED encoder, and joint contrastive-triplet learning with semi-hard negatives using a MobileNetV3 encoder. The report was updated after the challenge to include additional details.
The use of a frozen pretrained CED encoder for contrastive learning suggests leveraging strong audio representations without updating encoder weights, while the MobileNetV3 approach with semi-hard negatives indicates a focus on efficient on-device or low-latency inference. The combination of contrastive and triplet losses may improve embedding discrimination for vocal imitation queries.
Vocal imitation as a query modality for sound effects could enable more intuitive search in audio production tools, potentially reducing reliance on text metadata. Winning an AES challenge signals practical relevance for creative audio industries.
The techniques could be integrated into sound effect libraries or digital audio workstations to allow users to find sounds by humming or imitating them, improving workflow efficiency and user experience.
Next observable signals include publication of challenge results, release of code or model weights, and adoption of vocal imitation search in commercial audio software. Further research may compare these strategies on larger datasets or real-world user studies.