Cleaning up audio usually means cleaning up timelines and tweaking filters, but Meta believes it should be as simple as describing the sound you want. The company has released a new open-source AI model called SAM Audio that can isolate almost any sound from a complex recording using simple text prompts.
Users can bring out specific sounds like voices, instruments, or background noises without having to wade through complicated editing software. The model is now available through Meta’s Anything Playground segment, which includes other prompt-based image and video editing tools.
Overall, SAM Audio is designed to understand what sound you want to work with and cleanly separate it from everything else. According to Meta, this opens the door to faster audio editing for use cases such as music production, podcasting, film and television, accessibility tools and research.
For example, a creator might isolate vocals from a tape recording, remove traffic noise from a podcast, or delete a barking dog from an otherwise perfect recording – all by describing what the model should aim for.
This is how SAM Audio works
SAM Audio is a multimodal model that supports three different types of announcements. Users can describe a sound with text, click on a person or object in a video to visually identify the sound they want to isolate, or mark a time period when the sound first occurs. These prompts can be used individually or in combination, giving users granular control over what gets separated.
Under the hood, the system is based on Meta’s Perception Encoder Audiovisual Engine. It serves as the model’s ability to detect and understand sounds before cutting them out of the mix.
To improve audio separation assessment, Meta also introduced SAM Audio-Bench, a benchmark for measuring how well models handle speech, music and sound effects. It is accompanied by SAM Audio Judge, which evaluates how natural and accurate the separated audio signals sound to human listeners, even without comparison tracks.
Meta claims that these evaluations show that SAM Audio performs best when different prompt types are combined, and that it can process audio faster than real-time even at large scale.
However, the model has clear limitations. It does not support audio-based announcements, cannot perform full separation without announcements, and has problems with similar overlapping sounds, such as: B. isolating a single voice from a choir.
Meta says it plans to improve these areas and is already exploring real-world applications, including accessibility work with hearing aid manufacturers and organizations that support people with disabilities.
The launch of SAM Audio comes in the context of Meta’s broader AI push. The company is improving the speech intelligibility of its AI glasses for noisy environments, working on next-generation mixed reality glasses expected to launch in 2027, and developing conversational AI that could compete with ChatGPT, signaling a broader focus on AI models that understand sound, context and interaction.
