Xiaomi open-sources MiDashengLM-Spatial under Apache 2.0, an audio model that understands what a sound is and its direction, distance, and movement (report)

Verified (fact)

According to Pandaily, Xiaomi has open-sourced 'MiDashengLM-Spatial,' a spatial audio understanding model, under the Apache 2.0 license. The model adds a new encoder, 'Spatial-Dasheng,' to MiDashengLM-7B. On top of general audio understanding, it can also tell the direction a sound comes from, its distance, and its movement.

Needs fact-check (claim)

Understands both the 'what' and the 'where' of sounds (as reported). The collected item contains no published benchmark figures.

Why it matters (interpretation)

Most existing audio language models deal only with what a sound is. A model that also understands where a sound is and how it moves could be used in devices that need to know their surroundings, such as robots, smart devices, and AR glasses. The Apache 2.0 license, which allows commercial use, is also notable.

Sources

• Press Xiaomi Open-Sources MiDashengLM-Spatial, an Audio Model That Hears Both What and Where Pandaily