Web audio APIs and the low-level approach

“HTML5 Audio APIs - How Low can we Go?” (by Mark Boas for The Worm Hole) explains that there are currently two competing web APIs for audio. One is supported by Firefox, the other by Chrome and Safari. The former takes a low-level approach, the latter is higher-level.

The web audio community are a vibrant bunch. No sooner had the standard <audio> API been established, than developers were clamouring for more. Just playing audio wasn’t enough, we wanted to analyse, react to and manipulate our audio. Happily, the browser makers obliged with first Mozilla, then Google producing enhanced web audio APIs for their browsers - the only problem was, they were two very different implementations. The Audio Data API implemented in Firefox exposed the data at a fairly low level, while Webkit’s Web Audio API provided a higher level abstraction providing a number of predefined functions. Luckily, it didn’t take long for the JavaScript community to react and start bridging the gap between the two, by writing libraries that provided a common API ...
...
Mozilla’s approach was to expose a very comprehensive low level API, which potentially allows JavaScript developers to create all the functionality of Webkit’s Web Audio API and then some. As a result we get libraries like JSMad cropping up. What does JSMad do? Significantly, it allows you to play MP3s in Firefox [Neither Firefox or Opera support MP3 natively due to patent concerns]. Is JavaScript fast enough? Apparently so. This was a ‘this changes everything’ moment for me and since then a similar approach has been taken by pdf.js and more recently Broadway.js which decodes H.264 on the fly.

The author argues that Webkit should also adopt this approach and that it makes even sense for browser APIs in general:

• Browser vendors: create low-level APIs that serve as a stable and universal foundation for libraries.
• Library programmers: can experiment and come up with things that can’t be foreseen during the creation of a browser API.

The Component Model is such a low-level API, for extending the DOM:

The Component Model introduces comprehensive support for creating DOM elements. Examples include layout managers, combinations of Dojo and jQuery widgets, isolated widgets, such as Like/+1 buttons, and built-in HTML elements themselves.