Extensible Music Format (XMF)
The eXtensible Music Format (XMF) is a tree-structured container format developed by the MIDI Manufacturer's Association from 2001 to 2004. Historically it was promoted as a way to package together MIDI tunes and DLS instrument sounds for distribution, ensuring a consistent playback experience across environments<ref name="WhatXMFIsFor">eXtensible Music Format</ref>. It saw ample adoption on early "polyphonic" mobile phones, but as handsets achieved support for MP3 and other digital audio formats, the consumer market for instrument-based formats all but disappeared. Despite that it remains widely supported: even recent mobile architectures such as Google's Android include XMF playing technology<ref name="XMFInAndroid">SONiVOX Contributes audioINSIDE Technology to Google Open Handset Alliance and Android Platform</ref>.
File Structure
Internally, XMF files are composed of a header chunk and a collection of node chunks ordered in preorder. Nodes can be either of type folder (in which case they will have at least one child node) or contents (in which case they encapsulate a data stream, such as MIDI or DLS contents). Most data in XMF nodes is encoded in variable-length quantity (VLQ), a universal code that uses an arbitrary number of binary octets (eight-bit bytes) to represent an arbitrarily large integer. The Python code snippet below illustrates how a VLQ-encoded integer can be converted back to its native representation:
def vlq2int(data):
r'''Read one VLQ-encoded integer value from an input data stream.
'''
value = 0
while True:
# Read one numeric octet from the input stream
octet = ord(data.read(1))
# Appends the octet to the output value, discarding the continuation bit
value = (value << 7) + (octet & 0x7F)
# If the continuation bit is zero, this was the last octet
if (octet & 0x80) == 0:
break
return value
The diagram below illustrates the structure of the XMF format, displaying an example file's internals both in contiguous and tree forms.
ut_60-dls.xmf
Header Chunk
Root Node
Folder Node 1
Contents Node 1
ut.mid
Folder Node 2
Contents Node 2
mobile60.dls
The structure above is typical of several XMF authoring tools, including Beatnik's Mobile Sound Builder and Nokia's Audio Suite plugin for Steinberg Cubase SX. It's far from optimal, however, as it unnecessarily nests the media contents inside exclusive folder nodes, when they could very well be put directly under the root node (and in fact some mobile players are known to choke on those extra nodes).
Header Chunk
The table below relates the fields of a XMF header chunk, in order of occurrence.
| Field | Type | Value |
|---|---|---|
| Format ID | byte[4]
|
"XMF_" (0x584d465f)
|
| Format version | byte[4]
|
One of "1.00" (0x312E3030), "1.01" (0x312E3031) or "2.00" (0x322E3030)
|
| File Type ID | int32
|
Magic number indicating the XMF file type1 |
| File Type Revision ID | int32
|
Magic number indicating the XMF file type's revision1 |
| File size | VLQ
|
The file size in bytes |
| MTT length | VLQ
|
Length of the Metadata Types Table (MTT), in bytes |
| MTT Contents | N/A | Binary-encoded contents of the MTT |
| Tree Start Offset (TFO) | VLQ
|
The root node's absolute offset within the XMF file |
1. Included as of version 2.0 of the spec. Not present in files encoded to earlier versions.
Node Chunk
The table below relates the fields of a XMF node chunk, in order of occurrence.
| Field | Type | Value |
|---|---|---|
| Node length | VLQ
|
The node's length in bytes, including the length of all child nodes (for folder nodes) or encapsulated contents (for content nodes) |
| Item count | VLQ
|
Children node count, must be zero for content nodes |
| Node header length | VLQ
|
Length of this node's header, in bytes |
| Node metadata length | VLQ
|
Length of the node's metadata, in bytes |
| Node metadata contents | N/A | Binary-encoded node metadata contents |
| Reference type | VLQ
|
Indicates whether the following contents constitute an actual data buffer (0x1) or an offset to somewhere else within the file (0x2)
|
| Contents | N/A | Either a data buffer or an offset value within the file, depending on the reference type |
Simple Example Parser
As way of illustration, the Python script below decodes an input XMF file, printing various statistics about its contents:
from os import SEEK_CUR, SEEK_SET
class xmfnodemetadata(object):
def __str__(self):
return '\nMetadata Size : ' + str(self.size) + ' bytes'
class xmfnodeheader(object):
def __init__(self):
self.metadata = xmfnodemetadata()
def __str__(self):
return \
'\nHeader Size : ' + str(self.size) + ' bytes' + \
str(self.metadata)
class xmfnode(object):
def __init__(self):
self.header = xmfnodeheader()
self.children = []
def __str__(self):
return \
'\nNode Size : ' + str(self.size) + ' bytes' + \
str(self.header) + \
'\nReference Type : ' + hex(self.reftype) + \
'\nContents : ' + str(self.contents) + \
'\nChildren Count : ' + str(len(self.children)) + \
'\n'.join([str(child).replace('\n', '\n ') for child in self.children])
class mtt(object):
def __str__(self):
return '\nMTT Length : ' + str(self.size) + ' bytes'
class xmf10(object):
def __init__(self):
self.mtt = mtt()
def __str__(self):
return \
'\nFormat ID : ' + self.formatid + \
'\nFormat Version : ' + self.version + \
'\nFile Size : ' + str(self.size) + ' bytes' + \
str(self.mtt) + \
'\n' + \
'\nContent Nodes:' + \
'\n' + \
str(self.root)
def vlq2int(data):
r'''Read one VLQ-encoded integer value from an input data stream.
'''
value = 0
while True:
# Read one numeric octet from the input stream
octet = ord(data.read(1))
# Appends the octet to the output value, discarding the continuation bit
value = (value << 7) + (octet & 0x7F)
# If the continuation bit is zero, this was the last octet
if (octet & 0x80) == 0:
break
return value
def parsenode(data):
offset = data.tell()
node = xmfnode()
node.size = vlq2int(data)
childcount = vlq2int(data)
node.header.size = vlq2int(data)
node.header.metadata.size = vlq2int(data)
data.seek(offset + node.header.size, SEEK_SET)
node.reftype = vlq2int(data)
node.contents = '(folder)'
node.children = []
if childcount > 0:
node.children = [parsenode(data) for i in range(0, childcount)]
elif node.reftype == 1:
node.contents = data.read(4)
else:
raise Exception('Invalid type ' + hex(node.reftype))
data.seek(offset + node.size, SEEK_SET)
return node
def parse(path):
with open(path, 'rb') as data:
xmf = xmf10()
xmf.formatid = data.read(4)
xmf.version = data.read(4)
if xmf.version != '1.00':
raise Exception('Invalid version \"' + xmf.version + '\"')
xmf.size = vlq2int(data)
xmf.mtt.size = vlq2int(data)
data.seek(xmf.mtt.size, SEEK_CUR)
tfo = vlq2int(data)
data.seek(tfo, SEEK_SET)
xmf.root = parsenode(data)
return path + '\n' + str(xmf)
if __name__ == '__main__':
from sys import argv
parsed = parse(argv[1])
print(parsed)
Tools
The MMA provides a list of authoring tools for XMF; however, virtually every one of them is currently unsupported, their makers having either moved past the format or gone with it. Some can still be found around the Internet, however:
- The Mobile Sound Builder was Beatnik's solution for authoring and playing XMF files. It's hard to tell from their half-functioning company website whether it's still supported, but binaries for version 1.2 often turn up in Google searches;
- Nokia Audio Suite is a plug-in for Steinberg Cubase SX which adds the ability to author XMF files. It's a free download, however license keys for version 2.0 are issued only to users outside of Japan and US, while keys for version 1.1 aren't even available anymore – so effectively it can't be used by neither Japanese nor American developers;
- Faith reportedly<ref name="FaithmXMFTool">Mobile Game Audio Gets Boost from Faith West’s mXMFTool and New Royalty-Free Game Sounds Collection</ref> once provided the mXMFTool as a free download, however it was removed when their San Francisco office (Faith West) was closed. Currently it's nowhere to be found, though it seems you can talk them into mailing you a copy<ref name="FaithmXMFToolGet">Mobile eXtensible Music File</ref>;
- The Professional Audio Company apparently provided some sort of XMF-related tool in the past, but no further information seems to be available. The company relinquished their MMA Manufacturer ID<ref name="PACMMAID">MANUFACTURER'S ID NUMBERS</ref>, so it's very unlikely they still work with it;
- Crimson Technology still provides a demo version for their DLS tools, however it's unclear what XMF features (if any) they enable.
References
<references />