Invisible XML

The ixml flow

Data is an abstraction: there is no essential difference between the JSON

{"temperature": {"value": 21; "scale": "C"}}

and an equivalent XML

<temperature value="21" scale="C"/>

<temperature scale="C">21</temperature>

<temperature>
   <value>21</value>
   <scale>C</scale>
</temperature>

or indeed

Temperature: 21°C

since the underlying abstractions being represented are the same.

We choose which representations of our data to use, JSON, CSV, XML, or whatever, depending on habit, convenience, or the context we want to use that data in. On the other hand, having an interoperable generic toolchain such as that provided by XML to process data is of immense value. How do we resolve the conflicting requirements of convenience, habit, and context, and still enable a generic toolchain?

Invisible XML (ixml) is a method for treating non-XML documents as if they were XML, enabling authors to write documents and data in a format they prefer while providing XML for processes that are more effective with XML content. For example, CSS code like:

body {color: blue; font-weight: bold}

can be turned into XML like:

<css>
   <rule>
      <simple-selector name="body"/>
      <block>
         <property>
            <name>color</name>
            <value>blue</value>
         </property>
         <property>
            <name>font-weight</name>
            <value>bold</value>
         </property>
      </block>
   </rule>
</css>

or depending on choice, like this:

<css>
   <rule>
      <selector>body</selector>
      <block>
         <property name="color" value="blue"/>
         <property name="font-weight" value="bold"/>
      </block>
   </rule>
</css>

Method

The format of the textual document is described with a context-free grammar, which is used with a general parser to create an internal abstract structured document. The grammar is decorated as necessary with details of how the resulting abstract document should be serialised as XML.

temperature: value, -"°", scale.
     -value: "-"?, digit+.
     @scale: "C"; "F".
     -digit: ["0"-"9"].

With the input 21°C this would yield

<temperature scale="C">21</temperature>

Changing -value to @value would give

<temperature value="21" scale="C"/>

and changing @value and @scale to just value and scale would give

<temperature>
   <value>21</value>
   <scale>C</scale>
</temperature>

Learning

A thirty minute introduction to ixml video (followed by a 30 minute Q&A session).
The hands-on tutorial teaches you how to use ixml.
And there is an advanced tutorial, once you've learned the basics.

Resources

invisiblexml.org: The home for the specification, examples, implementations, etc.
The official Specification for Invisible XML formally describes the language.
The ixml grammar in ixml serialization.
The ixml grammar in XML serialization.
The working group where development happens. Feel free to join.

Papers

There are a number of papers on the development including:

Invisible XML: Introduces the concepts, and develops a notation to support them.
Data just wants to be (format) neutral: Discusses issues with automatic serialisation, and the relationship between Invisible XML grammars and data schemas.
Parse Earley, Parse Often: How to parse anything to XML: Discusses issues around grammar design, and in particular parsing algorithms used to recognise any document, and converting the resultant parse-tree into XML, and gives a new perspective on a classic algorithm.
On the Descriptions of Data: The Usability of Notations Discusses changes to the design following experience with using it, giving examples of its use to develop data descriptions, and in passing, suggests other output formats.
On the Specification of Invisible XML: Describes decisions made during the production of the specification of ixml.
A Pilot Implementation of ixml: Describes issues in implementing ixml, and proposes methods for round-tripping documents.
Roundtripping Invisible XML: Describes a method of roundtripping ixml, and shows some surprising consequences.

Examples of Use

Here are some published examples of ixml use in the wild:

Implementations

There are a number of implementations already available; see invisiblexml.org for details.