COM.CURLR.XML - A lightweight XML parser

The guiding principal behind curlr, and this web site, is to advance the use of Curl for the many web apis that are being published, particularly by Google, Yahoo and various other services. Many use XML as an exchange format and with the widespread adoption of AJAX, it is evident that XML processing is a central part of client side web applications (notwithstanding the increasing use of JSON, and Google's habit of wrapping their services in Javascript libraries).

The Curl language provides a SAX parser, and the not yet published Curl web services developer's kit builds on it to provide a stack of standards compliant XML and web services utilities - which I strongly recommend for enterprise application development.

For many applications, however, we don't need the rigor of a validating XML parser, or the type definitions of a DTD. Particularly when XML is being used only as an exchange format, we simply need a mechansim for transforming the XML data stream into structures that can be queried for the required information. The more efficient and less verbose this is, the better for the developer, for whom the XML data is simply a means to the desired end of building a useful and interesting application.

This, then, is the purpose of the code that I've implemented in the package COM.CURLR.XML - namely to provide a simple and lightweight set of classes that are easy to use in application code. The beta version of the package is now checked in to the curlr repository.

I had been wanting to implement a simple XML parser for some time - but as there's not yet a Curl implementation of lex / yacc or a general purpose scanner / parser (a project for another day), this was always going to be a handcrafted piece of code, and one I hadn't had time to write.

After some searching I found this Java Quick and Dirty XML parser, intended at the time it was written for mobile implementations but clearly suitable now for handling XML in many client applications. This is a very lightweight implementation of the basic XML standard, but the original code would be inefficient if implemented "as is" in Curl. Also, the parser is written as an event handler, with call-backs for start / end document, and start / end element events, these are also not necessary in the curlr implementation, which incrementally creates a tree structure of nodes, with a single root element.

So, the particulars differ from the original, but the principles still apply. The parser is non-validating, and does not handle the xml header - so the encoding must be known to the caller, and should be set correctly when the input stream is opened. The DOCUMENT tag is treated as a comment, but there are DOCUMENT definitions that the parser will not handle. This emphasizes the main usage constraint of the parser - that the content of the XML is expected by the parser.

There are four important classes included in the package:

• sadXMLParser - parses a stream of XML format data into a tree of nodes
• sadXMLNode - a node representing a tag in an XML document, it has search functions
• sadXMLDom - a DOM representing the XML document, supporting the basic W3C search methods
• sadXMLHttpRequest - a Curl implementation of the XMLHttpRequest object used extensively in AJAX applications.

In this article I'll introduce the first two classes, essential for all XML handling, and which I have used to generate and implement the Flickr api - so we can be reasonably sure that it works well enough. The other two are intended to parallel the usage of the AJAX XmlHttpRequest object; I'll present these in my next article, and show how you can implement AJAX applications with very similar Curl code.

The parser, sadXMLParser, has only one public class to be used called, unsurprisingly, parse. This takes a TextInputStream as an input argument, and returns the root node of a hierarchy on successful completion; an exception is thrown if the XML is badly formed.

The returned tree node structure is implemented by sadXMLNode, and is as simple as possible; a node has a parent (unless it's the root node), and nullable fields are assigned to the table of attributes (they are actually stored as an Arguments object), and the array of child nodes. All attributes are defined as name / value pairs with the value stored as a String.

The node class has several methods useful for searching the tree structure and for managing the tree structure. For familiarity of programming - but not compliance - I've included getters equivalent to those in the W3C level 1 spec. Here's the skeleton of the code:

The combination of the parser and the search methods of the tree nodes give us all we need to handle most application scenarios - the root node is, in effect, our XML Dom. If you're happier working with a Dom object using the standard search functions like getElementById, then you'll prefer to use sadXMLDom.

Let's look first at using the classes to examine the XML from a Flickr group's RSS feed - we'll be creating a simple RSS viewer in the next article, so this has some relevance. The following code parses the XML from an RSS feed from the Flickr Central group.

The code is quite simple - first we create a readable stream for the feed. The example chooses to do this by resolving the url into a HttpFile object. If this was unsuccessful, a MissingFileException would be thrown - so obviously the example is missing some defensive try/catch blocks. Once the file handle is obtained, we create the input stream with a call to the synchronous method http-read-open. This approach gives us access to the http response headers, and allows us to set the http request headers in the http-read-open call. If this isn't necessary, then the following even simpler code, using the read-open procedure will be sufficient to give a readable TextInputStream. In either case, the read open is where we would set the character encoding for the stream.

The parser is reentrant, so we would need only the one object for all subsequent XML processing. The parse method takes any TextInputStream as a parameter, and returns the root sadXMLNode of the parsed tree. The final statement using the to-String method will simply display the XML tree as text - minus the XML header, with an extra root node; this will be quite ugly, since the text of picture descriptions will contain HTML markup.

What next? Well, as with many applications, the XML itself is of not much use by itself, so we need to extract something that can be used or displayed. To get a list of the item titles we can write:

Which is mostly self-explanatory; search-children returns an array of sadXMLNodes, and we need to check that the return from get-child is not null - note that the content of a node is obtained from sadXMLNode.text, which will be the empty string if there is no conent.

The walk-nodes method provides a more powerful mechanism for searching the tree and building sets of nodes. It takes an anonymous procedure as input - which is applied to all nodes in the tree; when the procedure returns true, the traversal of the hierarchy is stopped. For the simple case of getting picture titles, we would use walk-nodes with a procedure as follows:

For the simple search walk-nodes provides no advantages, but it is useful for more complex searches - for example finding all nodes that have an attribute with a specified name or value.

So that's the 10 cent tour. Use the sadXMLParser and sadXMLNode classes to transform your XML into a tree of nodes, and use the node search methods to extract the data you need from the XML.

Next to come, we'll show the use of the sadXMLHttpRequest to build a simple RSS viewer.