Thursday 17 September 2009

Google App Engine (GAE) for Java and XStream

XStream is a powerful and often incredibly useful tool for turning Java classes into XML and back again. If have used it for all sorts of data transformation, configuration loading and saving, persistence needs over the past few years.

In my recent investigations of Google App Engine for Java (GAE/J) I have come across the problem of how to seed an application I am developing with the initial set of data that it needs. At the time of writing GAE doesn't have much in the way of a data loading/management API, especially on Java.

So I thought I would look at using my trusty old friend XStream and file upload to send data up into the application to populate the Datastore.

Unfortunately using XStream with GAE proved to be less than straightforward mainly due to the current security restrictions in the GAE environment. GAE is a sandboxed environment with white listing of a limited set of classes.

As a result there are a couple of issues with using XStream in GAE

  • The standard out of the box XStream uses classes that are not permitted by the sandboxed GAE environment

  • A common usage pattern for XStream is to use it to create an object input or output stream that can be used to read and write serializable objects, but this works by using subclassing of ObjectInputStream and ObjectOutputStream, again this is not allowed by the GAE security model


  • However all is not lost. I found a couple of interesting discussions about the security issues here and here the result of which was a JIRA entry with a number of patches that work around the Serialization security issues.

    I took the gae3-xstream.patch and applied it and tested it and found my problems were half solved. I had hit the second issue I listed above.

    I was trying and failing to create an ObjectInputStream in GAE to unmarshall an XStream representation of a serialized stream of objects.

    The serialized object representation had been created using

    ObjectOutputStream out = xstream.createObjectOutputStream(stringWriter);
    for (WavaBase wavaBase : wavaBases) {
        out.writeObject(wavaBase);
    }


    As a result you get a nicely formed XML document with an object-stream root.


    <object-stream>
        <wavabase>
            <id>F10K</id>
            ...
        <wavabase>
        <wavabase>
            <id>M10K</id>
            ...
        <wavabase>
        ...
    <object-stream>


    But this form of document requires use of an object stream to reverse the process.


    ObjectInputStream in = xstream.createObjectInputStream(inputStream);
    List<WavaBase> wavaBases = new ArrayList<WavaBase>();
    while(true) {
        WavaBase wavaBase = (WavaBase)in.readObject();
        if (wavaBase == null) break;
        wavaBases.add(wavaBase);
    }


    Here is where we hit the issue with XStream on GAE as we are not able to call createObjectInputStream() without getting a security violation.

    But all is not lost. If we adopt a different approach for creating our XML document containing the XStream serialized data we require we can work about the stream security problems.

    The alternative approach is to create a list or set of the data that we want to save and then to use toXml() to form the document, this doesn't require or use the custom input and output streams.

    To create the serialized object representation use

    List<WavaBase> clonedWavaBases = ...;
    ...
    xstream.toXML(wavaBases, stringWriter);


    which gives us a subtly different XML generated


    <list>
        <wavabase>
            <id>F10K</id>
            ...
        <wavabase>
        <wavabase>
            <id>M10K</id>
            ...
        <wavabase>
        ...
    <list>


    To turn the XML back into Java objects in the Google App Engine we call


    List<WavaBase> wavaBases = (List<WavaBase>)xstream.fromXML(inputStream);

    By-passing the issues with object stream handling.

    Using this approach has allowed me to load the data I need into my first GAE application Wava Calculator which is a simple GWT/GXT application for calculating age adjusted times for runners.

    1 comment:

    1. Hi, I tried your example

      java.security.AccessControlException: access denied (java.io.SerializablePermission enableSubclassImplementation)
      at java.security.AccessControlContext.checkPermission(AccessControlContext.java:323)
      at java.security.AccessController.checkPermission(AccessController.java:546)
      at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)

      ReplyDelete