R – Prepend BOM to XML response from Django

byte-order-markdjangoxml

I use Django's render_to_response to return an XML document. This particular XML document is intended for a flash-based charting library. The library requires that the XML document start with a BOM (byte order marker). How can I make Django prepent the BOM to the response?

It works to insert the BOM into the template, but it's inconvenient because Emacs removes it every time I edit the file.

I have tried to rewrite render_to_response as follows, but it fails because the BOM is being UTF-8 encoded:

def render_to_response(*args, **kwargs):
    bom = kwargs.pop('bom', False)
    httpresponse_kwargs = {'mimetype': kwargs.pop('mimetype', None)}
    s = django.template.loader.render_to_string(*args, **kwargs)
    if bom:
        s = u'\xef\xbb\xbf' + s
    return HttpResponse(s, **httpresponse_kwargs)

Best Answer

You're not really talking about a BOM (Byte Order Mark), since UTF-8 doesn't have a BOM. From your example code, the library expects the text to have 3 garbage bytes prepended for some inexplicable reason.

Your code is almost correct, but you must prepend the bytes as bytes, not characters. Try this:

def render_to_response(*args, **kwargs):
    bom = kwargs.pop('bom', False)
    httpresponse_kwargs = {'mimetype': kwargs.pop('mimetype', None)}
    s = django.template.loader.render_to_string(*args, **kwargs)
    if bom:
        s = '\xef\xbb\xbf' + s.encode("utf-8")
    return HttpResponse(s, **httpresponse_kwargs)

XML escape characters

There are only five:

"   &quot;
'   &apos;
<   &lt;
>   &gt;
&   &amp;

Escaping characters depends on where the special character is used.

The examples can be validated at the W3C Markup Validation Service.

Text

The safe way is to escape all five characters in text. However, the three characters ", ' and > needn't be escaped in text:

<?xml version="1.0"?>
<valid>"'></valid>

Attributes

The safe way is to escape all five characters in attributes. However, the > character needn't be escaped in attributes:

<?xml version="1.0"?>
<valid attribute=">"/>

The ' character needn't be escaped in attributes if the quotes are ":

<?xml version="1.0"?>
<valid attribute="'"/>

Likewise, the " needn't be escaped in attributes if the quotes are ':

<?xml version="1.0"?>
<valid attribute='"'/>

Comments

All five special characters must not be escaped in comments:

<?xml version="1.0"?>
<valid>
<!-- "'<>& -->
</valid>

CDATA

All five special characters must not be escaped in CDATA sections:

<?xml version="1.0"?>
<valid>
<![CDATA["'<>&]]>
</valid>

Processing instructions

All five special characters must not be escaped in XML processing instructions:

<?xml version="1.0"?>
<?process <"'&> ?>
<valid/>

XML vs. HTML

HTML has its own set of escape codes which cover a lot more characters.

Python – How to parse XML and count instances of a particular node attribute

I suggest ElementTree. There are other compatible implementations of the same API, such as lxml, and cElementTree in the Python standard library itself; but, in this context, what they chiefly add is even more speed -- the ease of programming part depends on the API, which ElementTree defines.

First build an Element instance root from the XML, e.g. with the XML function, or by parsing a file with something like:

import xml.etree.ElementTree as ET
root = ET.parse('thefile.xml').getroot()

Or any of the many other ways shown at ElementTree. Then do something like:

for type_tag in root.findall('bar/type'):
    value = type_tag.get('foobar')
    print(value)

And similar, usually pretty simple, code patterns.