Python – xml.parsers.expat.ExpatError on parsing XML

pythonxml

I am trying to parse XML with Python but not getting very far. I think it's due to wrong XML tree this API returns.

So this is what is returned by the GET request:

<codigo>3</codigo><valor></valor><operador>Dummy</operador>

The GET request goes here:

http://69.36.9.147:8090/clientes/SMS_API_OUT.jsp?codigo=ABCDEFGH&cliente=XX

This is the Python code I am using without any luck:

import urllib
from xml.dom import minidom

url = urllib.urlopen('http://69.36.9.147:8090/clientes/SMS_API_OUT.jsp?codigo=ABCDEFGH&cliente=XX')
xml = minidom.parse(url)
code = doc.getElementsByTagName('codigo')

print code[0].data

And this is the response I get:

xml.parsers.expat.ExpatError: junk after document element: line 1, column 18

What I need to do is retrieve the value inside the <codigo> element and place it in a variable (same for the others).

Best Answer

The main problem here is that the XML code being returned by that service doesn't include a root node, which is invalid. I fixed this by simply wrapping the output in a <root> node.

import urllib
from xml.etree import ElementTree

url = 'http://69.36.9.147:8090/clientes/SMS_API_OUT.jsp?codigo=ABCDEFGH&cliente=XX'
xmldata = '<root>' + urllib.urlopen(url).read() + '</root>'
tree = ElementTree.fromstring(xmldata)
codigo = tree.find('codigo').text

print codigo

You can use whatever parser you wish, but here I used ElementTree to get the value.

XML escape characters

There are only five:

"   &quot;
'   &apos;
<   &lt;
>   &gt;
&   &amp;

Escaping characters depends on where the special character is used.

The examples can be validated at the W3C Markup Validation Service.

Text

The safe way is to escape all five characters in text. However, the three characters ", ' and > needn't be escaped in text:

<?xml version="1.0"?>
<valid>"'></valid>

Attributes

The safe way is to escape all five characters in attributes. However, the > character needn't be escaped in attributes:

<?xml version="1.0"?>
<valid attribute=">"/>

The ' character needn't be escaped in attributes if the quotes are ":

<?xml version="1.0"?>
<valid attribute="'"/>

Likewise, the " needn't be escaped in attributes if the quotes are ':

<?xml version="1.0"?>
<valid attribute='"'/>

Comments

All five special characters must not be escaped in comments:

<?xml version="1.0"?>
<valid>
<!-- "'<>& -->
</valid>

CDATA

All five special characters must not be escaped in CDATA sections:

<?xml version="1.0"?>
<valid>
<![CDATA["'<>&]]>
</valid>

Processing instructions

All five special characters must not be escaped in XML processing instructions:

<?xml version="1.0"?>
<?process <"'&> ?>
<valid/>

XML vs. HTML

HTML has its own set of escape codes which cover a lot more characters.

Python – How to parse XML and count instances of a particular node attribute

I suggest ElementTree. There are other compatible implementations of the same API, such as lxml, and cElementTree in the Python standard library itself; but, in this context, what they chiefly add is even more speed -- the ease of programming part depends on the API, which ElementTree defines.

First build an Element instance root from the XML, e.g. with the XML function, or by parsing a file with something like:

import xml.etree.ElementTree as ET
root = ET.parse('thefile.xml').getroot()

Or any of the many other ways shown at ElementTree. Then do something like:

for type_tag in root.findall('bar/type'):
    value = type_tag.get('foobar')
    print(value)

And similar, usually pretty simple, code patterns.