Xml – How to execute XPath one-liners from shell

cross platformshellxmlxpath

Is there a package out there, for Ubuntu and/or CentOS, that has a command-line tool that can execute an XPath one-liner like foo //element@attribute filename.xml or foo //element@attribute < filename.xml and return the results line by line?

I'm looking for something that would allow me to just apt-get install foo or yum install foo and then just works out-of-the-box, no wrappers or other adaptation necessary.

Here are some examples of things that come close:

Nokogiri. If I write this wrapper I could call the wrapper in the way described above:

#!/usr/bin/ruby

require 'nokogiri'

Nokogiri::XML(STDIN).xpath(ARGV[0]).each do |row|
  puts row
end

XML::XPath. Would work with this wrapper:

#!/usr/bin/perl

use strict;
use warnings;
use XML::XPath;

my $root = XML::XPath->new(ioref => 'STDIN');
for my $node ($root->find($ARGV[0])->get_nodelist) {
  print($node->getData, "\n");
}

xpath from XML::XPath returns too much noise, -- NODE -- and attribute = "value".

xml_grep from XML::Twig cannot handle expressions that do not return elements, so cannot be used to extract attribute values without further processing.

EDIT:

echo cat //element/@attribute | xmllint --shell filename.xml returns noise similar to xpath.

xmllint --xpath //element/@attribute filename.xml returns attribute = "value".

xmllint --xpath 'string(//element/@attribute)' filename.xml returns what I want, but only for the first match.

For another solution almost satisfying the question, here is an XSLT that can be used to evaluate arbitrary XPath expressions (requires dyn:evaluate support in the XSLT processor):

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
    xmlns:dyn="http://exslt.org/dynamic" extension-element-prefixes="dyn">
  <xsl:output omit-xml-declaration="yes" indent="no" method="text"/>
  <xsl:template match="/">
    <xsl:for-each select="dyn:evaluate($pattern)">
      <xsl:value-of select="dyn:evaluate($value)"/>
      <xsl:value-of select="'&#10;'"/>
    </xsl:for-each> 
  </xsl:template>
</xsl:stylesheet>

Run with xsltproc --stringparam pattern //element/@attribute --stringparam value . arbitrary-xpath.xslt filename.xml.

Best Answer

You should try these tools :

xmlstarlet : can edit, select, transform... Not installed by default, xpath1
xmllint : often installed by default with libxml2-utils, xpath1 (check my wrapper to have --xpath switch on very old releases and newlines delimited output (v < 2.9.9)
xpath : installed via perl's module XML::XPath, xpath1
xml_grep : installed via perl's module XML::Twig, xpath1 (limited xpath usage)
xidel: xpath3
saxon-lint : my own project, wrapper over @Michael Kay's Saxon-HE Java library, xpath3

xmllint comes with libxml2-utils (can be used as interactive shell with the --shell switch)

xmlstarlet is xmlstarlet.

xpath comes with perl's module XML::Xpath

xml_grep comes with perl's module XML::Twig

xidel is xidel

saxon-lint using SaxonHE 9.6 ,XPath 3.x (+retro compatibility)

Ex :

xmllint --xpath '//element/@attribute' file.xml
xmlstarlet sel -t -v "//element/@attribute" file.xml
xpath -q -e '//element/@attribute' file.xml
xidel -se '//element/@attribute' file.xml
saxon-lint --xpath '//element/@attribute' file.xml

Related Solutions

Ruby – How to call shell commands from Ruby

This explanation is based on a commented Ruby script from a friend of mine. If you want to improve the script, feel free to update it at the link.

First, note that when Ruby calls out to a shell, it typically calls /bin/sh, not Bash. Some Bash syntax is not supported by /bin/sh on all systems.

Here are ways to execute a shell script:

cmd = "echo 'hi'" # Sample string that can be used

Kernel#` , commonly called backticks – `cmd`

This is like many other languages, including Bash, PHP, and Perl.

Returns the result (i.e. standard output) of the shell command.

Docs: http://ruby-doc.org/core/Kernel.html#method-i-60
```
value = `echo 'hi'`
value = `#{cmd}`
```
Built-in syntax, %x( cmd )

Following the x character is a delimiter, which can be any character. If the delimiter is one of the characters (, [, {, or <, the literal consists of the characters up to the matching closing delimiter, taking account of nested delimiter pairs. For all other delimiters, the literal comprises the characters up to the next occurrence of the delimiter character. String interpolation #{ ... } is allowed.

Returns the result (i.e. standard output) of the shell command, just like the backticks.

Docs: https://docs.ruby-lang.org/en/master/syntax/literals_rdoc.html#label-Percent+Strings
```
value = %x( echo 'hi' )
value = %x[ #{cmd} ]
```
Kernel#system

Executes the given command in a subshell.

Returns true if the command was found and run successfully, false otherwise.

Docs: http://ruby-doc.org/core/Kernel.html#method-i-system
```
wasGood = system( "echo 'hi'" )
wasGood = system( cmd )
```
Kernel#exec

Replaces the current process by running the given external command.

Returns none, the current process is replaced and never continues.

Docs: http://ruby-doc.org/core/Kernel.html#method-i-exec
```
exec( "echo 'hi'" )
exec( cmd ) # Note: this will never be reached because of the line above
```

Here's some extra advice: $?, which is the same as $CHILD_STATUS, accesses the status of the last system executed command if you use the backticks, system() or %x{}. You can then access the exitstatus and pid properties:

$?.exitstatus

For more reading see:

Bash – How to check if a directory exists in a Bash shell script

To check if a directory exists in a shell script, you can use the following:

if [ -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY exists.
fi

Or to check if a directory doesn't exist:

if [ ! -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
fi

However, as Jon Ericson points out, subsequent commands may not work as intended if you do not take into account that a symbolic link to a directory will also pass this check. E.g. running this:

ln -s "$ACTUAL_DIR" "$SYMLINK"
if [ -d "$SYMLINK" ]; then 
  rmdir "$SYMLINK" 
fi

Will produce the error message:

rmdir: failed to remove `symlink': Not a directory

So symbolic links may have to be treated differently, if subsequent commands expect directories:

if [ -d "$LINK_OR_DIR" ]; then 
  if [ -L "$LINK_OR_DIR" ]; then
    # It is a symlink!
    # Symbolic link specific commands go here.
    rm "$LINK_OR_DIR"
  else
    # It's a directory!
    # Directory command goes here.
    rmdir "$LINK_OR_DIR"
  fi
fi

Take particular note of the double-quotes used to wrap the variables. The reason for this is explained by 8jean in another answer.

If the variables contain spaces or other unusual characters it will probably cause the script to fail.

Best Answer

Related Solutions

Ruby – How to call shell commands from Ruby

Bash – How to check if a directory exists in a Bash shell script

Related Topic