Distributed XQuery using RPC

XRPC

XRPC is the distributed XQuery extension introduced by MonetDB/XQuery. It works over the web, i.e. it uses the HTTP protocol, and sends so-called SOAP messages, which contain data in an XRPC XML format.

Topics on the use of XRPC:

Distributed XQuery between multiple XQuery database servers
sending SOAP queries directly from your browser (or an application)
configuring the built-in HTTP Server (e.g. security settings)

Additional information can be found in the Reference Manual.

Distributed XQuery with XRPC

XRPC adds the execute at syntax to the XQuery language, which allows to call remote XQuery functions on other XML database servers. You can call any user-defined XQuery function, declared in an XQuery module, this way.

First, you must define the query you want to execute in an XQuery module and store it in a place such that both your own XRPC client and the XRPC server running on the remote host, on which you want to run your query, can access the module definition file.

Let's define an example module xrpc-mod.xq. with a simple function that sums two integers:

module namespace foo = "xrpc-test-function";

declare function foo:add($v1 as xs:integer, 
                         $v2 as xs:integer) as xs:integer
{
    $v1 + $v2
};

We can now write a query with an XRPC call to execute the above function from a mclient -lx prompt:

import module namespace test = "xrpc-test-function"
                at "http://www.monetdb.nl/XQuery/files/xrpc-mod.xq";

execute at {"localhost"} {test:add(100, 200)}

Of course, executing a function over XRPC at your localhost is not very exciting (you could as well just call the function directly). However, if you start a second Mserver on another machine, and replace localhost by that machine name, you will invoke the function remotely.

Complex Example

Anywhere in an XQuery, where a user-defined function is called, you can also call it remotely by wrapping it in an execute at. Thus, remote function application can be nested in for-loops, element construction, or even a function:

import module namespace test = "xrpc-test-function"
              at "http://www.monetdb.nl/XQuery/files/xrpc-mod.xq"

declare function bar($n as xs:integer) as xs:integer
{
   sum(for $i in 1 to $n,
           $host in ("foo", "bar"), 
       return (execute at {"localhost"} {test:add($i, 2)},
               execute at {$host} {test:add($i * 100, 200)}))
};
bar(100)

While the above suggests that 100 XRPC requests would be sent to both hosts foo and bar, MonetDB/XQuery is optimized to send only a single SOAP request to both of them, keeping even this complex query highly efficient.

XRPC as a SOAP API

The XRPC communication protocol is SOAP, and so it is easy to have your applications directly send XRPC requests as well. In this way, XRPC can also be used as a client-server API.

You can test sending SOAP requests via a demo included in each MonetDB/XQuery distribution. NOTE: before clicking the below test page link, first start an MonetDB/XQuery server on your local machine!

open the on-line XRPC demo

The demo shows how HTML pages can use the XRPC Javascript API to send XRPC requests directly to MonetDB/XQuery. The Javascript GUI allows you to write AJAX-style HTML pages with embedded XRPC requests, without need for server-side application code (asp, php, etc.).

The Administrative GUI is another example of an AJAX application that uses XRPC for database access. You can inspect its HTML sources to see how it works.

Many programming languages support sending and receiving of SOAP requests over HTTP. Sending XRPC requests to MonetDB/XQuery is easy; just make sure your XRPC request messages conform to the XRPC XML format described in the extended documentation of XRPC.

The Built-in HTTP server

As XRPC uses SOAP over HTTP, MonetDB/XQuery comes standard with a lightweight HTTP server, which serves the following purposes:

handling XRPC SOAP requests (i.e. HTTP POST request with local URI xrpc).
serving out documents. If you have a document X.XML stored at your MonetDB/XQuery server at host, you can just access this document under the URI http://host:50001/xrpc/doc/X.XML. That is, MonetDB/XQuery handles HTTP GET requests that start with xrpc/doc/ by serializing the XML document stored in its database. Note that 50001 is the default xrpc_port (this is mapi_port+1). You can override the xrpc_port value in MonetDB.conf.
any other GET request is just executed normally, i.e. for serving out files stored in the "${prefix}/share/MonetDB/xrpc" directory. Note that currently, the HTTP server does not allow directory listing, nor does it support CGI scripting.

HTTP directories and URLs

MonetDB/XQuery provides a standard place for putting XQuery Modules that are callable from outside: datadir/MonetDB/xrpc/export/. Here datadir is defined in the MonetDB.conf configuration file.

We have already put the example xrpc-mod.xq in the MonetDB distribution, such that you can also refer to it at the local file URL file://datadirMonetDB/xrpc/export/xrpc-mod.xq A disadvantage of this approach is that clients calling a function must be aware of the MonetDB installation directory (which determines datadir).

Because datadir/MonetDB/xrpc/export/ is served out by the built-in HTTP server, we can also use the URL http://127.0.0.1:50001/export/xrpc-mod.xq to refer to our example module.

Finally, from within XQuery queries there is support for the xrpc://host/URI URI naming scheme. This is a shorthand for http://host:xrpc_port/xrpc/URI, leaving it to MonetDB/XQuery to determine the actual port number.

Security Settings

For security reasons, the XRPC server does not execute functions unless they are trusted. It will only execute functions defined in those XQuery modules whose location (given by the at-hint) starts with one of the values listed in the MonetDB variable xrpc_trusted, defined in the MonetDB.conf file. Note that having xrpc_trusted empty has the special meaning that any XQuery module can be executed.