Sponger ontology mappers peform the the task of generating RDF instance data from extracted metadata (non-RDF) using ontologies associated with a given data source type. They are typically XSLT (using GRDDL or an in-built Virtuoso mapping scheme) or Virtuoso PL based. Virtuoso comes preconfigured with a large range of ontology mappers contained in one or more Sponger cartridges. Nevertheless you are free to create and add your own cartridges, ontology mappers, or metadata extractors.

Figure 9: Sponger architecture

Below is an extract from the stylesheet /DAV/VAD/rdf_cartridges/xslt/flickr2rdf.xsl, used for extracting metadata from Flickr images. Here, the template combines RDF metadata extraction and ontology mapping based on the FOAF and Dublin Core ontologies.

<xsl:template match="owner">
  <rdf:Description rdf:nodeID="person">
  <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/#Person" />
  <xsl:if test="@realname != ''">
  <foaf:name><xsl:value-of select="@realname"/></foaf:name>
  </xsl:if>
  <foaf:nick><xsl:value-of select="@username"/></foaf:nick>
  </rdf:Description>
  </xsl:template>
  <xsl:template match="photo">
  <rdf:Description rdf:about="{$baseUri}">
  <rdf:type rdf:resource="http://www.w3.org/2003/12/exif/ns/IFD"/>
  <xsl:variable name="lic" select="@license"/>
  <dc:creator rdf:nodeID="person" />
  ...

Cartridge Registry

Once a Sponger cartridge has been developed it must be plugged into the SPARQL engine by registering it in the Cartridge Registry, i.e. by adding a record in the table DB.DBA.SYS_RDF_MAPPERS, either manually via DML, or more easily through Conductor (Virtuoso's browser-based administration console), which provides a UI for adding your own cartridges. Sponger configuration using Conductor is described in detail later. For the moment, we'll focus on outlining the broad architecture of the Sponger.

The SYS_RDF_MAPPERS table definition is as follows:

create table DB.DBA.SYS_RDF_MAPPERS (
  RM_ID integer identity,         -- cartridge ID, designate order of execution
  RM_PATTERN varchar,             -- a REGEX pattern to match URL or MIME type
  RM_TYPE varchar default 'MIME', -- which property of the current resource to match: MIME or URL
  RM_HOOK varchar,                -- fully qualified PL function name e.g. DB.DBA.MY_CARTRIDGE_FUNCTION
  RM_KEY  long varchar,           -- API specific key to use
  RM_DESCRIPTION long varchar,    -- Cartridge description (free text)
  RM_ENABLED integer default 1,   -- 0 or 1 integer flag to include or exclude the given cartridge from Sponger processing chain
  RM_OPTIONS any,                 -- cartridge specific options
  RM_PID integer identity,        -- for internal use only
  primary key (RM_HOOK)
  );

Cartridge Invocation

The Virtuoso SPARQL processor supports IRI dereferencing via the Sponger. Thus, if the SPARQL query contains references to non-default graph URIs the Sponger goes out (via HTTP) to grab the RDF data sources exposed by the data source URIs and then places them into local storage (as Default or Named Graphs depending on the SPARQL query). Since SPARQL is RDF based, it can only process RDF-based structured data, serialized using RDF/XML, Turtle or N3 formats. As a result, when the SPARQL processor encounters a non-RDF data source, a call to the Sponger is triggered. The Sponger then locates the appropriate cartridge for the data source type in question, resulting in the production of SPARQL-palatable RDF instance data. If none of the registered cartridges are capable of handling the received content type, the Sponger will attempt to obtain RDF instance data via the in-built WebDAV metadata extractor.

Sponger cartridges are invoked during the aforementioned pipeline as follows:

When the SPARQL processor dereferences a URI, it plays the role of an HTTP user agent (client) that makes a content type specific request to an HTTP server via the HTTP request's Accept headers. The following then occurs:

• If the content type returned is RDF then no further transformation is needed and the process stops. For instance, when consuming an (X)HTML document with a GRDDL profile, the profile URI points to a data provider that simply returns RDF instance data.
• If the content type is not RDF (i.e. application/rdf+xml or text/rdf+n3 ), for instance 'text/plain', the Sponger looks in the Cartridge Registry iterating over every record for which the RM_ENABLED flag is true, with the look-up sequence ordered on the RM_ID column values. For each record, the processor tries matching the content type or URL against the RM_PATTERN value and, if there is match, the function specified in RM_HOOK column is called. If the function doesn't exist, or signals an error, the SPARQL processor looks at next record.
• If the hook returns zero, the next cartridge is tried. (A cartridge function can return zero if it believes a subsequent cartridge in the chain is capable of extracting more RDF data.)
• If the result returned by the hook is negative, the Sponger is instructed that no RDF was generated and the process stops.
• If the hook result is positive, the Sponger is informed that structured data was retrieved and the process stops.
• If none of the cartridges match the source data signature (content type or URL), the built-in WebDAV metadata extractor's RDF generator is called.
  

Figure 10: Sponger cartridge invocation flowchart