Thursday, February 3, 2011

Converting UML Models to OWL - Part 1: The Approach


Convert UML to OWL - why would you ever want to do this? One reason suffices: many enterprise models, that serve as either standards or enterprise schemas, are specified in UML. Increasingly, there is interest in having content of UML models re-purposed in RDF/OWL and the need for RDF/OWL to interoperate with systems built from UML Models.

UML Models are notoriously hard to exchange between UML tools, let alone be transformed into OWL. The exchange format XMI is not only is difficult to understand but also has vendor-specific extensions. The vagaries of MOF, CMOF and EMOF create their own challenges. Nonetheless we have done transformations of UML to OWL. Using a model-based transformation approach, based on SPARQL Rules, XMI models of UML models can be converted to OWL. UML class diagrams can be represented in OWL without information loss. The inverse, however, is not true and will require another blog series.

UML to OWL - Part 1 Contents

Part 1 of the series explains the basis of the approach. The complete series of blogs, as currently conceived, is as follows:
  • Converting UML Models to OWL - Part 1: The Approach
  • Converting UML Models to OWL - Part 2: Transforming UML Models to OWL Using SPARQL
  • Converting UML Models to OWL - Part 3: Examples of Industry UML Model Transformations
The content of this blog is organized as follows:
  1. Goals, Objectives and Requirements
  2. Backgrounder on XMI
  3. Backgrounder on MOF
  4. Solution Outline
  5. Overview of Semantic XML
  6. OCMOF - the OWL Representation of CMOF
  7. How the Transformations from UML to OWL Work
  8. Generation of UML Metaclasses
  9. Generation of UML Classes
  10. Generation of UML Class Superclass Relationships
  11. Generation of UML Packages
  12. Generation of UML Package Relationships
  13. Performance
  14. Concluding Remarks
Readers who are very interested in the detailed technical approach, should read all sections of this blog in order. Those who just need to have an overview of the approach could skip sections 9 through 12. Those who have deep knowledge of XMI and MOF may want to skip sections 2 and 3, but I would welcome their feedback on the accuracy of my statements.

Note that some diagrams may be too small to be viewed in the body of the document. Clicking on such a diagram will open a new window with a larger depiction of the diagram.

Goals, Objectives and Requirements

The OWL Models must faithfully represent packages and the logical models or class diagrams. Out of scope, currently, are all of the other UML models such as Interaction Diagrams and State Diagrams. The approach must be able to convert UML by processing XMI files from specific tools. This requires a strategy for converting from the XML structures of XMI to OWL models.

Backgrounder on XMI

XMI, the XML Metadata Interchange standard is a serialization format for UML Models. The main purpose of XMI is to define how the XML elements are organized within an XMI file. The XMI spec also defines a mechanism for how one XMI element references another, within and across XMI files. Such a mechanism is needed as it is a legal scenario for a single UML model to be serialized to more than one XMI file.

Backgrounder on MOF

MOF began at the time of CORBA and the need for IDL interfaces. MOF 1.4 resulted in its mapping to Java being codified in the Java Community Process (JCP) as the Java Metadata Initiative (JMI). MOF 2.0 was developed in tandem with UML 2.0. The separation of MOF into EMOF and CMOF was motivated by the influence of EMF's Ecore, and model-driven Java development. CMOF was more the motivation of meta model developers. CMOF stands for Complete Meta Object Facility and is an OMG standard for the UML 2 model interchange. More information can be found at this page on the OMG Website.

CMOF includes fully fledged associations, association generalization, property subsetting and redefinition, derived unions, and package merge. Typical XMI container structures look like the example below, from the CMOF UML Infrastructure Model. The basic idea is that a packagedElement owns other elements. A type attribute specifies the type of the packagedElement.
Things get a little busy with how IDs are used for associations and their member ends. That complication, we can leave for Part 2.


<?xml version="1.0" encoding="UTF-8"?>
<xmi:XMI xmi:version="2.1" xmlns:xmi="http://schema.omg.org/spec/XMI/2.1"
xmlns:cmof="http://schema.omg.org/spec/MOF/2.0/cmof.xml">
<cmof:Package xmi:id="_0" name="InfrastructureLibrary">
<ownedMember xmi:type="cmof:Package" xmi:id="Core" name="Core">
<ownedMember xmi:type="cmof:Package"
xmi:id="Core-Abstractions" name="Abstractions">
<packageImport xmi:type="cmof:packageImport"
xmi:id="Core-Abstractions-_packageImport.0"
importedPackage="Core-PrimitiveTypes"
importingNamespace="Core-Abstractions"/>
<ownedMember xmi:type="cmof:Package"
xmi:id="Core-Abstractions-Ownerships" name="Ownerships">
<packageImport xmi:type="cmof:packageImport"
xmi:id="Core-Abstractions-Ownerships-_packageImport.0"
importedPackage="Core-Abstractions-Elements"
importingNamespace="Core-Abstractions-Ownerships"/>
<ownedMember xmi:type="cmof:Class"
xmi:id="Core-Abstractions-Ownerships-Element" name="Element" isAbstract="true">
<ownedComment xmi:type="cmof:Comment"
xmi:id="Core-Abstractions-Ownerships-Element-_ownedComment.0"
annotatedElement="Core-Abstractions-Ownerships-Element">
<body>An element is a constituent of a model.
As such, it has the capability of owning other elements.</body>
</ownedComment>
<ownedRule xmi:type="cmof:Constraint"
xmi:id="Core-Abstractions-Ownerships-Element-not_own_self"
name="not_own_self" constrainedElement="Core-Abstractions-Ownerships-Element"
namespace="Core-Abstractions-Ownerships-Element">
<ownedComment xmi:type="cmof:Comment"
xmi:id="Core-Abstractions-Ownerships-Element-not_own_self-_ownedComment.0"
annotatedElement="Core-Abstractions-Ownerships-Element-not_own_self">
<body>An element may not directly or indirectly own itself.</body>
</ownedComment>
<specification xmi:type="cmof:OpaqueExpression"
xmi:id="Core-Abstractions-Ownerships-Element-not_own_self-_specification">
<language>OCL</language>
<body>not self.allownedElements()->includes(self)</body>
</specification>
</ownedRule>
...
<ownedAttribute xmi:type="cmof:Property"
xmi:id="Core-Abstractions-Ownerships-Element-ownedElement"
name="ownedElement" type="Core-Abstractions-Ownerships-Element"
upper="*" lower="0" isReadOnly="true" isDerived="true"
isDerivedUnion="true" isComposite="true"
association="Core-Abstractions-Ownerships-A_ownedElement_owner">
<ownedComment xmi:type="cmof:Comment"
xmi:id="Core-Abstractions-Ownerships-Element-ownedElement-_ownedComment.0"
annotatedElement="Core-Abstractions-Ownerships-Element-ownedElement">
<body>The Elements owned by this element.</body>
</ownedComment>
</ownedAttribute>
...

Figure 1: A sample of XMI


For more background on the history of MOF the following references may be of value: MOFLON, and Wikipedia.

Top

Solution Outline

Model-based transformation is the central idea of the approach. To implement it we have developed a metamodel of CMOF in OWL. Our strategy is to get out of XML into RDF Triples as soon as possible. Using an ontology of XML we convert XMI into a composite model of triples. XML is a simple enough structure for the composite object pattern - elements contain elements and elements have attributes. XML elements and attributes that make up the XMI file are transformed into OWL instances of the CMOF metamodel. Once we have the XMI in triples we can map constructs to classes and properties of a CMOF metamodel. This model then serves as the generator for model-based transformations to an OWL model of the UML.

Once these instances are loaded as "raw" RDF, rules fire to perform the transformations. Rules are associated with classes to ensure that instances of those classes are processed in an execution sequence. Using SPARQL Rules (SPIN), instances of a class are each processed through a binding mechanism specified by ?this variable. SPARQL Rules can be considered an approach that is similar to, or can be compared with, UML's Object Constraint Language (OCL) and the Query/View/Transformation (QVT) approach to transformations.

The benefits of the OWL and SPARQLRules model-based approach to transformation are:
  • Intimacy of the rules with RDF/OWL - triples are evaluated directly
  • Understandability - rules are smaller and expressed in the relevant contexts of the model
  • Enhanced Performance - evaluation of rules is localized to relevant instances
  • Customizability and Evolvability – transformations can be changed by modifying models and/or SPARQL rules
  • Ease of maintenance - rules are associated with the constructs they operate over
Top

Overview of Semantic XML

XMI is imported into the CMOF metamodel using TopBraid Composer's Semantic XML as a mapping method. With Semantic XML, TopBraid can automatically generate an OWL/RDF ontology from any XML file. Each distinct XML element name is mapped into a class, and the elements themselves become instances of those classes. A datatype property is generated for each attribute. Nesting of XML elements is represented in OWL using a composite:child property - an object pattern in OWL that is described at this blog entry.

The key idea of Semantic XML is that each of the generated OWL classes and datatype properties is annotated with an annotation property, sxml:element and sxml:attribute, respectively. These properties relate the OWL concepts to the XML serialization. Note that these annotations are also used if an OWL model needs to be serialized back to XML format.

If you import an XML file into an ontology that already contains classes and properties with Semantic XML annotations, then the loader will reuse those. The mapping is bi-directional and loss-less so that files can be loaded, manipulated and saved without losing structural information.

A video explaining how Semantic XML works is available at this link.

OCMOF - the OWL Representation of CMOF

The strategy for the transformation can be summarized as follows:
  • Use OWL classes to represent XMI Element Types
  • Use SPARQL Rules on those classes to generate CMOF Metaclasses
  • Use Metaclasses to make OWL Classes that represent the UML Model
An OWL metamodel of CMOF represents the kinds of containers, elements and attributes shown above. The metamodel was built by studying the UML Metamodel of UML 2.0 - the original motivation for this was to have an automated way of dealing with changes to UML. That will be a future consideration, for now this has proven to be a valuable way of doing verification and validation. The UML metamodel will be covered in Part 2 of this blog series, for Part 1, it is instructive perhaps to show a small piece of the XMI. Below is the XMI for Basic-Property from the UML model infrastructure.cmof.xmi.
<ownedAttribute xmi:type="cmof:Property"
xmi:id="Core-Basic-Class-ownedAttribute" name="ownedAttribute"
type="Core-Basic-Property" isOrdered="true"
upper="*" lower="0" isComposite="true"
association="Core-Basic-A_ownedAttribute_class">
<ownedComment xmi:type="cmof:Comment"
xmi:id="Core-Basic-Class-ownedAttribute-_ownedComment.0"
annotatedElement="Core-Basic-Class-ownedAttribute">
<body>The attributes owned by a class.
These do not include the inherited attributes.
Attributes are represented by instances of Property.</body>
</ownedComment>
</ownedAttribute>

Figure 2: A fragment of the XMI for the UML metamodel


As a example of XMI element mappings, the sxml:element maps the XMI element for ocmof:ownedAttribute as shown in the Turtle extract from the OWL model below.


ocmof:ownedAttribute
a owl:Class ;
rdfs:label "Attribute"^^xsd:string ;
rdfs:subClassOf ocmof:TypedThing , ocmof:NamedThing ;
rdfs:subClassOf
[ a owl:Restriction ;
owl:maxCardinality "1"^^xsd:nonNegativeInteger ;
owl:onProperty ocmof:isComposite
] ;
rdfs:subClassOf
[ a owl:Restriction ;
owl:maxCardinality "1"^^xsd:nonNegativeInteger ;
owl:onProperty ocmof:type
] ;
rdfs:subClassOf
[ a owl:Restriction ;
owl:maxCardinality "1"^^xsd:nonNegativeInteger ;
owl:onProperty ocmof:isDerivedUnion
] ;
rdfs:subClassOf
[ a owl:Restriction ;
owl:maxCardinality "1"^^xsd:nonNegativeInteger ;
owl:onProperty ocmof:isReadOnly
] ;
rdfs:subClassOf
[ a owl:Restriction ;
owl:maxCardinality "1"^^xsd:nonNegativeInteger ;
owl:onProperty ocmof:default
] ;
sxml:element "ownedAttribute"^^xsd:string .

Figure 3: ocmof:ownedAttribute in Turtle

The last line, sxml:element "ownedAttribute"^^xsd:string, is the mapping.

As a example of XMI attribute mappings, the sxml:attribute maps the XMI attribute for ocmof:isOrdered as shown in the Turtle extract from the OWL model below.


ocmof:isOrdered
a owl:DatatypeProperty ;
rdfs:domain ocmof:ownedAttribute ,
ocmof:ownedParameter ,
ocmof:OwnedEnd ;
rdfs:label "is ordered"^^xsd:string ;
rdfs:range xsd:boolean ;
sxml:attribute "isOrdered"^^xsd:string .

Figure 4: ocmof:isOrdered in Turtle

The last line, sxml:attribute "isOrdered"^^xsd:string, is the mapping.

The transformation to OWL results in the following class for uml:Core-Basic-Property.
Clicking on the image will open a larger image in a new window.

Core-Basic-Property Classes

Figure 5: A Generated Metaclass Example - uml:Core-Basic-Property

The diagram shows how the datatype properties of the class uml:Core-Basic-Property correspond to the XMI attributes given in the above fragment. For example isComposite becomes the property hasBooleanIsComposite. The prefix hasBoolean is customizable.

First an OWL model of CMOF XML Elements is used to generate instances of metaclasses to build OWL Classes for XMI Elements. The namespace prefix of ocmof has been used to denote all modeling constructs that makeup the CMOF metamodel. The prefix cmof is the namespace for all constructs generated from the import of the XMI files.

In the diagram below, we show the main classes of the metamodel. Classes like NamedThing and TypedThing have been introduced to optimize the work of the transformers. Constructs in XMI can typically be both named and typed. This kind of multiple inheritance is no problem for the transformations. The diagram is a partial view only. Clicking on the image will open a larger image in a new window.


OCMOF Classes

Figure 6: Some of the classes of the CMOF OWL model

As an alternate view, the diagram that follows is an HTML report of NamedThing in TopBraid Composer. This is automatically generated using SPARQL Web Pages (aka UISPIN)

NamedThing Summary generated by UISPIN

Figure 7: OCMOF NamedThing - an abstract class for the transformations

The diagram below shows more details of some ownedElements. Note how attributes of each of these classes relate to CMOF constructs.

Owned Elements

Figure 8: Some "ownedElement" OWL Classes in the OCMOF model

These ocmof classes serve as the starting point for generating ocmof meta-classes and instances of these classes that become the UML model transformed into OWL. The figure below shows the main metaclasses that are generated by rules on the ocmof classes.

CMOF Metaclasses

Figure 9: The key Meta-classes of the CMOF OWL model



How the Transformations from UML to OWL Work

Model-based transformations use rules associated with OWL Classes. OWL Metaclasses are built using a SPARQL rule for instances of TypedThing. The names of the metaclasses are determined from the value of the xmi:type attribute. A number of SPARQL Rules are defined on TypeThing. Priorities are set by the alphabetic ordering given by the first comment line of the rule. These rules look after the generation of:
  1. UML Metaclasses
  2. UML Classes
  3. UML Class Superclass Relationships
  4. UML Packages
  5. UML Package Relationships

Each rule will now be described.



Generation of UML Metaclasses

The first task is to create a metaclass and class for every type of element in the ingested XMI file. This is done using the SPARQL Rule below:

# STEP CMOF-SR-001 make UML Metaclass from CMOF type
CONSTRUCT {
?metaClassURI a rdfs:Class .
?metaClassURI rdfs:subClassOf cmof:MetaClass .
?metaClassURI rdfs:label ?metaClassLabel .
?typeURI a owl:Class .
?typeURI a ?metaClassURI .
?typeURI rdfs:subClassOf uml:Construct .
?typeURI rdfs:label ?classLabel .
}
WHERE {
?this xmi:type ?type .
FILTER (?type != "cmof:Property") .
BIND (o2o:localNameOfQName(?type) AS ?name) .
BIND (fn:concat("CMOF ", ?name) AS ?metaClassLabel) .
BIND (fn:concat("UML ", ?name) AS ?classLabel) .
BIND (xmi.common:makeUML-URI(?name) AS ?typeURI) .
BIND (xmi.common:makeCMOF-URI(?name) AS ?metaClassURI) .
}

Figure 10: The SPARQL Rules that make the metaclasses in the OCMOF model

What is going on in these rules? First we explain the "where" clause.

?this xmi:type ?type binds ?this to an instance of TypedThing. For each instance the rule is evaluated.

FILTER (?type != "cmof:Property") blocks further evaluation of the rule if the instance is of type cmof:Property. The reason for this will be explained in Part 2.

BIND (o2o:localNameOfQName(?type) AS ?name) extracts the name of the type from the QName.

BIND (fn:concat("CMOF ", ?name) AS ?metaClassLabel ) builds a label for the metaclass. The function fn:concat is from the JENA SPARQL Library. We use it here to prepend "CMOF" to the name we get from the type of the XMI Element.


BIND (fn:concat("UML ", ?name) AS ?classLabel) makes a class label from the name. We will be constructing both a metaclass and a class from the XMI type. We build a metaclass in order to say what kind of things can happen on the classes. In other words, the generated OWL model is a 3-level ontology. Likewise here we build a label for the UML Class.

BIND (xmi.common:makeUML-URI(?name) AS ?typeURI) builds a URI for the UML Class corresponding to type. This uses a function call to xmi.common:makeUML-URI whose job it is to build the correct namespace path for a UML construct URI. The implementation is shown below.

SELECT ?uri
WHERE {
BIND (xmi.common:baseURI() AS ?baseURI) .
BIND (smf:buildURI("{?baseURI}#{?arg1}") AS ?uri) .
}

where,
smf:buildURI("{?baseURI}#{?arg1}")) builds a URI for the name given in ?arg1 with a base URI supplied by the function xmi.common:baseURI().

BIND (xmi.common:makeCMOF-URI(?name) AS ?metaClassURI ) builds a URI for the metaclass corresponding to type. Likewise this constructs a namspace path for CMOF constructs.


Next we explain what is happening in the head of the rule with the Construct statements. These statements use the generated URIs to create instances of meta-classes and classes.


?metaClassURI a rdfs:Class
gives the metaClass its type.


?metaClassURI rdfs:subClassOf cmof:MetaClass
specifies that the metaclass is a sub-class of cmof:MetaClass - an abstract metaclass for all cmof classes.

?metaClassURI rdfs:label ?metaClassLabel
gives the metaclass a human label.

?typeURI a owl:Class
gives the UML Class a type

?typeURI a ?metaClassURI
gives the UML Class a more specific type so that it can have more properties than owl:Class provides.

?typeURI rdfs:subClassOf uml:Construct
specifies that the UML Class is a subclass of the abstract OWL Class uml:Construct/

?typeURI rdfs:label ?classLabel
gives the UML Class a human label.



Generation of UML Classes

Once we have the necessary metaclasses we can begin the work of creating instances of those classes. These instances will, of course, be classes (the meta-world can get confusing). This work is done the the SPARQL Rule below.

# STEP CMOF-SR-002 make UML Classes from CMOF elements
CONSTRUCT {
?type a rdfs:Class .
?type rdfs:subClassOf cmof:MetaClass .
?class a ?type .
?class rdfs:label ?name .
?class ocmof:hasCMOFbasis ?this .
?superURI a owl:Class .
?superURI a cmof:CategoryClass .
?superURI rdfs:label ?super .
?subURI a owl:Class .
?subURI a cmof:CategoryClass .
?subURI rdfs:subClassOf ?superURI .
?subURI rdfs:label ?sub .
?class rdfs:subClassOf ?mySuperClass .
}
WHERE {
?this xmi:type "cmof:Class" .
?this xmi:id ?name .
BIND (o2o:pathPart(?name, "-") AS ?path) .
OPTIONAL {
?path o2o:pairHyphenIncrementally ( ?super ?sub ) .
BIND (xmi.common:makeUML-URI(smf:buildString("CLASSES_{?super}")) AS ?superURI) .
BIND (xmi.common:makeUML-URI(smf:buildString("CLASSES_{?sub}")) AS ?subURI) .
BIND (xmi.common:makeCMOF-Resource("cmof:Class") AS ?type) .
BIND (xmi.common:makeUML-URI(?name) AS ?class) .
} .
BIND (xmi.common:makeUML-URI(smf:buildString("CLASSES_{?path}")) AS ?mySuperClass) .
}

Figure 11: The SPARQL Rules that make UML Classes

More details of this transformation will be given in Part 2 of this blog series. An interesting aspect of this particular rule to mention now is how it builds deep inheritance structures by using Property Functions to recurse over hyphenated names (more on the use of Property Functions, also known as Magic Properties, with TopBraid Composer can be found at this blog entry). These hyphenated names occur throughout the XMI metamodel of UML. For example Core-Basic-Class looks like:
<ownedMember xmi:type="cmof:Class" xmi:id="Core-Basic-Class" name="Class" superClass="Core-Basic-Type">
<ownedComment xmi:type="cmof:Comment" xmi:id="Core-Basic-Class-_ownedComment.0"
annotatedElement="Core-Basic-Class">
<body>A class is a type that has objects as its instances.</body>
</ownedComment>
<ownedAttribute xmi:type="cmof:Property" xmi:id="Core-Basic-Class-isAbstract"
name="isAbstract" type="Core-PrimitiveTypes-Boolean" default="false">
<ownedComment xmi:type="cmof:Comment" xmi:id="Core-Basic-Class-isAbstract-_ownedComment.0" annotatedElement="Core-Basic-Class-isAbstract">
<body>True when a class is abstract.</body>
</Attribute>

Figure 12: Example of Hyphenated Names in the UML Metamodel


How this is done in the SPARQL Rule is explained briefly below.


In the SPARQL Rule shown above in figure 11, the statement in the tail: ?path o2o:pairHyphenIncrementally ( ?super ?sub ) is a Property Function that returns two results: ?super and ?sub for every hypenated pair.
for each pair the statement: ?subURI rdfs:subClassOf ?superURI in the head of the rule builds superclass relationships.



Generation of UML Class Superclass Relationships

Once we have all of the UML Classes, the next rule can build the rdfs:subClassOf relationships.
# STEP CMOF-SR-005 - fixup the superclass of the root Classes
CONSTRUCT {
?class rdfs:subClassOf uml:Class .
}
WHERE {
?class a cmof:CategoryClass .
BIND (afn:localname(?class) AS ?className) .
FILTER fn:starts-with(?className, "CLASSES_") .
NOT EXISTS {
?class rdfs:subClassOf ?superClass .
} .
}
The result of executing the preceding UML Class rules is the UML Class Hierarchy shown in the diagram below.

UML Metamodel Class Hierarchy

Figure 13: Generated UML Metamodel Class Hierarchy

Top

Generation of UML Packages

# STEP CMOF-SR-020 - make Packages
CONSTRUCT {
?package rdfs:label ?name .
?package ocmof:hasCMOFbasis ?this .
?mySuperClass a owl:Class .
?mySuperClass rdfs:label ?path .
?superURI a owl:Class .
?superURI a cmof:CategoryClass .
?superURI rdfs:label ?super .
?subURI a owl:Class .
?subURI a cmof:CategoryClass .
?subURI rdfs:subClassOf ?superURI .
?subURI rdfs:label ?sub .
?package a ?mySuperClass .
?package a uml:Package .
}
WHERE {
?this xmi:type "cmof:Package" .
?this xmi:id ?name .
BIND (xmi.common:makeUML-URI(?name) AS ?package) .
BIND (o2o:pathPart(?name, "-") AS ?path) .
OPTIONAL {
?path o2o:pairHyphenIncrementally ( ?super ?sub ) .
BIND (xmi.common:makeUML-URI(smf:buildString("PACKAGES_{?super}")) AS ?superURI) .
BIND (xmi.common:makeUML-URI(smf:buildString("PACKAGES_{?sub}")) AS ?subURI) .
} .
BIND (xmi.common:makeUML-URI(smf:buildString("PACKAGES_{?path}")) AS ?mySuperClass) .
}


Generation of UML Package Relationships


# STEP CMOF-SR-024 - fixup the superclass of the root Packages
CONSTRUCT {
?packageClass rdfs:subClassOf uml:Package .
}
WHERE {
?this xmi:type "cmof:Package" .
?package ocmof:hasCMOFbasis ?this .
?package a ?packageClass .
NOT EXISTS {
?packageClass rdfs:subClassOf ?superClass .
} .
}
The result of executing the preceding UML Package rules is the UML Package Hierarchy shown in the diagram below.

UML Metamodel Packages Hierarchy

Figure 14: Generated UML Metamodel Package Hierarchy




Performance

As a measurement of the performance with the TopBraid Composer release 3.4.0, the conversion of the UML Infrastructure XMI took 38.611 seconds and generated 19,575 statements (RDF triples) on a DELL Studio XPS Laptop with 4GB of memory, running Windows 7. This translates to an inference speed of 507 TPS (Triples per second).

Concluding Remarks

Part 1 of this blog has introduced the power of model-based transformation using SPARQL Rules as a means to transform XMI to OWL. Our experience in doing this work confirms the extensibility and flexibility of this approach. The subject is a complex one requiring a grounding in the intricacies of UML Metamodeling, and a knowledge of SPARQL and SPARQL Rules. We have attempted to do that briefly in this blog - not an easy matter.

Part 2 of this blog series will discuss transforming UML Models to OWL Using SPARQL.


Wednesday, January 19, 2011

Running TopBraid Live in the Amazon EC2 cloud

A recent Dilbert strip inspired me to go through Dave Winer's EC2 for Poets tutorial as a geeky weekend project. It was surprisingly easy and inexpensive to get a computer image running in Amazon's "Elastic Computing Cloud" (EC2) and to then get a copy of TopQuadrant's TopBraid Live running in that image.

These images are cheap to run, as you can see on their price list. Note that the cost per hour of running a default Linux image is not eighty-five cents an hour when using servers in northern Virginia, but eight and a half cents. (It's an additional penny an hour when using their servers in California, Ireland, or Singapore.) If you're willing to spend a dollar or two an hour, you can get full control of some really large-scale computing power without spending much money unless you're planning some long-term use of it, in which case you'll want to compare the options with your requirements more closely than I did. You'll want one of the EBS images, which still offers you a wide choice of platforms. Otherwise, the only way to stop a running image is to terminate it, in which case it (and all your configuration of it) is gone. With an EBS image, once you stop running it, you can always restart it.

I started off with an Ubuntu Linux image, but I had trouble installing Sun Java on it. After starting up a Fedora image, installing Sun Java and Tomcat were easy, and once Tomcat was installed, the installation of TopBraid Live under Tomcat according to the TBL installation instructions was simple and straightforward.

The ease with which I switched my efforts from Ubuntu to Fedora was an important lesson in cloud use; it sure was easier than installing Fedora over Ubuntu on a laptop hard disk, and if I decided to switch back, going back to the Ubuntu EC2 image would take seconds, unlike reverting a hard disk once I'd overwritten the Ubuntu image with Fedora. You can initialize several different images in the EC2 cloud, and if you're not running any of them, Amazon only bills you for storage, which is pennies, so you can set up several to wait for you and start and stop them whenever you like. If you want more memory, cluster processing, 64-bit instead of 32-bit, or other additional resources, you can just try it and see how it goes. This means that when you plan out the deployment of TopBraid applications using 32- or 64-bit TBL, you don't have to worry too much about having an available machine with the best possible operating system and hardware configuration, because you can experiment with different cloud images until you find the setup that's best for you. (I never tried a Windows image or SUSE or one of the other Linux images, but if I was going to roll out a production TBL application I'd explore these further as part of my planning.)

It's pretty impressive to think about the kind of power and flexibility that you can get when you deploy cloud-based semantic web applications, and as I found out with my little experiment, the barriers to entry for trying it out are extremely low. After 16 hours of running one instance, 3 hours of running another, and 3.1 GB of data transfer in the last few weeks, I currently owe Amazon Web Services a total of $2.29. So, if you're holding off on trying TBL because you don't have the appropriate box on which to run it, Amazon's EC2 offers some nice options to try out.

Wednesday, December 29, 2010

How to: convert a spreadsheet to SKOS

In an earlier entry, we learned how SPARQL Rules can increase the quality of taxonomies and other controlled vocabularies stored using the W3C SKOS ontology. (As I wrote there, the Simple Knowledge Organization Systemvocabulary management specification is gaining popularity because, as a standard, it makes it easier to share taxonomies and thesaurii between different systems. It also guards investments in vocabulary development against the potential problems of dependence on a proprietary vendor format.)

TopQuadrant's Enterprise Vocabulary Net (EVN) vocabulary manager uses SKOS as its default format for storing data. Whether you use EVN or not, a first step in systematic management of vocabularies is often the conversion of vocabularies stored in ad hoc spreadsheets—an unfortunately very popular way to store them—to SKOS, so today we'll look at how TopBraid makes this conversion easy.

Below is an Excel spreadsheet with some data about a few Caniformia animals. (In the Linnaeus classification of animals, Caniformia is the suborder of Carnivora, which is an order of the Mammalia class.) It shows two families of this suborder and a few genuses and species of each family, with both the Latin and common name of each species.

Using a SPARQLMotion script, the basic steps of converting a spreadsheet like this to SKOS are:

  1. Read in the spreadsheet as a set of RDF triples.

  2. Use a CONSTRUCT query to convert the spreadsheet triples to SKOS triples. This is the step that varies the most from one conversion to another, because people can arrange spreadsheets any way they want, so the logic of the CONSTRUCT query has to infer the correct relationships between the values on the spreadsheet.

  3. Save the SKOS triples as an RDF file or in whatever format is appropriate to your applications that will use this data.

The following shows the SPARQLMotion script that I used to convert the spreadsheet above.

It has a module for each of the three steps listed above and an additional SetBaseURIStr module to set a ?baseURIStr variable. The script refers to the base URI of the output several times, and instead of hardcoding it in all those places, I decided to use this module to set this variable and to then reference the variable from other places so that resetting the base URI could be done in one place. The "set BaseURIString" module has a very simple SELECT query:

SELECT ?baseURIStr
WHERE {
LET (?baseURIStr := "http://example.com/taxonomies/animals") .
}

When you import an Excel file into TopBraid, the "Import Excel Cell Instances" SPARQLMotion module can pull triples from the spreadsheet with information such as the fact that a given cell has a row value of 7 (using zero-based counting), a column value of 0, a type value of "xsd:string", and "giant panda" as its contents. This level of detail can be useful for picking apart complex spreadsheets, but for simpler ones, if you instead use an "Import RDF from Workspace" module (in other words, if you have the script open the spreadsheet as if it were an RDF file), TopBraid uses the headings of the spreadsheet to identify more of the semantics of the data. For example, it would create triples saying that the thing identified as Row-6 has a commonName value of "giant panda" and a genus value of "Ailuropoda". This will be easier to convert to SKOS with a CONSTRUCT query.

There are five basic tasks that the conversion module must perform, all through the creation of triples:

  1. Declare that the dataset being created is an ontology.

  2. Import the standard W3C SKOS ontology so that we can reference its classes and properties.

  3. Declare a concept scheme. A SKOS vocabulary can have as many concept schemes as you like, but we'll just create one for our example.

  4. Declare concepts for each species, genus, and family found in the input triples, with a skos:broader property pointing from each one to either the appropriate broader concept or, if there is none, to the concept scheme created in the previous step.

  5. Create triples that attach any additional metadata to the appropriate concepts—in this case, to assign the common name value to each species concept. SKOS is very flexible, so if you had additional non-SKOS properties specific to your own applications that you wanted to assign to each concept, the steps would be similar to the ones for attaching the common name values from this spreadsheet to each concept.

Whe the "Import RDF from Workspace" module reads in a spreadsheet such as caniformia.xls, it uses the spreadsheet's filename to define a prefix for the spreadsheet's properties so that it can refer to those properties with names like caniformia:genus. After opening the spreadsheet directly in TopBraid Composer, I saw that the base URI created for the data and associated with the caniformia: prefix was file:///xls2skos/caniformia.xls, because I had it in a project named xls2skos. I wanted to use this prefix in my SPARQLMotion script's CONSTRUCT query, so I associated this URI with the caniformia: prefix in the Overview tab of the xls2skos.n3 file that stored the SPARQLMotion script.

The actual conversion takes place in the Apply Construct module that I named "convert XLSData". These modules can store multiple CONSTRUCT queries, so I used two. The first does the basic setup of the taxonomy being created, which are the first three of the five tasks listed above:

CONSTRUCT {
?baseURI a owl:Ontology .
?baseURI owl:imports <http://www.w3.org/2004/02/skos/core> .
<http://example.com/taxonomies/animals/caniformia> a skos:ConceptScheme .
<http://example.com/taxonomies/animals/caniformia> rdfs:label "Caniformia" .
}
WHERE {
LET (?baseURI := smf:buildURI("<{?baseURIStr}>")) .
}

The second query performs steps 4 and 5:

CONSTRUCT {
?speciesURI a skos:Concept .
?genusURI a skos:Concept .
?familyURI a skos:Concept .
?speciesURI skos:prefLabel ?speciesName .
?speciesURI skos:altLabel ?commonName .
?speciesURI skos:broader ?genusURI .
?genusURI skos:broader ?familyURI .
<http://example.com/taxonomies/animals/caniformia> skos:hasTopConcept ?familyURI .
}
WHERE {
?row caniformia:commonName ?commonName .
?row caniformia:species ?speciesName .
LET (?species := smf:encodeURL(?speciesName)) .
LET (?speciesURI := smf:buildURI("<{?baseURIStr}#{?species}>")) .
?row caniformia:genus ?genusName .
LET (?genus := smf:encodeURL(?genusName)) .
LET (?genusURI := smf:buildURI("<{?baseURIStr}#{?genus}>")) .
?row caniformia:family ?familyName .
LET (?family := smf:encodeURL(?familyName)) .
LET (?familyURI := smf:buildURI("<{?baseURIStr}#{?family}>")) .
}

In addition to creating a SKOS concept for each species, it creates one for each genus and family as well, using the skos:broader property to identify the connections between these concepts that make up the hierarchical taxonomy of terms.

One big decision to make with this query was how to create URIs that provided unique identifiers for each new concept being created. I knew that the species, family, and genus names must be unique, so I added those to the base URI after passing them to smf:encodeURL(), a SPARQLMotion extension function that escapes any characters that won't work well in a URI. If you have taxonomy data in a spreadsheet, there may already be a unique number or other form of ID assigned to some or all taxonomy terms that your conversion can grab so that you don't have to create URIs from the names on the spreadsheet like I did.

I also decided to use species names like "Canis lupus" as the skos:preferredLabel value in the output and to use labels from the spreadsheet's "common name" column like "gray wolf" as skos:altLabel values. If you wanted to to use common name values as preferred labels and species names as alternative labels, it would be a simple change to the query above.

My "convert XLSData" module also has its sml:replace value set to True so that it doesn't pass along the input triples to the final module, which saves the conversion result. This last module, which I named "Save as TDB", is an "Export to TDB" SPARQLMotion module that saves the conversion results using the Jena TDB format. I could have used a SPARQLMotion "Export to RDF" module, which saves triples as a Turtle or RDF/XML disk file, but I wanted to use the results of my conversion in EVN. EVN requires that you use Jena's TDB or SDB formats so that it can attach metadata to your work to support reporting and workflow tracking.

After running the conversion, here is one view of it in EVN:

This is a minimal example using a small spreadsheet with no non-SKOS metadata. If your spreadsheet includes columns for data that don't fit easily into the SKOS model, you can use TopBraid Composer to create a customized version of SKOS that includes your own properties, and your SPARQLMotion script's conversion module can then add triples for those properties to the result. Viewing them in EVN, they would appear under Custom Properties on the right. And of course, the screen shot above only hints at all that EVN lets you do with your controlled vocabulary once you convert it to SKOS.

Wednesday, November 10, 2010

Getting started with SPARQL Web Pages

In earlier entries on this blog, we've seen how SPARQL Rules attached to classes can let you identify constraint violations in instances of those classes and implement other kinds of business logic, all using the SPARQL standard. With release 3.4, which is now in beta, TopQuadrant products have another new application of SPARQL that lets you attach useful metadata to class definitions: descriptions of how you want the class's instances to look in a browser. We call this SPARQL Web Pages.

TopQuadrant vice president of product development Holger Knublauch has written several blog entries introducing SPARQL Web Pages under its original name, UISPIN, such as UISPIN: Creating HTML and SVG Documents with SPARQL, Charts and Business Reports with UISPIN, and UISPIN Example: Documenting SPIN Functions. In this blog entry, we'll see how to get started with a simple but useful example.

Address book information is not as simple to represent in RDF as one might think. A street address, city name, and postal code should be shown in a specific order, but RDF facilities for ordering the property values for a given instance can add annoying layers of complexity to a data model. Without ordering, though a simple address book entry can be difficult to read, like the following fake address shown in Turtle format:


a:Entry_1
rdf:type a:Entry ;
a:city "San Diego" ;
a:email "jerry122@hotmail.com" ;
a:firstName "Jerry" ;
a:homePhone "(738) 610-2019" ;
a:lastName "Snyder" ;
a:mobile "(702) 382-4712" ;
a:postalCode "39248" ;
a:region "CA" ;
a:streetAddress "3137 11th Ave." .

Using TopBraid Composer Maestro Edition, I created an RDF file with an Entry class for address book entries. I declared the properties shown above, with the Entry class as their domain, and then I added a few fake address book instances to the file. The file has a base URI of http://example.org/addressBook, which will be important later.

Next, I imported the html.rdf model from the UISPIN folder of the TopBraid project that is automatically added to every workspace. (In a production application, especially if I was working with a standard ontology instead of a hand-crafted little model defining an address book entry, I'd create a new file that imported the standard class and property definitions as well as html.rdf instead of adding the SPARQL Web page definitions directly into the file that defines the address book model.) The UISPIN folder also includes models to generate SVG, charts, and more; the html.rdf file lets you add HTML-generating SPARQL code to your classes. Below, you can see how I've added the ui:instanceView property from this file's model to the definition of my Entry class:

When I scroll the Class View down a little, you can see what I entered as the ui:instanceView property value. It's an HTML template, with instructions for plugging in instance data, that will be output whenever TopBraid sees an instance of this class:

It lays out an div element that begins with the address entry's mailing address and then has a small table showing the entry's email address and phone numbers, with these property names bolded in the output. You can take advantage of the full power of SPARQL in these templates, as you'll see in Holger's blog entries, but I kept things simple by mostly just using the spl:object() function to insert specific property values into various places in the HTML.

When I view the form tab of the Entry class and click on a row in the Instances view, I see a Resource Form for that instance, like I always did:

You can see a new Browser tab next to the Form tab, though, and it lets me see the instance view formatted according to the HTML template that I created in the ui:instanceView value:

Even better, TopBraid Live can serve up the data using these HTML templates outside of TopBraid Composer, so that sending a browser to the URLhttp://localhost:8083/tbl/uispin?_resource=http://example.org/addressBook%23Entry_1&_baseURI=http://example.org/addressBook displays the result in the browser (note the use of the base URI to specify the model with the data to display and the escaping of "#" as "%23"):

Of course, this HTML can also have links and reference CSS, Javascript, web services, and other applications on the TBL server where it's hosted, so you can build some very sophisticated user interfaces. Check out Holger's blog entries for further ideas on where you can take this, especially when you start incorporating SPARQL queries and their results into the templates.

Wednesday, October 27, 2010

TopQuadrant's new Enterprise Vocabulary Manager

EVN logo

The TopBraid platform can be used to build all kinds of applications and solutions. We've recently noticed one particular area where more and more customers needed help, and where semantic technology and our tools were a great fit: the management of multiple connected vocabularies spread out across an enterprise. To meet this need, we've created TopBraid Enterprise Vocabulary Net (EVN), a solution that works out of the box while having all the power of TopBraid Suite behind its customization capabilities.

The EVN product page has a long list of its features, which provide everything you need to manage taxonomies and thesaurii (and even create simple ontologies) in multi-user environments. The ability to review proposed changes before rolling them into production, with a choice of reports and other options for analyzing those changes and their potential impact, will be especially useful in larger organizations.

The use of EVN requires no knowledge of SKOS, RDF, or the related W3C standards, but the use of these standards behind EVN's graphical user interface is what makes EVN both flexible and scalable. The use of public standards for data, models, and application logic makes it much easier to integrate EVN with other systems than any other vocabulary management solutions we've seen in the marketplace. They also make it easier for EVN to let you set up an environment where different vocabularies in different parts of a large organization can work cooperatively with no need to merge those vocabularies into a single large, central vocabulary.

EVN is included in TopBraid Composer Maestro Edition release 3.4, which is now in beta, so you can try it without purchasing a separate product. For a quick overview of the features and what the product looks like, start with thescreenshot tour, or jump right in to the tutorial included with EVN's documentation.

Monday, October 4, 2010

How to: read RSS and RDFa from the web with a SPARQLMotion script

How do you get a SPARQLMotion script to read an RSS or Atom feed as RDF triples? How do you get a SPARQLMotion script to read triples that have been embedded into web pages using RDFa? The answer to both questions is the same: use the specialized SPARQLMotion module for the task. All you have to do is specify the URL of the file with the information you want to read.

To demonstrate both, we'll put together a short script that:

  1. Reads the RSS feed about technology news from Newsweek magazine

  2. Pulls the triples from the RDFa embedded in the Newsweek articles described in the feed

  3. Saves the extracted triples in a Turtle file

Along with Dublin Core properties such as dc:title and dc:description, RDFa attributes in Newsweek articles store additional RDF metadata using the Open Graph vocabulary developed by Facebook. This makes it easier for Facebook to incorporate additional information about news articles in their applications—for example, if people click the Facebook button next to a Newsweek article in order to share it with their Facebook friends.

It also makes it easier for you to use information about these articles in your own applications. The sample application below just saves the retrieved triples in a file, but you could also pass them to other SPARQLMotion modules that could have OpenCalais analyze the text, combine the triples with data from another source, create a new, specialized RSS feed or SPARQL endpoint, or send an email message based on the results of your processing. Retrieving the data is just the beginning.

To create this application, start by creating a new SPARQLMotion file called getnewsweektech. (For more detailed background on the steps involved in creating and running a SPARQLMotion script, see the PDF tutorial TopBraid Application Development Quickstart Guide.)

Create a new SPARQLMotion script in your getnewsweektech.n3 file. For its first module, select sml:ImportNewsFeed from thesml:ImportFromRemoteModules category and name it GetNewsweekTechNewsFeed. To configure it, you only need to set its sml:url value tohttp://feeds.newsweek.com/newsweek/technology?format=xml, a URL I learned about from Newsweek's web page about their RSS feeds.

Once this module pulls down the RSS data and TopBraid converts it to triples, your script will look through these triples for web page URLs provided as RSS link values and then retrieve the triples that are stored as RDFa in those web pages. The script can't pull the triples from all those web pages at once, so we'll use an IterateOverSelect module to drive the next step. We'll specify a SPARQL SELECT query in the IterateOverSelect module to find the RSS link values, and then for each result that this SELECT query finds, another module will retrieve the triples from the web page named by the link value.

Drag an Iterate over select module from the Control Flow section of the SPARQLMotion palette and name it GetArticleLinks. Paste the following query in as the value for its sml:selectQuery property:

PREFIX rss: <http://purl.org/rss/1.0/> 
SELECT ?articleURLString
WHERE {
?s a rss:item .
?s rss:link ?articleURL .
LET (?articleURLString := xsd:string(?articleURL)) .
}

The module that retrieves the RDFa needs a string version of the URL to specify where it should look for the RDFa, so the query above assigns a string version of each rss:item resource's rss:link value to the variable ?articleURLString. The script will execute the body of the IterateOverSelect (a separate module that we haven't created yet) once for each value bound to this variable. You're done configuring this module.

Next, we'll create the body of the IterateOverSelect. This can be a series of modules, but for this application we'll only need one. Drag anImport RDFa module from the Import from Remote section of the SPARQLMotion palette and name it ReadArticleRDFa. When configuring this new module, click the white triangle for its sml:url property's context menu and select Add SPARQL expression. This lets you add any combination of SPARQL keywords, symbols, function calls, and operators that returns a single value; for this, all you need here is the variable reference ?articleURLString. Each time this module retrieves triples from the RDFa in the web page at this URL, it will pass along the triples that it found to the next module. If this module has an sml:needsTidy property, set it to True to make it easier to read RSS that isn't well-formed XML.

For our script's last module, drag an Export to RDF File module from the palette's Export to Local section and call it SaveArticleTriples. Set its sml:targetFilePath value to newsweekTech.n3; it will write this file to the directory that holds the SPARQLMotion file with your script. Set the module's sml:baseURI to http://example.com/newsweek/tech/metadata or to any URI that you like.

All that's left is to connect up the four modules as shown below. When you add a connector out of your Get Article Links Iterate Over Selectmodule, TopBraid Composer will ask you whether your new connector is pointing at the body of the loop (the part to execute for each binding of the selected variable) or at the module that should take control of the script when the iteration is finished. Connect Get Article Links to the Read Article RDFa module with an sm:body link, because that's the part we want executed for each iteration, and connect Get Article Links to Save Article Triples with an sm:next link to transfer control (and the collected triples) there when the iteration is all done.

Select the Save Article Triples module and click the green triangle at the top of the workspace to execute the script up to that final module, and you should end up with a newsweekTech.n3 file in the same directory as your getnewsweektech.n3 file that holds the script. This new file will hold triples extracted from the various web pages named in the Newsweek tech news feed.

To branch out, you could substitute the names of other Newsweek feeds, or additional ones, and then collect all the triples together. You could drive the whole thing with a TopBraid Ensemble interface where an end user picks the category of Newsweek news (for example, their technology, politics, business, or entertainment categories) whose metadata should be retrieved. You could also find other publications that store RDFa metadata in their articles, or other websites, such as TopQuadrant's. And, as I mentioned earlier, you could combine this with other features of SPARQLMotion and TopBraid to make a very powerful application.

Monday, August 16, 2010

How to: Find SKOS constraint violations in AGROVOC with SPARQL Rules

The Simple Knowledge Organization System (SKOS) vocabulary management specification is gaining popularity because, as a standard, it makes it easier to share taxonomies and thesaurii between different systems. It also guards investments in vocabulary development against the potential problems of dependence on a proprietary vendor format.

The W3C makes an OWL ontology for SKOS available, which makes it easier to ensure that your vocabulary conforms to the standard. As a comment near the beginning of it tells us, though,

A number of semantic conditions are *not* expressed formally in this schema. These are:

S12
S13
S14
S27
S36
S46

For the conditions listed above, rdfs:comments are used to indicate the conditions.

The comment for S13 says that "skos:prefLabel, skos:altLabel and skos:hiddenLabel are pairwise disjoint properties". In plain English, this means that for a given concept, you can't use the same term for any two of these properties. For example, you shouldn't say that "dog" is both the preferred label and the alternate label for a given concept—it should be one or the other.

If these constraints are in the ontology as comments and not as something that can be implemented by executable code, how do you find violations of these constraints? The simplest way I've found is to use SPARQL Constraints (with SPIN). We've built on Paul Hermans' work to implement these constraints in the ontology at http://topbraid.org/spin/skosspin. It imports the W3C SKOS ontology and adds one rule to the skos:OrderedCollection class for constraint S36 and rules for the other five constraints to the skos:Concept class. For example, it adds the following three rules for constraint S13:

# Constraint S13: skos:prefLabel, skos:altLabel and skos:hiddenLabel 
# are pairwise disjoint properties.
ASK WHERE {
?this skos:prefLabel ?label .
?this skos:altLabel ?label .
}

ASK WHERE {
?this skos:prefLabel ?label .
?this skos:hiddenLabel ?label .
}

ASK WHERE {
?this skos:hiddenLabel ?label .
?this skos:altLabel ?label .
}

If any of these returns a boolean true, then we know that constraint S13 has been violated. (If you look at the skosspin ontology, you'll see these queries represented as triples, which is more difficult to read but easier to implement than rules expressed as SPARQL queries as shown above. TopBraid Composer can convert between the two formats, and so can a SPARQL Text to SPIN RDF Syntax Converter that Holger Knublauch has made available on the web.)

I tested this with the Food and Agriculture Organization of the United Nation's popular AGROVOC thesaurus, a vocabulary "designed to cover the terminology of all subject fields in agriculture, forestry, fisheries, food and related domains", and found over 1600 violations of constraint S13. Because this thesaurus has almost 29,000 concepts and preferred and alternate labels in multiple languages for most concepts, it's easier to violate these constraints than you might think, and I never would have found them without the ability to automate this search. For example, concept http://www.fao.org/aos/agrovoc#c_1135 has an English preferred label of "Buds", 14 preferred labels for other languages such as Farsi and Thai, and 24 alternate labels. Among these, the Slovak skos:prefLabel value and the Sloval skos:altLabel value are both "púèiky", so this concept violates constraint S13.

How do we find the violations? I tried it with the free edition of TopBraid Composer, because it has everything you need to define and use SPARQL Rules. (TopBraid Composer Maestro Edition's ability to use these rules from within applications has made it possible for me to add several nice features to applications for some of our clients.)

The screenshot below of TopBraid Composer's Navigator and Imports views shows that I created a checkAGROVOC project and added a checkAGROVOC.n3 ontology file to it. This ontology only does two things: it imports the ag_skos_080422.rdf file that I downloaded from the fao.org web site and it imports the skosspin ontology described above for its SPARQL Rules. (It imports a web version of the skosspin ontology and, because of its 62 meg size, the local copy of the actual AGROVOC thesaurus.)

With these two files imported, I opened TopBraid Composer's Problems view and clicked that view's "Refresh all problems of current TopBraid file" icon . A "Progress Information" message box told me that TopBraid Composer was "Checking SPIN constraints on skos:Concept", which took a few minutes because there were plenty to check.

After it finished checking, the Problems view said that there were Warnings and had a plus sign that I expanded to see the first few constraint violations:

(You may want to play with the column widths a bit, because the Location column is the one you really want to see.) Double-clicking anywhere on a specific warning line shows the details about that concept on the Resource Form, like this:

If you don't see the little yellow warning symbols on the Resource Form that show where the problems are, click the little "Display constraint violation warnings" icon at the top of TopBraid Composer . In this case, a bit of scrolling down when viewing concept c_1002 shows that the same term appears as both Hungarian skos:altLabel and skos:prefLabel property values for this concept.

The Problems view above only shows the first 100 of the 2,972 warnings. Clicking the context menu white triangle in the view's upper-right lets you configure Preferences for the view so that you can reset the number of items to be displayed; I had no problem with a figure of 3,000.

Now that we know what constraint has been violated, there are other ways to list the concepts that need to be corrected. For example, the following query in TopBraid Composer's SPARQL view lists the identifiers that have the same label for these two properties:

SELECT ?s ?label
WHERE {
?s skos:prefLabel ?label .
?s skos:altLabel ?label .
}

Once you execute this query you can export its results to a file and then use that as a reference point to address the issues in the vocabulary.

We could have started off by executing this query on the AGROVOC SKOS file, but remember, at the time we didn't know which constraints had been violated. Using SPARQL Rules as extra metadata for class definitions helps to automate the identification of quality issues with the data, letting us use other techniques to focus on the specific problems and how to fix them.

Are your SKOS vocabularies violating any of the six extra constraints described in the SKOS specification? As I mentioned, this all works with the free version of TopBraid Composer, which is available for Windows, Mac, and Linux, so you can try it yourself to find out. With the TopBraid Composer Maestro edition, you can build applications for end users who can then maintain these vocabularies with a web-based interface instead of using TopBraid Composer. The user interface for notification of constraint violations then becomes one of the many things you can customize to the needs of your end users. You can also define new constraints around your own shop's business rules—for example, to require that all labels begin with an upper-case letter—and you can set TopBraid Composer or your application to highlight these violations as soon as they occur, instead of checking in batch mode like I did above.

To summarize, what we've seen here is really just a starting point, and there are all kinds of places where you can take it to improve the consistency and value of your vocabularies.

This is a blog by TopQuadrant, developers of the TopBraid Suite, created to support the pursuit of our ongoing mission - to explode strange semantic myths, to seek out new models that support a new generation of dynamic business applications, to boldly integrate data that no one has integrated before.