README - DREFT(*) SKOS Thesaurus API Demonstrator
* - please see 

Author - nikki.rogers@bristol.ac.uk
Latest revision: 29.06.2004


*************
*** CONTENTS
*************

1. STATUS & TODOS
2. INSTALLATION OF AXIS AND SESAME
3. POST-INSTALLATION TEST
4. HOW DO I ADD A NEW REPOSITORY IMPLEMENTATION?
5. HOW DO I ACCESS THE DREFT API DIRECT - FROM A NON-SERVLET CONTEXT? 

*****************************************************************

FOR FURTHER INFORMATION PLEASE SEE:

* http://www.w3.org/2001/sw/Europe/reports/thes/8.7/
(this report describes the thesarus research prototype demonstrating the SKOS schema by means of the 
SKOS API web service and a demonstrator containing sample data, some simple clients for using the API, 
documentation and description of related work).

* http://www.w3.org/2001/sw/Europe/reports/thes/api/docs/
(Javadoc for the prototype API, as generated by Apache Axis).

* http://www.w3.org/2001/sw/Europe/reports/thes/1.0/guide/
(SKOS-Core 1.0 Guide)

*http://www.w3.org/2001/sw/Europe/reports/thes/dreft/
(Download area).


************
1 STATUS & TODOS
************


<p>
STATUS: current 'DREFT' implementation was completed August/September 2004. There remain some outstanding 'todo's' which
are to be found in various places in the source code. Also some outstanding issues which are listed in the associated documentation
for this release.  
</p>

<li>TODO: implement remaining SKOS Service methods (the majority are implemented now). 
<li>TODO: redeploy with axis (following a compile with debug switch on if i remember rightly!) so that parameter names are given in the SOAP 
instead of parameter in0/in1 etc.
  Dave: I tried compiling with javac and the Jikes java compiler with the -g option to get debugging information however in the output
  class files there was only method names and (return value, method) type information recorded - no parameter names.  This was
  confirmed by using javad on the output class file as well as just looking at the strings; the method names were not recorded at all.
 <li> TODO: improve javadoc in the source code
<li> TODO: implement at least some service methods as (wrapped message service) DOC/LIT style service (more recent version of Axis required)
<li> TODO: wrap up as a war with ant scripts to fetch required jars and to build and deploy all classes to servlet container.
<li> TODO: extend AxisFault to provide more informative messages from the service when exceptions
have been thrown 
<li> TODO: implement SKOS cross-thesauri mappings   
<li> TODO: add support for other RDF databases in addition to Sesame
<li> TODO: add support for other Relational databases in addition to MySQL


*****************************************************************
************
2. INSTALLATION OF AXIS AND SESAME
*************

My Op Sys is Fedora. And I use Eclipse for java development. If you want eclipse-specific instructions I've got notes - contact me.  These notes are specific to the jakarta-tomcat servlet container.

DOWNLOAD AXIS:
http://ws.apache.org/axis/download.cgi
I selected  Version 1.1 Sources - tar.gz
unzipped and tarred it and stuck it in my home directory


INSTALL AXIS:

NEED- tomcat (or other servlet container), java, ant installed

See install documentation at http://ws.apache.org/axis/java/install.html

- I copied the webapps/axis directory to my tomcat webapps directory

CHECK INSTALL
- start up tomcat
- browse to http://127.0.0.1:8080/axis/index.html
- click on 'Validate' - checks the needed jars can be found
e.g.
"The core axis libraries are present. 1 optional axis library is missing"
is ok
- go back to start page and click 'View  the list of deployed Web services'
- check you get some listed and can view the wsdl for them


OPTIONAL - TRY OUT AN AXIS DEMO TEST
-- the stockquote service

To deploy it:  WITHIN ECLIPSE:
- i made a new directory in swadwp8/axis/src and called it samples/stock and copied the files from the stock directory to it. Then rebuilt.

Then added the AdminClient program to 'run' classes within Eclipse (instead of from commandline) - set path to swadWP8/axis/src/samples/stock
(ie overriding the default directory).
And set the arguments:
	-lhttp://localhost:8080/axis/services/AdminService deploy.wsdd

Result - The terminal should display:
Processing file deploy.wsdd
<Admin>Done processing</Admin>

You should now see WSDL at this place
http://localhost:8080/axis/services/urn:xmltoday-delayed-quotes?wsdl

And run stock quote from in eclipse giving it right path and arguments:
(it should report 'XXX: 55.25' in the terminal)


OPTIONAL - TRY out the same axis test as above, but now also view the soap messages via the tool that comes with Axis - TCPMON:

****USING TCPMON

[have tomcat started on 8080]
re the stock quote sample

i) start up TCPMON - run it from within axis - (no arguments required). Add a listener - listen port = 8081, target port = 8080

ii) run the stock quote eg as per above supplying an extra argument re service port to target:
so the arguments to the program are:
-lhttp://localhost:8081/axis/servlet/AxisServlet -uuser1 -wpass1 XXX -uuser1 -wpass1 XXX
(note that by default the service option is localhost:8080 - see where this is coded in the axis.utils.Options.java class)

[and not default path - set the path to swadWP8/axis/src/samples/stock  as above]


You should see in tcpmon:

POST /axis/servlet/AxisServlet HTTP/1.0
Content-Type: text/xml; charset=utf-8
Accept: application/soap+xml, application/dime, multipart/related, text/*
User-Agent: Axis/#axisVersion#
Host: 127.0.0.1
Cache-Control: no-cache
Pragma: no-cache
SOAPAction: ""
Content-Length: 454
Authorization: Basic dXNlcjE6cGFzczE=

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <soapenv:Body>
  <ns1:getQuote soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="urn:xmltoday-delayed-quotes">
   <symbol xsi:type="xsd:string">XXX</symbol>
  </ns1:getQuote>
 </soapenv:Body>
</soapenv:Envelope>


HTTP/1.1 200 OK
Content-Type: text/xml;charset=utf-8
Date: Tue, 18 May 2004 14:15:40 GMT
Server: Apache-Coyote/1.1
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <soapenv:Body>
  <ns1:getQuoteResponse soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="urn:xmltoday-delayed-quotes">
   <ns1:getQuoteReturn xsi:type="xsd:float">55.25</ns1:getQuoteReturn>
  </ns1:getQuoteResponse>
 </soapenv:Body>
</soapenv:Envelope>

 
OPTIONAL - view the wsdl for the test service

viewing the wsdl for a service
*****************************
e.g
http://localhost:8081/axis/services/urn:xmltoday-delayed-quotes?wsdl





** INSTALL SESAME:
- download & install sesame 
[I did:
downloaded sesame and installed it in ~ at /sesame-1.0
Then I ran 'ant' from top level
It builds sesame war, jar, generates javadoc etc
I copied the war to tomcat webapps - in a newly created directory [SESAME_DIR] - & extracted it there]

- download and install mysql
[ I also copie the appropriate JDBC-driver file (i.e. mysql driver) to the directory [SESAME_DIR]/WEB-INF/lib/]

- create a user account for sesame on the MySQL server, and a database for holding the thesaurus database (e.g. I created one called thes_test) and finally grant the user all privileges on the concerning database(s).

e.g. 

mysql> GRANT ALL ON thes_test.* TO sesame@localhost;
Query OK, 0 rows affected (0.31 sec)
 
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.08 sec)

- check that you can connect to the database now as the user you created above:
[nr8262@ilrt-haako mysql_4]$ mysql -h 127.0.0.1 -u sesame
Welcome to the MySQL monitor.....
 
- now configure sesame: 
You need to create a file [SESAME_DIR]/WEB-INF/system.conf.  I renamed the provided example file in this location to system.conf.

You can clone existing repos to create new ones via the sesame admin tool, or you can do it by hand in the system.conf file. 
My system.conf entry for the thes_test database looks like this 
(notice the path to the skos rules file you need: entailment-skos-core-20040228.xml, or the latest version):

<repository id="rdbms-skos-db">
			<title>SKOS Inferencing Test DB</title>
			<sailstack>
				<!-- SyncRdfSchemaRepository prevents multiple concurrent
				transactions and reads during a transaction -->
				<sail class="org.openrdf.sesame.sailimpl.sync.SyncRdfSchemaRepository"/>
				<sail class="org.openrdf.sesame.sailimpl.rdbms.RdfSchemaRepository">
				<param name="jdbcDriver" value="com.mysql.jdbc.Driver"/>
				<param name="jdbcUrl" value="jdbc:mysql://127.0.0.1:3306/thes_inf_testdb"/>
				<param name="user" value="XXXthe user you createdXXXX"/>
				<param name="password" value=""/>

				<param name="use-inferencer" value="org.openrdf.sesame.sailimpl.rdbms.CustomInferenceServices"/>	
				<param name="org.xml.sax.driver" value="org.apache.xerces.parsers.SAXParser"/>
				<param name="rule-file" value="your_path_to_entailment-skos-core-20040228.xml"/>

			
			 	</sail>
			</sailstack>

			<!--Access Control List can contain zero or more 'user' elements-->
			<acl worldReadable="true" worldWritable="false">
				<user login="XXlogin_user_nameXX" readAccess="true" writeAccess="true"/>
			</acl>
		</repository>



- upload thesaurus test data to sesame:
Point a browser to the location where you have installed Sesame (e.g. 'http://MACHINE_NAME:8080/sesame/') & it should now display the Sesame web interface.
LOGIN as e.g.:
XXlogin_user_nameXX
XXlogin_passwordXX
(i.e. username & password as per top of your system.conf file).
You can for e.g. use copy and paste to upload some 'dummy' skos data (as at ) to the thes_test database you created (you will have given it a name in the sesame system.conf file - e.g. mine is called SKOS Inferencing Test DB):

In the browser, select this thes_test repository and press 'Go>>'. Click on the 'Add (copy-paste)'
*****Now copy and paste the sample skos thesaurus data given in skos_dummy_data.xml. 
and upload it. There should be no errors!

OK, now you're ready to integrate sesame and axis:





GETTING THE SKOS THES SERVICE INTEGRATED WITHIN YOUR AXIS INSTALLATION:


OK, you've got axis running in tomcat, now you need to get sesame, configure it with mysql & upload some test data. 


-- Make sure you place these SESAME JARS in the LIB for the axis webapp in your servlet container:

commons-fileupload-1.0.jar           
sesame.jar  
xercesImpl.jar
mysql-connector-java-2.0.14-bin.jar  
soap.jar    
xml-apis.jar


******************************************************************************
OBTAIN THE DREFT SOURCE CODE and BUILD IT:

For the zip file, please see:

http://www.w3.org/2001/sw/Europe/reports/thes/dreft/

Unzip this into your chosen 'skos' location.
You may use the build.properties file to compile your source code and deploy
it to tomcat if you wish. You must edit the build.properties file for your own system (i.e. 
your tomcat.base (path to tomcat) and manager.url for tomcat).
You must have a recent version of Ant installed. 

*******************************************************************************
*Configuration of the skos service:
--- we provide a configuration properties file which you must complete for a successful install for the service.
You will find this configuration file in the
		/configuration/sesame/   directory. It is called "rdf_database.properties"
Here is a sample:

# SESAME configuration 

syncSail=org.openrdf.sesame.sailimpl.sync.SyncRdfSchemaRepository
permSail =org.openrdf.sesame.sailimpl.rdbms.RdfSchemaRepository

# the databaseid is the same as the <repository id="rdbms-skos-db">  that 
# you chose when you configured sesame (see above - re  Sesame's system.conf file. )
databaseid=rdbms-skos4-db

# the databasename is the same as <title>SKOS Inferencing Test DB</title>
# the title you gave to the repository when you configured sesame
# (see above - re  Sesame's system.conf file. )
#
databasename=SKOS Inferencing Test DB

# the rulefile is the skos entailement file which should be
# in the same directory as this file
#
rulefile=entailment-skos-core-20040228.xml

# MYSQL configuration - database properties for 
# JDBC connections to the mysql backend

jdbc_driver=org.gjt.mm.mysql.Driver
jdbc_conn_url=jdbc:mysql://127.0.0.1:3306/thes_inf_testdb
jdbc_username=thes_admin
jdbc_passwd=


--As stated above, you must also ensure that the SKOS rule file
("entailment-skos-core-20040228.xml" or more recent)  is in the    /configuration/sesame/   directory.
*******************************************************************************


-- Now copy the .class files for the entire skos api over to the axis webapp classes directory.

-- Next, deploy the skos service using the Axis AdminClient tool & the deploy.wsdd that we provide with the code. 
Within eclipse this means you run adminclient with parameters:


 deploy.wsdd
 & set it to the appropriate directory from within Eclipse
(e.g.
/home/nr8262/webservices/jakarta-tomcat/jakarta-tomcat-4.1.29/webapps/axis/WEB-INF/classes/org/w3/y2001/sw/Europe/skos/
- i.e. the directory in which the deploy.wsdd can be found).
Running this tool will result in adding an entry for the skos service to Axis's server config file in the axis webapp directory in tomcat.

[you should see:
Processing file deploy.wsdd
<Admin>Done processing</Admin>]

-- To see if you successfully deployed the skos service try to view its wsdl: 
http://localhost:8080/axis/services/Service?wsdl   
[or here
http://localhost:8080/axis/services/SKOSThesaurusService?wsdl]


*****************************************************************
***********
3. POST-INSTALLATION TESTING
***********

If you have sesame installed and loaded with thesaurus data & the skos service correctly deployed to axis running in tomcat, 
you should now be ready to test the service by running an axis client. 


The process below describes how to run the 'skostestclient_getConcept' class to send a uri to the service and receive back the corresponding 
Concept object - its preferred and non-preferred terms will get dumped out to screen & you can view the soap messages via TCPMON.

- start tomcat
- start mysql server
[e.g. START THE SERVER
mysql]$ bin/mysqld_safe --user=mysql &
]

Then run a client class

-- Now try running the skostestclient_getConcept.java file which comes with the code.  
NOTE we currently hardcode the service's url to localhost port 8081 (not 8080) so you can view soap messages 
through TCPMON (see above re TCPMON).
 
This client call's the getConcept method on the service.
When you run the client you should see this output:

"
- here we go then
Usage: OK, for now it's ok cos everyting's harcoded, but soon it'll be a case of gimme something! 
Just about to invoke the call
Gonna look at that concept now
Concept preferred label: English cuisine
Concept Non preferred label: English dishes
Concept Non preferred label: English food
Process terminated
"

And through TCPMON you should see something like:


REQUEST MESSAGE:

POST /axis/services/Service HTTP/1.0
Content-Type: text/xml; charset=utf-8
Accept: application/soap+xml, application/dime, multipart/related, text/*
User-Agent: Axis/#axisVersion#
Host: 127.0.0.1
Cache-Control: no-cache
Pragma: no-cache
SOAPAction: ""
Content-Length: 779

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <soapenv:Body>
  <ns1:getConcept soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://www.w3.org/2001/sw/Europe/skos/ServiceAPI">
   <arg1 href="#id0"/>
  </ns1:getConcept>
  <multiRef id="id0" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns2:URI" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns2="http://www.w3.org/2001/sw/Europe/skos/namespace">
   <uri xsi:type="xsd:string">http:/example.com/Concept/0002</uri>
  </multiRef>
 </soapenv:Body>
</soapenv:Envelope>

RESPONSE MESSAGE:


HTTP/1.1 200 OK
Content-Type: text/xml;charset=utf-8
Date: Thu, 27 May 2004 13:42:46 GMT
Server: Apache-Coyote/1.1
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <soapenv:Body>
  <ns1:getConceptResponse soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://www.w3.org/2001/sw/Europe/skos/ServiceAPI">
   <getConceptReturn href="#id0"/>
  </ns1:getConceptResponse>
  <multiRef id="id0" soapenc:root="0" soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xsi:type="ns2:Concept" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns2="http://www.w3.org/2001/sw/Europe/skos/namespace">
   <uri xsi:type="ns2:URI" xsi:nil="true"/>
   <externalID xsi:type="xsd:string" xsi:nil="true"/>
   <internalID xsi:type="xsd:string" xsi:nil="true"/>
   <preferredLabel xsi:type="xsd:string">English cuisine</preferredLabel>
   <nonPreferredLabels xsi:type="xsd:string">English dishes</nonPreferredLabels>
   <nonPreferredLabels xsi:type="xsd:string">English food</nonPreferredLabels>
   <inScheme xsi:type="ns2:URI" xsi:nil="true"/>
   <scopeNote xsi:type="xsd:string" xsi:nil="true"/>
   <definition xsi:type="xsd:string" xsi:nil="true"/>
   <example xsi:type="xsd:string" xsi:nil="true"/>
  </multiRef>
 </soapenv:Body>
</soapenv:Envelope>


***********************
RUN THE WHOLE TEST SUITE ON THE REMOTE DREFT DEMO SERVICE
***********************

Run this class: 

skostestsuiteclient.java

and it will run through the majority of the operations on the remote service. You can also set it
to run through tests on your own local service.

See the accompanying file: TESTRESULTS for our own request/reponse messages, recorded in trials.




****************
***************************************
4. HOW DO I ADD A NEW REPOSITORY IMPLEMENTATION?
***************************************

(information for java developers who want to implement a new rdf backend
 in addition to the existing Sesame one)

You need to subclass AbstractRepo following the example we have given with 
SesameRepo. You might call your subclass "JenaRepo" for example.

You need to create a new directory under the "configuration" directory and call it
say "jena". Then create a new rdf_database.properties file in a similar way to how 
we have for the "sesame" configuration directory. Note that any new configuration files
your RDF database requires need to be placed in this same configuration directory - 
you can develop the XXXRepo's "configure" method appropriately, so that this repository may
be configured appropriately.

Finally, you need to edit the DemoService class to set this field to your XXXRepo, change:

	private final String repoClassName="org.w3.y2001.sw.Europe.skos.repository.SesameRepo";
to
	private final String repoClassName="org.w3.y2001.sw.Europe.skos.repository.XXXRepo";




*************************************
5. HOW DO I ACCESS THE DREFT API DIRECT - FROM A NON-SERVLET CONTEXT? 
*************************************

Please see the documentation in DemoService for information relating embedding Dreft in your application.
See also our sample: StandaloneServiceImplTest.java




 
