Task hopper

From LinkedGov

Scope

What part of LinkedGov is this and what should it do?

The task hopper serves potentially incorrect or problematic data to users for checking through the gamification framework.

Inputs/Outputs

What data is required for this component to work, and what will it return?

Inputs

What data does this component take in, and from where? Provide an example of it for formatting and illustration.

Every time the main database is modified, if a particular document contains an explicitly marked error, it sends the task hopper a notification of the error in the form of a URI to the issue named graph as well as the document graph.
- This will use an endpoint with a name like /task/import. There will also be a /task/new endpoint.
Returned tasks from games.

Outputs

What data does this component produce, and where does it go. Provide an example of it for formatting and illustration.

Modifications to the database in the form of SPARQL changesets.
Task descriptions for games.

User journey

What does the user see? What options do they have? What happens for each option (from the user's point of view)?

Functional spec

What is the process that this component carries out? Include diagrams and references to other LinkedGov components diagrams, if they are helpful.

External dependencies

Name	URL	Role
Jena (and ARQ)	jena.sourceforge.net	Parsing RDF and performing SPARQL requests
XOM	xom.nu	XML handling
Apache HTTP Components	hc.apache.org	HTTP.
Apache Commons-Lang	homepage	String manipulation and other common tasks not provided by the Java class libraries.
Maven	maven.apache.org	Build tool.
URIBulder	Groovy project	Constructing URIs. We've extracted one class from the library.
Apache Jersey	jersey.java.net	Web framework that makes providing RESTful services with content negotiation.

Deployment steps

Download source files

Setup eXist db

Download the jar

Install

Load data into eXist

Either install Ruby and run the scripts found in the linkedgov-examples repo,

or,

upload the files into eXist manually through the Admin interface (http://localhost:8080/exist/admin/admin.xql?panel=browse) or by using the Java Web client (http://localhost:8080/exist/webstart/exist.jnlp).

If you upload the files manually, you must make sure the RDF files do *not* have the extension ".rdf". Renaming the files locally on your machine to not include the extensions, does not prevent them from acquiring the .rdf inside eXist.

Run

Navigate to the eXist directory and run:

bin/startup.sh

Setup taskhopper

Navigate to the taskhopper directory after extraction and run:

mvn jetty:run -Djetty.port=8180

Maven will download the dependencies needed first time round as well as compile the source code, at which point the Jetty server will start running on port 8180.

Setup alpha-game

Run from inside the alpha-game directory:

python -m SimpleHTTPServer

You should then be able to access the index.html page at http://localhost:8000

API documentation

/task/new
- issue-uri parameter.
- graph-uri parameter.
- task-type parameter
- Add a task to the hopper.
- Returns XML document describing the task which contains a task ID.
- If there are errors, it will return a 403 Forbidden message.
- If there are internal errors, it will return a 500 Internal Server error.
- Errors will be accompanied by XML documents containing an error tag containing a description of the problem.
/task/import
- url parameter.
- Loads a URI, parses issues out of the document and loads those into the task hopper.
- Usage: internal.
/task/random
- type parameter.
- Retrieves a random task from the Task Hopper. If the type paramater is specified, it retrieves only a task of that type.
- Returns an XML document with root element rsp and zero or one task elements.
- Uses content negotiation to serve XML, JSON and JSONP.
/task/id (GET)
- Returns a description of the task with the relevant id.
- Uses content negotiation to serve XML, JSON and JSONP.
/task/id (POST)
- Endpoint for sending back completed tasks.
- action Parameter takes one of the following values:
  - null - Marks the value as null.
  - edit - Changes the value
    - You need to also provide the value you are changing it to in the parameter value.
- value parameter (used with action=edit, ignored with action=null)
  - Takes a replacement string for the value in question.
- Uses content negotiation to serve XML, JSON and JSONP.
/task/problem (GET)
- If, during a game or other use of the API, you encounter a problem, simply fire off a ping to /task/problem to record that you have had a problem.
- Uses content negotiation to serve XML, JSON and JSONP.

Use of the Task Hopper API

External

After we have received data, we process it to see if there are any problems in the data. Problems we identify get turned into tasks that get served up through the task hopper API.

Processing tasks is very simple: you load a task from the hopper and it returns you a task number with a description (in XML or JSON) of what is wrong. The description gives details of the issue, the source of the issue and gives example data and pointers to documentation. This task description is used to construct the interface for end users to fix the tasks.

After the user has selected what needs to be done with the data, they simply submit the data back to the task hopper API by POSTing the data to /task/id (e.g. /task/5).

Although data is stored and handled internally as RDF, when building games and apps around the task hopper, you don't have to know anything about RDF: the data is all exposed as XML or JSON/JSONP.

Internal

Implementation documentation

The TaskHopper is implemented in Java using Apache Jersey and a number of other libraries (listed at Task hopper#External dependencies).

Relevant tasks

These open tasks involve this component of LinkedGov:

ALPHA008 Black Box Task Hopper

Note: these are all cross-linked. If you remove a link from this list, please also remove the reference to this task in the page it links to.

XQuery examples

Create:

   for $task in collection("linkedgov-meta/taskhopper")/linkedgov-taskhopper
   return update insert <task id="1" /> into /linkedgov-taskhopper

Read all:

   for $task in collection("linkedgov-meta/taskhopper")/linkedgov-taskhopper/task
   return $task

Read specific:

   let $id := request:get-parameter("id", "1")
   for $task in collection("linkedgov-meta/taskhopper")/linkedgov-taskhopper/task[@id=$id]
   return $task

Delete:

   for $task in collection("linkedgov-meta/taskhopper")/linkedgov-taskhopper/task[@id='1']
   return update delete $task