Note: this is NOT the official version of the DAS specification.

Distributed Sequence Annotation System (DAS) Spec Review

Oct 1,2008

This is a working document and a proposal for a reworked DAS specification which hopes to clarify the DAS spec based on how DAS is being used in the community today and to include commands from the 1.53E spec and some of the 2.0 spec. Also the document has been adjusted to reflect changes in the use of DAS away from a solely genome centric protocol to a more open one encompassing other reference/coordinate systems such as protein sequences and structures. The spec also includes references to the DAS Registry which is essential for implementing an SOA architecture. Note: this is a technical document but should be readable and understandable by people without a deep understanding of broader technical issues and other system architectures. Note that we are proposing to change dtd for xsd

System Architecture

The Distributed Annotation System is a network of server and client software installations distributed across the web. The DAS protocol is a standard mechanism through which clients can communicate with servers in order to obtain various types of biological data. The protocol defines:

• the communication method
• the query model
• the data format

By enforcing these constraints, DAS allows a client to integrate data from many diverse sources implementing the protocol at a scaleable development cost.

The DAS network of servers comprises a registry, several reference servers and several annotation servers. Tying these together are the concepts of reference objects and coordinate systems.

Reference Objects

Reference objects are items of data with stable identifiers that are targets for annotation. At the most abstract level a reference object might be an annotatable concept or idea (e.g. a particular gene), but usually describes a biological unit within which annotations can be positioned. For example, “P15056” refers to a protein sequence upon which annotations can be based. Similarly, “chromosome 21” refers to a DNA sequence.

Individual reference objects can in fact have several versions, and it is important to recognise that annotations based upon different versions of the same reference entity are not necessarily equivalent.

Annotations

Annotations are pieces of information that are always attributed to a reference object. Annotations are usually positional, that is they refer to a specific location within a reference object. An exon within a genomic sequence is an example. Annotations can also be non-positional, in which case they can be considered as information attributed to the whole of the reference object. For example, the description of a protein or gene.

Coordinate Systems

A coordinate system is a stable, logical set of reference objects. A coordinate system provides a mechanism to uniquely identify reference objects that share identifiers, such as chromosomes. For example, chromosome 21 might identify several reference objects from different species’, but only one within the NCBI 36 human assembly. Thus, “human NCBI 36 chromosomes” is a coordinate system containing 25 reference objects.

Coordinate systems are formally described using four properties:

• The category or type of annotatable entity. For example a chromosome, contig or protein sequence
• The authority responsible for defining the coordinate system. For example NCBI or UniProt
• The version, for coordinate systems containing entities that are not versioned (e.g. genomic assemblies)
• The species, for coordinate systems containing only entities from a single organism

Of these, category and authority are required.

Some example coordinate systems:

Reference & Annotation Servers

A reference server is a DAS server that provides core data for the reference objects in a particular coordinate system. For example, the reference server for “UniProt Protein sequence” provides the actual sequence for each UniProt entry. It does this by implementing the DAS sequence command. So that clients can discover the available reference objects in a coordinate system, a reference server must also list them via the entry_points command.

Annotation servers are specialized for returning lists of annotations for the reference objects within a coordinate system. This is done by implementing the DAS features command.

Note: The distinction between reference and annotation servers is conceptual rather than physical. That is, a single server instance can in fact play both roles by offering both sequences and annotations of those sequences.

Note: A server may support multiple coordinate systems.

The DAS Registry

The DAS registry is a special component of DAS, fulfilling the following roles:

1. Catalogues and describes the capabilities and coordinate systems of DAS services
2. Allows discovery of available DAS services via both human and programmatic interfaces
3. Automatically validates registered DAS sources to ensure that they correctly implement the protocol
4. Periodically tests DAS sources and notifies their administrators if they are unavailable
5. Provides a mechanism for activating or highlighting individual DAS services in clients

Clients

A DAS client typically integrates data from a number of DAS servers, making use of the different data types. For example, a client might implement the following procedure for a particular sequence location:

1. Contact DAS registry to find reference and annotation servers for a particular genomic assembly
2. Obtain sequence from the reference server
3. Obtain sequence features from each of the annotation servers
4. Display the annotations in the context of the sequence

See this example in diagrammatic form: PNG image

Client/Server Interactions

The DAS is web-based. Clients query the reference and annotation servers using the HTTP protocol (see RFC2616) by sending a formatted URL request to the server. Servers process the request and return a response in the form of a formatted XML document (see W3C Extensible Markup Language) according to a predefined schema.

The Request

All DAS requests take the form of a hierarchical URL. Each URL has a site-specific prefix, followed by a standardized path and query string. The standardized path begins with the string /das. This is followed by URL components containing the data source name and a command. Should put some guidance or specify the ability for servers to accept encoded URLs For example:

http://das.sanger.ac.uk/das/ccds_mouse/features?segment=1:174405453,174408689
^^^^^^^^^^^^^^^^^^^^^^^ ^^^ ^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  site-specific prefix  `das`  `data src`  `command`   `arguments`

</code>

In this case, the site-specific prefix is http://das.sanger.ac.uk. The request begins with the standardized path /das, and the data source, in this case /ccds_mouse. This is followed by the command /features, which requests a list of features, and a query string providing named arguments to the /features command.

Thus, a single DAS server hosts one or more DAS sources, allowing it to provide different types of information, and/or information in several coordinate systems. This example source provides consensus CDS transcripts for mouse chromosomes, but the same server provides a number of other sources, including a similar source for human along with sources containing very different types of data.

More information on the format of the request and the various available commands is given [#commands below].

The query string portion of the request (the “?” symbol rightward) can be POSTed to the URL following conventional HTTP standards. Since some queries can be quite large, this is the recommended way of argument passing.

’’‘DELETE: The request may be replaced with a SOAP-style XML-encapsulated document in future versions of this specification.

The Response

The response from the server to the client consists of a standard HTTP header with DAS status information within that header, followed optionally by XML content that contains the answer to the query. The DAS status portion of the header consists of three lines. The first is X-DAS-Version and gives the current protocol version number, currently DAS/1.6. The second line is X-DAS-Status and contains a three digit status code which indicates the outcome of the request. The third is X-DAS-Capabilities, which describes the parts of of the spec the server implements.

Here is an example HTTP header: (provided by Web server)

HTTP/1.1 200 OK                          
Date: Sun, 12 Mar 2000 16:13:51 GMT          
Server: Apache/1.3.6 (Unix) mod_perl/1.19    
Last-Modified: Fri, 18 Feb 2000 20:57:52 GMT 
Connection: close                            
Content-Type: text/plain                     
X-DAS-Version: `DAS/1.6`
X-DAS-Status: 200
X-DAS-Capabilities: error-segment/1.0; unknown-segment/1.0; unknown-feature/1.0; ...
data follows...

</code>

The defined status codes are listed in Table 1.

The HTTP/1.0 protocol allows web clients to request byte-level compression of the response by sending the HTTP header Accept-Encoding header. Web servers that are capable of it can reply with a Content-transfer-encoding header and a compressed body. Implementors of DAS clients and servers may wish to implement this HTTP feature.

X-DAS-Capabilities

The X-Das-Capabilities header provides an extensible list of the capabilities that the server provides. This can be used by

those writing experimental extensions to DAS to flag clients that those extensions are available. Capabilities have the form CapabilityName/Version and are separated by semicolon, space, as in “capabilityA/1.0; capabilityB/1.4; capabilityC/1.0”. The following standard capabilities are present in the DAS/1.6 protocol:

Reference Object IDs

The ID used by a client or server to refer to a reference object can contain any set of printable characters (including the space character), but not the colon character (“:”), which is reserved for separating reference IDs from sequence ranges (see below). The newline, tab and carriage return characters are also reserved for future use.

A data source that uses the colon character for its internal IDs must map this character to another one on the way in and on the way out. For example:

   Client request       server's internal id         Response to client

   gi-123456       -->  gi:123456                    --->  gi-123456

   gi-123456:1,1000 --> gi:123456 start=1 stop=1000  --->  gi-123456:1,1000

The Queries

This section lists the queries recognized by reference and annotation servers. Each of these queries begins with some site-specific prefix, denoted here as PREFIX. The other meta-variable used in these examples is DSN, which is a symbolic data source (as seen in the [#request above example.])

Sources Command

Retrieve the list of data sources for a server.

Scope: Reference and annotation servers.

Command: sources

Format:

PREFIX/das/sources

</code>

Description: This query returns the list of data sources that are available from this server. In particular the following information for a DAS server is important:

• The email address of the maintainer of a DAS source
• The coordinate system of the provided data
• Different properties that allow further description of a source

Arguments: none

Response:

The response to the sources command is the “DASSSOURCE” XML-formatted document:

<?xml version='1.0' encoding='UTF-8' ?>
<?xml-stylesheet type="text/xsl" href="das.xsl"?>
<SOURCES>
 <SOURCE uri="URI" title="title" doc_href="URL" description="description">
    <MAINTAINER email="email address" />
    <VERSION uri="URI" created="date">
      <COORDINATES uri="uri" source="data type" authority="authority" test_range="ID">coordinate string</COORDINATES>      
      <CAPABILITY type="das1:command" query_uri="URL" />
      <PROP name="key" value="value" />     
     </VERSION>
   </SOURCE>
</SOURCES>

Format:

Example Responses

• http://www.dasregistry.org/das/sources - the listing of sources at the DAS registration server
• http://www.ensembl.org/das/sources - the DAS sources provided by Ensembl. The DAS registration server uses this listing to automatically update the registration for these DAS servers.
• http://vega.sanger.ac.uk/das/sources - the VEGA DAS sources

Entry Points Command

Retrieve the list of reference objects for a data source

Scope: Reference servers.

Command: entry_points

Format:

PREFIX/das/DSN/entry_points

</code>

Description: This query returns the list of sequence entry points available and their sizes in base pairs.

Arguments:

ref (deprecated)
If a sequence reference ID is provided in the ref argument, the query will return the components of the sequence (its subsequences) rather than the list of top-level entry point sequences. This argument is DEPRECATED, and superseded by the “component” category of the features request.

type (deprecated)
For ACEDB servers, the type parameter provides the class of the reference sequence, Sequence by default. DEPRECATED

Response:

The response to the entry_points command is the “DASEP” XML-formatted document:

Format:

<?xml version=”1.0” standalone=”no”?> <!DOCTYPE DASEP SYSTEM “http://www.biodas.org/dtd/dasep.dtd”>

</code>

;

<!DOCTYPE> (required; one only)

The doctype indicates which formal DTD specification to use. For the entry_points query, the doctype DTD is “http://www.biodas.org/dtd/dasep.dtd”.