Introduction

The Arcadia-funded COAR Notify Project is developing and accelerating community adoption of a standard, interoperable, and decentralised approach to linking research outputs hosted in the distributed network of repositories with resources from external services such as overlay-journals and open peer review services, using linked data notifications. As part of this project, COAR is funding the development of platforms and systems to support the exchange of linked data notifications across partner organisations and the workflows to manage notifications in those platforms and systems. As the largest adopted repository platform in the World one of the first platforms to be addressed is DSpace the implementation of which has been entrusted to 4Science.

The application can receive and send LDN messages concerning items with external systems. The LDN system is the protocol of message exchanging; the Quality Assurance system is the mechanism used to approve or reject item updates.

How to enable

Configuration properties involved:

Relation with the Quality Assurance Correction Service

The LDN system, as a message exchanging i/o system, has an inbox and an outbox. Every LDN message refers to a Notify Service: all the Notify Services are configured manually from the admin application form. A Notify Service is just like an authority labelled on LDN messages.

The Quality Assurance system is the implementation of item metadata updates operations. A Quality Assurance Event contains informations for item metadata updates: QAEvent are stored into QAEvent solr collection. All of the QAEvent are shown on an administration form. Every QAEvent can be accepted, ignored or removed: if accepted some metadata of the referred item are modified, if ignored or removed nothing about the item is modified. Every QAEvent has a property named source : qa events created by processing an LDNMessage has source="coar-notify".

To process an LDN message means to create a QAEvents; as soon as the QAEvent is accepted the referred item is updated. We do it by routing LDN Messages from the queue to the LDN Router. The LDN Router is designed into the spring file /dspace/config/spring/api/ldn-coar-notify.xml on the bean ldnRouter. The router de-multiplex two attributes read from the LDN Message (two string values inside the array called type) content to java classes called Processors: see org.dspace.app.ldn.processor.* package to view them all.

The match between LDN message type and the QAEvent topic (suddenly created by the evaluation of the ldn message from the queue) is configured onto ldn-coar-notify.xml spring file. Every Processor owns a list of actions. An action is often an email to be sent and an action to create the qa event. The LDN processor receives the QAEvent topic as a parameter and creates the relative QAEvent.

LDN Autodiscovery mechanism

Third party system can retrieve the location of the repository LDN InBox via the LDN autodiscovery mechanism, nevertheless to be able to interact with DSpace they need to be approved by a Repository Administrators and listed in the LDN Services Directory (see next paragraph); otherwise their messages will be flagged as untrusted and not processed at all.

LDN Services Directory

You need to register the external services to trust incoming LDN messages / send LDN messages to:

Administrators can do that using the menu LDN Services. New/Edit Notify Service with this form:

LDN Inbox queueing

LDN incoming messages are stored into the ldn_message database table. The incoming LDN messages is stored into a logical queue. The queue_status column of the table contains the status of the LDN message inside the queue. All the possible queue_status values are described into the java class org.dspace.app.ldn.LDNMessageEntity as integer constants.

If valued as queued the LDN message will be processed. 

LDN Message logical queue is managed by:

• The LDN Message Extractor is an asynchronous DSpace task that reads the oldest manageable LDN message and enroute it to the LDN Router; the extractor instance ends as soon as the LDN message extracted is routed and processed or failed. For each tentative the attempts counter of the message is increased by 1 and the timeout is increased by X minutes, where X= the value of the configuration property ldn.processor.queue.msg.timeout | defaulted to 60. Cron configuration at property ldn.queue.extractor.cron

• The LDN Message Timeout Checker  is an asynchronous DSpace task that looks for timed-out messages with attempts less than Y, where Y= the value of the configuration property ldn.processor.max.attempts. Cron configuration at property ldn.queue.timeout.checker.cron

Please consider that this means that the corresponding QAEvent is not automatically created as soon as the LDN Message is received. LDN Message (accepted by the inbox controller and stored into ldn_message table) => Extractor (started runtime as a batch) => Routing => Processor => Action that creates the QA - see class org.dspace.app.ldn.action.LDNCorrectionAction

Offer, Acknowledgement and Announce

Considering these possible scenarios here at: COAR Notify Protocol: Example Scenarios

We have to keep the user updated about the item situation. We do it with colored boxes on its landing (handle) page.

When the Offer message (being it review, endorsement or ingest) has been sent as an outgoing LDN message, and nothing else about it has been received, the yellow box is shown.

When the Offer message has been followed by a related incoming Acknowledgement message: if the ack is an accept/tentative accept the box shown is blue, if the ack is a tentative rejct the box shown is red.

When Offer message has been followed by a related Announce incoming message, there are no boxes of this kind shown, because it’s expected for the boxes about qa to be shown!

Level of Trust

It’s a number 0 < # < 1. Triggers an automatic approval of QAEvent once the LDN message is extracted. On qaevents.xml spring configuration files the bean qaScoreEvaluation describes three different boundaries defaults to manage the level of trust:

<property name="scoreToReject" value="0.3" />
<property name="scoreToIgnore" value="0.5" />
<property name="scoreToApprove" value="0.8" />

score <= rejection deletes the QAEvent;  score <= ignore discards the QAEvent;  score >= approval accepts the QAEvent automatically, so nobody will see it from the Quality Assurance Page because it’s created and right after approved.

WORK IN PROGRESS Test calls: Request Review (send automatic LDN Message on item submission)

Let's configure a new Notify Service such as:

Test calls: Announce Relationship

Let's trigger an Announce Relationship (see COAR official pattern documentation here).

Firstly configure a Notify Service such as:

By clicking the + New green button fill the form as:

Pick the following POST call from the Postman project (see below to download it and import it to your installed PostMan instance): let's emulate the Notify Service external system sending us an LDN Message!

Let's evaluate the request' json body (some field is commented) according to the COAR official documentation of this pattern:

{
"@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
],
"actor": {
    "id": "https://research-organisation.org",
    "name": "Research Organisation",
   "type": "Organization"
},
"context": {
    "id": "http://localhost:4000/handle/123456789/1119", //IMPORTANT!! this is the handle link to the item from the referred local repository 
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
        "id": "https://another-research-organisation.org/repository/datasets/item/201203421/data_archive.zip",
        "mediaType": "application/zip",
        "type": [
            "Article",
           "sorg:Dataset"
        ]
    }
},
"id": "urn:uuid:2f4ec582-109e-4952-a94a-b7d7615a8c69", // IMPORTANT!! this must be a unique value because it's going to be stored as uuid into the ldn_message table.
"object": {
    "as:object": "newValue",
    "as:relationship": "somethingElse", // as "http://purl.org/vocab/frbr/core#supplement" or something else
    "as:subject": "https://research-organisation.org/repository/item/201203/421/",
    "id": "http://localhost:4000/handle:123456789/1119",
    "type": "Relationship"
},
"origin": {
    "id": "https://research-organisation.org/repository",
    "inbox": "2f4ec582-109e-4952-a94a-b7d7615a8c69",  // through this field we reach the configured Notify Service entity by matching this field to the one named LDN Inbox URL
    "type": "Service"
},
"target": { // MUST be valid but it is not rilevant
    "id": "https://another-research-organisation.org/repository",
    "inbox": "https://another-research-organisation.org/inbox/",
    "type": "Service"
},
"type": [
    "Announce",
    "coar-notify:RelationshipAction"
]
}

See below the successful response:

This means the LDN Message has been enqueued. Please remember the enqueued ldn messages must be evaluated by the Extractor.

Postman Project

Here’s the Postman collection for all test purposes: 

Coar Notify.postman_collection.json