Page History
...
/server/api/config/logicitemfilters
...
Introduction
4Science has developed the integration with the COAR system. The application can receive and send messages concerning items with externl systems. The LDN system is the protocol of message exchanging; the Quality Assurance system is the mechanism used to approve or reject item updates.
How to enable
Configuration properties involved:
coar-notify.enabled = true|false
If false the Coar logo in the footer linked to the informative page will appear.
ldn.enabled = true|false
If false the LDN inbox is disabled, so the system is not “listening”
Relation with the Quality Assurance Correction Service
The LDN system, as a message exchanging i/o system, has an inbox and an outbox. Every LDN message refers to a Notify Service: all the Notify Services are configured manually from the admin application form. A Notify Service is just like an authority labelled on LDN messages.
The Quality Assurance system is the implementation of item updates operations. A Quality Assurance Event contains informations for item metadata updates: QAEvent are stored into qaevent solr collection. All of the qaevent are shown on an administration form. Every qaevent can be accepted, ignored or removed: if accepted some metadata of the referred item are modified, if ignored or removed nothing about the item is modified.
To process an LDN message means to create a QA Events; as soon as the QAEvent is accepted the referred item is updated.
The match between LDN message type and the QA event topic is configured onto ldn-coar-notify.xml spring file. Every ldn message type has an action related, which owns a list of actions. An action is often an email to be sent and an LDNProcessor to call. The LDN processor receives the qa event topic as a parameter and creates the relative qa event.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
LDN Autodiscovery mechanism
Third party system can retrieve the location of the repository LDN InBox via the LDN autodiscovery mechanism, nevertheless to be able to interact with DSpace they need to be approved by a Repository Administrators and listed in the LDN Services Directory (see next paragraph); otherwise their messages will be flagged as untrusted and not processed at all.
LDN Services Directory
You need to register the external services to trust incoming LDN messages:
...
Every ldn message is going to have a link referring to the stored notify service. At time of writing it is the LDN Inbox URL. If the incoming LDN message has the ldn inbox url equals to one of the stored Notify Service' ldn_url - the ldn message is stored and queued as a trusted message, otherwise it’s flagged as untrusted.
Untrusted ldn messages are not processed.
LDN Inbox queueing
LDN incoming messages are stored into the ldn_message database table. The incoming ldn messages is stored into a logical queue. The queue_status column of the table contains the status of the ldn message inside the queue. If valued as queued the ldn message will be processed. The LDN Message Extractor is an asynchronous DSpace task that reads the oldest manageable ldn message and route it to be processed; the extractor instance ends as soon as the ldn message extracted is routed and processed or failed to be.
Please consider that this means that the corresponding QAEvent is not automatically created! The corresponding QAEvent is created by the processor instance of the ldn message, invoked by the extractor.
LDN message processing
All the possible incoming ldn messages patterns are documented in the official link: COAR Notify Protocol: Notification Patterns Every ldn message json ends with the array named Type. Inside the DSpace spring configuration settings we store a list of ldn message types to be routed to certain processors. See dspace/config/spring/api/ldn-coar-notify.xml the bean ldnRouter and its incomingProcessors map property. When the extractor finds an ldn message with a Type mismatching any of the processors, the ldn message queue status moves to unmapped.
Unmapped ldn messages are not processed.
At time of writing the processing of an ldn message corresponds to the creation of a QA Event.
LDN Messages Queue is maintained by two different tasks:
Message Extractor
Timeout Checker
The Extractor pick an ldn message from the queue, assigns a processor and launches it. If the processor ends successfully the ldn message is out out of the queue, updating its queue_status to processed. Also, queue_last_start_time and queue_timeout are updated: queue_last_start_time is updated with the current timestamp, queue_timeout is updated with a value current timestamp + the configuration value of the key ldn.processor.queue.msg.timeout. Also, increases queue_attempts by 1. The extractor does not consider the ldn messages with attempts already done >= the value of the configuration key ldn.processor.max.attempts
The Timeout looks for timed-out ldn messages with queue_status as processing. If attempts < ldn.processor.max.attempts the message is enqueued again, moving its status to queued; otherwise its status is set to failed.
Offer, Acknowledgement and Announce
Considering these possible scenarios here at: COAR Notify Protocol: Example Scenarios
We have to keep the user updated about the item situation. We do it with colored boxes on its landing (handle) page.
When the Offer message (being it review, endorsement or ingest) has been sent as an outgoing LDN message, and nothing else about it has been received, the yellow box is shown.
When the Offer message has been followed by a related incoming Acknowledgement message: if the ack is an accept/tentative accept the box shown is blue, if the ack is a tentative rejct the box shown is red.
When Offer message has been followed by a related Announce incoming message, there are no boxes of this kind shown, because it’s expected for the boxes about suggestions to be shown!
Automatic meaning
Automatic is a flag true/false of Inbound Patterns and are strictly linked to item filter. The Inbound Pattern is inbound from the NotifyService point of view. So they are configuration for ldn messages of the DSpace outbox. Automatic triggers an Outbox ldn message targeting the notify service and targeting the item in submission. The automatic flag involves only the submission phase of an item. If no item filter is set - the flag is applied on all submissions.
Level of Trust
it’s a number 0 < # < 1. Triggers an automatic approval of QA Event once the ldn message is extracted. On qaevents.xml spring configuration files the bean qaScoreEvaluation describes three different boundaries defaults to manage the level of trust:
<property name="scoreToReject" value="0.3" /><property name="scoreToIgnore" value="0.5" /><property name="scoreToApprove" value="0.8" />
<= rejection deletes the qaevent;
<= ignore discards the qaevent;
>= approval accepts the qaevent automatically, so nobody will see it from the Quality Assurance Page because it’s created and right after approved.
Test calls
Here’s the Postman collection for test purposes: Coar Notify.postman_collection.json
Integration with the submission system
….
the validation will check the configured item filter
Notification is sent only after that the item has been archived, after the workflow. Please note that the LDN messages are sent in an asynchronous way by a job that by default is executed every five minutes
LDN InBox
location
it is asynchronous, store the message in the db, process it via a scheduled job
…configuration of the frequency…
How to configure what happen once a LDN message is processed: the LDN Actions
Document how the LDN routing works and can be configured (spring xml file to use, snippet of examples/default configuration). List the available LDN Actions and document their configuration options
Document how to enable the automatic processing (spring xml file to use, snippet of examples/default configuration)
References
[1] Bollini, Andrea, Buso, Irene, Lombardi, Corrado, Maffei, Stefano, Mornati, Susanna, Shearer, Kathleen, Walk, Paul, Klein, Martin, & Rodrigues, Eloy. (2023, June 14). Implementing the COAR Notify Protocol in DSpace 7. Open Repositories 2023 (OR2023), Stellenbosch, South Africa. Zenodo. https://doi.org/10.5281/zenodo.8091621
...
Overview
Content Tools