Summary
Reference:
User Story
A professor wishes to add the reference to a scientific article, Irrespective of whether he chooses ORCID or VIVO, the information he will enter in either of these platforms will be mutually updated,
Issue
![](/download/attachments/199525065/image2020-12-15_14-46-10.png?version=1&modificationDate=1608057970320&api=v2&effects=drop-shadow)
About Kafka
Goal of using Kafka
![](/download/attachments/199525065/image2020-12-15_3-48-34.png?version=1&modificationDate=1608018514995&api=v2&effects=drop-shadow)
What is Kafka?
see also https://kafka.apache.org/intro
Messaging system | Event streaming |
---|
Event streaming thus ensures a continuous flow and interpretation of data so that the right information is at the right place, at the right time. | - To publish (write) and subscribe to (read) streams of events, including continuous import/export of your data from other systems.
- To store streams of events durably and reliably for as long as you want.
- To process streams of events as they occur or retrospectively.
|
| |
ORCID to VIVO Dataflow through Kafka
![](/download/attachments/199525065/image2020-12-15_9-16-26.png?version=1&modificationDate=1608038186683&api=v2&effects=drop-shadow)
Description
Prerequisite
Dataflow execution
Results
![](/download/attachments/199525065/image2020-12-16_5-17-39.png?version=1&modificationDate=1608110259801&api=v2)
In summary
- It has been shown that it is possible to use Kafka to populate VIVO from ORCID
Several points require special attention
- The ORCID ontology needs to be refined and clarified.
- The mapping between ORCID and VIVO also needs to be worked on
- The structure of the Kafka message has to be designed to respect the add/delete/modify record actions
- Several minor bugs need to be fixed in the scripts.
Future Plan
- Building a POC VIVO → Kafka → ORCID
- Proving the architecture to operate in event-driven and real-time mode
- Getting POCs to Java
- Redesigning the mapping process, ORCID ontology structure and message structure