Persistent Identifier

It is good practice to use Persistent Identifiers to address items in a digital repository. There are many different systems for Persistent Identifiers: Handle , DOI , urn:nbn, purl and many more. It is far out of the scope of this document to discuss the differences of all these systems. For several reasons the Handle System is deeply integrated in DSpace, and DSpace makes intensive use of it. With DSpace 3.0 the Identifier Service was introduced that makes it possible to also use external identifier services within DSpace.

DOIs are Persistent Identifiers like Handles are, but as many big publishing companies use DOIs they are quite well-known to scientists. Some journals ask for DOIs to link supplemental material whenever an article is submitted. Beginning with DSpace 4.0 it is possible to use DOIs in parallel to the Handle System within DSpace. By "using DOIs" we mean automatic generation, reservation and registration of DOIs for every item that enters the repository. These newly registered DOIs will not be used as a means to build URLs to DSpace items. Items will still rely on handle assignment for the item urls.

DOI Registration Agencies

To register a DOI one has to enter into a contract with a DOI registration agency which is a member of the International DOI Foundation. Several such agencies exist, with each of them having different target groups and policies. Most of the registration agencies charge fees for registering DOIs. To make it quite clear: to register DOIs with DSpace you have to enter into a contract with a DOI registration agency.

DataCite is an international initiative to promote science and research, and a member of the International DOI Foundation. The members of DataCite act as registration agencies for DOIs. Some DataCite members provide their own APIs to reserve and register DOIs; others let their clients use the DataCite API directly. Starting with version 4.0 DSpace supports the administration of DOIs by using the DataCite API directly or by using the API from EZID (which is a service of the University of California Digital Library). This means you can administer DOIs with DSpace if your registration agency allows you to use the DataCite API directly or if your registration agency is EZID.

Crossref is another international DOI registration agency. Since DSpace 10.0 DSpace is able to register DOIs with Crossref.

General architecture

DSpace contains several features that interacts or uses DOIs. DSpace is build in a way that these features should work with any supported DOI registration agency. The configuration of those features and configuration of the DOI registration agency to use, are seaparted from each other. So most of this article is valid for all registration agencies currently supported.

General configuration of DOIs in DSpace

dspace.cfg

After you enter into a contract with a DOI registration agency, they'll provide you with user credentials and a DOI prefix. You have to enter these in the dspace cfg. Here is a list of DOI configuration options in dspace.cfg:

identifier.doi.user

identifier.doi.user = user123

identifier.doi.password

identifier.doi.password = top-secret

identifier.doi.prefix

identifier.doi.prefix = 10.5072

identifier.doi.namespaceseparator

identifier.doi.namespaceseparator = dspace-

Identifier Service

The Identifier Service manages the generation, reservation and registration of identifiers within DSpace. You can configure it using the config file located in [dspace]/config/spring/api/identifier-service.xml. In the file you should already find the code to configure DSpace to register DOIs. Just read the comments and remove the comment signs around the two appropriate beans.

After removing the comment signs the file should look something like this (I removed the comments to make the listing shorter):

<!--
    Copyright (c) 2002-2010, DuraSpace.  All rights reserved
    Licensed under the DuraSpace License.

    A copy of the DuraSpace License has been included in this
    distribution and is available at: http://www.dspace.org/license
-->

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

    <bean id="org.dspace.identifier.service.IdentifierService"
          class="org.dspace.identifier.IdentifierServiceImpl"
          autowire="byType"
          scope="singleton"/>

    <bean id="org.dspace.identifier.DOIIdentifierProvider"
        class="org.dspace.identifier.DOIIdentifierProvider"
        scope="singleton">
        <property name="configurationService"
            ref="org.dspace.services.ConfigurationService" />
        <property name="DOIConnector"
            ref="org.dspace.identifier.doi.DOIConnector" />
    </bean>

    <bean id="org.dspace.identifier.doi.DOIConnector"
        class="org.dspace.identifier.doi.DataCiteConnector"
        scope="singleton">
        <property name='DATACITE_SCHEME' value='https'/>
        <property name='DATACITE_HOST' value='mds.test.datacite.org'/>
        <property name='DATACITE_DOI_PATH' value='/doi/' />
        <property name='DATACITE_METADATA_PATH' value='/metadata/' />
        <property name='disseminationCrosswalkName' value="DataCite" />
    </bean>
</beans>

If you use other IdentifierProviders beside the DOIIdentifierProvider there will be more beans in this file. If you use another DOI registration agency than DataCite, the lines about the DOIConnector will differ. For Crossref these lines will simply look like this:

<bean id="org.dspace.identifier.doi.DOIConnector"
            class="org.dspace.identifier.doi.crossref.CrossRefConnector"/>

Please pay attention to configure the property DATACITE_HOST. Per default it is set to the DataCite test server. To reserve real DOIs you will have to change it to mds.datacite.org. Ask your registration agency if you're not sure about the correct address.

Sending metadata updates to your DOI registration agency

DSpace should send updates to your DOI registration agency whenever the metadata of an item changes. To do so, you must enable the DOIConsumer in your dspace.cfg (or local.cfg). You should remove the comments in front of the two following properties or add them to the local.cfg:

event.consumer.doi.class = org.dspace.identifier.doi.DOIConsumer
event.consumer.doi.filters = Item+Modify_Metadata

Then you should add 'doi' to the property event.dispatcher.default.consumers. After adding it, this property may look like this:

event.dispatcher.default.consumers = versioning, discovery, eperson, harvester, doi

DOIs and Item Level Versioning

If you enabled Item Level Versioning you may enable the VersionedDOIIdentifierProvider instead of the DOIIdentifierProvider. The VersionedDOIIdentifierProvider ensures that newer versions of the same Item gets a DOI looking as the DOI of the first version of and item, extended by a dot and the version number. With DSpace 6 this also became the default for handles if Item Level Versioning is enabled. In the configuration file [dspace]/config/spring/api/identifier-service.xml you'll find the possiblity to enable the VersionedDOIIdentifierProvider. If you want to use versioned DOIS, please comment out the DOIIdentifierProvider as only one of both DOIProviders should be enabled at the same time.

Command Line Interface

To make DSpace resistant to outages of DOI registration agencies we decided to separate the DOI support into two parts. When a DOI should be generated, reserved or minted, DSpace does this in its own database. To perform registration and/or reservation against the DOI registration agency a job has to be started using the command line. Obviously this should be done by a cron job periodically. In this section we describe the command line interface, in case you ever want to use it manually. In the next section you'll see the cron job that transfers all DOIs designated for reservation and/or registration.

The command line interface in general is documented here: Command Line Operations.

The command used for DOIs is 'doi-organiser'. You can use the following options:

Currently you cannot generate new DOIs with this tool. You can only send information about changes in your local DSpace database to the registration agency.

'cron' job for asynchronous reservation/registration

When a DOI should be reserved, registered, deleted or its metadata updated, DSpace just writes this information into its local database. A command line interface is supplied to send the necessary information to the registration agency. This behavior makes it easier to react to outages or errors while using the API. This information should be sent regularly, so it is a good idea to set up a cron job instead of doing it manually.

There are four commands that should be run regularly:

• Update the metadata of all items that have changed since their DOI was reserved.
• Reserve all DOIs marked for reservation
• Register all DOIs marked for registration
• Delete all DOIs marked for deletion

In DSpace, a DOI can have different states (see the following table). After updating an item's metadata the state of its assigned DOI is set back to the last state it had before. So, e.g., if a DOI has the state "to be registered" and the metadata of its item changes, it will be set to the state "needs update". After the update is performed its state is set to "to be registered" again. Because of this behavior the order of the commands above matters: the update command must be executed before all of the other commands above.

* An item in this state is not visible in DataCite

The cron job should perform the following commands with the rights of the user your DSpace installation runs as:

[dspace]/bin/dspace doi-organiser -u -q
[dspace]/bin/dspace doi-organiser -s -q
[dspace]/bin/dspace doi-organiser -r -q
[dspace]/bin/dspace doi-organiser -d -q

The doi-organiser sends error messages as email and logs some additional information. The option -q tells DSpace to be quiet. If you don't use this option the doi-organiser will print messages to stdout about every DOI it successfully reserved, registered, updated or deleted. Using a cron job these messages would be sent as email.

In case of an error, consult the log messages. If there is an outage of the API of your registration agency, DSpace will not change the state of the DOIs so that it will do everything necessary when the cron job starts the next time and the API is reachable again.

The frequency the cron job runs depends on your needs and your hardware. The more often you run the cron job the faster your new DOIs will be available online. If you have a lot of submissions and want the DOIs to be available really quickly, you probably should run the cron job every fifteen minutes. If there are just one or two submissions per day, it should be enough to run the cron job twice a day.

To set up the cron job, you just need to run the following command as the dspace UNIX user:

crontab -e

The following line tells cron to run the necessary commands twice a day, at 1am and 1pm. Please notice that the line starting with the numbers is one line, even it it should be shown as multiple lines in your browser.

# Send information about new and changed DOIs to the DOI registration agency:
0 1,13 * * *  [dspace]/bin/dspace doi-organiser -u -q ; [dspace]/bin/dspace doi-organiser -s -q ; [dspace]/bin/dspace doi-organiser -r -q ; [dspace]/bin/dspace doi-organiser -d -q

Limitations of DSpace's DOI support

That means if you want to use other applications or even more than one DSpace installation to register DOIs with the same prefix, you'll have to use a unique namespace separator for each of them. Also you should not generate DOIs manually with the same prefix and namespace separator you configured within DSpace. For example, if your prefix is 10.5072 you can configure one DSpace installation to generate DOIs starting with 10.5072/papers-, a second installation to generate DOIs starting with 10.5072/data- and another application to generate DOIs starting with 10.5072/results-.

DOIs will be used in addition to Handles. This implementation does not replace Handles with DOIs in DSpace. That means that DSpace will still generate Handles for every item, every collection and every community, and will use those Handles as part of the URL of items, collections and communities.

DSpace currently generates DOIs for items only. There is no support to generate DOIs for Communities and Collections yet.

When using DSpace's support for DOIs probably not all information would be restored when using the AIP Backup and Restore (see https://github.com/DSpace/DSpace/issues/5203). The DOIs included in metadata of Items will be restored, but DSpace won't update the metadata of those items at your registration agency anymore. You can even get problems when minting new DOIs after you restored older once using AIP.

Specific configuration to use the DataCite API to register DOIs with DSpace

If you use a DOI registration agency that lets you use the DataCite API directly, you can follow the instructions above to configure DSpace. In case EZID is your registration agency the configuration of DSpace is documented here: Configure DSpace to use EZID service for registration of DOIs.

To use DOIs within DSpace you have to configure several parts of DSpace:

• enter your DOI prefix and the credentials to use the API from DataCite in dspace.cfg,
• configure the script which generates some metadata,
• activate the DOI mechanism within DSpace,
• configure a cron job which transmits the information about new and changed DOIs to the registration agency.

Some configurations properties are specific to DataCite:

Additional information on DOI registration via DataCite can be found in the OR2024 presentation "Creating DOIs with rich metadata using DSpace".

Metadata conversion with DataCite

To reserve or register a DOI, DataCite requires that metadata be supplied which describe the object that the DOI addresses. The file [dspace]/config/crosswalks/DIM2DataCite.xsl controls the conversion of metadata from the DSpace internal format into the DataCite format:
[dspace]/config/crosswalks/DIM2DataCite.xsl

To change the Publisher, data manager and/or hosting Institution that will be send to DataCite as part of the metadata, use  the crosswalk.dissemination.DataCite.* properties in local.cfg.

If you want to know more about the DataCite Schema, have a look at the documentation. If you change this file in a way that is not compatible with the DataCite schema, you won't be able to reserve and register DOIs anymore. Do not change anything if you're not sure what you're doing. To get the XML on which the XSLT processor will start, use the following command:

To get the XML that will be send to DataCite replace 'dim' with 'DataCite'. If the DOI is not stored in the metadata, DSpace will add it automatically as identifier. So don't worry if the XML produced by this command does not contain the DOI. Once the DOI is stored in the metadata, it should also be contained in the XML.

Specific configuration to use EZID service to register DOIs with DSpace

The EZID IdentifierProvider operates synchronously, so there is much less to configure. You will need to un-comment the org.dspace.identifier.EZIDIdentifierProvider bean in config/spring/api/identifier-service.xml to enable DOI registration through EZID.

In config/dspace.cfg you will find a small block of settings whose names begin with identifier.doi.ezid.  You should uncomment these properties and give them appropriate values.  Sample values for a test account are supplied.

Back in config/spring/api/identifier-service.xml you will see some other configuration of the EZIDIdentiferProvider bean. In most situations, the default settings should work well.  But, here's an explanation of options available:

• EZID Provider / Registrar settings: By default, the EZIDIdentifierProvider is configured to use the CDLib provider (ezid.cdlib.org) in the EZID_SCHEME, EZID_HOST and EZID_PATH settings. In most situations, the default values should work for you. However, you may need to modify these values (especially the EZID_HOST) if you are registered with a different EZID provider. In that situation, please check with your provider for valid "host" and "path" settings. If your provider provides EZID service at a particular path on its host, you may set that in EZID_PATH.
  
  • NOTE: As of the writing of this documentation, the default CDLib provider settings should also work for institutions that use Purdue (ezid.lib.purdue.edu) as a provider. Currently, Purdue and CDLib currently share the same infrastructure, and both ezid.cdlib.org and ezid.lib.purdue.edu point to the same location.
    
• Metadata mappings: You can alter the mapping between DSpace and EZID metadata, should you choose.  The crosswalk property is a map from DSpace metadata fields to EZID fields, and can be extended or changed.  The key of each entry is the name of an EZID metadata field; the value is the name of the corresponding DSpace field, from which the EZID metadata will be populated.
• Crosswalking / Transforms: You can also supply transformations to be applied to field values using the crosswalkTransform property.  Each key is the name of an EZID metadata field, and its value is the name of a Java class which will convert the value of the corresponding DSpace field to its EZID form. The only transformation currently provided is one which converts a date to the year of that date, named org.dspace.identifier.ezid.DateToYear.  In the configuration as delivered, it is used to convert the date of issue to the year of publication.  You may create new Java classes with which to supply other transformations, and map them to metadata fields here.  If an EZID metadatum is not named in this map, the default mapping is applied:  the string value of the DSpace field is copied verbatim.

Limitations of EZID DOI support

DOIs will be used in addition to Handles. This implementation does not replace Handles with DOIs in DSpace. That means that DSpace will continue to generate Handles for every item, every collection and every community, and will use those Handles as part of the URL of items, collections and communities.

Currently, the EZIDIdentifierProvider has a known issue where it stores its DOIs in the dc.identifier field, instead of using the dc.identifier.uri field (which is the one used by DataCite DOIs and Handles).  See https://github.com/DSpace/DSpace/pull/1006 for more details. This will be corrected in a future version of DSpace.

DSpace currently generates DOIs for items only. There is no support to generate DOIs for Communities and Collections yet.

Specific configuration when using DSpace with Crossref as registration agency

Some configurations properties are specific to Crossref:

Metadata conversion with Crossref

Crossref more or less has one metadata schema per content type they support. Metadata stored in DSpace is transformed into metadata following Crossref's schemas by using XSL. You can find and change those XSLTs at dspace/config/crosswalks/crossref/...

Adding support for other Registration Agencies

If you want DSpace to support other registration agencies, you just have to write a Java class that implements the interface DOIConnector ([dspace-source]/dspace-api/src/main/java/org/dspace/identifier/doi/DOIConnector.java). You might use the DataCiteConnector ([dspace-source]/dspace-api/src/main/java/org/dspace/identifier/doi/DataCiteConnector.java) as an example. After developing your own DOIConnector, you configure DSpace as if you were using the DataCite API directly. Just use your DOIConnector when configuring the IdentifierService instead of the DataCiteConnector.

Configuring pre-registration of Identifiers

Why mint in submission?

Users often want to see what DOI they will  get so they can alter their PDF, coverpage, other metadata, and so on.

This feature should ensure that users can see their future DOI, and if necessary, a warning that if certain conditions are not met, the DOI will not be registered after approval.

Keeping a DOI in pending status does use up an integer from the total DOI namespace, but it also ensures that the submitter, reviewers, administrators etc know what the DOI will be if it is ever registered in the future.

If this is really not desired, eg. there are many item types which should never get a DOI, then there is a way to configure a filter that avoids minting a new PENDING DOI at all unless conditions are met in submission.

Enable the Identifiers step

See Submission User Interface#Configuringthe%22Identifiers%22step

Configure filters and behaviour

To enable this feature and configure the exact way it works, edit the ${dspace.dir}/dspace/modules/identifiers.cfg  configuration file

Administrator registration

If an item does not have a DOI at all, or if an item has a MINTED or PENDING DOI, a user with ADMIN rights over the item may queue the DOI registration from the Item Status page. No filters will be applied to this action. This requires identifiers.item-status.register-doi  to be true  in identifiers configuration (see above)

Standardizing the format and location of DOI metadata

As of DSpace 10 the default practice for storing DOI metadata has changed:

• DOIs minted from within DSpace will be stored in dc.identifier.doi metadata
• DOIs of documents imported from crosswalks will be stored in dc.relation.hasversion

In both cases, DOIs will now be stored in the HTTPS URL form https://doi.org/10.<whatever>, rather than just plain 10.<whatever> or doi:10.<whatever> or the old HTTP URL form http://dx.doi.org/10.<whatever>.

In previous versions, DOIs could end up inconsistently in dc.identifier or dc.identifier.doi or dc.identifier.uri, and may have had different formats, in both cases depending where they came from.

The doi-migrate command line tool helps to migrate DOI metadata between fields and normalize their format. It can move existing DOI metadata from the older fields and formats to this newer standard, but can also move to other fields if desired.

[dspace]/bin/dspace doi-migrate [options]

Usage

Selecting which DOIs to migrate:

--only-prefix <doi-prefix>     Only migrate DOIs with the given prefix
--except-prefix <doi-prefix>   Only migrate DOIs without the given prefix

Choosing which fields to affect:

-f, --from <field-name>    Move DOIs from this field
-t, --to <field-name>      Move DOIs into this field
(--from and --to can be the same if only the formatting of DOIs should be adjusted)

Operating modes:

-n, --dry-run  Don’t actually perform a migration, just generate a CSV (on standard output)
--rehydrate    Perform the migrations from a previous --dry-run invocation (CSV read from standard input)

You must either give both --to and --from plus one of --only-prefix and --except-prefix, OR --rehydrate.

The tool recognizes DOIs in any of the following formats:

• a plain DOI such as 10.1234/56789
• an old-style DOI URL like http://dx.doi.org/10.1234/56789 (including if HTTPS was used)
• a new-style DOI URL like https://doi.org/10.1234/56789 (including if plain HTTP was used)
• a DOI URI like doi:10.1234/56789

It normalizes all of these to a new-style DOI URL with HTTPS (https://doi.org/10.1234/56789)

How-to guide for a typical institution

The tool is set up to assume that all of your DOI minted in-house have the same prefix, and all other prefixes are external. (The prefix is the part of the DOI before the / character.) This example will assume an institution whose own publications have the prefix 10.1729. The institution’s minted DOIs have so far been placed in dc.identifier.

First we will create a CSV file of all the in-house DOIs that need to be migrated. We will move them into dc.identifier.doi field in the new format.

dspace doi-migrate --dry-run --from dc.identifier --to dc.identifier.doi --only-prefix 10.1729 > in-house-dois.csv

This will create a file in-house-dois.csv that can be checked in any spreadsheet software, including Microsoft Excel, Apple Numbers, Google Sheets, or LibreOffice Calc. Check that all of the proposed changes to metadata look correct. The columns are in the order: Item UUID; field name that the metadata will be moved from; current value of that metadata field on the item; field name that the metadata will be moved to; changed/normalized value of the metadata field on that item.

Having checked that the proposed changes look okay, run the migration with:

dspace doi-migrate --rehydrate < in-house-dois.csv

If there are multiple prefixes for publications whose DOIs were minted locally, this process can be repeated several times.

Then we will migrate all the DOIs for publications which were imported from external sources like ArXiv. The importers used to store these in dc.identifier.doi without the HTTPS URL; following the new DSpace standard, we will move them to dc.relation.hasversion. The usage is very similar:

dspace doi-migrate --dry-run --from dc.identifier.doi --to dc.relation.hasversion --except-prefix 10.1729 > imported-dois.csv

Again, we check the imported-dois.csv file to make sure that everything looks to be in order. Then we actually run the migrations:

dspace doi-migrate --rehydrate < imported-dois.csv