Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Login as an Administrative user
  • In Sidebase, select "Export" → "Metadata".  Type in the Community/Collection name.
    • Alternatively, browse
    Browse
    • to the Community or Collection you wish to export, and
    click
    • then go to "Export
    Metadata" link to export to a downloadable CSV
    • In XMLUI, "Export Metadata" can be found in the "Context" menu on a Community/Collection homepage
    • In JSPUI, "Export Metadata" can be found in the "Admin Tools" menu on a Community/Collection homepage
    Or search for items within your repository, and click "Export Metadata" in the search results to export to a downloadable CSV
  • In XMLUI, perform a search, and click on "Export Search Metadata" in the "Context" menu. By default, this option is only available to Administrators (xmlui.search.metadata_export=admin), but you can optionally allow any logged in user to export this metadata (xmlui.search.metadata_export=user), or anyone (xmlui.search.metadata_export=anonymous). 
  • In JSPUI, perform a search, and click on the "Export Metadata" button above the search results.
    • " → "Metadata".  That Community/Collection will be preselected.
  • Click "Export".  A new Process will be created (in "Processes" menu).  Once completed, download the resulting CSV.
Note
titleExporting search results to CSV was not added until DSpace 7.3

As of DSpace 7.3, it is possible to Export search results to a CSV (similar to 6.x).When logged in as an Administrator, after performing a search a new "Export search results as CSV" button appears. Clicking it will export the metadata of all items in your search results to a CSV.  This CSV can then be used to perform batch metadata updates (based on the items in your search results). - Release Notes#7.3ReleaseNotes

Please see below documentation for more information on the CSV format and actions that can be performed by editing the CSV.

...

The following table summarizes the basics.

Command used:

[dspace]/bin/dspace metadata-export

Java class:

org.dspace.app.bulkedit.MetadataExport

Arguments short and (long) forms):

Description

-f or --file

Required. The filename of the resulting CSV.

-i or --id

The Item, Collection, or Community handle or Database ID to export. If not specified, all items will be exported.

-a or --all

Include all the metadata fields that are not normally changed (e.g. provenance) or those fields you configured in the [dspace]/config/modules/bulkedit.cfg to be ignored on export.

-h or --help

Display the help page.

To run the batch editing exporter, at the command line:

...

  • First, complete all editing of the CSV and save your changes
  • Login as an Administrative User
  • Click In sidebar, select "Import" → "Metadata" and select drag & drop the CSV file
    • In XMLUI, "Import Metadata" can be found under the "Administrative" menu on any page
    • In JSPUI, "Import Metadata" can be found under the "Administer" menu (under your user account dropdown). On the Adminstration Tools page, select "Import Metadata" from the "Content" dropdown
  • After uploading the CSV, you will be presented with a summary of all changes that will be performed in the system. You can review these changes and choose whether to apply them or cancel.

Command Line Import

Note
titleValidate a Batch Metadata CSV was not added until DSpace 7.3

As of DSpace 7.3, it is now possible to validate a Batch Metadata CSV before applying changes (similar to 6.x).  When uploading a CSV for batch updates (using "Import" menu), a new "Validate Only" option is selected by default. When selected, the uploaded CSV will only be validated & you'll receive a report of the detected changes in the CSV.  This allows you to verify the changes are correct before applying them.  (NOTE: applying the changes requires re-submitting the CSV with the "Validate Only" option deselected)  - Release Notes#7.3ReleaseNotes


Command Line Import

The following table summarizes the basics.

Command used:

[dspace]/bin/dspace metadata-import

Java class:

org.dspace.app.bulkedit.MetadataImport

Arguments short and (long) forms:

Description

-f or --file

Required. The filename of the CSV file to load.

-s or --silent

Silent mode. The import function does not prompt you to make sure you wish to make the changes.

-e or --email

The email address of the user. This is only required when adding new items.

-w or --workflow

When adding new items, the program will queue the items up to use the Collection Workflow processes.

-n or --notify

when adding new items using a workflow, send notification emails.

-t or --template

When adding new items, use the Collection template, if it exists.

-h or --help

Display the brief help page.

Silent Mode should be used carefully. It is possible (and probable) that you can overlay the wrong data and cause irreparable damage to the database.

...

Info
titleTips to Simplify the Editing Process

 When editing a CSV, here's a couple of basic tips to keep in mind:

  1. The "id" column MUST remain intact. This column also must always have a value in it.
  2. To simplify the CSV, you can simply remove any columns you do NOT wish to edit (except for "id" column, see #1). Don't worry, removing the entire column won't delete metadata (see #3)
  3. When importing a CSV file, the importer will overlay the metadata onto what is already in the repository to determine the differences. It only acts on the contents of the CSV file, rather than on the complete item metadata. This means that the CSV file that is exported can be manipulated quite substantially before being re-imported. Rows (items) or Columns (metadata elements) can be removed and will be ignored. 
    1. For example, if you only want to edit "dc.subject", you can remove ALL columns EXCEPT for "id" and "dc.subject" so that you can just manipulate the "dc.subject" field. On import, DSpace will see that you've only included the "dc.subject" field in your CSV and therefore will only update the "dc.subject" metadata field for any items listed in that CSV.
  4. Because removing an entire column does NOT delete metadata value(s), if you actually wish to delete a metadata value you should leave the column intact, and simply clear out the appropriate row's value (in that column).

...


Editing Collection Membership

...

  • It's possible the CSV was not saved properly after editing. Check that the edits are in the CSV, and that there were no backend errors in the DSpace logs (which would be an indication of an invalid or corrupted CSV file)
  • Depending on the version of DSpace, you may be encountering this known bug with processing linebreaks in CSV filesDS-3245: https://github.com/DSpace/DSpace/issues/6600
  • If you are setting a new embargo date in the CSV, ensure that the embargo lift date is a future date.  It's been reported that past dates may cause DSpace to ignore item changes.

Batch Editing, Entities and Relationships

Consider the following page for this topic: Configurable Entities

Background about entities and virtual metadata

  • In DSpace 7, we have entities. Entities are items with an entity type (there can still be items without an entity type).

  • Two entities can be linked to each other. For this purpose relations are defined, which indicate the relationship between the entities. Relationships between two entities are defined by the metadata schema relation. The relation reflects how two entities are related to each other, for example isPersonOfProject or isPublicationOfAuthor.

  • Until the introduction of entities, we could only link items to each other by inserting DOIs or URLs of other items into metadata fields dc.relation.*. What is new about the linking between entities in DSpace 7 is that UUIDs are entered into the fields, i.e. internal identifiers of other entities that DSpace can easily resolve. DSpace "knows" which entities are linked to each other and how.

  • On the item view of an entity (remember: an entity is an item with an entity type) metadata of other entities can be displayed. DSpace refers to this as virtual metadata. Virtual metadata does not belong to the item in whose item view it is displayed, but to a linked entity. They can be changed only in the linked entity. As an example: we have the entities journal and journal issue. All journal issues display the title of the journal in their item view. This title is stored only in the journal and is only (dynamically) displayed in the issues.

Admin CSV export

  • Virtual metadata is exported with the entities in which it is included. For example, when you export projects, you see a column for the project.investigator field. Here, the names of two people have been included as virtual metadata. However, the names are not stored in the project, but exported from the respective person entities at the time of export. The specification ::virtual:: marks this. The specifications ::8585:: and ::27946:: are examples for this documentation and represent IDs of the relations. The specification ::600 comes from the DSpace Authority, which is also specified due to technical circumstances.

  • The relation itself is also included in the CSV export, in the relation.isPersonOfProject field. Additionally, a relation.isPersonOfProject.latestForDiscovery field is created. This field has internal reasons in DSpace and should help to make things faster discoverable. In the fields you again see the ::virtual::8585::600 specification, which are already explained above. Instead of the values of individual metadata fields, you now have the UUIDs of the items that are linked. You can always get these UUIDs from the URL of the item view of an item.

An example heading row of the CSV export file (project entity):

Code Block
id,collection,dc.title,project.investigator,relation.isPersonOfProject,etc,etc,etc.

Subsequent example row in the CSV export file (project entity):

Code Block
350,2292,Project title,"Smith, John::virtual::8585::600||Doe, Jane::virtual::27946::600","d89c1eb1-2e7c-4912-a1eb-f27b17fd6848::virtual::8585::600||e3595b14-6937-47b9-b718-1972cb683943::virtual::27946::600"


Admin CSV import

  • As always, only the columns and rows that will be changed should be specified. You do not want to import the columns that contain virtual metadata, because they are not stored in the imported items, but in the linked items. So in the above example you don't want to import the project.investigator column, but delete it from the CSV.

  • To link one item to another you need to create a corresponding column of the relation metadata schema, so in our example above relation.isPersonOfProject. All columns of the form relation.*.latestForDiscovery are created and updated automatically, so you don't want to import them. If you want to create a new relation, of course you don't know the ID of the relation, you can replace it with a +, then DSpace will assign it on its own. Of course, people can also be removed from the column or completely new relations can be created for new items, even if there are no old ones to be taken over.

An example heading row for the CSV import file (project entity):

Code Block
id,collection,dc.title,relation.isPersonOfProject,etc,etc,etc.

Subsequent example row for the CSV import file (project entity):

Code Block
350,2292,Project title,"d89c1eb1-2e7c-4912-a1eb-f27b17fd6848::virtual::8585::600||e3595b14-6937-47b9-b718-1972cb683943::virtual::+::600"