Upgrading from DSpace 6.x or lower

Additional Upgrade Steps

Before Upgrade/Migration

Export your Solr Statistics from your existing DSpace site

If you are using SOLR Statistics and wish to retain your statistical data in the latest version of DSpace, then you MUST export your old Solr statistics data from your existing site.

1. From your old version of DSpace, dump your authority and statistics Solr cores.  (Only necessary if you want to keep both your authority records and/or SOLR Statistics)

   [dspace]/bin/dspace solr-export-statistics -i authority
   [dspace]/bin/dspace solr-export-statistics -i statistics

   The dumps will be written to the directory [dspace]/solr-export.  This may take a long time and require quite a lot of storage.  In particular, the statistics core is likely to be huge, perhaps double the size of the content of solr/statistics/data.  You should ensure that you have sufficient free storage.
   
   This is not the same as the disaster-recovery backup that was done above.  These dumps will be reloaded into new, reconfigured cores later.
   If you were sharding your statistics data, you will need to dump each shard separately. The index names for prior years will be statistics-YYYY (for example:  statistics-2017 statistics-2018 etc.)  The current year's statistics shard is named statistics and you should dump that one too.

2. Move your old Solr cores to a safe location in case of trouble with the upgrade procedure.  If you leave them in place, you will get a mixture of old and new files that the new Solr will refuse to load.

Perform Upgrade/Migration

Choose your upgrade method

Perform the DSpace upgrade or migration based on your preferred upgrade method.  There are two main options:

1. (RECOMMENDED) Install the new version of DSpace and migrate your old data into it.  This process is described in Migrating DSpace to a new server
2. Perform a manual upgrade of your site.  This process is described in Upgrading DSpace

In most situations, you may find it easier  to follow the steps in Migrating DSpace to a new server because it will allow you to work with a fresh installation  of the new DSpace release before migrating your data into that installation.  That process is often easier if you plan to upgrade from a very old DSpace release.

Update your DSpace configuration (if performing manual upgrade)

If you choose option two (the manual upgrade), please keep in mind that you will need to remove/modify some of your existing DSpace configuration.  These are some of the configurations that used to exist in DSpace 6.x which are no longer valid in DSpace 7 or later:

1. Review your customized configurations (recommended to be in local.cfg): As mentioned above, we recommend any local configuration changes be placed in a local.cfg Configuration File.  With any major upgrade some configurations may have changed. Therefore, it is recommended to review all configuration changes that exist in the config directory, and its subdirectories, concentrating on configurations your previously customized in your local.cfg.  See also the Configuration Reference.
2. Remove obsolete configurations.  With the removal of the JSPUI and XMLUI, a large number of server-side (backend) configurations were made obsolete and were therefore removed between the 6.x and 7.0 release. A full list can be found in the Release Notes.
3. Remove BTE Spring configuration: If it exists, remove the [dspace]/config/spring/api/bte.xml Spring Configuration.  This file is no longer needed as the BTE framework was removed in favor of Live Import from external sources.
4. Migrate or recreate your Submission configuration.  As of DSpace 7, the submission configuration has changed.  The format of the "item-submission.xml" file has been updated, and the older "input-forms.xml" has been replaced by a new "submission-forms.xml".   You can choose to either start fresh with the new v7 configuration files, or you can use the steps below to migrate your old configurations into the new format.  See the Submission User Interface for more information

   1.  First, create a temporary folder to copy your old v6 configurations into

      # Example of creating a [dspace]/config/temp folder for this migration
      # You must replace [dspace] with the full path of your DSpace 7 installation.
      cd [dspace]/config
      mkdir temp

   2. Copy your old (v5 or v6) "item-submission.xml" and "input-forms.xml" into that temporary folder

   3. Run the command-line migration script to migrate them to v7 configuration files

      # This example uses [dspace] as a placeholder for all paths.
      # Replace it with either the absolute or relative path of these files
      [dspace]/bin/dspace submission-forms-migrate -s [dspace]/config/temp/item-submission.xml -f [dspace]/config/temp/input-forms.xml

   4. The result will be two files. These are valid v7 configurations based on your original submission configuration files.
      1. 1. [dspace]/config/item-submission.xml.migrated 
         2. [dspace]/config/submission-forms.xml.migrated 
   5. These "*.migrated" files have no inline comments, so you may want to edit them further before installing them (by removing the ".migrated" suffix).  Alternatively, you may choose to copy sections of the *.migrated files into the default configurations in the [dspace]/config/ folder, therefore retaining the inline comments in those default files.
5. City IP Database file for Solr Statistics has been renamed. The old [dspace]/config/GeoLiteCity.dat file is no longer maintained by its provider.  You can delete it.  The new file is named GeoLite2-City.mmdb by default. If you have configured a different name and/or location for this file, you should check the setting of usage-statistics.dbfile in [dspace]/config/modules/usage-statistics.cfg (and perhaps move your custom setting to local.cfg).
6. tm-extractors media filtering (WordFilter) no longer exists:  the PoiWordFilter plugin now fulfills this function.  If you still have WordFilter configured, remove from dspace.cfg and/or local.cfg all lines referencing org.dspace.app.mediafilter.WordFilter and uncomment all lines referencing org.dspace.app.mediafilter.PoiWordFilter.
7. Re-configure Solr URLs:  change the value of solr.server to point at your new Solr external service.  It will probably become something like solr.server = https://localhost:8983/solr.  Solr only needs to be accessible to the DSpace backend, and should not be publicly available on the web.  It can either be run on localhost or via a hostname (if run on a separate server from the backend). Also review the values of
   1. discovery.search.server
   2. oai.solr.url
   3. solr.authority.server
   4. solr-statistics.server
8. Sitemaps are now automatically generated/updated: A new sitemap.cron setting exists in the dspace.cfg which controls when Sitemaps are generated. By default they are enabled to update once per day, for optimal SEO.  See Search Engine Optimization docs for more detail
   1. Because of this change, if you had a system cron job which ran "./dspace generate-sitemaps", this system cron job can be removed in favor of the new sitemap.cron setting.

After Upgrade/Migration

Import your Solr Statistics from your existing DSpace site

If you are using SOLR Statistics and wish to retain your statistical data in the latest version of DSpace, then you MUST import your old Solr statistics data from your existing site.  (This step is using the same data that you exported above)

1. Reload Solr Statistics

   1. Load authority and statistics from the CSV dumps that you made earlier in Step 2 above.

      [dspace]/bin/dspace solr-import-statistics -i authority
      [dspace]/bin/dspace solr-import-statistics -i statistics

      This could take quite some time.
      
      If you had sharded your statistics, you will need to load the dump of each shard separately into the "statistics" core. DSpace 7 does not support Solr shards at this time.  Unfortunately, this will involve renaming all CSV export files to remove the year (e.g. rename "statistics-2012_export_2013-12_5.csv" to "statistics_export_2013-12_5.csv") and rerunning "[dspace]/bin/dspace solr-import-statistics -i statistics".  More advice on this process can be found in this dspace-tech mailing list thread.
      
      

   2. For the Statistics core(s) only, upgrade legacy DSpace Object Identifiers (pre-6.4 statistics) to UUID Identifiers.

      [dspace]/bin/dspace solr-upgrade-statistics-6x -i statistics

      Again If you had sharded your statistics, you will need to run this for each shard separately. See also  SOLR Statistics Maintenance#UpgradeLegacyDSpaceObjectIdentifiers(pre-6xstatistics)toDSpace6xUUIDIdentifiers

2. (Optional) Set up IP to City database for location-based statistics.  If you wish to (continue to) record the geographic origin of client activity, you will need to install (and regularly update) one of the following:
   
   1. Either, a copy of  MaxMind's GeoLite City database (in MMDB format)
      1. NOTE: Installing MaxMind GeoLite2 is free.  However, you must sign up for a (free) MaxMind account in order to obtain a license key to use the GeoLite2 database.
      2. You may download GeoLite2 directly from MaxMind, or many Linux distributions provide the geoipupdate tool directly via their package manager.  You will still need to configure your license key prior to usage.
      3. Once the "GeoLite2-City.mmdb" database file is installed on your system,  you will need to configure its location as the value of  usage-statistics.dbfile  in your local.cfg configuration file . 
      4. You can discard any old GeoLiteCity.dat database(s) found in the config/ directory (if they exist).
      5. See the "Managing the City Database File" section of  SOLR Statistics  for more information about using a City Database with DSpace.
   2. Or, you can alternatively use/install DB-IP's City Lite database (in MMDB format)
      • This database is also free to use, but does not require an account to download.
      • Once the "dbip-city-lite.mmdb" database file is installed on your system,  you will need to configure its location as the value of  usage-statistics.dbfile  in your local.cfg configuration file . 
      • See the "Managing the City Database File" section of  SOLR Statistics  for more information about using a City Database with DSpace.

Update your Handle Server Configuration

Older versions of DSpace used an older Handle Server.  If you are using the built-in Handle Server with hdl.handle.net URLs, then you'll need to add the follow to the end of the server_config section of your [dspace]/handle-server/config.dct file (the only new line is the "enable_txn_queue" line)

"case_sensitive" = "no"
"storage_type" = "CUSTOM"
"storage_class" = "org.dspace.handle.HandlePlugin"
"enable_txn_queue" = "no"

Alternatively, you could re-run the ./dspace make-handle-config script, which is in charge of updating this config.dct file.