Page History
Version 5.0
Infowarning |
---|
Support for DSpace 5 .0 is still in the planning stages and will be released sometime in late (likely Nov/Dec) 2014 |
Table of Contents:
Table of Contents | ||||||||
---|---|---|---|---|---|---|---|---|
|
New features in DSpace 5.0 (WIP)
ended on January 1, 2023. See Support for DSpace 5 and 6 is ending in 2023 |
Tip | ||
---|---|---|
| ||
DSpace 5.0 can be downloaded immediately from: See the DSpace 5.x Release Notes for more information. |
Info | ||
---|---|---|
| ||
DSpace 5.x Release Notes have been incorporated into the DSpace 5.x documentation wiki. The content of that page, and this one, is similar, but it's not a perfect duplicate. This page is for coordinating and planning DSpace Release 5.0, and for communicating this status information to the community. This Release Status page is a historical document, with much of the same material. |
Table of Contents:
Table of Contents | ||||||||
---|---|---|---|---|---|---|---|---|
|
New features in DSpace 5.0
Excerpt | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Excerpt | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
DSpace 5.0 ships with a number of new features. Certain features are automatically enabled by default while others require deliberate activation. | The following non-exhaustive list contains the major new features in 5.0 that are enabled by default: Easier Upgrading to 5.x from any previous DSpace version
| XMLUI and JSPUI new features
JSPUI new features
XMLUI new features
New CRUD endpoints for REST web service API
UI support for metadata batch import from various bibliographic formats
Kindly contributed by the Greek National Documentation Centre/EKT Solr libraries were upgraded to version 4.10.2 (JSPUI, XMLUI and OAI modules)
| | The following list contains features that are included in the DSpace 5.0 release, but need to be enabled manually. | Review the documentation for these features carefully, especially if you are upgrading from an older version of DSpace. XMLUI new features
|
For all UIs
- Batch import from Zip files uploaded via the admin interface of either UI
Easier Database upgrades, thanks to the inclusion of Flyway, an open-source database migration toolA feature-complete REST API, supporting both read and write operations- Linked (Open) Data support
- Metadata for all DSpace Objects
Improved thumbnail generation (via ImageMagick/Ghostscript)- Google Analytics integration now tracks "downloads" of bitstreams
- A place to add third-party plugins
For JSPUI only
HTML5-based drag and drop upload interface for uploading bitstreams during submissionPer-item configurable visual indicators for browse and search results
For XMLUI only
Mirage2 theme, utilizing many modern web design tools and technologies, facilitating a "responsive" web design, adapting to any screen sizeORCID integration via an authority plugin, and a new authority cacheSerpa/Romeo lookupRendering MathML code in abstractsError pages (including page not found) are all styled with your own site stylesCollection drop down box performance improvements for repositories with a large number of collections(Optional) ability to generate citation cover pages on PDFsAn aspect for adding Google Analytics web usage information to item pages
Release TODO list
...
|
Release TODO list
...
- Check documentation for all new features, add links to them the list of new features above. Incomplete list of missing docs:
Jira server DuraSpace JIRA
...
serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-1641 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-
...
- put Tomcat-specific code to JSPUI's web.xml
- just document this Tomcat 7-specific warning
On DevMtg 2014-11-05 we decided to document for RC1 and decide before RC2
2168 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-2175 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-2183
- Current list of release blockers:
Jira server DuraSpace JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution maximumIssues 20 jqlQuery project = DS AND priority = Blocker AND fixVersion = 5.0 AND statusCategory != Complete serverId c815ca92-fd23-34c2-8fe3-956808caf8c5
Seems like we have 2 options:Jira server DuraSpace JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key DS-2249 - put Tomcat-specific code to JSPUI's web.xml
- just document this Tomcat 7-specific warning
On DevMtg 2014-11-05 we decided to document for RC1 and decide before RC2
- Update the list of contributors on the Release Notes in Documentation (see "TBA" in text)
- Draft up announcement for mailing lists
- Update DSpace.org
Automatic Solr upgrade
Note | ||||||||
---|---|---|---|---|---|---|---|---|
This has now been done in
|
Findings
Solr is distributed on Maven Central as a .war file, which includes the Lucene .jars. No Lucene version relevant to DSpace is available from Maven Central.
Solr is a wrapper for Lucene providing concurrency. Lucene can work with index files directly if we don't need concurrent access (i.e. before we start up Solr).
DSpace 1.6 was the first version to use Solr for statistics. In order to upgrade to DSpace 5, we need to upgrade the Solr index format in two steps. The second step is easy, upgrading from DSpace 3 (Solr 3.5) to DSpace 5 means just running the Solr 4.10.2 optimize command, which can be run even concurrently when the UIs are running. However, Solr 4.4 doesn't understand the legacy index formats used in Solr before 3.5. Therefore, in the first step, we need to use Solr 3.5 or Lucene 3.5 to upgrade from the older index versions to 3.5.
Suggested upgrade
We need to avoid the conflict of having both Solr 3.5 and 4.10.2 in the DSpace classpath. Therefore I suggest to run the first step before the Solr 4.10.2 webapp is started as part of DSpace 5.
To do that, we need to:
- Detect the oldest segment version using SegmentInfo.getVersion() of Lucene 4.10
- General logic is in this area of the CheckIndex script: https://github.com/apache/lucene-solr/blob/lucene_solr_4_10/lucene/core/src/java/org/apache/lucene/index/CheckIndex.java#L426
- If that oldest segment version is < 3.5 go to 3. Otherwise go to 5.
- get lucene-core-3.5.0.jar from Maven Central
- run the IndexUpgrader class of Lucene 3.5 (or optimize())
- start up DSpace in order to start up Solr 4
- run the IndexUpgrader class of Lucene 4.10.2
The manual steps would be:
Code Block |
---|
wget "http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/3.5.0/lucene-core-3.5.0.jar" -O lucene-core-3.5.0.jar
# check index version, see table below:
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex /dspace/solr/search/data/index/
# upgrade index version:
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /dspace/solr/search/data/index/
# check index version again, should be "version=3.5 format=FORMAT_3_1 [Lucene 3.1+]" |
DSpace/Solr/Lucene versions and compatibility
DSpace | Solr | Lucene | Used Lucene index version | version reported by CheckIndex | Supported Lucene index versions |
5.0 | 4.10.1 | 4.10.2 | 4.10 | (before optimize) versions=[4.4.0 .. 4.10.2] format= | |
4.0 | 4.4.0 | 4.4 | 4.4 | version=4.4.0 format= | |
3.0 | 3.5.0 | 3.5 | version=3.5 format=FORMAT_3_1 [Lucene 3.1+] | LUCENE_20, LUCENE_21, LUCENE_22, LUCENE_23, LUCENE_24, LUCENE_29, LUCENE_30, LUCENE_31, LUCENE_32, LUCENE_33, LUCENE_34, LUCENE_35 | |
1.8 | 3.3.0 | 3.3 | ??? | version=3.3 format=FORMAT_3_1 [Lucene 3.1+] | |
1.7 | 1.4.1 | 2.9.3 (2.9.3 951790 - 2010-06-06 01:30:55) | ??? | version=FORMAT_DIAGNOSTICS [Lucene 2.9] | |
1.6 | 1.3.0 | 2.4-dev (2.4-dev 691741 - 2008-09-03 15:25:16) | ??? |
Luke
https://code.google.com/p/luke/wiki/Compatibility
Java version
Minimum version of JVM required to run Luke 1.0.1 and earlier is 1.5.
Minimum version of JVM required to run Luke 3.x and 4.x is 1.6.
Lucene Java indexes
Luke 1.0.1 should be able to open indexes built using the following versions of Lucene-Java:
- 3.0.x
- 2.x (any version)
Starting with version 4.x Luke cannot open indexes created with Lucene 3.0.0 or earlier.
Starting with version 3.1.0 Luke releases will use the same numbering as Lucene releases, to avoid confusion.
Solr upgrade (WIP)
Findings
Solr is distributed on Maven Central as a .war file, which includes the Lucene .jars. No Lucene version relevant to DSpace is available from Maven Central.
Solr is a wrapper for Lucene providing concurrency. Lucene can work with index files directly if we don't need concurrent access (i.e. before we start up Solr).
DSpace 1.6 was the first version to use Solr for statistics. In order to upgrade to DSpace 5, we need to upgrade the Solr index format in two steps. The second step is easy, upgrading from DSpace 3 (Solr 3.5) to DSpace 5 means just running the Solr 4.4 optimize command, which can be run even concurrently when the UIs are running. However, Solr 4.4 doesn't understand the legacy index formats used in Solr before 3.5. Therefore, in the first step, we need to use Solr 3.5 or Lucene 3.5 to upgrade from the older index versions to 3.5.
Suggested upgrade
We need to avoid the conflict of having both Solr 3.5 and 4.4 in the DSpace classpath. Therefore I suggest to run the first step before the Solr 4.4 webapp is started as part of DSpace 5.
To do that, we need to:
- detect the index version using getCurrentVersion(index_dir)
- for any version older than LUCENE_35 go to 3, otherwise go to 7
- get the Solr 3.5 war from Maven Central
- extract the lucene-core-3.5.0.jar from the Solr 3.5 war
- run the IndexUpgrader class of Lucene 3.5 (or optimize())
- start up DSpace in order to start up Solr 4
- run the IndexUpgrader class of Lucene 4.4
The manual steps would be:
Code Block |
---|
mkdir solr-3.5.0
cd solr-3.5.0
wget "http://search.maven.org/remotecontent?filepath=org/apache/solr/solr/3.5.0/solr-3.5.0.war" -O solr-3.5.0.war
unzip solr-3.5.0.war
cd WEB-INF/lib/
# check index version, see table below:
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex /dspace/solr/search/data/index/
# upgrade index version:
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /dspace/solr/search/data/index/
# check index version again, should be "version=3.5 format=FORMAT_3_1 [Lucene 3.1+]" |
DSpace/Solr/Lucene versions and compatibility
DSpace | Solr | Lucene | Used Lucene index version | Supported Lucene index versions |
5.0 | 4.4.0 | 4.4 | 4.4 | |
4.0 | 4.4.0 | 4.4 | 4.4 | |
3.0 | 3.5.0 | 3.5 | LUCENE_20, LUCENE_21, LUCENE_22, LUCENE_23, LUCENE_24, LUCENE_29, LUCENE_30, LUCENE_31, LUCENE_32, LUCENE_33, LUCENE_34, LUCENE_35 | |
1.8 | 3.3.0 | 3.3 | ??? | |
1.7 | 1.4.1 | 2.9.3 (2.9.3 951790 - 2010-06-06 01:30:55) | ??? | |
1.6 | 1.3.0 | 2.4-dev (2.4-dev 691741 - 2008-09-03 15:25:16) | ??? |
Luke
https://code.google.com/p/luke/wiki/Compatibility
Java version
Minimum version of JVM required to run Luke 1.0.1 and earlier is 1.5.
Minimum version of JVM required to run Luke 3.x and 4.x is 1.6.
Lucene Java indexes
Luke 1.0.1 should be able to open indexes built using the following versions of Lucene-Java:
- 3.0.x
- 2.x (any version)
Starting with version 4.x Luke cannot open indexes created with Lucene 3.0.0 or earlier.
Starting with version 3.1.0 Luke releases will use the same numbering as Lucene releases, to avoid confusion.
Possible JIRA tickets to resolve (automated list)
This is the list of JIRA tickets ("New Features" and Improvements) currently scheduled for 5.0. There is no guarantee they will all be completed in time for 5.0 release, and tickets may be dropped/added to the list at any time. But, if you'd like to volunteer to help with one, please let us know by attaching a comment to the ticket in question.
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Changes
The following changes have already been made to the codebase and will be released in 5.0. Please note that the below listing is dynamically generated, so it will be changing as we continue to add new features, improvements and bug fixes to the 5.0 release.
...
- Peter Dietz (Longsight)
- Hardy Pottinger (U of Missouri)
- Ivan Masár
- Mark H. Wood (Indiana University)
- Robin Taylor (University of Edinburgh)
- Pascal-Nicolas Becker (Technische Universität Berlin(Technische Universität Berlin)
- Andrea Schweer (Library Consortium of New Zealand)
All Release Team membership information is as of November 4, 2014. Please volunteer (by emailing Tim Donohue), if you are interested in joining the team!
...
Your contributions are welcome now! Code and documentation need not be finished, so long as it is working and we can all see what it is for. Time is set aside for fixing, polishing, and integration. We have some general Code Contribution Guidelines available, but you are also welcome to ask questions on the dspace-devel mailing list.
Date | Milestone | What it means |
---|---|---|
October 06 | Deadline for feature pull requests | If you wish to contribute features to DSpace 5.0, you must submit a pull request by this date. |
October 08 | Weekly developers' meeting devoted to review of feature pull requests for 5.0 | The entire hour's meeting will be used to discuss proposed features submitted by the deadline. |
October 15 | Weekly developers' meeting devoted to review of feature pull requests for 5.0 | The entire hour's meeting will be used to discuss proposed features submitted by the deadline. |
October 31 | Feature freeze | DSpace 5.0 is considered feature-complete on this date. Only bugfixes will be pulled between this date and final release. |
November 6 | Release Candidate 1 tagged | A DSpace 5.0 Release Candidate will be available for wider testing. |
November 10 | Release Candidate 2 tagged | 5.0-rc2 was released to address missing optional artifacts from 5.0-rc1 (Mirage2 and LNI) |
November 10-21 | Testathon | Intensive public testing of the 5.0 Release Candidate is invited. The Release Team will focus on getting problems resolved. |
December |
19 | Release Candidate |
3 tagged | An updated DSpace 5.0 Release Candidate will be available for wider testing. |
January 15 | DSpace 5.0 is publicly released | DSpace 5.0 is released for download and general use. |
Release Process needs to proceed according to the following Maven release process: Release Procedure