DSpace Discovery

DSpace Discovery is an Maintained Addon for DSpace XMLUI that replaces the default Search and Browse behavior with Apache Solr.

Proposal For Inclusion into DSpace 1.7.0

Recent work on porting DSpace to an Asyncronous build process prooved too large a task to be completed in DSpace 1.7 with all the other significant changes, with this in mind. It is proposed that a version of Discovery be delivered within the DSpace 1.7.0 codebase and initially maintained there for the 1.7.x development path. 

This proposal includes the following features:

• Discovery will be compiled/installed into DSpace 1.7.0 XMLUI but left disabled by default
• The Appropriate Solr Search Core will be available and prepared to be started
• All dependencies for turning on Discovery will be available without recompilation by doing the following steps:
• Solr will be upgraded to the latest 1.4.1 release
• Documentation will be included which outlines exactly how to turn on Discovery and run it

Introduction Video

http://www.youtube.com/v/abRSXTUEwws

Documentation

*Discovery Configuration in DSpace 1.7.0
*Discovery Install in pre 1.7.0 HowTo

Design Premis for Discovery

The Design premis behind Discovery is to keep as much the implementation of Search and Browse independent of DSpace as possible. The basis for this is to twofold. (a) to reduce cost in maintaining any customized code and (b) to repurpose third party solutions wherever possible (a.k.a. standing on shoulder of giants).  So, the basic tenants are:

1. Keep as much of the customization and configuration in Solr as possible.
2. Keep it as generic as possible.
3. Keep it as simple as possible
4. In cases where configuration is outside Solr, Provide pluggability to replace functionality easily at end user deployment.
5. Align Search/Browse capabilities with Solr capabilities, not other way around.  This means, possibly abandon certain strategies for navigating via Browse if it proves these do not fit well with solr.

RoadMap

Discovery is currently and addon for DSpace that still requires significant addition of configuration files to support.  Planned releases will initially coincide with DSpace Scheduled Releases. Eventually, once completely stabilized.  Discovery may be included into DSpace releases as a replacement for DSpace Search and Browse out of the box.

Issue Management

Subversion Access

http://scm.dspace.org/svn/repo/modules/dspace-discovery

Examples in Production

• Dryad Data Repository: http://www.datadryad.org/search?query=&rpp=10&group_by=none&sort_by=score&order=DESC&submit=Go

Discovery in DSPace 1.8.0

New Discovery back end (unconfirmed)

The goal of this task was twofold.

1. Ensure that discovery doesn't have to use solr as a backend, in other words ensure that user can plug in their own backend implementation.
2. Rewrite some of the code to make it more logical (example: the sidebar filters for a community page where rendered because the CommunityRecentSubmissionsTransformer extended the AbstractsFiltersTransformer => logically these are 2 completely different things).

This task focuses completely on these 2 point. Please bear mind that almost nothing in the UI has been changed. While programming this task I also had a minor feature request which I implemented namely the following:

The autocomplete filters can be configured to not split words on a space, this is particularly useful for authors.

The rewrite of the back end is explained below.

1.   A new discovery sub module has been created named "dspace-discovery-solr" this module is the only module that contains any solr dependencies and contains the old " SolrServiceImpl" class and the spring-dspace-addon-discovery-services.xml spring file. This module is used by default and ensures that a discovery out of the box will still work. None of the other discovery modules have dependencies on this module so it can be replaced very easily.
2.  The "dspace-discovery-provider" module has lost the all the solr dependencies as explained and the "SearchService" interface has undergone some changes, the search methods don't require a solrQuery anymore and do not return a solrResult. Instead 2 new objects have been created
   • DiscoverQuery: will hold all query information
   • DiscoverResult: will contain the resulting dspace objects and/or the facets
   • By doing things this way any programmer can implement the search service and write their own backend.
3. All the user interface classes have been rewritten to support these new objects.
4. One logical change was also made namely the following:
   The abstractFilters class has been transformed into the "SidebarFacetsTransformer". The facets will still appear on the same places, but unlike earlier where the only reason the community had sidebar facets was that the "CommunityRecentSubmissions" class extended the filters class the SidebarFacetsTransformer is now called on each community page. I did this because it just makes more sense that way.
5. I also added an extra method to indexing implementation of solr, named " indexItemFieldCustom" should it ever be required to adjust some simple indexing task the entire " SolrServiceImpl" does not need to be overwritten. One simply extends the class and implement that method (a boolean is returned which can prevent further indexing of the metadata field).

There is one thing that remains unfinished and that is the related items, I'm still thinking on the best way to implement that with the DiscoveryQuery/DiscoveryResult objects, if anybody has some suggestion I'm always willing to listen.

Other Resources

• http://lucene.apache.org/solr/
• AJAX Integration: http://github.com/evolvingweb/ajax-solr http://solrjs.solrstuff.org/
• Integration With DSpace REST project for unified Search and Browse in the REST webapplication as well.
• Access Control Request Handler for Solr Access control for Solr Service, Documents and Fields.
• General specifications
• FieldType JAVADOC
• Predefined FieldTypes in SolR
• http://wiki.github.com/evolvingweb/ajax-solr/reuters-tutorial-step-9 Integrate AutoSuggest, Tag Clouds, Google Maps and dynamic Paging into DSpace Search Results.