This documentation refers to an earlier version of Islandora. https://wiki.duraspace.org/display/ISLANDORA/Start is current.

Islandora utilizes the Solr open source search platform to enable flexible and configurable indexing and searching. Solr uses the Lucene Java search library at its core for full-text indexing and search and offers hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling as additional features.

The Fedora Generic Search Service, or GSearch, is a search service installed with Fedora that allows for automatic updating of the Lucene/Solr index. GSearch relies on JMS to receive messages that are sent when Fedora objects are ingested, modified or purged. This keeps the Lucene index in sync with the Fedora repository.

How Islandora uses Solr/Lucene and Gsearch

Islandora makes it possible to use the power of Solr/Lucene for discovery. Gsearch is used as a method for keeping indexes current. When an item is ingested, the FOXML is transformed by an XSLT file stored in Gsearch into a format that can be read by Solr's schema and returned based on the request handlers in our custom solrconfig.xml.

The Islandora Solr Search module is packaged with files that will support Islandora solution packs, but these can be modified if you are familiar with Solr.  Specifically the Solr schema and corresponding Gsearch XSLT are a good starting point even if you do not use the Solr Search module.  Additional information about Solr is presented in Chapter 4: Search and Discovery in Islandora (The Solr Module).

While Solr is not required to run Islandora, it is recommended.

Installing Solr and GSearch

1. Download fedoraGSearch from SourceForge.net and extract the contents of the compressed file. Copy the file fedoragsearch.war to the webapps folder.

cp ~/fedoragsearch.war $FEDORA_HOME/tomcat/webapps/

2. Download Solr to your local environment and unpack the downloaded file.

3. Create a directory for Solr. These instructions presume that it will be installed at ~/opt/solr:

mkdir -p /opt/solr

4. Drag or mv the .war file that is located under the dist directory of your unpacked download to the newly created /opt/solr directory.

5. Copy the directory apache-solr-x.x.x/example/solr to $FEDORA_HOME/solr and /exmaple/webapps/solr.war to $FEDORA_HOME/tomcat/webapps

cp -r apache-solr-x.x.x/example/solr $FEDORA_HOME/solr

cp apache-solr-x.x.x/example/webapps/solr.war $FEDORA_HOME/tomcat/webapps/

6. Edit the file $FEDORA_HOME/server/config/fedora-users.xml to add an additional user:

<user name="fgsAdmin" password="[password]">
   <attribute name="fedoraRole">
     <value>administrator</value>
   </attribute>
  </user>

7. Edit the file $FEDORA_HOME/tomcat/webapps/fedoragsearch/FgsConfig/fgsconfig-basic.properties

configDisplayName=configProductionOnSolr
gsearchPass= [password]
local.FEDORA_HOME=/usr/local/fedora
namesOfRepositories=FgsRepos

namesOfIndexes=FgsIndex
[keep name in order not to modify GUI for GSearch]

fedoraBase=http://localhost:8080
[to access objects from index in GUI for GSearch]

fedoraPass=[password]
indexEngine=Solr
indexBase=http://localhost:8080/solr
indexDir=${local.FEDORA_HOME}/solr/data/index
indexingDocXslt=foxmlToSolr

Gsearch properties files are sensitive.  A space after the text at the end of a line may cause that entry to fail.

8. Navigate to $FEDORA_HOME/tomcat/webapps/fedoragsearch/FgsConfig/and run Apache ant on fgsconfig-basic.xml

cd $FEDORA_HOME/tomcat/webapps/fedoragsearch/FgsConfig/

ant -f fgsconfig-basic.xml

9. Rename the current Solr schema.xml file to schema.bak

mv $FEDORA_HOME/solr/conf/schema.xml $FEDORA_HOME/solr/conf/schema.bak

10.  Copy the new, ant-generated Solr schema into place as schema.xml

cp $FEDORA_HOME/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/conf/schema-x.x.x-for-fgs-x.x.xml $FEDORA_HOME/solr/conf/schema.xml

11. Restart Fedora.

$FEDORA_HOME/tomcat/bin/shutdown.sh

$FEDORA_HOME/tomcat/bin/startup.sh

12. Test your installation by going to http://localhost:8080/fedoragsearch/rest

  1. Login using fgsAdmin 
  2. Click on updateindex
  3. Click on updateIndex fromFoxmlFiles

Installing & Configuring the Solr Search Module

The Islandora Solr search module allows you to search the Solr index.  The Islandora Solr Sample Configuration module provides default display profiles to the module. The module makes four new blocks available; two for search and two for display.  The other block is called the Advanced Search Block and does fielded searches against the Solr index.  Both blocks would use whatever request handler is configured in the module settings. For information on how to configure the Solr module, see Chapter 4 - Search and Discovery in Islandora (The Solr Module).

Installation Steps:

  1. Verify that fedoraGSearch and Solr are both installed and running.
  2. Download the islandora_solr_search module and install as a Drupal module.
  3. Then, download and uncompress the Apache Solr php client and copy the Solr directory under Apache from the archive to the islandora_solr_search module's folder.
  4. Log in to your Drupal site to enable the islandora_solr_search module.
    1. Administer > Modules and enable Islandora Solr Search

8 Comments

  1. It seems like there are some steps between step 6 and step 8. $FEDORA_HOME/tomcat/fedoragsearch does not have a FgsConfig directory. Also there is no conf directory in step 9 since step 3 creates the /opt/solr directory and step 4 adds the war file to the /opt/solr directory.

    1. Following up on this issue: it turns out that the installation requires gsearch 2.3 (I had downloaded 2.2 which doens't have any of the files. I followed up on the main 7.x-1.1 page to have the dl link point to 2.3 instead and that should resolve issues for folks).

  2. Actually, coming back to this, I realized there is no mention of solrconfig.xml (which defines the handlers). gsearch2.3 doesn't seem to have a solrconfig.xml file (and the xml file provided by solr does not have the handlers). Any suggestion on this would be helpful (that or just packaging solrconfig, schema.xml files with the module).

    1. This is a little confusing without clear indication of which versions of gsearch and solr we should be using with which version of Fedora Commons. I just failed an install and will be trying again tomorrow.

      EDIT: I used gsearch 2.6 and solr 4.2.0 and got it working. The instructions that come packaged with gsearch provide a little more clarity.

  3. A couple of notes:

    I was not able to get Solr running properly without configuring the Solr home location in Tomcat, which is not included in the steps here.  Following the instructions at the Solr wiki's page on Tomcat installation I created a solr.xml context file in $CATALINA_HOME/conf/Catalina/localhost, and that fixed the problem. That would fit in around step 6 above. 

    Using Solr 4.2.0 (and GSearch 2.6), the 'example' solr directory contains a subfolder 'collection1' that contains the 'conf' and 'data' folders referenced above. 

    Also, not sure why the step of creating the '/opt/solr' directory here is necessary?  The Solr wiki page on Tomcat installation shows installing into /opt, but in that case several of the other configurations would be different (i.e., not pointing to $FEDORA_HOME/solr).

  4. A couple of notes regarding Step 7 - Editing fgsconfig-basic.properties

    We recently went through an install with Gsearch 2.6 and in the log files upon start up, found some of the variables were not resolving correctly. We had to make the following additions to the fgsconfig-basic.properties and re-run the 'ant -f fgsconfigbasic.xml'  script:

    gsearchUser=[fgsuser added in previous step]
    fedoraBase=http://localhost:8080
    fedoraAppName=fedora
    gsearchBase=http://localhost:8080
    gsearchAppName=fedoragsearch
    fedoraVersion=[your version of Fedora, in our case the value was 3.6.2]
    fedoraUser=[your fedora admin user, usually fedoraAdmin] 
    objectStoreBase=${local.FEDORA_HOME}/data/objectStore

     

    Depending on your setup, some of these variables might not be used, but when looking at the log files such as fedoragsearch.daily.log, we were seeing some obvious variables that were not set. Just a heads up for others.

     

  5. My Gsearch 2.6 fgsconfig http://pastebin.com/xhFC9hPv

    I also update webapps/solr/WEB-INF/web.xml to have JNDI property for solr/home