Islandora utilizes the Solr open source search platform to enable flexible and configurable indexing and searching. Solr uses the Lucene Java search library at its core for full-text indexing and search and offers hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling as additional features.
The Fedora Generic Search Service, or GSearch, is a search service installed with Fedora that allows for automatic updating of the Lucene/Solr index. GSearch relies on JMS to receive messages that are sent when Fedora objects are ingested, modified or purged. This keeps the Lucene index in sync with the Fedora repository.
How Islandora uses Solr/Lucene and Gsearch
Islandora makes it possible to use the power of Solr/Lucene for discovery. Gsearch is used as a method for keeping indexes current. When an item is ingested, the FOXML is transformed by an XSLT file stored in Gsearch into a format that can be read by Solr's schema and returned based on the request handlers in our custom solrconfig.xml.
The Islandora Solr Search module is packaged with files that will support Islandora solution packs, but these can be modified if you are familiar with Solr. Specifically the Solr schema and corresponding Gsearch XSLT are a good starting point even if you do not use the Solr Search module. Additional information about Solr is presented in Chapter 4: Search and Discovery in Islandora (The Solr Module).
While Solr is not required to run Islandora, it is recommended.
Installing Solr and GSearch
1. Download fedoraGSearch from SourceForge.net and extract the contents of the compressed file. Copy the file fedoragsearch.war to the webapps folder.
cp ~/fedoragsearch.war $FEDORA_HOME/tomcat/webapps/
2. Download Solr to your local environment and unpack the downloaded file.
3. Create a directory for Solr. These instructions presume that it will be installed at ~/opt/solr:
4. Drag or mv the .war file that is located under the dist directory of your unpacked download to the newly created /opt/solr directory.
5. Copy the directory apache-solr-x.x.x/example/solr to $FEDORA_HOME/solr and /exmaple/webapps/solr.war to $FEDORA_HOME/tomcat/webapps
cp -r apache-solr-x.x.x/example/solr $FEDORA_HOME/solr
cp apache-solr-x.x.x/example/webapps/solr.war $FEDORA_HOME/tomcat/webapps/
6. Edit the file $FEDORA_HOME/server/config/fedora-users.xml to add an additional user:
<user name="fgsAdmin" password="[password]">
<attribute name="fedoraRole">
<value>administrator</value>
</attribute>
</user>
7. Edit the file $FEDORA_HOME/tomcat/webapps/fedoragsearch/FgsConfig/fgsconfig-basic.properties
configDisplayName=configProductionOnSolr
gsearchPass= [password]
local.FEDORA_HOME=/usr/local/fedora
namesOfRepositories=FgsRepos
namesOfIndexes=FgsIndex
[keep name in order not to modify GUI for GSearch]
fedoraBase=http://localhost:8080
[to access objects from index in GUI for GSearch]
fedoraPass=[password]
indexEngine=Solr
indexBase=http://localhost:8080/solr
indexDir=${local.FEDORA_HOME}/solr/data/index
indexingDocXslt=foxmlToSolr
8. Navigate to $FEDORA_HOME/tomcat/webapps/fedoragsearch/FgsConfig/and run Apache ant on fgsconfig-basic.xml
cd $FEDORA_HOME/tomcat/webapps/fedoragsearch/FgsConfig/
ant -f fgsconfig-basic.xml
9. Rename the current Solr schema.xml file to schema.bak
mv $FEDORA_HOME/solr/conf/schema.xml $FEDORA_HOME/solr/conf/schema.bak
10. Copy the new, ant-generated Solr schema into place as schema.xml
cp $FEDORA_HOME/tomcat/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/conf/schema-x.x.x-for-fgs-x.x.xml $FEDORA_HOME/solr/conf/schema.xml
11. Restart Fedora.
$FEDORA_HOME/tomcat/bin/shutdown.sh
$FEDORA_HOME/tomcat/bin/startup.sh
12. Test your installation by going to http://localhost:8080/fedoragsearch/rest
- Login using fgsAdmin
- Click on updateindex
- Click on updateIndex fromFoxmlFiles
Installing & Configuring the Solr Search Module
The Islandora Solr search module allows you to search the Solr index. The Islandora Solr Sample Configuration module provides default display profiles to the module. The module makes four new blocks available; two for search and two for display. The other block is called the Advanced Search Block and does fielded searches against the Solr index. Both blocks would use whatever request handler is configured in the module settings. For information on how to configure the Solr module, see Chapter 4 - Search and Discovery in Islandora (The Solr Module).
Installation Steps:
- Verify that fedoraGSearch and Solr are both installed and running.
- Download the islandora_solr_search module and install as a Drupal module.
- Then, download and uncompress the Apache Solr php client and copy the Solr directory under Apache from the archive to the islandora_solr_search module's folder.
- Log in to your Drupal site to enable the islandora_solr_search module.
- Administer > Modules and enable Islandora Solr Search
8 Comments
Ashok Modi
It seems like there are some steps between step 6 and step 8. $FEDORA_HOME/tomcat/fedoragsearch does not have a FgsConfig directory. Also there is no conf directory in step 9 since step 3 creates the /opt/solr directory and step 4 adds the war file to the /opt/solr directory.
Ashok Modi
Following up on this issue: it turns out that the installation requires gsearch 2.3 (I had downloaded 2.2 which doens't have any of the files. I followed up on the main 7.x-1.1 page to have the dl link point to 2.3 instead and that should resolve issues for folks).
Ashok Modi
Actually, coming back to this, I realized there is no mention of solrconfig.xml (which defines the handlers). gsearch2.3 doesn't seem to have a solrconfig.xml file (and the xml file provided by solr does not have the handlers). Any suggestion on this would be helpful (that or just packaging solrconfig, schema.xml files with the module).
Aaron Collie
This is a little confusing without clear indication of which versions of gsearch and solr we should be using with which version of Fedora Commons. I just failed an install and will be trying again tomorrow.
EDIT: I used gsearch 2.6 and solr 4.2.0 and got it working. The instructions that come packaged with gsearch provide a little more clarity.
Aaron Brenner
A couple of notes:
I was not able to get Solr running properly without configuring the Solr home location in Tomcat, which is not included in the steps here. Following the instructions at the Solr wiki's page on Tomcat installation I created a solr.xml context file in $CATALINA_HOME/conf/Catalina/localhost, and that fixed the problem. That would fit in around step 6 above.
Using Solr 4.2.0 (and GSearch 2.6), the 'example' solr directory contains a subfolder 'collection1' that contains the 'conf' and 'data' folders referenced above.
Also, not sure why the step of creating the '/opt/solr' directory here is necessary? The Solr wiki page on Tomcat installation shows installing into /opt, but in that case several of the other configurations would be different (i.e., not pointing to $FEDORA_HOME/solr).
Kristian Allen
A couple of notes regarding Step 7 - Editing fgsconfig-basic.properties
We recently went through an install with Gsearch 2.6 and in the log files upon start up, found some of the variables were not resolving correctly. We had to make the following additions to the fgsconfig-basic.properties and re-run the 'ant -f fgsconfigbasic.xml' script:
gsearchUser=[fgsuser added in previous step]
fedoraBase=http://localhost:8080
fedoraAppName=fedora
gsearchBase=http://localhost:8080
gsearchAppName=fedoragsearch
fedoraVersion=[your version of Fedora, in our case the value was 3.6.2]
fedoraUser=[your fedora admin user, usually fedoraAdmin]
objectStoreBase=${local.FEDORA_HOME}/data/objectStore
Depending on your setup, some of these variables might not be used, but when looking at the log files such as fedoragsearch.daily.log, we were seeing some obvious variables that were not set. Just a heads up for others.
Steve
My Gsearch 2.6 fgsconfig http://pastebin.com/xhFC9hPv
I also update webapps/solr/WEB-INF/web.xml to have JNDI property for solr/home
Benjamin Wiens
Step 12: Change the a href from http://192.168.56.101:8080/fedoragsearch/rest to http://localhost:8080/fedoragsearch/rest