Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note: This implementation will display only words or partial words when using autocomplete funcionality because JSPUI indexes fields splitting it them in words and removing some suffixes, this is bad for searching full text like author names. In a future section I will describe how to set custom text indexers like "full text" to our custom fields.

Once finished our textboxes must work like this:

...

  • Wiki Markup
    \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/jqueryui/*{_}jquery-1.7.2.min.js{_}*
  • Wiki Markup
    \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/jqueryui/*{_}jquery-ui-1.8.21.custom.min.js{_}*
  • Wiki Markup
    \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/jqueryui/*css/...*

Wiki Markup
We'llNow edit \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/search/advanced.jsp

Adding javascript functions

It's time to create a javascript library file that will contain a pair of functions.  First one will patch autocomplete default behavior and will bold every part of result text that match our typed words. Second one will replaceSAX expression on our texbox everytime a keyword it's typed in it, giving the autocomplete effect.

Code Block
languagejavascript
titleautocomplete.js
linenumberstrue
collapsetrue

/**
 * Functions used on advanced search page
 */
function monkeyPatchAutocomplete() {
    // don't really need this, but in case I did, I could store it and chain
    var oldFn = $.ui.autocomplete.prototype._renderItem;

    $.ui.autocomplete.prototype._renderItem = function( ul, item){
    	var term = this.term.split(' ').join('|');
    	var re = new RegExp("(" + term + ")", "gi") ;
    	var t = item.label.replace(re,"<b>$1</b>");
    	return $( "<li></li>" )
    	.data( "item.autocomplete", item )
    	.append( "<a>" + t + "</a>" )
    	.appendTo( ul );
    };

}

function replaceSAXExpression(baseUrl , limit ,fieldName, fieldText){
	var field = $('#'+ fieldName).val();
	$('#' + fieldText).val('');
	$('#' + fieldText).autocomplete({
		source: function( request, response ) {
			$.ajax({
				url: baseUrl +"/terms?terms=true&terms.limit=" + limit + "&terms.sort=count&terms.regex.flag=case_insensitive&terms.fl=" + field + "&terms.regex=.*" + request.term + ".*&wt=json",
				dataType: "json",
				data: {
					style: "full",
					maxRows: 5,
					name_startsWith: request.term,
				},
				success: function( data ) {
 					response( $.map( data.terms[field], function( item ) {
						if(!$.isNumeric(item)){
							return{
    							label: item,
    							value: "\"" + item + "\"",
         					}
         				}else {
         					return null;
         				}
         			}
				));}
			});
		},
		minLength: 1,
		select: function( event, ui ) {

		},
		open: function() {
			$( this ).removeClass( "ui-corner-all" ).addClass( "ui-corner-top" );
		},
		close: function() {
			$( this ).removeClass( "ui-corner-top" ).addClass( "ui-corner-all" );
		}
	});
}

 

Creating a new SOLR core

Wiki Markup
Now we must create a new SOLR core inside \[dspace-source\]/solr directory. To do so we'll use one of the existing cores and copy it's structure with this command:

...

Code Block
<cores adminPath="/admin/cores">
 <core name="search" instanceDir="search" />
 <core name="statistics" instanceDir="statistics" />
 <core name="searchJSPUI" instanceDir="searchJSPUI" />  <!-- Add this line-->
</cores> 

Wiki Markup
Now we'll modifyoverwrite our new solr core configuration file \[dspace-source\]/dspace/solr/searchJSPUI/conf/schema.xml to handle lucene generated indexes.

 If no modification to search indexes has been done on dspace.cfg file this configuration file will be enough:

schema.xml

Note

If we have added new search fields we must add them to schema.xml file in order to be accessible by SOLR and allow autocomplete funcionality. Example in schema.xml:

Code Block

<field name="publisher" type="text" indexed="true" stored="true" />
<!-- field must have the same name value as we defined in search index in dspace.cfg -->

Wiki Markup
Then we'll edit \[dspace-source\]/dspace/config/dspace.cfg and modify default JSPUI search index folder:&nbsp;

Code Block
# Where to put search index files
search.dir = ${dspace.dir}/solr/searchJSPUI/data/index

So now our SOLR core is set up, new uploaded items indexes will be stored inside it's data folder. Proceed with next step:

Creating SearchJSONServlet.java

Wiki Markup
Now we'll create the servlet responsible of reading autocomplete petitions and reponse with solr JSON structures. We could ommit this intermediate class but solr it's blocked from external querys due to security concerns, so this inner class will act as a bridge from client browser querys and internal solr server.
We'll create \[dspace-source\]/dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/servlet/SearchJSONServlet.java class and then will fill it's content with this code:

Code Block
languagejava
titleSearchJSONServlet.java
linenumberstrue
collapsetrue
package org.dspace.app.webui.servlet;

import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLDecoder;
import java.sql.SQLException;
import java.util.Map;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

import org.apache.log4j.Logger;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.Term;
import org.apache.lucene.queryParser.ParseException;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.queryParser.TokenMgrError;
import org.apache.lucene.search.BooleanQuery;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.Sort;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.search.TopScoreDocCollector;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;
import org.dspace.app.webui.util.JSPManager;
import org.dspace.authorize.AuthorizeException;
import org.dspace.content.Bitstream;
import org.dspace.content.Bundle;
import org.dspace.content.DCValue;
import org.dspace.content.Item;
import org.dspace.core.ConfigurationManager;
import org.dspace.core.Constants;
import org.dspace.core.Context;
import org.dspace.core.LogManager;
import org.dspace.handle.HandleManager;
import org.dspace.search.CustomQueryParser;
import org.dspace.search.DSIndexer;
import org.dspace.search.DSQuery;
import org.dspace.usage.UsageEvent;
import org.dspace.utils.DSpace;

public class SearchJSONServlet extends DSpaceServlet
{
    /** log4j category */
    private static Logger log = Logger.getLogger(SearchJSONServlet.class);

    /* Create an DSpaceServlet Servlet */
   public SearchJSONServlet()
    {
        super();
    }

    protected void doDSPost(Context context, HttpServletRequest request,
            HttpServletResponse response) throws ServletException, IOException,
            SQLException, AuthorizeException
    {
        doDSGet(context, request, response);
    }

    protected void doDSGet(Context context, HttpServletRequest request,
            HttpServletResponse response) throws ServletException, IOException,
            SQLException, AuthorizeException
    {
        String pathString = request.getPathInfo();
        String queryString = request.getQueryString();

        //Build localhost solr query path
        String scheme = request.getScheme();             // http
        String serverName = "127.0.0.1";                 // Solr can only be accessed from localhost
        int serverPort = request.getServerPort();        // 80
        String parentContextPath = "";                   // Change this value if dspace webpages are placed in a different folder other than html service root folder
        String contextPath = "/solr";
        String servletPath = "/searchJSPUI";

        StringBuffer url =  new StringBuffer();
        url.append(scheme).append("://").append(serverName);
        if (serverPort != 80)
            url.append(":").append(serverPort);
        url.append(parentContextPath).append(contextPath).append(servletPath);
        url.append(pathString + "?");
        url.append(queryString);
        String solrPath = url.toString();

        //Get response from solr server and send it as servlet reponse
        StringBuilder jsonResponse = new StringBuilder();
        URL sorlURL = new URL(solrPath);
        URLConnection solrConnection = sorlURL.openConnection();
        BufferedReader in = new BufferedReader(new InputStreamReader(solrConnection.getInputStream()));
        String inputLine;
        while ((inputLine = in.readLine()) != null)
            jsonResponse.append(inputLine + "\n");
        in.close();
        StreamResult result = new StreamResult(response.getOutputStream());
        response.setHeader("Content-Length", String.valueOf(jsonResponse.length()));
        response.getOutputStream().flush();

        byte[] buf = jsonResponse.toString().getBytes();
        response.setContentLength(buf.length);
        ServletOutputStream servletOut = response.getOutputStream();
        servletOut.write(buf);
    }
}

This code captures input query string and builds a conformant SOLR query string that is send to SOLR server, it's reponse it's directly returned via result variable to reponse output stream.

Wiki Markup
Now we'll link our new servlet with it's appropriate url, to do so we we'll add this lines in&nbsp;\[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/WEB-INF/web.xml

Code Block
languagehtml/xml
titleweb.xml
collapsetrue
<servlet>
  	<servlet-name>searchJSON</servlet-name>
  	<servlet-class>org.dspace.app.webui.servlet.SearchJSONServlet</servlet-class>
</servlet>
<servlet-mapping>
        <servlet-name>searchJSON</servlet-name>
  	<url-pattern>/searchJSON/*</url-pattern>
</servlet-mapping>