Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Using

...

jquery

...

and

...

jquery-ui

...

libraries

...

we'll

...

try

...

to

...

setup

...

the

...

autocomplete

...

functionality

...

on

...

every

...

textbox

...

of

...

advanced

...

search

...

page,

...

we'll

...

take

...

advantage

...

of

...

Discovery

...

Solr

...

cores

...

to

...

add

...

an

...

additional

...

search

...

core

...

for

...

JSPUI.

...

Note

...

:

...

This

...

implementation

...

will

...

display

...

only

...

words

...

or

...

partial

...

words

...

when

...

using

...

autocomplete

...

funcionality

...

because

...

JSPUI

...

indexes

...

fields

...

splitting

...

them

...

in

...

words

...

and

...

removing

...

some

...

suffixes,

...

this

...

is

...

bad

...

for

...

searching

...

full

...

text

...

like

...

author

...

names.

...

In

...

a

...

future

...

section

...

I

...

will

...

describe

...

how

...

to

...

set

...

custom

...

text

...

indexers

...

like

...

"full

...

text"

...

to

...

our

...

fields.

...

Once

...

finished

...

our

...

textboxes

...

must

...

work

...

like

...

this:

Image Added

Wiki Markup
Administrators that use XMLUI interface will notice that Discovery functionality strongly increases searching and browsing by the use of layered navigation and textbox autocomplete among others. To achieve these goals an Apache Solr search server is deployed in \[dspace-source\]/dspace/solr to take care of all query-reponse handling. 
JSPUI interface use an Apache Lucene Directory (located on  \[dspace\]/search ) to search and browse by, but it leaks of previously mentioned functionalities.

...

We'll

...

use

...

a

...

servlet

...

class

...

to

...

query

...

a

...

SOLR

...

core

...

to

...

generate

...

a

...

JSON

...

responses,

...

this

...

responses

...

will

...

be

...

wired

...

to

...

a

...

textbox

...

with

...

autocomplete

...

functionality.

...

Here

...

is

...

a

...

simple

...

diagram

...

of

...

this

...

dataflow.

Image Added
 

First steps: Preparing JSPUI advanced search page

Wiki Markup
First of all we'll add jquery and jquery-ui libraries to advanced search .jsp page. We can add them on head section of advanced search page \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/search/advanced.jsp, or use a on/off switch approach to attach them as described on [Adding jQuery (or other script library) support on JSPUI|DSpaceKB:Adding jQuery (or other script library) support on JSPUI]. In this manual we'll attach jquery libraries by appending them on head section.

...

First

...

we

...

must

...

download

...

jquery

...

and

...

jquery-ui

...

script

...

libraries

...

from

...

jQuery

...

UI

...

webpage

...

,

...

we'll

...

download

...

it's

...

full

...

version

...

so

...

we

...

can

...

use

...

more

...

functionalities

...

on

...

future

...

improvements.

...

All

...

files

...

are

...

zipped

...

inside

...

a

...

structure

...

like

...

this:

...

  • Folder

...

  • -

...

  • css

...

  • Folder

...

  • -

...

  • js

...

  • Folder

...

  • -

...

  • development-bundle

...

  • File

...

  • -

...

  • index.html

...

We'll

...

use

...

only

...

js

...

and

...

css

...

folders

...

for

...

our

...

purposes.

...

I'll

...

describe

...

selected

...

folders

...

content:

...

  • js

...

  • :

...

  • Contains

...

  •  jquery-1.7.2.min.js

...

  • and jquery-ui-1.8.21.custom.min.js

...

  • script

...

  • files

...

  • css:

...

  • Contains

...

  • ui-lightness

...

  • folder,

...

  • here

...

  • are

...

  • all

...

  • necessary

...

  • .css

...

  • style

...

  • files

...

  • and

...

  • images

...

  • to

...

  • display

...

  • ui-lightness

...

  • visual

...

  • theme,

...

  • (name

...

  • will

...

  • vary

...

  • if

...

  • we

...

  • selected

...

  • another

...

  • visual

...

  • theme

...

  • on

...

  • jQuery

...

  • UI

...

  • page)

...

Wiki Markup
We'll create a new folder inside \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/*jqueryui* and copy all previously described files inside it.. Structure will look like this:

...

  • Wiki Markup
    \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/jqueryui/*{_}jquery-1.7.2.min.js{_}*

...

  • Wiki Markup
    \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/jqueryui/*{_}jquery-ui-1.8.21.custom.min.js{_}*

...

  • Wiki Markup
    \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/jqueryui/*css/...*

...

Wiki Markup
Now edit \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/search/advanced.jsp
\\

{:=|=|=
Code Block
language
html/xml
title
advanced.jsp
collapse
true
}
...
<%
 //Add this code on top section
 //Generate JSPUI Solr path url
 String scheme = request.getScheme();             // http
 String serverName = request.getServerName();     //request.getServerName();     // hostname.com
 int serverPort = request.getServerPort();        // 80
 String parentContextPath = "";                   //Change this value if dspace not installed on html root folder
 String contextPath = "/jspui";
 String servletPath = "/searchJSON";
 // Reconstruct original requesting URL
 StringBuffer url =  new StringBuffer();
 url.append(scheme).append("://").append(serverName);
 if (serverPort != 80)
 url.append(":").append(serverPort);
 url.append(parenContextPath).append(contextPath).append(servletPath);
 String solrPath = url.toString();

%>
...
<!-- Add this lines after <dspace:layout> tag-->
<link type="text/css" href="<%=request.getContextPath()%>/static/js/jqueryui/css/smoothness/jquery-ui-1.8.20.custom.css" rel="stylesheet" />
<script type="text/javascript" src="<%=request.getContextPath()%>/static/js/jqueryui/jquery-ui-1.8.20.custom.min.js"></script>
<script type="text/javascript" src="<%=request.getContextPath()%>/static/js/jqueryui/jquery-1.7.2.min.js"></script>
<script type="text/javascript" src="<%=request.getContextPath()%>/static/js/autocomplete.js"></script>
<script type="text/javascript">
window.onload = function() {
   monkeyPatchAutocomplete();
   replaceSAXExpression('<%=solrPath %>', 10,'tfield1', 'tquery1');
   replaceSAXExpression('<%=solrPath %>', 10,'tfield2', 'tquery2');
   replaceSAXExpression('<%=solrPath %>', 10,'tfield3', 'tquery3');
}
</script>
...
<!-- Add onchange event call for every search combobox -->
<select name="field1" id="tfield1" onchange="JavaScript:replaceSAXExpression('<%=solrPath%>', 10 ,'tfield1', 'tquery1');">
<select name="field2" id="tfield2" onchange="JavaScript:replaceSAXExpression('<%=solrPath%>', 10 ,'tfield2', 'tquery2');">
<select name="field3" id="tfield3" onchange="JavaScript:replaceSAXExpression('<%=solrPath%>', 10 ,'tfield3', 'tquery3');">
{code}

h2. Adding javascript functions

Adding javascript functions

Wiki Markup
It's time to create a javascript library file that will contain a pair of functions. &nbsp;First one will patch autocomplete default behavior and will bold every part of result text that match our typed words. Second one will replace SAX expression used in autocomplete textbox everytime we change select value (we must search by other index). It will reside in \[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/autocomplete.js

{:=|=
Code Block
language
javascript
title
autocomplete.js
linenumberstrue
collapsetrue
|collapse=true|linenumbers=true}
/**
 * Functions used on advanced search page
 */
function monkeyPatchAutocomplete() {
    // don't really need this, but in case I did, I could store it and chain
    var oldFn = $.ui.autocomplete.prototype._renderItem;

    $.ui.autocomplete.prototype._renderItem = function( ul, item){
    	var term = this.term.split(' ').join('|');
    	var re = new RegExp("(" + term + ")", "gi") ;
    	var t = item.label.replace(re,"<b>$1</b>");
    	return $( "<li></li>" )
    	.data( "item.autocomplete", item )
    	.append( "<a>" + t + "</a>" )
    	.appendTo( ul );
    };

}

function replaceSAXExpression(baseUrl , limit ,fieldName, fieldText){
	var field = $('#'+ fieldName).val();
	$('#' + fieldText).val('');
	$('#' + fieldText).autocomplete({
		source: function( request, response ) {
			$.ajax({
				url: baseUrl +"/terms?terms=true&terms.limit=" + limit + "&terms.sort=count&terms.regex.flag=case_insensitive&terms.fl=" + field + "&terms.regex=.*" + request.term + ".*&wt=json",
				dataType: "json",
				data: {
					style: "full",
					maxRows: 5,
					name_startsWith: request.term,
				},
				success: function( data ) {
 					response( $.map( data.terms[field], function( item ) {
						if(!$.isNumeric(item)){
							return{
    							label: item,
    							value: "\"" + item + "\"",
         					}
         				}else {
         					return null;
         				}
         			}
				));}
			});
		},
		minLength: 1,
		select: function( event, ui ) {

		},
		open: function() {
			$( this ).removeClass( "ui-corner-all" ).addClass( "ui-corner-top" );
		},
		close: function() {
			$( this ).removeClass( "ui-corner-top" ).addClass( "ui-corner-all" );
		}
	});
}{code}
\\

h2. Creating a new SOLR core


Creating a new SOLR core

Wiki Markup
Now we must create a new SOLR core inside \[dspace-source\]/solr directory. To do so we'll use one of the existing cores and copy it's structure with this command:

{
Code Block
}
bash$ cp -R [dspace-source]/dspace/solr/search [dspace-source]/dspace/solr/searchJSPUI
bash$ mkdir [dspace-source]/dspace/solr/searchJSPUI/data
bash$ mkdir [dspace-source]/dspace/solr/searchJSPUI/data/index
{code}

Wiki Markup
Edit \[dspace-source\]/dspace/solr/solr.xml and add one extra line with our new search core.

{
Code Block
}
<cores adminPath="/admin/cores">
 <core name="search" instanceDir="search" />
 <core name="statistics" instanceDir="statistics" />
 <core name="searchJSPUI" instanceDir="searchJSPUI" />  <!-- Add this line-->
</cores> 
{code}

Wiki Markup
Overwrite new solr core configuration file \[dspace-source\]/dspace/solr/searchJSPUI/conf/schema.xml to handle lucene generated indexes.

...

If

...

no

...

modification

...

to

...

search

...

indexes

...

has

...

been

...

done

...

on

...

dspace.cfg

...

file

...

this

...

configuration

...

file

...

will

...

be

...

enough:

...

schema.xml

Note

If we have added new search fields we must add them to schema.xml file in order to be accessible by SOLR and allow autocomplete funcionality. Example in schema.xml:

] {note}If we have added new search fields we must add them to schema.xml file in order to be accessible by SOLR and allow autocomplete funcionality. Example in schema.xml: {code}
Code Block

<field name="publisher" type="text" indexed="true" stored="true" />
<!-- field must have the same name value as we defined in search index in dspace.cfg -->
{code}\\ {note}


Wiki Markup
Then we'll edit \[dspace-source\]/dspace/config/dspace.cfg and modify default JSPUI search index folder:

{
Code Block
}
# Where to put search index files
search.dir = ${dspace.dir}/solr/searchJSPUI/data/index
{code}

So

...

now

...

our

...

SOLR

...

core

...

is

...

set

...

up,

...

new

...

uploaded

...

items

...

indexes

...

will

...

be

...

stored

...

inside

...

it's

...

data

...

folder.

...

Proceed

...

with

...

next

...

step:

...

Creating

...

SearchJSONServlet.java

Wiki Markup
Now we'll create the servlet responsible of reading autocomplete petitions and reponse with solr JSON structures. We could ommit this intermediate class but solr it's blocked from external querys due to security concerns, so this inner class will act as a bridge from client browser querys and internal solr server.
We'll create \[dspace-source\]/dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/servlet/SearchJSONServlet.java class and then will fill it's content with this code:

{:=|=
Code Block
language
java
title
SearchJSONServlet.java
linenumberstrue
collapsetrue
|collapse=true|linenumbers=true}package org.dspace.app.webui.servlet;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.sql.SQLException;

import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.xml.transform.stream.StreamResult;

import org.apache.log4j.Logger;
import org.dspace.authorize.AuthorizeException;
import org.dspace.core.Context;

public class SearchJSONServlet extends DSpaceServlet
{
 private static final long serialVersionUID = 1L;
 /** log4j category */
 private static Logger log = Logger.getLogger(SearchJSONServlet.class);

 /**
 * Create an DSpaceServlet Servlet
 */
 public SearchJSONServlet()
 {
 super();
 }

 protected void doDSPost(Context context, HttpServletRequest request,
 HttpServletResponse response) throws ServletException, IOException,
 SQLException, AuthorizeException
 {
 doDSGet(context, request, response);
 }

 protected void doDSGet(Context context, HttpServletRequest request,
 HttpServletResponse response) throws ServletException, IOException,
 SQLException, AuthorizeException
 {
 String pathString = request.getPathInfo();
 String queryString = request.getQueryString();

 //Build localhost solr query path
 String scheme = request.getScheme();             // http
 String serverName = "127.0.0.1";                 // Solr can only be accessed from localhost
 int serverPort = request.getServerPort();        // 80
 String contextPath = "/solr";
 String servletPath = "/searchJSPUI";

 StringBuffer url =  new StringBuffer();
 url.append(scheme).append("://").append(serverName);
 if (serverPort != 80)
 url.append(":").append(serverPort);
 url.append(contextPath).append(servletPath);
 url.append(pathString + "?");
 url.append(queryString);
 String solrPath = url.toString();

 //Get response from solr server and send it as servlet reponse
 StringBuilder jsonResponse = new StringBuilder();
 URL sorlURL = new URL(solrPath);
 URLConnection solrConnection = sorlURL.openConnection();
 BufferedReader in = new BufferedReader(new InputStreamReader(solrConnection.getInputStream()));
 String inputLine;
 while ((inputLine = in.readLine()) != null)
 jsonResponse.append(inputLine + "\n");
 in.close();
 StreamResult result = new StreamResult(response.getOutputStream());
 response.setHeader("Content-Length", String.valueOf(jsonResponse.length()));
 response.getOutputStream().flush();

 byte[] buf = jsonResponse.toString().getBytes();
 response.setContentLength(buf.length);
 ServletOutputStream servletOut = response.getOutputStream();
 servletOut.write(buf);
 }
}
{code}

This

...

code

...

captures

...

input

...

query

...

string

...

and

...

builds

...

a

...

conformant

...

SOLR

...

query

...

string

...

that

...

is

...

send

...

to

...

SOLR

...

server,

...

it's

...

reponse

...

it's

...

directly

...

returned

...

via

...

result

...

variable

...

to

...

reponse

...

output

...

stream.

Wiki Markup
Now we'll link our new servlet with it's appropriate url, to do so we we'll add this lines in&nbsp;\[dspace-source\]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/WEB-INF/web.xml

{:=|=|=
Code Block
language
html/xml
title
web.xml
collapse
true
}<servlet>
        <servlet-name>searchJSON</servlet-name>
        <servlet-class>org.dspace.app.webui.servlet.SearchJSONServlet</servlet-class>
</servlet>
<servlet-mapping>
        <servlet-name>searchJSON</servlet-name>
        <url-pattern>/searchJSON/*</url-pattern>
</servlet-mapping>{code}

h2. &nbsp;Check results

After we package and install our modified version of JPSUI interface in our webserver, we will check a pair aspects to be sure everthing works as expected.

First we'll browse locally to [

 Check results

After we package and install our modified version of JPSUI interface in our webserver, we will check a pair aspects to be sure everthing works as expected.

First we'll browse locally to http://127.0.0.1/solr/

...

and

...

must

...

appear

...

three

...

SOLR

...

cores

...

as

...

hyperlinks:

...

  1. search
  2. statistics
  3. searchJSPUI.

Note: If there's

...

no

...

desktop

...

environment

...

installed

...

on

...

your

...

server

...

you

...

can

...

use

...

"links"

...

package

...

to

...

browse

...

it

...

on

...

text

...

mode.

...

We'll

...

click

...

on

...

last

...

one

...

and

...

try

...

to

...

access

...

it's

...

admin

...

webpage

...

and

...

then

...

to

...

schema

...

browser,

...

it

...

must

...

three

...

categories

...

on

...

left

...

column

...

bar:

...

fields,

...

dynamic

...

fields

...

and

...

field

...

types.

...

Other

...

possible

...

scenarios

...

If

...

this

...

modification

...

is

...

done

...

over

...

a

...

running

...

instance

...

with

...

data

...

in

...

it

...

we

...

must

...

follow

...

this

...

steps

...

after

...

run

...

"ant

...

update":

...

  • Verify

...

  • that

...

  • running dspace.cfg

...

  • contains

...

  • this

...

  • configuration

...

  • line:

...

  •  
    Code Block
    
    search.dir = ${dspace.dir}/solr/searchJSPUI/data/index
    

...

  • Wiki Markup
    run an index rebuild to populate \[dspace\]/solr/searchJSPUI/data/index folder with current search indexes (recommended) or copy them directly from \[dspace\]/search