(Jump to other KnowledgeBases)
DSpace Resources
DSpace Website
DSpace Documentation
DSpace Wiki
DSpace Demo Site
Contribute to KnowledgBase
Page History
...
Using
...
jquery
...
and
...
jquery-ui
...
libraries
...
we'll
...
try
...
to
...
setup
...
the
...
autocomplete
...
functionality
...
on
...
every
...
textbox
...
of
...
advanced
...
search
...
page,
...
we'll
...
take
...
advantage
...
of
...
Discovery
...
Solr
...
cores
...
to
...
add
...
an
...
additional
...
search
...
core
...
for
...
JSPUI.
...
Note
...
:
...
This
...
implementation
...
will
...
display
...
only
...
words
...
or
...
partial
...
words
...
when
...
using
...
autocomplete
...
funcionality
...
. JSPUI
...
interface indexes
...
fields
...
by splitting
...
them
...
into words
...
and
...
removing
...
some
...
suffixes,
...
this
...
is
...
bad
...
for
...
searching
...
full
...
text
...
terms like
...
author
...
names.
...
In
...
a
...
future
...
section
...
I
...
will
...
describe
...
how
...
to
...
set
...
custom
...
text
...
indexers
...
like
...
"full
...
text"
...
to
...
our
...
fields in JSPUI.
...
Once
...
finished
...
our
...
textboxes
...
must
...
work
...
like
...
this:
...
Administrators that use XMLUI interface will notice that Discovery functionality strongly increases searching and browsing by the use of layered navigation and textbox autocomplete among others. To achieve these goals an Apache Solr search server is deployed in [dspace-source]/dspace/solr
...
to
...
take
...
care
...
of
...
all
...
query-reponse
...
handling.
...
JSPUI interface use an Apache Lucene Directory (located on [dspace]/search ) to search and browse by, but it leaks of previously mentioned functionalities.
We'll use a servlet class to query a SOLR core to generate JSON responses, this responses will be wired to textboxes with autocomplete functionality. Here is a simple diagram of this dataflow.
First steps: Preparing JSPUI advanced search page
First of all we'll add jquery and jquery-ui libraries to advanced search .jsp page. This libraries have the necessary code to implement autocomplete behavior. We can add them on head section of advanced search page [dspace-source]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/search/advanced.jsp,
...
or
...
use
...
a
...
on/off
...
switch
...
approach
...
to
...
attach
...
them
...
as
...
described
...
on
...
...
...
...
...
...
...
...
...
JSPUI. In this manual we'll use second approach because it's the fastest way to disable scriptaculous libraries to avoid colliding with jquery (both use the same notation $ for calling functions ).
We'll edit [dspace-source]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/search/advanced.jsp
| Code Block | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| |
| =
| |
| =
| ||||||||||
} ... <% //Add this code on top section //Generate JSPUI Solr path url String scheme = request.getScheme(); // http String serverName = request.getServerName(); //request.getServerName(); // hostname.com int serverPort = request.getServerPort(); // 80 String parentContextPath = ""; //Change this value if dspace not installed on html root folder String contextPath = "/jspui"; String servletPath = "/searchJSON"; // Reconstruct original requesting URL StringBuffer url = new StringBuffer(); url.append(scheme).append("://").append(serverName); if (serverPort != 80) url.append(":").append(serverPort); url.append(parenContextPathparentContextPath).append(contextPath).append(servletPath); String solrPath = url.toString(); %> ... <!-- Add this lines after <dspace:layout> tag--> <link type="text/css" href="<%=request.getContextPath()%>/static/js/jqueryui/css/smoothness/jquery-ui-1.8.20.custom.css" rel="stylesheet" /> <script type="text/javascript" src="<%=request.getContextPath()%>/static/js/jqueryui/jquery-ui-1.8.20.custom.min.js"></script> <script type="text/javascript" src="<%=request.getContextPath()%>/static/js/jqueryui/jquery-1.7.2.min.js"></script> url.toString(); %> ... <dspace:layout locbar="nolink" titlekey="jsp.search.advanced.title" scriptaculous="off" jquery="on"> <script type="text/javascript" src="<%=request.getContextPath()%>/static/js/autocomplete.js"></script> <script type="text/javascript"> window.onload = function() { monkeyPatchAutocomplete(); monkeyPatchAutocomplete( replaceSAXExpression('<%=solrPath %>', 10,'tfield1', 'tquery1'); replaceSAXExpression('<%=solrPath %>', 10,'tfield1tfield2', 'tquery1tquery2'); replaceSAXExpression('<%=solrPath %>', 10,'tfield2tfield3', 'tquery2tquery3'); } </script> ... <!-- Add id field and onchange event call for every search combobox, add id field on search textbox too--> <select name="field1" id="tfield1" onchange="JavaScript:replaceSAXExpression('<%=solrPath %>solrPath%>', 10 ,'tfield3tfield1', 'tquery3tquery1'); } </script> ... <!-- Add onchange event call for every search combobox --> "> <input type="text" name="query1" id="tquery1" value="<%=StringEscapeUtils.escapeHtml(query1)%>" size="30" /> ... <select name="field1field2" id="tfield1tfield2" onchange="JavaScript:replaceSAXExpression('<%=solrPath%>', 10 ,'tfield1tfield2', 'tquery1');">tquery2');"> <input type="text" name="query2" id="tquery2" value="<%=StringEscapeUtils.escapeHtml(query2)%>" size="30"/> ... <select name="field2field3" id="tfield2tfield3" onchange="JavaScript:replaceSAXExpression('<%=solrPath%>', 10 ,'tfield2tfield3', 'tquery2tquery3');"> <select<input type="text" name="field3query3" id="tfield3tquery3" onchangevalue="JavaScript:replaceSAXExpression('<%=solrPath%>', 10 ,'tfield3', 'tquery3');"> {code} h2. Adding javascript functions It's time to create a javascript library file that will contain a pair of functions. First one will patch autocomplete default behavior and will bold every part of result text that match our typed words. Second one will replace SAX expression used in autocomplete textbox everytime we change select value (we must search by other index). It will reside in \[dspace-source\<%=StringEscapeUtils.escapeHtml(query3)%>" size="30"/> |
Adding javascript functions
It's time to create a javascript library file that will contain a pair of functions. First one will patch autocomplete default behavior and will bold every part of result text that match our typed words. Second one will replace SAX expression used in autocomplete textbox everytime we change select value (we must search by other index). It will reside in [dspace-source]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/static/js/autocomplete.js
| Code Block | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| =
| |
| =
| |||||||||||
|collapse=true|linenumbers=true} /** * Functions used on advanced search page */ function monkeyPatchAutocomplete() { // don't really need this, but in case I did, I could store it and chain var oldFn = $.ui.autocomplete.prototype._renderItem; $.ui.autocomplete.prototype._renderItem = function( ul, item){ var term = this.term.split(' ').join('|'); var re = new RegExp("(" + term + ")", "gi") ; var t = item.label.replace(re,"<b>$1</b>"); return $( "<li></li>" ) .data( "item.autocomplete", item ) .append( "<a>" + t + "</a>" ) .appendTo( ul ); }; } function replaceSAXExpression(baseUrl , limit ,fieldName, fieldText){ var field = $('#'+ fieldName).val(); $('#' + fieldText).val(''); $('#' + fieldText).autocomplete({ source: function( request, response ) { $.ajax({ url: baseUrl +"/terms?terms=true&terms.limit=" + limit + "&terms.sort=count&terms.regex.flag=case_insensitive&terms.fl=" + field + "&terms.regex=.*" + request.term + ".*&wt=json", dataType: "json", data: { style: "full", maxRows: 5, name_startsWith: request.term, }, success: function( data ) { response( $.map( data.terms[field], function( item ) { if(!$.isNumeric(item)){ return{ label: item, value: "\"" + item + "\"", } }else { return null; } } ));} }); }, minLength: 1, select: function( event, ui ) { }, open: function() { $( this ).removeClass( "ui-corner-all" ).addClass( "ui-corner-top" ); }, close: function() { $( this ).removeClass( "ui-corner-top" ).addClass( "ui-corner-all" ); } }); }{code} \\ h2. Creating a new SOLR core Now we must create a new SOLR core inside \ |
Creating a new SOLR core
Now we must create a new SOLR core inside [dspace-source
...
]/solr
...
directory.
...
To
...
do
...
so
...
we'll
...
use
...
one
...
of
...
the
...
existing
...
cores
...
and
...
copy
...
it's
...
structure
...
with
...
this
...
command:
| Code Block |
|---|
} bash$ cp -R [dspace-source]/dspace/solr/search [dspace-source]/dspace/solr/searchJSPUI bash$ mkdir [dspace-source]/dspace/solr/searchJSPUI/data bash$ mkdir [dspace-source]/dspace/solr/searchJSPUI/data/index {code} Edit \ |
Edit [dspace-source
...
]/dspace/solr/solr.xml
...
and
...
add
...
one
...
extra
...
line
...
with
...
our
...
new
...
search
...
core.
| Code Block |
|---|
} <cores adminPath="/admin/cores"> <core name="search" instanceDir="search" /> <core name="statistics" instanceDir="statistics" /> <core name="searchJSPUI" instanceDir="searchJSPUI" /> <!-- Add this line--> </cores> {code} |
Overwrite
...
new
...
solr
...
core
...
configuration
...
file
...
[dspace-source
...
]/dspace/solr/searchJSPUI/conf/schema.xml with this file schema.xml to handle lucene indexes generated by JSPUI.
If no modification to search indexes has been done on dspace.cfg file this configuration file will be enough.
| Note | |||
|---|---|---|---|
If we have added new search fields we must add them to schema.xml file in order to be accessible by SOLR and allow autocomplete funcionality. Example in schema.xml: to handle lucene generated indexes. If no modification to search indexes has been done on dspace.cfg file this configuration file will be enough: [^schema.xml] {note}If we have added new search fields we must add them to schema.xml file in order to be accessible by SOLR and allow autocomplete funcionality. Example in schema.xml: {code}
|
Then we'll
...
edit
...
[dspace-source
...
]/dspace/config/dspace.cfg
...
and
...
modify
...
default
...
JSPUI
...
search
...
index
...
folder:
| Code Block |
|---|
} # Where to put search index files search.dir = ${dspace.dir}/solr/searchJSPUI/data/index {code} |
So
...
now
...
our
...
SOLR
...
core
...
is
...
set
...
up,
...
new
...
uploaded
...
items
...
indexes
...
will
...
be
...
stored
...
inside
...
it's
...
data
...
folder.
...
Proceed
...
with
...
next
...
step:
...
Creating
...
SearchJSONServlet.java
...
Now
...
we'll
...
create
...
the
...
servlet
...
responsible
...
of
...
reading
...
autocomplete
...
petitions
...
and
...
reponse
...
with
...
solr
...
JSON
...
structures.
...
We
...
could
...
ommit
...
this
...
intermediate
...
class
...
but
...
solr
...
it's
...
blocked
...
from
...
external
...
querys
...
due
...
to
...
security
...
concerns,
...
so
...
this
...
inner
...
class
...
will
...
act
...
as
...
a
...
bridge
...
from
...
client
...
browser
...
querys
...
and
...
internal
...
solr
...
server.
...
We'll
...
create
...
[dspace-source
...
]/dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/servlet/SearchJSONServlet.java
...
class
...
and
...
then
...
will
...
fill
...
it's
...
content
...
with
...
this
...
code:
| Code Block | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| =
| |
| =
| |||||||||||
|collapse=true|linenumbers=true}package org.dspace.app.webui.servlet; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.net.URL; import java.net.URLConnection; import java.sql.SQLException; import javax.servlet.ServletException; import javax.servlet.ServletOutputStream; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import javax.xml.transform.stream.StreamResult; import org.apache.log4j.Logger; import org.dspace.authorize.AuthorizeException; import org.dspace.core.Context; public class SearchJSONServlet extends DSpaceServlet { private static final long serialVersionUID = 1L; /** log4j category */ private static Logger log = Logger.getLogger(SearchJSONServlet.class); /** * Create an DSpaceServlet Servlet */ public SearchJSONServlet() { super(); } protected void doDSPost(Context context, HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException, SQLException, AuthorizeException { doDSGet(context, request, response); } protected void doDSGet(Context context, HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException, SQLException, AuthorizeException { String pathString = request.getPathInfo(); String queryString = request.getQueryString(); //Build localhost solr query path String scheme = request.getScheme(); // http String serverName = "127.0.0.1"; // Solr can only be accessed from localhost int serverPort = request.getServerPort(); // 80 String contextPath = "/solr"; String servletPath = "/searchJSPUI"; String servletPath = "/searchJSPUI"; String parentContextPath = ""; //Change this value if your dspace installation is not on base web server directory StringBuffer url = new StringBuffer(); url.append(scheme).append("://").append(serverName); if (serverPort != 80) url.append(":").append(serverPort); url.append(parentContextPath).append(contextPath).append(servletPath); url.append(pathString + "?"); url.append(queryString); String solrPath = url.toString(); //Get response from solr server and send it as servlet reponse StringBuilder jsonResponse = new StringBuilder(); URL sorlURL = new URL(solrPath); URLConnection solrConnection = sorlURL.openConnection(); BufferedReader in = new BufferedReader(new InputStreamReader(solrConnection.getInputStream())); String inputLine; while ((inputLine = in.readLine()) != null) jsonResponse.append(inputLine + "\n"); in.close(); StreamResult result = new StreamResult(response.getOutputStream()); response.setHeader("Content-Length", String.valueOf(jsonResponse.length())); response.getOutputStream().flush(); byte[] buf = jsonResponse.toString().getBytes(); response.setContentLength(buf.length); ServletOutputStream servletOut = response.getOutputStream(); servletOut.write(buf); } } {code} |
This
...
code
...
captures
...
input
...
query
...
string
...
and
...
builds
...
a
...
conformant
...
SOLR
...
query
...
string
...
that
...
is
...
send
...
to
...
SOLR
...
server,
...
it's
...
reponse
...
it's
...
directly
...
returned
...
via
...
result
...
variable
...
to
...
reponse
...
output
...
stream.
...
Now
...
we'll
...
link
...
our
...
new
...
servlet
...
with
...
it's
...
appropriate
...
url,
...
to
...
do
...
so
...
we
...
we'll
...
add
...
this
...
lines in [dspace-source
...
]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/WEB-INF/web.xml
| Code Block | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| =
| |
| =
| |
| =
| |||||||||||
}<servlet> <servlet-name>searchJSON</servlet-name> <servlet-class>org.dspace.app.webui.servlet.SearchJSONServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>searchJSON</servlet-name> <url-pattern>/searchJSON/*</url-pattern> </servlet-mapping>{code} h2. Check results After we package and install our modified version of JPSUI interface in our webserver, we will check a pair aspects to be sure everthing works as expected. First we'll browse locally to [ |
Check results
After we package and install our modified version of JPSUI interface in our webserver, we will check a pair aspects to be sure everthing works as expected.
First we'll browse locally to http://127.0.0.1/solr/ or http://127.0.0.1:8080/solr/
...
(depending on your webserver port and configuration) and there must appear three SOLR cores as hyperlinks:
- search
- statistics
- searchJSPUI.
Note: If there's no desktop environment installed on your server you can use "links" package to browse it on text mode or even use "curl" program to send http request to our web server.
We'll click on last one and try to access it's admin webpage and then to schema browser, there must be three categories on left column bar: fields, dynamic fields and field types.
Other possible scenarios
If this modification is done over a running instance with data in it we must follow this steps after run "ant update":
- Verify that running dspace.cfg contains this configuration line:
Code Block search.dir = ${dspace.dir}/solr/searchJSPUI/data/index
...
- run
...
- an
...
- index
...
- rebuild
...
- to
...
- populate
...
- [dspace
...
- ]/solr/searchJSPUI/data/index
...
- folder
...
- with
...
- current
...
- search
...
- indexes
...
- (recommended)
...
- or
...
- copy
...
- them
...
- directly
...
- from
...
- [dspace
...
- ]/search
Overview
Content Tools