Page History

← ALL HOW TOs

...

...

References

• Introduction  (youtube)

Terminology

...

Indexing Documents

• index via...
  • Request Handlers & Update Handlers (via HTTP POST/PUT)
    
    • default:  XML, Binary, JSON, CVS, etc.
    • can define own handlers in config
  • Index Handlers
    
    • import from databases
  • Solr Cell framework (???)
  • custom Java application to ingest data through Solr's Java Client and other apps
• update processors
  • signature
  • logging
  • indexing

Request Handlers

<!--  solr.SearchHandler  -->
<requestHandler name="standard" class="solr.SearchHandler">               <!-- /select -->
<requestHandler name="search" class="solr.SearchHandler" default="true">
<requestHandler name="permissions" class="solr.SearchHandler" >
<requestHandler name="document" class="solr.SearchHandler" >

<!--  solr.UpdateRequestHandler  -->
<requestHandler name="/update" class="solr.UpdateRequestHandler"  />

<!--  other handlers  -->
<requestHandler name="/replication" class="solr.ReplicationHandler" startup="lazy" />
<requestHandler name="/analysis/field" startup="lazy" class="solr.FieldAnalysisRequestHandler" />
<requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
<requestHandler name="/admin/ping" class="solr.PingRequestHandler">

...

To see what a requestHandler returns, change the value of qt from /select to the name of the handler in the solr admin Query page (https://cals-che-repo-dev.library.cornell.edu/solr/#/development/query).  NOTE: You will need to change the host to your solr admin host and may need to change the name of the core from development to the name or your core.

...

Querying

• receive XML, JSON, CSV, or binary (via HTTP GET)
• request handlers (via HTTP GET)
  
  • default:  /admin, /select, /spell
  • can define own handlers in config
• search components
  • query
  • spelling
  • faceting
  • highlighting
  • statistics
  • debug
  • clustering

• search process  (see Common Query Parameters)

  	description	default	example
  qt	selects Request Handler for a query using /select	DisMaxRequestHandler
  defType	selects a Query Parser for the query	parser configured in Request Handler
  q	field_name:field_value with * as wildcard to search for	*:*	q=title:*Archery*
  fq	filters query by applying an additional query to the initial query's results, caches the results (same syntax as q)	*:*	fq=popularity:[10TO*]& fq=section:0
  sort	sort field	score desc
  start	an offset into the query results where the returned response should begin	0	start=0
  rows	the number of rows to be displayed at one time	10	rows=20
  fl	fields to return in result	all	fl=id, name
  df	default field name (I think) that indicates field to serch	all indexed fields	df=description
  wt	selects a Response Writer for formatting the query response	xml | json	wt=json
  qf	list of fields and the "boosts" to associate with each of them when building DisjunctionMaxQueries  (see also SOLR df and qf explanation)	all indexed fields are required (???)	qf=title^20 description^10

  
  

...

Features

• High Level
  • Advanced Full-Text Search
  • Optimized for High Volume Web Traffic
  • Standards Based Open Interfaces - XML, JSON, HTTP
  • Comprehensive HTML Admin Interfaces
  • Service statistics exposed over JMX for monitoring
  • Near Real-time indexing and Adaptable with XML configuration
  • Linearly scalable, auto index replication, auto, extensible plugin architecture
• Specific Features
  • faceting
  • highlighting
  • spell checking
  • query-re-ranking
  • transforming
  • suggestors
  • more like this
  • pagination
  • grouping & clustering
  • spatial search
  • components
  • real time (get & update)
  • labs

...

Configuration

• schema.xml
  • field types
  • etc.
• solrconfig.xml
  • register Request Handlers for querying the index
  • register Update Handlers for indexing documents
  • register Event Handlers for searcher events (e.g. queries to execute to warm new searches)
  • activate version-dependent features in Lucene
  • Lib directives indicates where Solr can find JAR files for extensions
  • Index management settings
  • Enable JMX instrumentation of Solr MBeans
  • Cache-management settings
• solr.xml
• core.properties

Fields

Defined in schema.xml

Hydra Types: 

defined by <types><fieldType>...</></>

...

NOTE: letter indicates the postfix indicator that sets the type for Hydra dynamic fields.  Ex. name_tsi means that name has type="text"

Hydra Field Def Parameters:

defined by <fields><dynamicField>...</></>

...

NOTE: letter indicates the postfix indicator that sets that to true for Hydra dynamic fields.  Ex. name_tsi means that name has stored=true/indexed="true"

Examples for values of stored and indexed:

...

...

Solr Cloud Features

• horizontal scaling (for sharding and replication)
• elastic scaling
• high availability
• distributed indexing
• distributed searching
• central configuration for entire cluster
• automatic load balancing
• automatic failover for queries
• zookeeper integration for coordination & configurations

CRUD

Create

Read

Return all results with search term = "book"

...

http://localhost:8983/solr/#/development/select?q=book

Update

Delete

NOTE: Examples use stream.body to show how to do this through a URL.  Usually done via HTTP POST.

...

• In Solr UI, select core to effect from selection box on left side menu
• select Documents on left side menu
• set Document Type = XML
• set Doucment(s) text area to `<delete><query>*:*</query></delete>`
• leave commit within and overwrite as defaults
• Submit

...

• More Query Examples

Search for a specific field, category, containing a search term, book

http://localhost:8983/solr/#/development/select?q=category:book

Search for price between 0 and 400, inclusive

http://localhost:8983/solr/#/development/select?q=price:[0 TO 400]

Limit search results to return only fields id, name, and price.

http://localhost:8983/solr/#/development/select?q=book&fl=id,name,price

Return facets for a specific field, category, with counts for each value of category based on the search results.

http://localhost:8983/solr/#/development/select?q=book&fl=id,name,price&facet=on&facet.field=category

...

<lst name="facet_counts">
  <lst name="facet_queries" />
  <lst name="facet_fields">
    <lst name="category">
      <int name="book">10</int>
      <int name="video">2</int>
      <int name="audio">2</int>
    </lst>
  </lst>
  <lst name="facet_dates"/>
</lst>

Return facets for a specific field, category, with specific value for category, book, with counts for each value of category based on the search results.

http://localhost:8983/solr/#/development/select?q=book&fl=id,name,price&facet=on&facet.field=category&fq=category:electronics

...