Page History

Added in DSpace 3.0 is an optional statistics engine using Elastic Search Elasticsearch, which may be enabled as an alternative to the default SOLR Statistics engine (based on Apache SOLR). The motivation for adding Elastic Search Elasticsearch was to find an alternative statistics processing engine that would handle the workload of a large amount of statistics data. Additionally, the Elastic Search Elasticsearch statistics display offers another method for creating statistical queries against your data. Elastic Search Elasticsearch Usage Statistics has been contributed by Peter Dietz of Ohio State University's Knowledge Bank. The data source for Elastic Search Elasticsearch Statistics is DSpace Usage Events, where Usage Event is a view or download of a DSpace Object (Bitstream, Item Page, Collection Page, Community Page). Elastic Search Elasticsearch Statistics is bundled with DSpace, and requires no additional installation of software, it just needs to be enabled. Elastic Search Elasticsearch is only available for use with XMLUI. 

What data is being recorded?

The default information below is what DSpace will record about a Usage Event. In DSpace 3.0 the fields of data collected is not configurable through a configuration setting.

Information about the User Requesting the Content

• IP Address

• Time of Request
• DNS / Hostname
• User Agent
• isBot, a flag that DSpace thinks that user is a robot or not
• Geographical Information about where the user is located: 
  • Continent
  • Country
  • Country Code
  • City
  • Geographical Latitude/Longitude

Information about the DSpace Resource that was used

• DSpace Object ID
• DSpace Object Type: (Item, Bitstream, Collection, or Community)
• If it is relevant, we also store the hierarchy of where this object exists within DSpace
  
  • Owning Community
  • Owning Collection
  • Owning Item

Enabling

...

Elasticsearch Statistics

Elastic Search Elasticsearch Statistics is disabled by default in DSpace 3.0, the following steps will enable Elastic Search Elasticsearch so that you can collect data, and present statistics reports.

...

Modify dspace/config/xmlui.xconf, and uncomment the aspect, Statistics Elastic SearchElasticsearch. 

        <!--
             If you prefer to use "Elastic SearchElasticsearch" Statistics, you can uncomment the below
             aspect and COMMENT OUT the default "Statistics" aspect above.
             You must also enable the ElasticSearchLoggerEventListener.
        -->
        <!-- <aspect name="Statistics - Elastic SearchElasticsearch" path="resource://aspects/StatisticsElasticSearch/" /> -->

...

<!-- Elastic SearchElasticsearch -->
<!--<bean class="org.dspace.statistics.ElasticSearchLoggerEventListener">
        <property name="eventService">
            <ref bean="dspace.eventService" />
        </property>
    </bean>-->

...

After making these two changes, you will then need to rebuild and restart DSpace.

Importing Legacy Data into

...

Elasticsearch Statistics

Once Elastic Search Elasticsearch Statistics has been enabled, it will begin adding all new Usage Events to its data store. To import your legacy data, you will need to import the data from the dspace.log files. There is no tool yet that converts SOLR statistics data to Elastic Search Elasticsearch statistics data.

From the (Windows / Linux) terminal, you will need to use the DSpace Command Launcher to convert the dspace.log files to a statistics log format. Then you will need to import the statistics log format files into DSpace Statistics.

The Log Converter program converts log files from dspace.log into an intermediate format that can be inserted into Elastic Search Elasticsearch Statistics.

-i or --in

-o or --out

-m or --multiple

-n or --newformat

-v or --verbose

-h or --help

...

The Log Importer program takes the intermediate format data produced in the previous step, and imports it into Elastic Search Elasticsearch Statistics.

-i or --in

-m or --multiple

-s or --skipdns

-v or --verbose

-h or --help

An example form of this command would be [dspace]/bin/dspace stats-log-importer-elasticsearch -i statistics.log -m

Viewing Data in

...

Elasticsearch Statistics

In XMLUI, while logged in as an administrator, the Context Panel will have an additional "View Statistics" link when you browse to a Community, Collection, or Item.

...