The migration-utils offers optional server metrics using Prometheus for those who have long running migrations.
To enable metrics collection, the --enable-metrics
command line option must be used when running the migration-utils. With this option on, the migration-utils exposes an endpoint that can be used by Prometheus to collect metrics at http://localhost:8080/prometheus
.
java -jar target/migration-utils-6.0.0-SNAPSHOT-driver.jar --enable-metrics --source-type=akubra --target-dir=target/test/ocfl --objects-dir=src/test/resources/legacyFS/objects --datastreams-dir=src/test/resources/legacyFS/datastreams
Example Metrics Collection Setup
The following is an example of how you could setup Prometheus and Grafana using Docker to collect and monitor metrics. It is a quick and easy setup, and not intended to demonstrate a production ready install. This setup assumes that you already have Docker and Docker Compose installed on your system.
Linux Setup
Create a Prometheus config file, named prometheus.yml
, that looks similar to the following:
scrape_configs: - job_name: 'fcrepo' scrape_interval: 30s metrics_path: '/prometheus' static_configs: - targets: ['localhost:8080']
Next, create a Docker compose file in the same directory, named docker-compose.yml
,that looks like the following:
version: '3.8' services: prometheus: image: prom/prometheus ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml network_mode: "host" grafana: image: grafana/grafana ports: - "3000:3000" network_mode: "host"
Mac Setup
Create a Prometheus config file, named prometheus.yml
, that looks similar to the following:
scrape_configs: - job_name: 'migration-utils' scrape_interval: 30s metrics_path: '/prometheus' static_configs: - targets: ['host.docker.internal:8080']
Next, create a Docker compose file in the same directory, named docker-compose.yml
,that looks like the following:
version: '3.8' services: prometheus: image: prom/prometheus ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml grafana: image: grafana/grafana ports: - "3000:3000"
Collecting Metrics
From within this directory, execute docker-compose up
to startup the services. You can confirm the services started by browsing to http://localhost:9090, the Prometheus endpoint, and http://localhost:3000, the Grafana endpoint.
Note, this will run the process in the foreground. If you prefer to run it in the background, use the -d
flag.
The final bit of setup is to connect Grafana to Prometheus as follows:
- Log into Grafana with the default credentials:
admin
/admin
- Navigate to Configuration -> Datasources
- Add a Prometheus datasource
- For the Prometheus URL, if you're on linux specify http://localhost:9090, and if you're on Mac use http://host.docker.internal:9090. For amazon linux 2 use http://prometheus:9090
- Save the configuration
Now that you have the datasource added, you can create a dashboard to display metrics. Here is an example dashboard with some pre-configured graphs: