TODO: Update configuration documentation (+local file scanning)
The Ingest Server coordinates the distribution of packages throughout the Chronopolis network, and can perform additional services if needed (examples below). This is done through a HTTP API which both Intake and Replication services can interact with and receive updates about data they are processing.
Links
- Gitlab: https://gitlab.umiacs.umd.edu/chronopolis/chronopolis-core
- Builds: http://adaptci01.umiacs.umd.edu/resource/chronopolis
- Chronopolis-Core
- API/Developer DocumentationAPI Documentation: https://chronopolis-docs.umiacs.io/
Installation
Prerequisites
...
Administration
Database Setup
Download the schema from the CI server
...
Database initialization can be done through the flyway-maven-plugin provided that you have a PostgresQL database which you can connect to and create/drop tables for. The flyway plugin will create load the first version of the schema and baseline the database at 1.0 so that all migrations can be applied. This requires the chronopolis-core repository to be cloned so that the flyway plugin can be run.
To run the flyway plugin, cd
into the ingest-rest
directory and use flyway:baseline
and provide configuration for the flyway.user
, flyway.password
, and flyway.url
configuration parameters. Once the database is baselined, the ingest server can be started and will apply all migrations.
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
psql (8.4.20) Type "help" for help. postgres=# CREATE USER chron WITH PASSWORD 'secret-password'; CREATE ROLE postgres=# CREATE DATABASE ingest; CREATE DATABASE postgres=# GRANT ALL PRIVILEGES ON DATABASE ingest to chron; GRANT postgres=# \c ingest; psql (8.4.20) You are now connected to database "ingest". ingest=# SET ROLE chron; SET ingest=> \i /tmp/schema_pg.sql[chronopolis-core] $ cd ingest-rest [chronopolis-core/ingest-rest] $ ../mvnw -Dflyway.user=postgres -Dflyway.password=mysecretpassword -Dflyway.url=jdbc:postgresql://172.17.0.2/ingest flyway:baseline |
Preparing the DB for Schema Migrations
...
- Download and untar/unzip the Flyway Command Line Tool
- The Ingest Server currently uses Flyway 45.2.04; if possible the binary for that version should be used
- Edit the conf/flyway.conf
- some properties follow the same pattern as our application properties (connecting to the database)
specify the version which you are creating the baseline (using the MAJOR.MINOR number of the ingest server version)
Code Block language bash title Flyway Configuration Example # # Copyright 2010-2015 Axel Fontaine # ... # Jdbc url to use to connect to the database flyway.url=jdbc:postgresql://localhost/ingest # Fully qualified classname of the jdbc driver (autodetected by default based on flyway.url) # flyway.driver= # User to use to connect to the database (default: <<null>>) flyway.user=chron # Password to use to connect to the database (default: <<null>>) flyway.password=my-postgresql-password ... flyway.baselineVersion=3.0
Use the flyway bash script to update the database
Code Block language bash title Flyway Baseline Migration $ ./flyway baseline
...