Versions Compared

Old Version 1

changes.mady.by.user Unknown User (robertwilliams)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (robertwilliams)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

A guide to deploying SimplyE with using Docker containers.

Contents:

...

Table of Contents

...

indent

...

15px

Purpose

So you're deploying your library's circulation manager. Awesome! If you'd like to get up and running quickly, we recommend using our Docker image.
If you're already familiar with Docker and/or would like to contribute to our Docker builds, you can find our build files at NYPL-Simplified/circulation-docker.

...

  • Running the Circulation Manager
    • [Prep work

    • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#cm-prep][Creating Circulation Manager containers

      • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#cm-host][Running Scripts

      • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#cm-scripts][Deploying the App

      • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#cm-app][Environment Variables

      • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#cm-env][Evaluating Success

  • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#cm-success]Support Containers for Testing and Development

    • [Creating a Postgres container

    • https://github.com/NYPL-Simplified/Simplified/wiki/Deployment:-Quickstart-with-Docker#pg][Creating an Elasticsearch container

...

.

...

...

Prep Work

  1. Install Docker. Docker has step-by-step instructions to grab its most up-to-date version. Depending on your package manager, you could also install a slightly older version with: sudo apt-get install docker-ce or sudo yum install docker-ce.
  2. Create any dependent, temporary containers (optional) for integrations like Elasticsearch and Postgres. We don't recommend using containers in the long-term for holding or maintaining data. However, if you just want to get a sense of how your Circulation Manager will work, containers are a quick option. Instructions for integrating Elasticsearch and Postgres via Docker can be found below.
  3. Get the Docker images for the Library Simplified Circulation Manager. Run:
   $ sudo docker pull nypl/circ-

...

webapp && sudo docker pull nypl/circ-scripts

...

Running Circulation Manager containers
 Anchor
_a8kwy94iu1be
_a8kwy94iu1be
Running scripts
To deploy an app filled with your library's books, you'll need to run a number of scripts. Read the environment variable details below before running this script; you will likely need to alter it to meet your needs.
 Anchor
_55slrod1jusc
_55slrod1jusc
Example docker run script
$    $ sudo docker run -d --name circ-scripts \ -e TZ="US/Central" \ -e SIMPLIFIED_DB_TASK='init'  \ -e SIMPLIFIED_PRODUCTION_DATABASE='postgres://[username]:[password]@[host]:[port]/[database_name]' \ nypl/circ-scripts
 Anchor
_wq87rm9mbm2z
_wq87rm9mbm2z
What It Does
The example above runs this resulting container in detached mode (-d), linked to the (-e) SIMPLIFIED_PRODUCTION_DATABASE and calling it "circ-scripts". With the (-e) optional argument TZ, you can pass a Debian-system timezone representing your local time zone, which will cause timed scripts to run according to your local time. If the database you've connected in your configuration has never been used before, use -e to set the optional argument SIMPLIFIED_DB_TASK to 'init'. This will keep track of the state of the database you've created and create an alias on your Elasticsearch cluster, allowing database updates to be easily managed with scripts.
 Anchor
_qvgs9qpwxl8c
_qvgs9qpwxl8c
Running Scripts
Once you've given your scripts some time to run (~30 minutes should be enough time to start having works move through the import process), you'll want to refresh your cached materialized views so they show up in your deployed app.```sh$ 
   $ sudo docker exec circ-scripts /var/www/circulation/core/bin/run refresh_materialized_
...
views
 Anchor
_b9bwrvuw3ilv
_b9bwrvuw3ilv
Troubleshooting
You'll want to check the logs of your container. For example:
# check logs of the database task and running supervisor processes$ sudo docker logs circ-scripts # check logs of cron and scripts$ sudo docker exec circ-scripts cat /var/log/cron.log | less$ sudo docker exec circ-scripts ls /var/log/simplified$ sudo docker exec circ-scripts cat /var/log/simplified/overdrive_monitor_full | less # The log directory can also be found on the production server. # Its location can be found using this command.$ sudo docker inspect circ-scripts \ --format='range $mount := .Mountsif eq $mount.Destination "/var/log"$mount.Sourceendend'
You can hop into a running container at any time with the command: $ sudo docker exec -it circ /bin/bash
Docker has fantastic documentation to get more familiar with its command line tools, like docker exec and docker inspect. We recommend you check them out.
 Anchor
_hwt9asvxe7jl
_hwt9asvxe7jl
Deploying the App
Using an nypl/circ-deploy container deploys the OPDS feeds expected by the SimplyE client applications. Read the environment variable details below before running the following script; you will likely need to alter it to meet your needs.
 Anchor
_3xw8zwfy5mpu
_3xw8zwfy5mpu
Example docker run script
   $ sudo docker run --name circ-
...
webapp \ -d -p 80:80 \ -e SIMPLIFIED_PRODUCTION_DATABASE='postgres://[username]:[password]@[host]:[port]/[database_name]' \ -e SIMPLIFIED_DB_TASK="
...
auto" \ nypl/circ-
...
webapp
 Anchor
_5gvpvxf6ewi0
_5gvpvxf6ewi0
What It Does
The script above runs the container in detached mode (-d), binding its port 80 to your server's port 80 (-p), connecting it to your PostgreSQL database -e SIMPLIFIED_PRODUCTION_DATABASE) and calling it "circ-deploy". Unless you've been running a scripts container for while, when you visit your server through a browser, you'll see a very sparse OPDS feed. If the database you've connected in your configuration has never been used before, use -e to set the optional argument SIMPLIFIED_DB_TASK to 'init'. This will keep track of the state of the database you've created and create an alias on your Elasticsearch cluster, allowing database updates to be easily managed with scripts.
 Anchor
_bl63udg5pnoo
_bl63udg5pnoo
Troubleshooting
You'll want to check the logs of your container (/var/log/nginx/error.log and /var/log/libsimple/uwsgi.log) to troubleshoot:
# check logs of the database task and running supervisor processes$ sudo docker logs circ-deploy # check logs inside the container$ sudo docker exec circ-deploy cat /var/log/nginx/error.log | less$ sudo docker exec circ-deploy cat /var/log/libsimple/uwsgi.log | less # restart the application$ sudo docker exec circ-deploy touch uwsgi.ini
You can hop into a running container at any time with the command: $ sudo docker exec -it circ /bin/bash
Docker has fantastic documentation to get more familiar with its command line tools, like docker exec and docker inspect. We recommend you check them out.
 Anchor
_2tm8gnmdm5y1
_2tm8gnmdm5y1
Environment Variables
 Anchor
_6kgxy5pl9riu
_6kgxy5pl9riu
SIMPLIFIED_CONFIGURATION_FILE
Optional The full path to configuration file in the container. Using the volume options -v, e.g. YOUR_LOCAL_DIRECTORY_WITH_CONFIG_FILE:/etc/simplified, it should look something like -e SIMPLIFIED_CONFIGURATION_FILE=/etc/simplified/YOUR_CONFIGURATION_FILENAME.json.
 Use Configuration is now held in the database and accessed via an administrative interface at /admin, so you probably don't need this.
If you do, use this documentation to create the JSON file for your particular library's configuration. If you're unfamiliar with JSON, you can use this JSON Formatter & Validator to validate your configuration file.
 Anchor
_eaq7n8qpz3z3
_eaq7n8qpz3z3
SIMPLIFIED_DB_TASK
RequiredOptional. Performs a task against the database at container runtime. Options are:
  • ignore : Does nothingauto : Either initializes or migrates the database, depending on if it is new or not. This is the default value.
  • ignore : Does nothing.
  • init : Initializes the app against a brand new database. If you are running a circulation manager for the first time every, use this value to set up an Elasticsearch alias and account for the database schema for future migrations.
  • migrate : Migrates an existing database against a new release. Use this value when switching from one stable version to another.
 Anchor
_4gwt1zdywmyr
_4gwt1zdywmyr
SIMPLIFIED_PRODUCTION_DATABASE
Required. The URL of the production PostgreSQL database for the application.
 Anchor
_5pklaqswpau8
_5pklaqswpau8
SIMPLIFIED_TEST_DATABASE
Optional in v2.x only. The URL of a PostgreSQL database for tests. This optional variable allows unit tests to be run in the container.
 Anchor
_u9l543x735gq
_u9l543x735gq
TZ
Optional. Scripts container only. The Applies to circ-scripts only. The time zone that cron should use to run scheduled scripts–usually the timezone of the library or libraries on this the circulation manager , selected according instance. This value should be selected according to Debian-system timezone options. This value allows scripts to run at ideal times.
 Anchor
_szz2qk9xinew
_szz2qk9xinew
Evaluating Success
If your Docker containers are running successfully, you should have a /var/log/simplifieddirectory full of logfiles in your circ-scripts container, and you should be able to visit your server's domain and see an OPDS feed from circ-deploy. If either of these things aren't occurring, use the troubleshooting details above to check var/log/cron.log or the logfiles in /var/log/simplified for circ-scripts and/or /var/log/libsimple/uwsgi.log or /var/log/nginx/error.log.
 Anchor
_1d9npow2mt2s
_1d9npow2mt2s
Support Containers (for use in development or testing)
 Anchor
_1ovt2xiwqda3
_1ovt2xiwqda3
Elasticsearch
While we do not recommend you run Elasticsearch from a Docker container permanently, you may want to get up and running with a throwaway search index. Elasticsearch isn't installed via the Dockerfile, so the fastest way to connect to it will be through another container. Here's how:
  1. Get the Docker image for Elasticsearch v1.x:
  2. $ sudo docker pull elasticsearch:1
  3. Create an Elasticsearch container, and grab its IP Address. Run, running the following commands:
$ sudo docker run -d --name es elasticsearch:
...
1   # create an elasticsearch container
$ sudo docker 
...
ps  # confirm that it's running
...
; note its IP address
$ sudo docker inspect es --format="range .NetworkSettings.Networks.
...
IPAddressend"
 
  1. Add Record the Elasticsearch URL to your configuration filefor use when configuring the Search integration in the Circulation Manager Admin interface. When you run sudo docker ps, you'll see a single running container called es. Use the IP that comes from running inspect to update your your config.json file with the proper Elasticsearch location. You should end up with something like "http://172.17.0.2:9200".
 Anchor
_9zl93qfes5pb
_9zl93qfes5pb
Postgres
While we do not recommend you run Postgres from a Docker container permanently, you may want to get up and running with a throwaway database. Postgres isn't installed via the Dockerfile, so the best way to connect to Postgres will be through another container. Here's how:
  1. Get the Docker image for Postgres 9.4 or 9.5:
  2. $ sudo docker pull postgres:9.5
  3. Create a Postgres container, and grab its IP Address. Run, running the following commands:
$ sudo docker run -d --name pg postgres:9.
...
5   # create a postgres container
$ sudo docker 
...
ps  # confirm that it's running
...
; note its IP address
$ sudo docker inspect pg --format="range .NetworkSettings.Networks.
...
IPAddressend"
  1. Create a Postgres database. Run, running the following commands:
$ docker exec -u postgres pg psql -c "create user simplified with password 'test';"  # create a user and password
$ docker exec -u postgres pg psql -c "create database simplified_circ_db;"  # create database
$ docker exec -u postgres pg psql -c "grant all privileges on database simplified_circ_db to simplified;"
$ docker exec -u postgres pg psql -d simplified_circ_db -c "create extension pgcrypto;"
  1. Add the Postgres URL to your configuration file. In config.json, add the appropriateproduction_url. You should end up with something like database environment variable as shown above. It will be of the form "postgres://simplified:test@172.17.0.3:5432/simplified_circ_db", following the "postgres://<USERNAME>:<PASSWORD>@<HOST>:<PORT>/<DATABASE_NAME>" format.
...