When running the Harvester using cron, add the following line to the beginning of the crontab file content to ensure the data are properly encoded:
LANG=en_US.UTF-8
| Panel |
|---|
0 */4 * * * /path/to/runIngest.sh |
stub
The data harvester is convenient to run as a scheduled process utilizing cron. Once you have established your workflow for ingesting data publications cron can be utilized to schedule the task on a regular basis. For example the default run script can be added to cron to run every 4 hours and ingest and score new data.
...
The above example will execute the bash script every four hours. Alternatively, you could separate out the ingest process.
...