Contribute to the DSpace Development Fund
The newly established DSpace Development Fund supports the development of new features prioritized by DSpace Governance. For a list of planned features see the fund wiki page.
This page contains information about how demo.dspace.org server is setup/configured. This demo.dspace.org server is managed jointly by the DSpace Committer Team. Any Committer can request server access.
If major issues occur or something needs to be installed requiring root access, contact Tim Donohue or "sysadmin AT duraspace DOT org"
Updating / Upgrading DSpace installation
To ensure we are consistently updating DSpace in the same manner, please perform the following steps when updating any configuration
or making any customization to DSpace.
(If you have updates/suggestions, please let us know – we can change these processes, but we just need to make sure we are all consistently following the same general steps)
Make all Configuration/File Changes in '~/dspace-src/' FIRST
The ~/dspace-src/
folder is a Git clone of the DSpace GitHub Repository: https://github.com/DSpace/DSpace/
In this local Git repository, we are running off of a local branch named "demo". You can see all the branches by running
git branch
Changes that you wish to keep should be committed to this local "demo" branch.
At any one time, you can compare this 'demo' branch to any version of DSpace. For example, to compare 'demo' to DSpace 1.8.2 run:
cd ~/dspace-src git checkout demo git diff dspace-1.8.2
WARNING: If you make direct config edits to ~/dspace/config/ you can expect that they may be overwritten in future (unless you also copy them to ~/dspace-src/dspace/config/)
You have been warned! Again, if your changes don't make it to ~/dspace-src/ then THEY WILL BE LOST during the next update!
Upgrade DSpace Source
If you are upgrading to the next stable version of DSpace, you can use git merge
to help you merge all changes.
WARNING THE FOLLOWING IS UNTESTED! PLEASE UPDATE WHEN WE UPGRADE TO 3.0-RC1!!
For Example:
cd ~/dspace-src git checkout demo git merge dspace-3.0-rc1 (or 'git merge master')
NOTE: You should make sure to pay close attention to whether any Conflicts occurred. If so, you will need to resolve them manually.
Resolving Conflicts: Here are some hints on how to resolve / manage conflicts encountered during a merge:
If there were a lot of conflicts and you just want to accept the "master" or tagged version (and overwrite any local changes), you can use:
git checkout --theirs [full-path-to-file] git add [full-path-to-file]
If you need to completely delete a file that caused conflict, just use:
git rm [full-path-to-file]
If you need to abort an in-process merge that had conflicts, just run:
git merge --abort
Rebuild DSpace
cd ~/dspace-src mvn clean package
Push out Updates
WARNING: this overwrites existing configs in ~/dspace/config/
sudo service tomcat stop cd ~/dspace-src/dspace/target/dspace-[version]-build/ ant update sudo service tomcat start
Double check everything still looks to be working.
Also make sure your changes made it to ~/dspace/
(and that you didn't remove previous settings, especially configs)
An easy way to double check config changes is to do a 'diff' of the latest dspace.cfg with the most recent '.old' one.
Recommended to commit your changes to "demo" local branch
Assuming your changes are already over in ~/dspace-src/
this is easy...
cd ~/dspace-src/ git commit [file] # OR, to commit all changed files git commit -a
General Server Setup
Here's an overview of how everything is setup on the 'demo.dspace.org' server:
Base Software
- The server is currently running on DuraSpace's Amazon EC2 account with the following configurations:
- Amazon AMI (EBS Image)
- Ubuntu Linux 12.04 LTS server (64-bit)
- Separate EBS volume mounted at /home/dspace
- All DSpace requirements/software is installed in the 'dspace' user's home directory (
/home/dspace
):- Java:
/usr/lib/jvm/java-1.7.0-openjdk-amd64
(NOTE: We also have Oracle Java 7 installed at/usr/lib/jvm/java-7-oracle
but it is not used by default) - Ant:
~/ant
(installed manually) - Maven:
~/maven
(installed manually) - PostgreSQL:
~/pgsql
(compiled from source) - Tomcat:
~/tomcat
(installed manually) - DSpace Source:
~/dspace-src
- DSpace Install:
~/dspace
- Java:
Website / Splash page
- Tomcat is configured to run on port 80
- It is using 'jsvc' to start as 'root' and then switch to being owned by 'dspace'
- Tomcat's startup settings are all located in
~dspace/.profile
- DSpace Webapps are run from
~/dspace/webapps/
(configured in Tomcat's server.xml) - The main "splash" page (http://demo.dspace.org) is generated by Tomcat and is located at:
~/tomcat/webapps/ROOT/index.html
- Its content is also managed via the GitHub Repository at: http://github.com/DSpace/demo.dspace.org
- Info on updating & pushing to GitHub can be found in the README at https://github.com/DSpace/demo.dspace.org
'dspace' user account
- The 'dspace' user's
~/.profile
initializes PATH/environment - The 'dspace' user has limited 'sudo' access on system.
- The 'dspace' user's
~/bin/
includes various useful scripts
Server Backups
- A server snapshot (backup) is generated nightly
- Snapshots from the last 4 nights are kept
- Every Sunday a weekly snapshot is generated and kept for two weeks
- On the first of the month a monthly snapshot is generated and kept for two months
Who to Contact
Only a DuraSpace employee can do the following:
- Install extra Ubuntu software packages on server
- Reboot server
- Upgrade Ubuntu to next version of Ubuntu
- Create Snapshots of server volumes and restore server based on one of those Snapshots
Contact sysadmin@duraspace.org or Tim Donohue if you need any of these tasks performed.
Getting SSH access to demo.dspace.org
This is how you provide a DSpace Committer with command-line access to this server.
- Have Committer generate an SSH Key on their computer and send you their PUBLIC Key.
- Append their PUBLIC Key on the end of the 'dspace' user's
~/.ssh/authorized_keys
fileNOTE: Please add a comment regarding who's key this is, so that it makes it easier to clean up later on. For example:
# Tim Donohue's SSH Key ssh-rsa ....
They should now be able to connect as follows:
ssh dspace@demo.dspace.org
Starting / Stopping DSpace (and related services)
The 'dspace' user can easily start/stop PostgreSQL and Tomcat using the corresponding service scripts:
sudo service postgresql start sudo service tomcat start sudo service handle-server start sudo service tomcat stop sudo service postgresql stop sudo service handle-server stop
Important Cron Jobs
Obviously, you can get the latest information on the existing Cron jobs by logging into the demo.dspace.org server and running:
crontab -l
However, here's a brief overview of a few of the more important Cron jobs.
Reset DSpace Content (based on AIPs) every Saturday
EVERY SATURDAY NIGHT (currently at 23:59 UTC), all existing DSpace content is automatically REMOVED and reset to the AIPs located at ~/AIP-restore/
This is controlled by the ~/bin/reset-dspace-content
script (source code in GitHub)
This is a BASH script that essentially does the following:
- Stops Tomcat
- Backs up current DB & Assetstore to
~/tmp/data-backup
(This backup is performed just in case something goes wrong and we
need to quickly restore DSpace.) - Deletes existing DB & Assetstore
- Resets DSpace back to a 'fresh_install' state (by restoring database tables, sequences, registries, and initial Admin user acct)
- Imports the AIPs in
~/AIP-restore
into DSpace as default content (This also autocreates the demo EPeople and Groups)- SEE README in ~/AIP-restore/ for info on updating these AIPs
- Refreshes all Indexes (Lucene/DB & Discovery) & Restarts Tomcat
- A log of the entire 'reset' process is written to
~/AIP-restore/reset-dspace-content.log
How to update the set of demo AIPs
The set of demo AIPs are all stored in the ~/AIP-restore/
directory.
To update these AIPs, you must use the DSpace AIP Backup & Restore tools as described at:
AIP Backup and Restore
You can regenerate / update these AIPs by doing the following:
- Install a fresh (empty) copy of DSpace on your local server.
- Configure it to have the same handle prefix as demo.dspace.org (handle prefix: 10673) & setup an initial administrative user (ideally 'dspacedemo+admin@gmail.com' which is the Demo Administrator on demo.dspace.org).
Download the existing AIPs from this directory, e.g.
scp dspace@demo.dspace.org:~/AIP-restore/* .
Use the downloaded AIPs to "restore" content to your local server's empty DSpace, e.g.
\[dspace\]/bin/dspace packager -r -a -f -t AIP -e [admin-email] -i 10673/0 /full/path/to/SITE@10673-0.zip
- Update your DSpace's content as you see fit (adding/removing/changing objects)
Export a fresh set of AIPs, by performing a full SITE export e.g.
\[dspace\]/bin/dspace packager -d -a -t AIP -e [admin-email] -i 10673/0 -o includeBundles=ORIGINAL,LICENSE -o passwords=true SITE@10673-0.zip
- The above example just exports ORIGINAL & LICENSE bundles into AIPs, and also exports user passwords into AIPs (so that they can also be restored).
Upload those newly updated AIPs to demo.dspace.org, e.g.
scp . dspace@demo.dspace.org:~/AIP-restore/
- NOTE: Before putting them on demo.dspace.org, you may want to do your own test restore using these AIPs, just to ensure there are no issues.
Reset "News" sections every night
Since the "News" sections are editable via the JSPUI, there is a cron job that automatically resets them each night.
It's a rather simple cron job that just copies the original "news-*" files from the ~/dspace-src/
directory:
05 0 * * * cp $HOME/dspace-src/dspace/config/news-* $HOME/dspace/config/ > /dev/null
Reset Demo User Passwords every hour
Since people have been known to change our demo user passwords on this demo.dspace.org server, we now reset them to the default password every hour.
This functionality is just a simple set of SQL UPDATE commands that are run via the ~/bin/reset-demo-passwords
script.
kompewter IRC bot
The kompewter IRC bot is on the server at ~/kompewter
.
It's source code is managed in Peter Dietz's GitHub acct at: https://github.com/peterdietz/jenni
Starting / Stopping kompewter
To start kompewter just run:
~/kompewter/jenni
(This only keeps it alive for the remainder of your SSH session. To keep it running after logoff, use nohup or run it in a screen seesion and detach.)
Java Profiling using YourKit
Remote Profiling Using YourKit
Full instructions available at: http://www.yourkit.com/docs/95/help/profiling_j2ee_remote.jsp
In order to locate potential memory issues in DSpace, we've installed YourKit on demo.dspace.org at ~/yjp/
.
It can be accessed remotely so that we can perform various Java profiling tasks.
On your local computer:
- Download & Install YourKit Profiler. Put in your open source license key (available to all DSpace Committers).
- Open up YourKit, select "Connect to remote application..." option.
- Point it at "demo.dspace.org:10001" and start doing some profiling!
- If it's not running, start it using ~/yjp/bin/yjp.sh
- If needed, logs are in ~/.yjp/log/