Contribute to the DSpace Development Fund
The newly established DSpace Development Fund supports the development of new features prioritized by DSpace Governance. For a list of planned features see the fund wiki page.
This documentation is meant to show how to set up your computer to test DSpace Github Pull Requests (PRs). PRs are codes including proposal for new features or bug corrections.
Work in progress
This page is still being continuously updated and improved. It is a work in progress.
Getting ready
Step 1: Install required software
To start working on PR review and testing, the following applications are needed:
- GitHub login. This lets you add a comment to any PR you test, letting developers know if you found problems or if all your tests were successful.
- GIT https://git-scm.com/downloads
- Install GitHub CLI. This makes it easier to "checkout" (i.e. download) the PR code so you can test it.
- Install Docker Desktop. This is the easiest way to get a DSpace backend installed locally for testing new (frontend or backend) PRs.
- Windows 11: ensure Hyper-V are turned on in the Windows Features (TO ADD). May require administrator rights on the computer.
- Once installed, you will be required to restart your computer. When first launched, an error may appear mentioning that WSL for Windows is needed. If so, install WSL. When Installing WSL you will be asked for a Linux Distro.
- NOTE: Docker recently changed their service agreement, but Docker remains free for personal use or non-commercial open source projects like DSpace.
NOTE for Windows 10: Docker Desktop can sometimes become a bit "memory-hungry" when it is running. It's best to either disable the "Start Docker Desktop when you log in" setting (so that it only runs when needed). Alternatively, if you are using the default WSL2 backend, you can create a ".wslconfig" file in your user directory and limit the amount of memory you let Docker use (at least 4-8GB is recommended if you can spare it):
.wslconfig[wsl2] memory=8GB
Congratulations! You have installed the required software!
Step 2: Set up DSpace Docker
This step must be followed after all software listed in step one are installed. For additional details and more up-to-date information on DSpace Docker files and configurations, refer to the documentation on DSpace Github (Docker Compose files for DSpace Backend and Angular Docker Compose files)
GitHub CLI authentication
This operation is done once. It is must be done before beginning to install either DSpace Backend or Frontend.
Actions | Command line | Purpose | Software |
---|---|---|---|
GitHub CLI (gh) authentication | gh auth login | Git authentication using github credentials Needed to use Git command. If this is your first time using GitHub CLI (gh), you likely will need to authenticate with it using your GitHub.com login. | Git |
Also possible to use Git Bash:
- Install Git Bash & verify the installation by running "
git version
" from Command Prompt or Powershell - No need to authenticate with Github credentials.
Installing Backend
3-step process (details in table below):
- Cloning the GitHub repository
- Getting the DSpace Backend images from the Docker Registry
- Starting the Docker Backend Containers
Actions | Command line | Purpose | Software |
---|---|---|---|
Clone the backend codebase | # Run from command-line where ever you want the "DSpace" directory of code to be created. gh repo clone DSpace/DSpace # Once downloaded, change into that directory cd DSpace | Before cloning carefully select your parent folder. Getting all the files needed to run DSpace Backend. This downloads all the code from https://github.com/DSpace/DSpace into a local directory named "DSpace". | Git |
Start Docker Desktop | If it is not already running, start Docker Desktop on your machine. It MUST be running for any "docker compose" or "docker" commands to work.
| Docker | |
Pull down the latest backend images | # This command should be run from the "DSpace" code directory docker compose -f docker-compose.yml -f docker-compose-cli.yml pull | Pull down the latest backend images from DockerHub. This downloads all the prebuilt Docker images so you don't need to rebuild them locally. | Docker command line |
Start the Docker DSpace Backend Containers (& automatically install our Entities Test Data) | # First start the backend with the test data docker compose -p d7 -f docker-compose.yml -f dspace/src/main/docker-compose/db.entities.yml up -d # [OPTIONAL] Run "logs -f" to watch the logs for everything to start up. docker compose -p d7 -f docker-compose.yml -f dspace/src/main/docker-compose/db.entities.yml logs -f # (Click Ctrl+C to exit logs view) | This command creates the 3 Docker containers for the Backend (dspacedb for the database ; dspacesolr for the solr indexes ; dspace for the backend server ; angular for the frontend UI) and then starts them up. Warning: dspacedb is the only Docker container that is not started automatically (the two others, dspacesolr and dspace, are) The backend will be started with the test data and may take a few minutes. It should be completed before moving to the next action. # Wait a few minutes until http://localhost:8080/server/ responds. | Docker command line |
Download, install and reindex the test data assetstore (of files) | # Finally, download the test data assetstore (of files) and install it, reindexing all content docker compose -p d7 -f docker-compose-cli.yml -f dspace/src/main/docker-compose/cli.assetstore.yml run dspace-cli | Docker command line | |
Check system is up and running | DONE! At this point, you should be able to go to http://localhost:8080/server/ and see a backend similar to https://demo.dspace.org/server/
| Web browser (http://localhost:8080/server/ ) The Hal Browser page is displayed |
Installing Frontend
3 step process (details in table below):
- Cloning the GitHub repository
- Getting the DSpace FrontEnd images from the Docker Registry
- Starting the Docker DSpace Frontend Containers
Actions | Command line | Purpose | Software |
---|---|---|---|
Clone the frontend codebase | # Run from command-line where ever you want the "dspace-angular" directory of code to be created. gh repo clone DSpace/dspace-angular # Once downloaded, change into that directory cd dspace-angular | Getting all files needed to run DSpace frontend (Angular) First, clone the frontend codebase using GitHub CLI (gh). This downloads all the code from https://github.com/DSpace/dspace-angular to a local directory named "dspace-angular". You may wish to run this command from the same parent directory where you cloned the backend (just to keep this new "dspace-angular" folder next to the "DSpace" folder where the backend code resides). | Git |
Ensure Docker Desktop is running | If it is not already running, start Docker Desktop on your machine. It MUST be running for any "docker compose" or "docker" commands to work. On Windows, if Docker Desktop is running, you'll see the Docker whale logo in your taskbar's notification area.Using Docker, pull down the latest frontend images from DockerHub. This downloads all the prebuilt Docker images so you don't need to rebuild them locally. | ||
Pull down the latest frontend images | # This command should be run from the "dspace-angular" code directory docker compose -f docker/docker-compose.yml pull | Using Docker, pull down the latest frontend images from DockerHub. This downloads all the prebuilt Docker images so you don't need to rebuild them locally. | Docker command line |
Start Docker Container for DSpace frontend/UI | docker compose -p d7 -f docker/docker-compose.yml up -d # Optionally run "logs -f" to watch the logs for everything to start up. docker compose -p d7 -f docker/docker-compose.yml logs -f # (Click Ctrl+C to exit logs view) | This command will create the Docker Container for the Frontend and then start it up. | Docker command line |
Check UI is up and running |
| Web Browser (http://localhost:4000/) DSpace front page is displayed. |
Step 3: Ensure everything is up and running
Checking Docker Desktop Containers
Your Docker Desktop Containers page should look like the image below, with the d7 project containing at least 4 containers: dspacedb for the database ; dspacesolr for the solr indexes ; dspace for the backend server ; angular for the frontend UI.
TODO: ALSO ADD PRINT SCREEN OF UI AND BACKEND
Checking DSpace URL
Once you start the containers either through command line or Docker UI, the following URLs should be active.
- Backend: http://localhost:8080/server/
- Angular UI: http://localhost:4000/home
- OAI: http://localhost:8080/server/oai/request?verb=Identify
Starting Testing PRs
How to find PR for testing
If you are less technical, or just getting started, we recommend beginning your testing (or code review) with Pull Requests that contain the "1 Approval" label. These are smaller Pull Requests that need a very quick test or review (usually a small bug fix or improvement).
You are welcome to test any open pull request. Even if the pull request is already assigned to other testers, it can help speed up its completion/acceptance if others also test it.
We recommend testing whichever pull requests are of interest to you. You may find one relating to a bug that you'd also seen, or something that sounds easy for you to test, etc. The choice is up to you.
How to select a PR to test
Things important to consider before selecting a issue to test:
- No merge conflict detected (merge conflict tag)
- No changes have been requested by a reviewer (changes requested tags)
- No "work in progress" tag
- All automated test have passed without error
- Looking for an easy one? Look for the "1 Approval" tag (see links above)
Understanding what to test
For testing:
- Look if a Github Issue is linked to the PR, it usually gives more information on the feature to test.
- If test instructions are unclear, put a comment asking for instructions
- Tester do not need to be assign a PR to test it.
If the PR needs a specific configuration, it can be change: recommendation is to use a local.cfg containing the config value for the test. i.e, if the feature change in the PR use emails, it is necessary to configure SMTP using you SMTP reference.
Testing the DSpace Pull Request (PR)
Testing a Frontend PR | Testing a Backend PR | |
---|---|---|
Move to relevant directory | dspace-angular | DSpace |
Checkout the PR code | Find the number of the Pull Request
| Find the number of the Pull Request
|
Rebuild based on that PR's code | docker compose -p d7 -f docker/docker-compose.yml build Note: Build duration (Angular): 3-4 minutes with a good bandwith/connexion This will create a new Docker Image including the PR code. | docker compose -f docker-compose.yml -f docker-compose-cli.yml build Note: Build duration (backend): environ 5-10 minutes with a good bandwith/connexion This will create a new Docker Image including the PR code. |
Start services needed | Frontend can be start using command line or Docker Software UI. Commandline: Ensure you are positioned in dSpace-angular folder (cd command to dspace-angular) Run following command: This command will start the container using the new Docker image created. | Backend can be start using command line or Docker Software UI. Commandline: Ensure you are positioned in DSpace folder (cd to DSpace if not) Run following command: This command will start the container using the new Docker image created. |
Once you are ready to check the PRs: open Dev tool of you preferred browser
If an error occurs, check the log, either backend or frontend log can be accessed using Docker by clicking on the containers Name (i.e: dspace-angular or dspacedb, etc.)
Reporting test results
The PR's description should describe how to test it and what you should see.
- If you notice any odd behavior, check for errors in the UI using your browser's DevTools. See Troubleshoot an error#FindingtheErrorMessageintheUserInterface
- Based on what you find, add a comment to the PR. If it works, congratulate the developer. If it doesn't, let them know the error you saw (and what you clicked on when that error occurred).
4 Comments
Carolyn Sullivan
Re: Installing Backend, Step 2: Pull down the latest backend images
A note: if you're getting the error 'The Compose file './docker-compose.yml' is invalid because: services.dspacesolr.build contains unsupported option: 'additional_contexts'' and you already have docker compose V2 installed, run the command as docker compose, not docker-compose (otherwise it'll try to use docker compose V1 if you haven't uninstalled it)
Carolyn Sullivan
What are the hardware requirements for running this? I don't have any difficulty launching the backend, but when I try to run the front-end, the Docker interface shows that it's using 326% of CPU, which seems both unlikely and terrifying.
Pascal-Nicolas Becker
If you know git a little bit, you can easily checkout pull requests with git, without the need to install the github client. Every Git repository does have a
.git
directory in its root directory, which contains a fileconfig
. In this config file there is one section for each remote you added to that git repository. For example if you added a remote "DSpace" you should have a section like this:To be able to checkout pull requests, you have to add a line between url and fetch (the position is important if I remember correctly). After editing the section would look like this:
Then you need to run
git fetch
once, to fetch all pull requests. No you should see all pull requests as remote branches of the "DSpace" repository being labeled likeDSpace/pr/1
.git switch DSpace/pr/1
orgit checkout DSpace/pr/1
lets you then work with the prs as if they were normal branches. Of course you can do the same for dspace-angular.Pascal-Nicolas Becker
Looks like there are even easier ways of doing this without editing you git configuration:
(code snippet via https://stackoverflow.com/a/30584951).