Testing Tickets
External Projects
Hydra (instructions)
...
Project
...
Tested by
...
RC-1 Testing
...
Notes
Islandora
...
Project
...
Tested by
...
Success? RC-1
...
Notes
...
API-X
...
Project
...
Tested by
...
Success? RC-1
...
Notes
...
Testing Plan
Code Block |
---|
git clone https://github.com/fcrepo4/fcrepo4
cd fcrepo4
git checkout 5.0.0-RC |
Sanity Builds
Scripts
https://github.com/awoods/fcrepo-build-scripts
...
linux
...
mac
...
...
Note (18 January 2018): The victims database warnings are due to the victi.ms site currently returning a 503 error. Jared Whiklo has reported this issue upstream: https://github.com/victims/victims-web/issues/155
One-Click Run
cd fcrepo-webapp; mvn clean install -Pone-click
...
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar
...
Manual Tests
Info |
---|
All of the below should take place in the HTML UI and non-vagrant tests should run against fcrepo-webapp-plus. |
- Create nested containers
- Create binary resources
- Run fixity on binary
- Update Properties: Perform SPARQL-Update on container
- Update Properties: Perform SPARQL-Update on binary
- Delete container
- Delete binary
- Use transactions
- Create versions
- View versions
- Rollback versions
Database Tests
With Tomcat7 deployment, run above manual tests with alternate backend databases (Configuring JDBC Object Store)
...
fcr:backup/fcr:restore Functionality
These tests are designed to ensure the proper function of the 'fcr:backup/fcr:restore' features by testing them against various Fedora configurations. The validity of the 'restore' can only be determined by crawling the repository and verifying the successful retrieval of the repository's content.
If the anticipated Fedora release is not backwards compatible with the previous version of Fedora, then the "From Fedora Version" should be the previous version. Otherwise, it is sufficient to test the fcr:backup/fcr:restore functionality using the same version.
See: RESTful HTTP API - Backup and Restore
# Backup
curl -X POST localhost:8080/rest/fcr:backup
# Restore
curl -X POST -d "/path/to/backup/directory" localhost:8080/rest/fcr:restore
Resources
- These python scripts - fcrepo-testing - can be used to load RDF content and binary content to a Fedora repository and verify the integrity of the loaded resources. Output from the load process can be used to verify the integrity of a 'restored' repository. See the README for more info.
- This script can be used to walk your repository, failing if a non-success response is encountered.
...
Tested by
...
Platform
...
Container
(Tomcat/Jetty)
...
Database
Backend
...
From Fedora
Version
...
To Fedora
Version
...
Number of
RDF Resources
...
Number of
Binaries
...
Size of Backup (du -h .)
...
Success RC1?
...
Notes
...
...
...
Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, thereby mimicking an 'in place upgrade.' All succeeded.
RC2 test used backup of 471 from RC1 test.
...
...
...
Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, thereby mimicking an 'in place upgrade.' All succeeded.
RC2 test used backup of 473 from RC1 test.
...
...
...
Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, thereby mimicking an 'in place upgrade.' All succeeded.
RC2 test used backup of 474 from RC1 test.
...
...
...
...
Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, as verification. All succeeded.
For RC2 test, the back up that was restored in this test was actually created from a postgres backend repo.
...
...
...
Performed a GET test on all resources in 4.7.5 RC2 before /fcr:restore, as verification. All succeeded.
For RC1 test, the back up that was restored in this test was actually created from a mysql backend repo.
Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2934 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2938 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2932
Defer to 5.1
Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2935 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2936 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2937
RC-2 Testing
Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2961 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2962 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2963 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2964 Jira server DuraSpace JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-2966
External Projects
Samvera (instructions)
Project | Tested by | Success? RC-1 | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|---|
valkyrie | Support merged into master branch |
Islandora (testing ticket)
Project | Tested by | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|
CLAW | Discovered FCREPO-2961 with regards to authentication RC-3 works once https://github.com/Islandora-CLAW/Syn/pull/14 is merged. |
API-X
Project | Tested by | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|
fcrepo-api-x-integration | ||||
fcrepo-api-x-demo (Docker) |
Testing Plan
Code Block |
---|
git clone https://github.com/fcrepo4/fcrepo4
cd fcrepo4
git checkout fcrepo-5.0.0-RC-3
# or pull pre-built resources from https://github.com/fcrepo4/fcrepo4/releases/tag/fcrepo-5.0.0-RC-2 |
Sanity Builds
Scripts
https://github.com/awoods/fcrepo-build-scripts
Project | Command | Platform | Tested By | RC 1 | RC 2 | RC 3 | Notes |
---|---|---|---|---|---|---|---|
fcrepo4 | mvn clean install | linux | Build successful on Ubuntu Server 16.04 LTS (HVM) Build successful on Fedora Linux 27; openjdk version "1.8.0_191"; Maven 3.5.0 (Red Hat 3.5.0-6) | ||||
fcrepo4 | mvn clean install | mac | Build successful | ||||
fcrepo4 | mvn clean install | windows | Build successful |
Note (18 January 2018): The victims database warnings are due to the victi.ms site currently returning a 503 error. Jared Whiklo has reported this issue upstream: https://github.com/victims/victims-web/issues/155
API-Test Suite
Start Fedora with the following command
Code Block mvn jetty:run -Dfcrepo.external.content.allowed=fcrepo-http-api/src/test/resources/allowed_external_paths.txt -Dfcrepo.auth.webac.userAgent.baseUri=http://example.com/ -pl fcrepo-webapp/
Run the API-Test suite with the following command
Code Block java -jar target/testsuite-1.0-SNAPSHOT-shaded.jar -c config-local.yml
Where "config-local.yml" is something like the following
No Format default: rooturl: http://localhost:8080/rest/ root-controller-user-webid: http://example.com/fedoraAdmin root-controller-user-name: fedoraAdmin root-controller-user-password: fedoraAdmin permissionless-user-webid: http://example.com/testuser permissionless-user-name: testuser permissionless-user-password: testpass broker-url: tcp://127.0.0.1:61616 topic-name: fedora queue-name: implementation-name: Fedora implementation-version: 5.0.0-RC tomcat: rooturl: http://localhost:8080/fcrepo/rest/ root-controller-user-webid: http://example.com/fedoraAdmin root-controller-user-name: fedoraAdmin root-controller-user-password: secret3 permissionless-user-webid: http://example.com/testuser permissionless-user-name: testuser permissionless-user-password: password1 broker-url: tcp://127.0.0.1:61616 topic-name: fedora queue-name: implementation-name: Fedora implementation-version: 5.0.0-RC
Platform | Test Suite Version | Tested by | Results RC-2 (#pass,fail,skip) | Results RC-3 (#pass,fail,skip) | Notes | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Fedora Linux 27 | f3dd201 | Total tests run: 238, Failures: 12, Skips: 23 | Total tests run: 238, Failures: 12, Skips: 23 | ||||||||||||||||||||
Ubuntu | Total tests run: 220, Failures: 13, Skips: 16 | ||||||||||||||||||||||
Mac 10.14 | Total tests run: 238, Failures: 44, Skips: 16 |
| |||||||||||||||||||||
Mac OSX 10.13.6 | a40ce88 | Danny Bernstein | Total tests run: 238, Failures: 12, Skips: 23 |
|
One-Click Run
cd fcrepo-webapp; mvn clean install -Pone-click
Command | Platform | Tested By | RC-1 | RC-2 | RC-3 | Notes |
---|---|---|---|---|---|---|
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Linux | Ran one-click and successfully performed manual tests | ||||
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Fedora Linux 27 | Randall Floyd | Built and ran one-click, successfully performed manual tests | |||
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Mac | Build and start One-Click run successfully | ||||
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Windows |
Manual Tests
Info |
---|
All of the below should take place in the HTML UI and non-vagrant tests should run against fcrepo-webapp. |
- Create nested containers
- Create binary resources
- Run fixity on binary
- Update Properties: Perform SPARQL-Update on container
- Update Properties: Perform SPARQL-Update on binary
- Delete container
- Delete binary
- Use transactions
- Create versions
- View versions
WebAC Tests
Use the following recipes to create WebAC-protected resources
Default ACL
Code Block |
---|
# root container should be read-only by default
curl -i -u testuser:testpass -X PUT http://localhost:8080/rest/test-default
# expect 403
# create a resource with admin
curl -i -u fedoraAdmin:fedoraAdmin -X PUT http://localhost:8080/rest/test-default
# expect 201
# resources inheriting the default acl should be read-only
curl -i -u testuser:testpass -X GET http://localhost:8080/rest/test-default
# expect 200
curl -i -u testuser:testpass -X PUT http://localhost:8080/rest/test-default/child
# expect 403 |
Platform | Tested by | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|
Mac | Danny Bernstein | |||
Fedora Linux 27 | ||||
Simple read-only
Code Block |
---|
# create a resources with admin
curl -i -u fedoraAdmin:fedoraAdmin -X PUT http://localhost:8080/rest/test-read-only
curl -i -u fedoraAdmin:fedoraAdmin -X PUT http://localhost:8080/rest/test-read-only/child
# put read-only acl on parent
echo "
@prefix acl: <http://www.w3.org/ns/auth/acl#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
<#restricted> a acl:Authorization ;
acl:agent 'testuser' ;
acl:mode acl:Read;
acl:default <http://localhost:8080/rest/test-read-only> ;
acl:accessTo <http://localhost:8080/rest/test-read-only> .
" | curl -i -u fedoraAdmin:fedoraAdmin -d @- -X PUT -H "Content-Type: text/turtle" http://localhost:8080/rest/test-read-only/fcr:acl
# Verify expected access privileges on target and child - ie read but not write
curl -i -u testuser:testpass -X GET http://localhost:8080/rest/test-read-only/child
# expect 200
curl -i -u testuser:testpass -X POST http://localhost:8080/rest/test-read-only/child
# expect 403 |
Platform | Tested by | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|
Mac | Danny Bernstein | |||
Fedora Linux 27 | ||||
Simple read-write
Code Block |
---|
# create a resources with admin
curl -i -u fedoraAdmin:fedoraAdmin -X PUT http://localhost:8080/rest/test-read-write
curl -i -u fedoraAdmin:fedoraAdmin -X PUT http://localhost:8080/rest/test-read-write/child
# put read-only acl on parent
echo "
@prefix acl: <http://www.w3.org/ns/auth/acl#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
<#restricted> a acl:Authorization ;
acl:agent 'testuser' ;
acl:mode acl:Read, acl:Write;
acl:default <http://localhost:8080/rest/test-read-write> ;
acl:accessTo <http://localhost:8080/rest/test-read-write> .
" | curl -i -u fedoraAdmin:fedoraAdmin -d @- -X PUT -H "Content-Type: text/turtle" http://localhost:8080/rest/test-read-write/fcr:acl
# Verify expected access privileges on target and child - ie read and write
curl -i -u testuser:testpass -X GET http://localhost:8080/rest/test-read-write/child
# expect 200
curl -i -u testuser:testpass -X POST http://localhost:8080/rest/test-read-write/child
# expect 201 |
- Create resource to protect
- Create ACL on protected resource (read/write)
- Create child of protected resource
- Verify expected access privileges on target and child
Platform | Tested by | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|
Mac | ||||
Fedora Linux 27 | ||||
External Content
Create and verify all three modes of external content handling: proxy, redirect, copy
Code Block |
---|
#Proxy
curl -i -H"Link: <file:///local/file/storage/photo.png>; rel=\"http://fedora.info/definitions/fcrepo#ExternalContent\"; handling=\"proxy\"; type=\"image/png\"" -XPUT -ufedoraAdmin:fedoraAdmin http://localhost:8080/rest/test_proxy_local_file
#verify that the binary is returned.
curl -i -u fedoraAdmin:fedoraAdmin -XGET -ufedoraAdmin:fedoraAdmin http://localhost:8080/rest/test_proxy_local_file
#remove the file one disk, repeat the previous command and verify that an error code is returned. |
Code Block |
---|
#Redirect
curl -i -H"Link: <https://duraspace.org/wp-content/themes/duraspace/assets/images/fedora/fedora-logo-black.png>; rel=\"http://fedora.info/definitions/fcrepo#ExternalContent\"; handling=\"redirect\"; type=\"image/png\"" -XPUT -ufedoraAdmin:fedoraAdmin http://localhost:8080/rest/test_redirect_remote_file
#Get the file and verify a 307 is returned
curl -i -ufedoraAdmin:fedoraAdmin http://localhost:8080/rest/test_redirect_remote_file |
Code Block |
---|
#Copy
curl -i -H"Link: <https://duraspace.org/wp-content/themes/duraspace/assets/images/fedora/fedora-logo-black.png>; rel=\"http://fedora.info/definitions/fcrepo#ExternalContent\"; handling=\"copy\"; type=\"image/png\"" -XPUT -ufedoraAdmin:fedoraAdmin http://localhost:8080/rest/test_copy_remote_file
#verify copied file returns 200.
curl -i -ufedoraAdmin:fedoraAdmin http://localhost:8080/rest/test_copy_remote_file
|
Handling (proxy,redirect,copy) | Platform | Tested by | Success? RC-2 | Success? RC-3 | Notes |
---|---|---|---|---|---|
proxy, redirect, copy | Mac | Danny Bernstein | |||
Database Tests
With Tomcat8 deployment, run above manual tests with alternate backend databases (Configuring JDBC Object Store)
Database | Platform | Tested by | Success RC1? | Success RC2 | Success? RC-3 | Notes |
---|---|---|---|---|---|---|
MySQL | macOS | High Sierra 10.13.6, MySQL 8.0.13 | ||||
PostgreSQL | macOS | High Sierra 10.13.6, PostgreSQL 9.6.2 | ||||
PostgreSQL | linux | Ubuntu 16.04, PostgreSQL 9.4.20 | ||||
MySQL5.6 | linux | Yinlin Chen | Ubuntu 16.04, Mysql 5.7.24 | |||
PostgreSQL | windows | version 11.0-2 | ||||
MySQL5.6 | windows |
fcr:backup/fcr:restore Functionality
These tests are designed to ensure the proper function of the 'fcr:backup/fcr:restore' features by testing them against various Fedora configurations. The validity of the 'restore' can only be determined by crawling the repository and verifying the successful retrieval of the repository's content.
If the anticipated Fedora release is not backwards compatible with the previous version of Fedora, then the "From Fedora Version" should be the previous version. Otherwise, it is sufficient to test the fcr:backup/fcr:restore functionality using the same version.
See: RESTful HTTP API - Backup and Restore
# Backup
curl -X POST localhost:8080/rest/fcr:backup
# Restore
curl -X POST -d "/path/to/backup/directory" localhost:8080/rest/fcr:restore
Resources
- These python scripts - fcrepo-testing - can be used to load RDF content and binary content to a Fedora repository and verify the integrity of the loaded resources. Output from the load process can be used to verify the integrity of a 'restored' repository. See the README for more info.
- This script can be used to walk your repository, failing if a non-success response is encountered.
Tested by | Platform | Container (Tomcat/Jetty) | Database Backend | From Fedora | To Fedora | Number of RDF Resources | Number of Binaries | Size of Backup (du -h .) | Success RC2? | Success? RC-3 | Notes |
---|---|---|---|---|---|---|---|---|---|---|---|
OSX 10.13.6 | Jetty | File-simple | 5.0.0-RC-2 | 5.0.0-RC-2 | 2821 | 0 | 500K | ||||
Danny Bernstein | OSX 10.13.6 | Jetty | File-simple | 5.0.0-RC-2 | 5.0.0-RC-2 | 2564 | 2564 | 2.6 GB | |||
Linux | Tomcat | Postgres | |||||||||
Linux | Tomcat | Postgres | |||||||||
Linux | Tomcat | Postgres | |||||||||
Linux | Tomcat | Mysql | |||||||||
Linux | Tomcat | Mysql 5.6 | |||||||||
Linux | Tomcat | Postgres |
NB: "Success" is measured not by receiving a "204 No Content" message after the 'fcr:restore' command, but by performing a GET on every resource in the repository and receiving "200 OK" messages.
Vagrant Tests
Code Block |
---|
vagrant destroy
vagrant up |
Test steps | Tested By | Success RC2? | Success? RC-3 | Notes |
---|---|---|---|---|
FEDORA_AUTH=true |
NB: "Success" is measured not by receiving a "204 No Content" message after the 'fcr:restore' command, but by performing a GET on every resource in the repository and receiving "200 OK" messages.
Vagrant Tests
Code Block |
---|
vagrant destroy
vagrant up |
...
FEDORA_AUTH=true
FEDORA_AUDIT=false
...
...
audit disabled
triplestore is staying in sync
solr
triplestore reindexing
...
FEDORA_AUTH=false
FEDORA_AUDIT=false
...
...
auth (disabled as expected)
audit disabled
triplestore is staying in sync
solr
triplestore reindexing
...
FEDORA_AUTH=true
FEDORA_AUDIT=true
...
...
Triple Store
Auth
Reindexing
Solr (after a "vagrant box update" and "vagrant destroy" it seems to work now).
However: if I add an RDF triple to a node (in my case <> dc:title "blah") , I'm am not seeing the change reflected in solr. The triple is however present in fuseki.
Is this expected behavior? ie are only some triples propagated to solr?
The above turns out not to be an issue. Updates to solr are collected and then execute in batch operations. I just needed to wait a few seconds.
Audit events: with this caveat
I am seeing this stacktrace in the karaf logs upon adding a resource. Not sure if it is a problem. Audit events are being logged properly as far as I can tell in /audit and fuseki is also receiving the audit triples.
sudo tail -f /opt/karaf/data/log/karaf.log:
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.fcrepo.client.FcrepoOperationFailedException: HTTP operation failed invoking info:fedora/audit with statusCode: -1 and message: URI does not specify a valid host name: info:fedora/audit
at org.fcrepo.client.FcrepoClient.executeRequest(FcrepoClient.java:218)[152:org.fcrepo.client.fcrepo-java-client:0.2.1]
at org.fcrepo.client.FcrepoClient.executeRequest(FcrepoClient.java:204)[152:org.fcrepo.client.fcrepo-java-client:0.2.1]
at org.fcrepo.client.RequestBuilder.perform(RequestBuilder.java:77)[152:org.fcrepo.client.fcrepo-java-client:0.2.1]
at org.fcrepo.camel.FcrepoProducer.getMetadataUri(FcrepoProducer.java:251)[201:org.fcrepo.camel.fcrepo-camel:4.5.0]
at org.fcrepo.camel.FcrepoProducer.getUri(FcrepoProducer.java:225)[201:org.fcrepo.camel.fcrepo-camel:4.5.0]
at org.fcrepo.camel.FcrepoProducer.doRequest(FcrepoProducer.java:194)[201:org.fcrepo.camel.fcrepo-camel:4.5.0]
at org.fcrepo.camel.FcrepoProducer.process(FcrepoProducer.java:156)[201:org.fcrepo.camel.fcrepo-camel:4.5.0]
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[58:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145)[58:org.apache.camel.camel-core:2.18.0]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[
I confirmed that this behavior exists in in the 4.7.4
...
FEDORA_AUTH=false
FEDORA_AUDIT=true
...
...
Manual Tests
Same as above, plus:
...
- Verify audit events are in triplestore
- Verify resources are in triplestore
- Verify resources are in Solr
- Verify authorization works for the two auth-enabled configurations
Verify reindexing to triplestore works
Code Block vagrant ssh sudo service tomcat7 stop sudo rm -rf /etc/fuseki/databases/test_data/* sudo service tomcat7 start curl -XPOST localhost:9080/reindexing/ -H"Content-Type: application/json" -d '["activemq:queue:triplestore.reindex"]'
Backwards Compatibility Tests
- Start 45.70.4 0 one-click
- Load sample datasets via /fcr:restore
- Run test scripts on 45.70.40
- Stop 45.70.40
- Start RC one-click
- Run test scripts on RC
- ReStart RC
- Run test scripts on RC
Tested by | Success RC1RC2 | Success RC2 | Notes |
---|---|---|---|
Did not run camel_toolbox_tests or authz_tests. | |||
? RC-3 | Notes | ||
Resources
[1] Testing scripts
...