MacOS 10.11.6
Used data from https://github.com/fcrepo4-labs/fcrepo-sample-dataset/tree/master/additional_datasets
brew install postgres
gem install fcrepo_wrapper
Create config file .fcrepo_wrapper:
port: 8984 enable_jms: false |
via
mvn clean install
Clone https://github.com/fcrepo4-labs/fcrepo-sample-dataset and extract:
cd fcrepo-sample-dataset/additional_datasets/ tar xzvf lubm/LUBM_02.tar.gz tar xzvf 10kIndirectContainers/10kIndirectContainers.tar.gz |
Start postgres: pg_ctl -D db/postgres start
fcrepo_wrapper clean
Start Fedora 4.7.0: fcrepo_wrapper
Confirm Fedora up/empty at: http://localhost:8984/rest/
curl -X POST -d "/full/path/to/fcrepo-sample-dataset/additional_datasets/LUBM_02/" "http://localhost:8984/rest/fcr:restore"
Restore extracted reference dataset(s):
curl -X POST -d "/full/path/to/fcrepo-sample-dataset/additional_datasets/LUBM_02/" "http://localhost:8984/rest/fcr:restore" curl -X POST -d "/full/path/to/fcrepo-sample-dataset/additional_datasets/10kIndirectContainers/" "http://localhost:8984/rest/fcr:restore" |
Confirm 100 "objects" and 10000 "memberCollection" loaded at: http://localhost:8984/rest/
From "objects" node in UI, create an "albums" container. Add one binary to it, uploading an image file.
Directory (11360) will vary.
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode export --resource http://localhost:8984/rest/11360/objects/albums --dir /Users/atz/repos/import_export_test/ --binaries java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar -x .json -l application/ld+json --mode export --resource http://localhost:8984/rest/11360/objects/albums --dir /Users/atz/repos/import_export_test/ --binaries |
+ Repeated (successful) exports do not alter target dir after first run
Export:
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode export --resource http://localhost:8984/rest/ --dir /Users/atz/repos/import_export_test/ --binaries |
Deleting everything at http://127.0.0.1:8984/rest/11360 fails:
WARN 16:23:22.499 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a000004:e85c:588f9f9e:a9 in state RUN WARN 16:31:53.376 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a000004:e85c:588f9f9e:a9 in state SCHEDULE_CANCEL WARN 16:31:57.952 (arjuna) ARJUNA012095: Abort of action id 0:ffff0a000004:e85c:588f9f9e:a9 invoked while multiple threads active within it. WARN 16:32:08.391 (arjuna) ARJUNA012108: CheckedAction::check - atomic action 0:ffff0a000004:e85c:588f9f9e:a9 aborting with 1 threads active! WARN 16:32:08.391 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a000004:e85c:588f9f9e:a9 in state CANCEL_INTERRUPTED WARN 16:32:46.082 (arjuna) ARJUNA012120: TransactionReaper::check worker Thread[Transaction Reaper Worker 0,5,main] not responding to interrupt when cancelling TX 0:ffff0a000004:e85c:588f9f9e:a9 -- worker marked as zombie and TX scheduled for mark-as-rollback WARN 16:32:59.021 (arjuna) ARJUNA012110: TransactionReaper::check successfuly marked TX 0:ffff0a000004:e85c:588f9f9e:a9 as rollback only WARN 16:33:25.371 (arjuna) ARJUNA012113: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] missed interrupt when cancelling TX 0:ffff0a000004:e85c:588f9f9e:a9 -- exiting as zombie (zombie count decremented to 0) |
Then, manually called kill -9
on fedora java process. Cleaned out installation and restarted Fedora.
Import:
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode import --resource http://localhost:8984/rest/ --dir /Users/atz/repos/import_export_test/ --binaries --auditLog |
Finishes successfully with log message in importexport_audit_20170130T170006.log:
... 2017-01-30 17:10:11,021 Finished import... 20006 resources imported |
But simultaneously, Fedora crashes again. Looks like a normal shutdown but was unprompted (no "Fedora is shutting down." message):
[Thread-0] INFO org.simplericity.jettyconsole.DefaultJettyManager - Shutting down Fedora Repository Deployable Web Application 4.7.0.. [Thread-0] INFO org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@1f39269d{HTTP/1.1}{0.0.0.0:8984} [Thread-0] INFO / - Closing Spring root WebApplicationContext [Thread-0] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.s.j.JettyConsoleWebappContext@1e643faf{/,file:/private/var/folders/6q/13dm9dcn1qx_gw9x66lhl7jc0000gq/T/fcrepo-webapp-4.7.0-jetty-console.jar_8984/webapp/,UNAVAILABLE}{/private/var/folders/6q/13dm9dcn1qx_gw9x66lhl7jc0000gq/T/fcrepo-webapp-4.7.0-jetty-console/fcrepo-webapp-4.7.0-jetty-console.jar} [Thread-0] INFO org.simplericity.jettyconsole.DefaultJettyManager - Shutdown of Fedora Repository Deployable Web Application 4.7.0 complete. |
After restart, appears empty. So import failed. On second identical attempt, succeeded with the same "Finished import... 20006 resources imported" message.
Export (Test 0) again:
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode export --resource http://localhost:8984/rest/11360/objects/albums --dir /Users/atz/repos/import_export_test/ --binaries |
Diff of jsonld files:
--- a/rest/11360/objects/albums.jsonld +++ b/rest/11360/objects/albums.jsonld @@ -1,9 +1,9 @@ [ { "@id" : "http://localhost:8984/rest/11360/objects/albums", - "@type" : [ "http://fedora.info/definitions/v4/repository#Container", "http://fedora.info/definitions/v4/repository#Resource", "http://www.w3.org/ns/ldp#RDFSource", "http://www.w3.org/ns/ldp#Container" ], + "@type" : [ "http://fedora.info/definitions/v4/repository#Container", "http://www.w3.org/ns/ldp#Container", "http://www.w3.org/ns/ldp#RDFSource", "http://fedora.info/definitions/v4/repository#Resource" ], "http://fedora.info/definitions/v4/repository#created" : [ { "@type" : "http://www.w3.org/2001/XMLSchema#dateTime", - "@value" : "2017-01-30T22:23:50.14Z" + "@value" : "2017-01-31T01:37:09.859Z" } ], "http://fedora.info/definitions/v4/repository#createdBy" : [ { "@value" : "bypassAdmin" @@ -13,7 +13,7 @@ } ], "http://fedora.info/definitions/v4/repository#lastModified" : [ { "@type" : "http://www.w3.org/2001/XMLSchema#dateTime", - "@value" : "2017-01-30T22:24:05.691Z" + "@value" : "2017-01-31T01:38:32.169Z" } ], "http://fedora.info/definitions/v4/repository#lastModifiedBy" : [ { "@value" : "bypassAdmin" diff --git a/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld b/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld index 029aa6b..01e1490 100644 --- a/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld +++ b/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld @@ -1,9 +1,9 @@ [ { "@id" : "http://localhost:8984/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee", - "@type" : [ "http://fedora.info/definitions/v4/repository#Binary", "http://fedora.info/definitions/v4/repository#Resource", "http://www.w3.org/ns/ldp#NonRDFSource" ], + "@type" : [ "http://www.w3.org/ns/ldp#NonRDFSource", "http://fedora.info/definitions/v4/repository#Binary", "http://fedora.info/definitions/v4/repository#Resource" ], "http://fedora.info/definitions/v4/repository#created" : [ { "@type" : "http://www.w3.org/2001/XMLSchema#dateTime", - "@value" : "2017-01-30T22:24:05.694Z" + "@value" : "2017-01-31T01:38:32.179Z" } ], "http://fedora.info/definitions/v4/repository#createdBy" : [ { "@value" : "bypassAdmin" @@ -16,7 +16,7 @@ } ], "http://fedora.info/definitions/v4/repository#lastModified" : [ { "@type" : "http://www.w3.org/2001/XMLSchema#dateTime", - "@value" : "2017-01-30T22:24:05.694Z" + "@value" : "2017-01-31T01:38:32.179Z" } ], "http://fedora.info/definitions/v4/repository#lastModifiedBy" : [ { "@value" : "bypassAdmin" |
So the salient differences for JSON+LD are datestamps and the order of @type
elements.
But for Turtle, the diff is semantically distinct. The datestamps differ in the same way, but the objects have acquired new "ldp:Container , ldp:RDFSource"
and "ldp:NonRDFSource
" values, respectively.
Diff (condensed):
--- rest/11360/objects/albums.ttl -<http://localhost:8984/rest/11360/objects/albums> a fedora:Container , fedora:Resource ; +<http://localhost:8984/rest/11360/objects/albums> a fedora:Container , ldp:Container , ldp:RDFSource , fedora:Resource ; ... --- rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.ttl -<http://localhost:8984/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee> a fedora:Binary , fedora:Resource ; +<http://localhost:8984/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee> a ldp:NonRDFSource , fedora:Binary , fedora:Resource ; |
Fedora unresponsive during large (failing) delete operation in UI. This is probably a known problem.