This page details some of the ways that uploading a file might fail, the client errors received, files persisted to disk, etc. as a result. The primary objective is to understand how Fedora fails under different circumstances, and identify any bugs that can be fixed to improve reliability.
Summary
- Uploads that completed successfully and without error message could all be transferred to another VM and read successfully.
- Filling up the disk or undeploying Fedora result in files being written to the Fedora home directory but not tracked by the repository.
- Filling up the disk and then attempting to update Fedora results in an unresponsive repository.
General procedure
- Run Fedora 4 in Tomcat on one VM.
- Upload a large file from a second VM using curl. 4GB files were used so transfer times would be long enough to interrupt the upload process.
- Upload a second file, and interrupting the upload in various ways to simulate application, network or hardware failures.
- Copy the Fedora home directory to the second VM and run a separate Fedora instance to verify that the files could be retrieved
Fill up disk
- Before the second upload, add files to the filesystem containing the Fedora home directory so there isn't enough disk space for the upload to finish.
- The client receives a 400 Bad Request error.
- The file uploads to a temporary directory and is then copied over to the Fedora home directory.
- When the disk is full, any attempt to create containers makes Fedora become unresponsive, and unable to be restarted.
- Freeing up disk space allows Fedora to handle updates and be started again.
Undeploy or redeploy Fedora
- While the second upload is ongoing, undeploy the Fedora webapp (by removing the fcrepo.war file from Tomcat's webapps directory).
- The upload finishes and the client receives a 500 Internal Server Error.
- The file is moved to the Fedora home directory, but does not show up in the REST API
Kill Tomcat
- While the second upload is ongoing, kill Tomcat's java process on the client VM.
- The client dies with a non-0 exit status.
- The file is not written to the Fedora home directory.
Kill Client
- While the second upload is ongoing, kill curl on the client VM.
- If the checksum parameter is specified, the upload fails.
- But if the checksum parameter is not specified, then the partial upload is treated as a complete file and added to the repository.
Block Client connections
- Before the second upload, block the client IP on the server VM using IPTables firewall.
- The second upload times out.
