...
- Danny Bernstein
- David Wilcox
- Jared Whiklo
- Peter Winckles
- Daniel Lamb
- Ben Pennell
- Thomas Bernhart
- Calvin Xu
- Mike Ritter
Agenda
- Announcements
- 2021-04 Fedora 6 10 Camel Toolbox Sprint
- North American User Group
- Pilots / Testing
- Pre release short list of bug fixes and improvements
Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3289 Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3620 Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3561 Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3638 Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3672
- Any new tickets/issues critical to be considered for the release
?Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3690 Jira server LYRASIS JIRA serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 key FCREPO-3691 - others
- Your topic
- Migration-util performance issue in testing
- Updates to fcrepo Modeshape Modeshape CI https://github.com/fcrepo-exts/modeshape/pull/23
- fcrepo-java-client question
- migration util still have performance issue after tuning java heap size and turning off checksum validation
Tickets
In Review
Expand Jira server DuraSpace JIRA jqlQuery filter=13100 serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 Please squash a bug!
Expand Jira server DuraSpace JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution maximumIssues 20 jqlQuery filter=13122 serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 Tickets resolved this week:
Expand Jira server DuraSpace JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution maximumIssues 20 jqlQuery filter=13111 serverId c815ca92-fd23-34c2-8fe3-956808caf8c5 Tickets created this week:
Expand Jira server DuraSpace JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution maximumIssues 20 jqlQuery filter=13029 serverId c815ca92-fd23-34c2-8fe3-956808caf8c5
Notes
- Alpha release of fcrepo-migration-validator will land soon
- Will be reaching out to a handful of organizations to find testers, including UVA and UW Madison
- RC-1 Release of fcrepo
- Concerns around performance
- AWS
- Search Index
- MYSql
- Going to give it 3 weeks to do some testing on these issues
- Migration utils is running slower according to Calvin Xu
- You can disable checksums by passing it a --no-checksum-validation flag, which should speed it back up to where it was in the previous release
- Concerns around performance
- Moving off of Jira
- Need to confirm if the existing ticket history can be moved along or preserved in some way
- Also need to confirm Kanban functionality works with Github's permissions scheme
- https://github.com/fcrepo-exts/modeshape/pull/2
- PR seems benign, and stops null pointer errors when deleting objects from Seth's repo
- It's unclear as to how the objects were created or what is making the NPE pop
- Should probabaly add some logging to cover the case when null still happens
Actions
- Announcements
- North American User Group
- Pilots / Testing
- Performance testing by Andrey, Peter and Jared
- Andrey and Jared were seeing fairly bad performance
- Andrey has performed the same tests on Fedora 4 and 5, and is seeing significantly worse performance on 6
- Will check to see what the specs of the machine running the tests were
- Peter and Danny have been unable to reproduce the poor performance
- Andrey and Jared were seeing fairly bad performance
- Including search index commits in same DB commit as the other db commits seems to maintain normal performance
- Search queries are fast except when getting back RDF types. Danny is working on resolving this in a separate PR.
- Comparing POSTs and PUTs - Jared reported that PUTs were a little bit slower
- Single object vs Multi-object transactions
- At the moment all transactions assume they are multi-object
- It may reduce the database latency significantly if single object transaction commits were implemented, but at the cost of considerable complexity
- what percentage of the latency is due to database? To determine how much of a concern this is.
- Calvin testing migration-util with larger heap, checksum turned off
- Initially faster, slowed back down to 25k per day after about 1.5 million objects
- Previous runs used about 30% cpu, but not it is only using about 7%
- It seems to still be processing image files, which are not particularly large
- UVA originally ran their migration before alpha2, had significantly slower performance in their second migration after alpha2
- Ben C created a ticket to look into slow resumption of migration-util, which iterates back through all the Fedora 3 objects.
- Performance testing by Andrey, Peter and Jared
- Pre release short list of bug fixes and improvements
- Any new tickets/issues to be considered for the release
- Other topics
- fcrepo-java-client
- Need to follow through on java 11 update and update of integration tests.
- Thomas will take a look at the existing two tickets for updating it
- Modeshape
- Danny will look at the gh actions PR
- Will update Seth's PR to run gh actions to make sure tests pass
- We will review the unreleased updates to modeshape since the last release was cut, to determine if we want to sync up our fork
- ocfl-java database
- Peter will be checking on its usage of postgres for inventory caching and distributed object locking, as there are additional costs incurred by its use of separate processes per connection that he had not realized
- fcrepo-java-client
Actions
- Look through migration util commits since December to see if there are any potential causes of performance drop off.