Some thoughts about the last week or so...
- We're getting close to being done with the migration of trackers from SourceForge to JIRA. The last major item is to move the task items, which we use to indicate work that's actually in progress. I was able to convert the code Chris wrote to migrate the other trackers (bugs, refactorings, new feature requests) to create the Jelly script for importing the SF tasks into JIRA, but in the process of testing, discovered that the SF export file was incomplete. This activity is now waiting on a service request with SourceForge, which hopefully will be completed soon and we can finish up our move to JIRA.
- I started looking into Zotero as it is one of the items on the list of integrations for the innovation challenge. I focused mainly on how Fedora could be used to store the information collected by Zotero. Doing this on the client is not very practical, but with the upcoming server storage capabilities of Zotero 2.0, this becomes interesting. In discussions with Eddie he pointed out that he was focusing more on allowing Zotero to easily harvest content out of Fedora using unAPI, and he gave a good overview of that work in the architecture meeting.
- After a discussion in last week's architecture meeting, I updated the fedora-client.jar to include the version of the Fedora server with which it was released. So we now have a fedora-3.0-client.jar, or at least we will as soon as my changes go into trunk.
- Looking again at the messaging client and the changes made to allow GSearch (or any other application) to make asynchronous broker connections, I realized that there's still a use case for a synchronous connection. Since a synchronous connection is implied in the MessagingClient interface, that is still the default, but passing in a parameter to start() allows the client to return while the connection is still being negotiated.
- Moving toward the goal of having a web-based administrative client for Fedora (as well as the possibility of a hosted Fedora application) I've started to investigate web-based GUI frameworks. The first of those is Adobe's Flex. I was able to create a simple application that has some admin client-type features with a small amount of xml in a relatively short amount of time.
The capabilities to connect to REST or SOAP-based web services quickly is impressive. The drawbacks of requiring a Flash player and the development tool being a commercial product do dampen my enthusiasm a bit, but I was happy to find that applications can be built using only the open-source pieces of Flex. Of course, Open Laszlo is a completely open-source alternative to Flex, so that's next on my list to consider.
Early this week I wrote up an overview of Amazon's web services offerings and posted it here. This included information about Amazon S3, EC2, EBS, and a bit about SimpleDB and SQS as well as information regarding how Fedora has been integrated with these services thus far, and where we might consider going. Discussions with the team during the newly started architecture meeting helped add several ideas to the list. In conjunction with this, I also ran some simple tests with EC2 using instance storage, EBS, and S3 as storage behind Fedora to see how the various storage types compared performance-wise. It turned out that the best performance came from EBS, then instance storage, followed after a while by S3.
My other focus this week was around an issue that has come up a couple of times regarding the updater in GSearch. In the default installation scenario GSearch is installed into the same Tomcat as Fedora, and Fedora starts the message broker on startup. Unfortunately, if the Tomcat server happens to start GSearch first, it's not able to connect to the message broker. To get around this the Fedora messaging client (which is used by GSearch) now spawns a thread to do the connection and performs several retries if the first attempt is not successful. This allows GSearch to let Fedora start and get the broker going before connecting. As a side note, testing this out required using a separate Tomcat to run GSearch, so it verified the ability to run GSearch external to Fedora.
My main focus this week was getting an EC2 image set up which properly builds, installs, and starts a Fedora server on instance startup. Most of this work was server configuration and scripting. I started by recreating the image I made last week (which provides a checkout of trunk) with some updates. I used that as my base, then added scripting to grab and update the provided install.properties file, set up the environment, update the code, install Fedora, convert the demos using the current IP, and start the Fedora server. Adding that to the server startup provided the desired results. I can now start an instance with either a configA or configB properties file, then log in and run the system tests, which pass with no further setup.
I'm considering my initial pathfinding prototype complete at this stage since I believe the capabilities of EC2 have been well enough established. There are features that are lacking, such as the ability to choose a database other than the default and have that started and configured for you, and the ability to select the JVM version to use. Based on what I've done so far, though, these things can certainly be accomplished, it's just a matter of taking the time to do it. I'd like to use the architecture meeting on Tuesday to discuss where we plan to go with AWS.
I also spent some time investigating the Amazon EBS, just doing basic setup, mounting, moving from instance to instance, etc. There's not a lot to this, really, but it's something that we will very likely use in conjunction with EC2.