Page History

...

• Speed is limited by the network making iI/o O much slower than local disk
• File size is limited to 5GB

...

• Authentication to S3 is done through use of a an Access Key ID and Secret Access Key pair, or through use of an X.509 certificate
• Connecting to S3 is done via SOAP or REST. Amazon provides libraries in several languages to make use of S3 easier. Third-party libraries are also available; these tend to be more full-featured.
• All files stored in S3 are considered objects. Every object can have metadata associated with it as a list of name-value pairs.
• All objects are stored in buckets, which are similar to folder folders on a file system. Bucket names must be unique across all of S3. This is necessary to allow access to buckets using subdomains (i.e. if you have a bucket named fedora, the content in that bucket will be available at https://fedora.s3.amazonaws.com/^{Image Removed})
• The name of each object is its key. Keys must be unique within a bucket. Keys can be named in such a way as to suggest a folder structure.

Resources
Main page - http://www.amazon.com/gp/browse.html?node=16427261^{Image Removed}
Getting Started Guide - http://docs.amazonwebservices.com/AmazonS3/2006-03-01/gsg/^{Image Removed}
Developer Guide - http://docs.amazonwebservices.com/AmazonS3/2006-03-01/^{Image Removed}
Technical FAQ - http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1109&categoryID=55^{Image Removed}
Java REST library for S3 - http://developer.amazonwebservices.com/connect/entry.jspa?externalID=132&categoryID=47^{Image Removed}
JetS3t, open source toolkit for S3 (includes a Swing GUI for working with S3 files) - https://jets3t.dev.java.net/^{Image Removed}

Amazon Elastic Compute Cloud (EC2)

...

• Server images allow starting/stopping pre-configured server instances
• Server instances can be started/stopped as needed, requiring only slightly more time than is required to boot/shutdown the OS
• Root access to running server instances
• Provides the ability to run server instances in multiple locations by specifying the "Availability Zone" of the instance on startup
• Elastic IP addresses allow you to map a static IP to any running instance, so your IP stays consistent even as server instances come and go
• No up-front costs, pay only for what you use

...

• Server images are called Amazon Machine Images (AMIs). AMIs are stored on S3, then registered with EC2.
• A server Instance is an AMI which has been started and is running. All instances of an AMI are identical at startup. All changes to an instance and any information stored on an instance are lost when an instance fails or is shut down.
• Amazon provides a set of command line tools for working with AMIs and instances. Instances can be accessed via SSH.
• AMIs can be created from scratch on a local machine or by modifying an existing AMI. There is a listing of publicly available AMIs, some provided by Amazon, others provided by various companies or individuals.
• To create an AMI from an existing AMI you create an instance of the existing AMI, log in and modify it as required, create an AMI bundle (i.e. produce the image based on the instance), upload it to S3, then register the AMI with EC2.
• Parameters can be passed to an instance on startup, either as a text string or as a file. These parameters can be used to start or configure an instance in a certain way.

Resources
Main page - http://www.amazon.com/gp/browse.html?node=201590011^{Image Removed}
Getting Started Guide - http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/^{Image Removed}
Developer Guide - http://docs.amazonwebservices.com/AWSEC2/2008-05-05/DeveloperGuide/^{Image Removed}
Technical FAQ - http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1145&categoryID=100^{Image Removed}
EC2 Setup Video (Windows) - http://s3.amazonaws.com/AmazonEC2Tutorial/AmazonEC2Tutorial.wmv^{Image Removed}
ElasticFox, excellent Firefox Extension for EC2 - http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609^{Image Removed}
List of public AMIs - http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=101^{Image Removed}

Amazon Elastic Block Store (EBS)

...

Benefits

• Up to 1TB disk storage per volume, unlimited volumes
• Persist beyond the life of an EC2 instance
• Designed to perform faster than EC2 instance storage
• Can be disconnected and moved between instances
• Automatically disconnects when an EC2 instance fails or is shut down
• Many EBS volumes can be mounted to a single instance at one time
• Reliable: EBS instances are automatically replicated (within its availability zone) - on the order of 10x more reliable than typical disk drives
• Snapshots of an EBS volume can be made to S3, allowing creation of new EBS volumes with a known dataset
• Snapshots of a volume after the first only store deltas, keeping the S3 footprint small.

...

• Amazon provides command line tools for using EBS along with the EC2 tools. These tools allow you to create, attach, detach, and delete volumes as well as create and delete snapshots.

Resources
Main page - http://www.amazon.com/gp/browse.html?node=689343011^{Image Removed}
Feature Guide - http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1667&ref=featured^{Image Removed}

Amazon SimpleDB

What is it?
A simplified relational database service which removes the need for a schema by limiting queries to a single table (called a domain) and allowing variable columns per row.

Resources
Main page - http://www.amazon.com/gp/browse.html?node=342335011^{Image Removed}

Amazon Simple Queue Service (SQS)

What is it?
A simple queue service made available to handle very basic queueing needs. The only functions are CreateQueue, SendMessage, ReceiveMessage, and DeleteMessage. There is no publish/subscribe support.

Resources
Main page - http://www.amazon.com/Simple-Queue-Service-home-page/b?ie=UTF8&node=13584001^{Image Removed}

What has been done so far using AWS?

...

• Created a private AMI which starts up with a running Fedora.
• Data is stored to S3 via a a mount provided by using JungleDisk.
• Created tutorials that allow people to try out Fedora by creating an instance once they are given permissions to access the AMI

links:
http://www.jisc.ac.uk/whatwedo/programmes/programme_rep_pres/repositories_sue/fedorazon.aspx^{Image Removed}
http://www.ukoln.ac.uk/repositories/digirep/index/Fedorazon_How_to_Guides^{Image Removed}
http://docs.google.com/View?docid=dfccf5n6_51dwmk8rdb^{Image Removed}

Integration Challenge

...