The Software Devotional

Amazon Web Services April Fools Comes Early

Not without a sense of humor, the Amazon Web Services group posted about a great new service it will soon be offering, if but for a single day (Apr 1). Based on customer feedback, Amazon has "designed and [is] now introducing the Floating Amazon Cloud Environment, or FACE for short" to deliver "durable, unmanned helium-filled blimps with a capacity of 65,536 small EC2 instances, or a proportionate number of larger instances." An additional feature added for the FACE environment is support for RFC 1149, which describes a "method for the encapsulation of IP datagrams in avian carriers." Go Amazon.

Free SimpleDB Pricing and Public Beta

Free SimpleDB Pricing Tier

Though the announcement showed up in my mail today, Amazon announced yesterday that a new free pricing tier will be made available for its SimpleDB service. According to Amazon's SimpleDB page, the first 25 machine hours, 1GB of data transfer, and 1GB of data storage used each month will be free of charge for at least the first 6 months (starting Dec 1, 2008).

From Amazon: You can get started with SimpleDB for free and without risk. For at least the first six months of the public beta (starting December 1, 2008), you pay no charges on the first 25 Machine Hours, 1 GB of Data Transfer, and 1 GB of Storage that you consume every month. In most use cases, approximately 2,000,000 GET, QUERY, or QUERYWITHATTRIBUTES API requests can be completed per month before incurring any usage charges. Many applications should be able to operate perpetually within this free tier, such as a daily web-site analysis and traffic reporting tool, a web indexing service, or an analytics utility for online marketing programs.

I have read elsewhere that this process is being spun as a
'trial' for the SimpleDB product. I don't believe that Amazon views this free tier as a traditional trial period and I don't think they are trying to convince more people to help test their software while it is still in beta. It appears to me that Amazon is putting a toe in the water to determine how successful and how costly a free tier might be for SimpleDB. If Amazon can capture the hearts and minds of budding mISVs (when cost is a paramount concern), then they are likely to subsequently capture the pocketbooks of the successful mISVs as well.

In addition to the new free pricing tier, SimpleDB's monthly storage cost per GB has been reduced from $1.50 to $0.25 - 83%.  No doubt, this can be attributed to Google's and Microsoft's plethora of cloud computing announcements this year. Further, at $1.50, SimpleDB's storage cost was way out of line compared with S3's cost of only $0.15 per GB.

Recall that pricing for Amazon's SimpleDB is based on 3 factors: machine utilization, data transfer (in and out), and data storage.  After the first 25 machine hours (free), each machine hour costs $0.14.  The first GB of data transferred in and out are free, and the initial pricing tiers are $0.10 per GB inbound and $0.17 per GB outbound.  After the first GB of storage used per month (free), each additional GB costs $0.25 per month. A rough calculation shows the monthly savings to be a little under $4. Still, many web hosting providers charge several dollars each month for database storage - no need for that any longer.

SimpleDB Enters Public Beta Period

In conjunction with the news of the free pricing tier, Amazon also announced that SimpleDB, while still in beta, is open to all developers. The closed beta, opened to a select group of developers late last year, has given way to a public beta period.

This is good news for me, as I am speaking at VSLive in Dallas next week discussing, among other things, SimpleDB. I despise going to conferences to learn of technologies that have no hope of impacting my development environment for many months or even years to come. There is a time and place for learning how the future will shape up, but for a hands-on coding session, I personally think it best to have the technologies available at my fingertips. This makes SimpleDB available to all the attendees.

The Official Email

Dear AWS Developer,

Amazon SimpleDB is now available in public beta; any business or developer may now sign up and start using this service immediately. Amazon SimpleDB is a web service providing the core database functions of indexing and querying. We're also excited to announce a free tier of usage for SimpleDB, as well as lower prices for storing your data. In addition, Amazon SimpleDB customers will now have access to AWS Premium Support, providing access to a 24/7, one-on-one, fast-response support channel.

Pricing Updates

To help you get started with Amazon SimpleDB, we are providing a free usage tier for at least the next six months. Each month, there is no charge for the first 25 machine hours, 1 GB of data transfer (in and out), and 1 GB of storage that you use. Standard pricing will apply beyond these usage levels, and free usage does not accumulate over time.

In addition, beginning today, Amazon SimpleDB customers will now enjoy significantly reduced storage pricing, only $0.25 per gigabyte-month. This new rate reflects an 83% reduction in storage pricing. For more information, see aws.amazon.com/simpledb.

AWS Premium Support Now Available for SimpleDB

Customers can now also sign up for AWS Premium Support of Amazon SimpleDB. AWS Premium Support offers technical assistance from the Amazon Web Services team whenever and as frequently as customers need it. Whether for operational issues or technical questions, Amazon SimpleDB customers now have access to developer support engineers 24/7, and can count on predictable response time and personalized support. To sign up for AWS Premium Support, visit aws.amazon.com/premiumsupport.

New Features for Amazon SimpleDB Coming Soon

We are also very pleased to announce plans to release two enhancements in the next few weeks that will make Amazon SimpleDB even easier to use. These enhancements will include:

• SQL-like SELECT API: With this feature, you will be able to employ a very familiar SQL-like query language when constructing queries and building applications. The SELECT API will help you be more productive with Amazon SimpleDB right from the start.
• Batch Put: Enables you to streamline the upload of multiple items or attributes, with a single batch API request rather than multiple single item calls.

We are pleased to share these exciting announcements with you, and invite you to visit aws.amazon.com/simpledb for full details.

Sincerely,

The Amazon SimpleDB Team

 

New AWS High-CPU EC2 Instance Types Announced

Amazon announced today the availability of new instance types for EC2. Previously, EC2 instances were heavier on memory allocation than the new High-CPU instances. The new instance types, dubbed "High-CPU Instances" scale the processor count, or EC2 Compute Units, more quickly than memory. The proportion of CPU/memory in the High-CPU instances is considerably higher. Processor intensive applications that don't require substantial memory allocations can now be run on fewer instances, meaning they can be operated at a lower cost.  Using the "Standard" instance types, the cost per EC2 Compute Unit (without respect to other factors) was $0.10/hour. The High-CPU Instance types cost only $0.04/hour to operate.  Again, the rough pricing is subjective to the application's need for other resources such as RAM, disk space, I/O performance, and perhaps the advantage of running on a 64 bit platform. Keep this in mind when you compare the per-Compute Unit prices above.

Take a look at the instance types that were added:

High CPU Medium Instance
	$0.20 / instance hour
	1.7 GB of memory
	5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
	350 GB of instance storage
	32-bit platform
	I/O Performance: Moderate
High-CPU Extra Large Instance
	$0.80 / instance hour
	7 GB of memory
	20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
	1690 GB of instance storage
	64-bit platform
	I/O Performance: High

Compare the "High-CPU Extra Large Instance" to the standard "Extra Large Instance" shown below:

High CPU Medium Instance
	$0.20 / instance hour
	15 GB memory
	8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each)
	1,690 GB instance storage
	64-bit platform
	I/O Performance: High

Amazon defines a compute unit as follows (taken from here). Refer to Amazon's description of instance types for more details.

EC2 Compute Unit (ECU) - One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

Get some. :)

Amazon's SimpleDB Beta Invitation

I just received my invitation to begin using Amazon's SimpleDB service, currently in Limited Beta. SimpleDB was launched earlier this year, and is the latest in Amazon's arsenal of services in the cloud. Like all of Amazon's other web services, you pay only for what you use.

From: Amazon Web Services

Greetings from Amazon Web Services,

Thank you for signing up for Amazon SimpleDB. You now have immediate access to Amazon SimpleDB and can build innovative and entrepreneurial applications using this and other AWS web services.

Any usage charges (as applicable to the services you are using) will be billed to your account on your next billing date. Please see the Account Activity area of the AWS web site for detailed account information:

http://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=activity-summary

To build your application and make valid web service requests, you'll need your Access Identifiers. Obtain and learn more about Access Identifiers on the AWS web site:

http://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key

Thank you for your participation in the Amazon Web Services community. See http://aws.amazon.com for more information.

Sincerely,
Amazon Web Services

This message was produced and distributed by Amazon Web Services LLC, 1200 12th Avenue South, Seattle, Washington 98144-2734

SimpleDB uses a different paradigm than most developers are accustomed to. In many cases, you will find that the differences are intended to simplify the architecture and use of SimpleDB. Relational databases have largely failed in terms of management simplicity - even a reasonably sized SQL Server instance requires a full-time DBA to constantly care for it. Using SimpleDB, you not only get the benefit of using a fully managed service (Amazon will manage the service for you), but the interface is simple enough to avoid the problems that typically cause problems with relational database performance.

Using SimpleDB, developers group data into domains (think tables), and domains are composed of items (think rows). Each item is composed of attribute/value pairs, similar to a relational database's columns.  However, unlike relational databases, it is trivial to add new attributes to an item at any time and an attribute can contain multiple values.  For example, an attribute named "size" could contain the values "S", "M", "L", "XL", and "XXL". Data is queried and manipulated using familiar REST-like PUT, GET, and DELETE requests.

There are several main differences between SimpleDB and a traditional relational database system. First, items are not defined a priori - attributes are added when necessary to items, making them more flexible. Second, the data is not stored in a tabular data structure.  Rather, the data is stored hierarchically. This format is more flexible, and supports higher performance with little grooming required. Third, all data is stored as text. This puts the responsibility for domain integrity squarely on the developer - there will be no runtime errors warning of a violation of a field's intended data type. Consult the online Getting Started Guide for more juicy software details. The guide contains sample code all its operations in C#, VB.NET, Perl, and PHP. If you try reading the guide using IE (I use IE7), don't be discouraged by the abhorrent look of the pages as their functionality do not seem to be affected. Open the page in Firefox if you care how it looks.

Amazon charges separately for what it calls machine utilization, which translates to query cost, data transfer, and storage of the items stored in SimpleDB. The charge for machine utilization is $0.14 per "Amazon SimpleDB Machine Hour".  Typical queries require near-negligible time to execute. Inbound data transfer costs $0.10/GB, while outbound transfer is based on tiered pricing: $0.17/GB for the first 10 TB/month with lower priced tiers costing $0.10/GB over 150TB. I will be paying $0.17/GB. Finally, data storage is charged at the flat rate of $1.50/GB/month.

One drawback that will need to be considered when designing an application that uses SimpleDB is that each item attribute value is limited to 1024 bytes. Larger items should be stored in the S3 system with only a pointer stored in SimpleDB. The limitation is likely intended for to help with managing performance of the SimpleDB system, but it is also important to consider that the data storage cost of SimpleDB is 10 times the cost of S3.

For most attributes this will not be a problem. However, there are many cases where it makes more sense to store larger chunks of data in the database. For example, in an online retail sales application, the description of many items could go well over 1024 bytes. It doesn't seem reasonable that you would store a series of items, such as item descriptions for a retail sales site, separately in S3. A possible solution to this issue is to create several attributes named Description_01, Description_02, Description_03, etc., using as many as necessary to store the required production description. This functionality could easily be abstracted into a SimpleDB library.

I am signed up. When I can find time, I will convert portions of this site to SimpleDB to give it a spin. I will certainly build a sample app and share my experiences here.

DevX Article: Building Applications in the Cloud with Amazon Web Services

Article: Building Applications in the Cloud with Amazon Web Services

You know I love my Amazon.  Being so much more than a bookseller, several years ago, Amazon redirected its expertise in building big things to building a set of online services known collectively as Amazon Web Services. I was really excited about AWS's first offering, Simple Storage Service (S3). I wrote one of the first .NET libraries for interacting with S3 and published my work at DevX in May of 2006.

A lot has changed since then, but Amazon's work in the cloud has not.  Since S3, Amazon released its Elastic Compute Cloud (EC2), SimpleDB, and Simple Queue Service (SQS).  Each of these offerings are production class services offered by a company that has built one of the biggest online retail engines in the world.  It seems they have some experience in building big things.

This afternoon DevX launched a special report on Software as a Service (SaaS) and featured an article I wrote as part of the report. The article is a high level perspective on Amazon's offerings in the SaaS and cloud-based services market. Read it.

I plan to continue to write about Amazon's offerings and will probably pick back up on writing a freeware AWS library for .NET if there is interest. Let me know.

I Love Amazon.com

I was involved in an email thread recently where the following question was posed: "In your opinion, what are the top 5 'great' companies?" I hadn't considered the question before, but I was a little surprised when Amazon.com popped into my head without evening thinking about it.  I had to think for a moment to come up with the other 4. I didn't have to think too hard to come up with a couple more, but I was still surprised at how quickly and naturally Amazon.com came to mind.

I thought for a bit about why Amazon was so natural a choice for me, and I realized it goes way back for me. I have always loved reading books. These days, I focus mainly on science, technology, software, and the business of software, but I read some fiction as well. I have always loved popping into a brick-and-mortar bookstore and look around for a while.

When Amazon was first launched, I was an old school book buyer. I would go into a store, peruse the books on the shelves, read through the table of contents, and then scan a couple of chapters to gauge the quality of the book. Once I found a book I liked, I would go home and order it on Amazon.  It felt a little dirty at first, but the savings at Amazon were compelling. Still, it felt a little dirty.

As Amazon continued to improve, I clicked that lever again, and again, and again. Like Skinner's rats, I was being conditioned to depend on that sugary taste that made me feel good. Every experience with Amazon was perfect. Click. Every experience with Amazon got even better. Click.

As I used Amazon more and more, and as their site became more and more popular, I began depending heavily on their recommendations and reviews. While these services improved, Amazon's client base began to grow substantially, which further improved the recommendations and reviews. Before long, I realized my habit of needing to see and touch a book before buying it had slowly faded to the point where I was depending solely on reviews of books. I also found that Amazon's recommendations were suggesting many of the books I ended up purchasing.

Fast forward to today.  Amazon is no longer just a book seller.  It is not longer even just an online retailer. Amazon has built a set of online cloud services that give common developers amazing scalability and reliability at an astonishingly low price. Start-ups all over the country are using AWS to completely build their online applications.

Here are some of the reasons I love my Amazon:

Free Shipping

Well, not totally free. I am a subscriber to Amazon Prime, meaning 95% of what I order comes to me with 2 shipping for free. For customers who don't subscribe to Amazon Prime, Amazon offers Super Saver Shipping, which means free shipping (ground shipping) for most orders over $25. Because of the number of Amazon distribution centers, I normally get items that are heavily stocked in only 1 day.

No Sales Tax

I live in Tennessee, where we have a rather steep 9.25% sales tax. It may not last forever, but I appreciate no sales tax while it lasts.

Amazon Prime

Amazon Prime is a discount shipping program. For $79 a year, we get overnight shipping for $3.99/item and 2-day shipping for free. You can even share this service with other members of your household.

One-Click Ordering

I never realized how much I dislike going through a traditional online order process. Even with retailers that I have purchased from in the past, there is still lots of information to provide with each order. Amazon provides a way to order an item with a single click. Amazon wasn't the first retailer to offer this service, but they have certainly not stopped innovating. Amazon added a feature where you can select your shipping address and payment method from a single drop-down before you click to order. Try One-Click ordering using Amazon Prime for a great combination.

MP3 Downloads

I love my iPod, but have never liked Apple's DRM terms. When Amazon began selling DRM-free in MP3 format, I never looked back. Songs are either $0.89 or $0.99 and Amazon provides a nice player for samples and a great downloader tool.

Amazon Sales Rank

When I am looking for a new book, or trying to compare two books, I have found the Amazon Sales Rank very revealing in uncovering great books. Find a book on Amazon.com and scroll down to the Product Details section.  You will see how well the item is selling in its major category and in its specific category.  For example, the book A Thousand Splendid Suns currently shows that it is ranked #30 in Books, #1 in Books/Literature & Fiction/Literary, and #6 in Books/Literature & Fiction/Contemporary. I decided to purchase this book after seeing it ranked #1 in books for a while - I recommend it.

Amazon Marketplace

My wife orders lots of books for our children for school. In many cases, the books she purchases are not new, and are readily available in used form. Amazon does a great job of making such books available in its Amazon Marketplace.  The Marketplace allows sellers to sell their items through Amazon. Once you select an item, Amazon will display text such as "129 New and Used Available from $11.89." In many cases, you can get used items in great condition when, even with the flat shipping rate for books, makes them less expensive than the same new item directly from Amazon.

Amazon Web Services

Amazon has built a great set of cloud services known collectively as Amazon Web Services (AWS) that make it easy for developers to build great applications at a very low start-up cost. From its Simple Storage Service (S3), to the Simple Queue Service (SQS), the Elastic Compute Cloud (EC2), to its SimpleDB, Amazon has made available the building blocks of most enterprise applications in a hosted fashion, allowing developers to develop new products, not new plumbing projects.

I also have a wish list for Amazon:

Search Only Amazon Prime

In some cases, if the item isn't eligible for Amazon Prime, I will probably search for it locally. In such cases, I would really to enable a search option (it would be on by default for me) that would only return items eligible for Amazon Prime when I perform a search.

Better Better Together

I know the guys at Amazon are smart, so I just can't understand why I get some of the items listed in the Better Together area. If you don't know about Better Together, it will offer the opportunity to purchase two items together, occasionally at a discount. The problem is, I have seen too many examples of items listed in Better Together which seem mutually exclusive (two very similar digital cameras), or completely unrelated. I understand that this feature is probably implemented this by observing which items have been purchased together in the past. However, it seems that it used to work better than it does today. I want a better Better Together.