Is Amazon EC2 + S3 a disruptive technology?
24. Aug, 2006 
7 Comments
I vividly remember talking to Werner Vogels a year and a half ago when we went out for lunch when I was in Seattle for a visit. Werner’s job was to go to Amazon to enable global-scale computing. Sure, they were already doing global-scale computing for their e-commerce properties and partners but I had no idea that what he really wanted to do was enable global-scale computing for everybody!
Today, Amazon announced their Elastic Compute Cluster:
Just as Amazon Simple Storage Service (Amazon S3) enables storage in the cloud, Amazon EC2 enables “compute” in the cloud. Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon’s proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use.
Capacity planning and operations management are things that are often ignored at startup companies. When all of a sudden traffic spikes and you’re unable to scale until the new servers arrive next week you’re screwed. But the ability to add additional capacity in minutes is an enormous leveling force. Also, the ability to scale up and down is just as important. You can greatly reduce your operational costs by not paying for the compute power until you need it.
Makes me want to do a startup company now 
Update: It looks like someone has already setup a public app. You can look at this really simple Rails application running live on EC2. Really, really cool.
Update2: Jon Udell has now made a screencast showing a Python application running identically on both a local machine and a machine in EC2.
Ning was the first “web 2.0″ approach to application hosting, with their own Google GFS/BigTable/GData-like approach to data storage.
It’s no great surprise that Amazon are doing this. It will be even less of a surprise when Microsoft do it – after all, they’re a platform company. And because Microsoft do it, Google will have to do it too to stay in the game (MS won’t let you mashup with Google services, so Google need an outlet for all their services)
The interesting thing will be what languages each company use. Ning are PHP, Javascript and Ruby people. Google are Python and Java. Will Microsoft allow any .Net language?
Thinking out loud, is this why Danny Thorpe is now at Microsoft?
I had no idea that Ning is in the infrastructure business. I thought they were just another hosting provider with a friendlier front-end. I did a quick Google to see if they had a scaling story, but I can’t find it. Do you have a link, Rich?
I can totally see MSFT getting into this game as well.
As for Danny, yes he’s working on enabling cross-browser, cross-web-services-platform mash-ups (ostensibly via widgets for the time being).
Ning and EC2 are similar in that they both want you to host your apps on their servers. They’re different in how they approach that. Whereas Ning provide a storage/query engine and an environment within which to code, EC2 is more like a QoS-based virtualization provider.
So EC2 is much lower level than Ning. Hence EC2 can be used for more than just web apps. But the Ning solution intrinsically gives you a source-code management system with branching aimed at web apps. Do people have a requirement for hosted non-web apps like EC2? I doubt it.
I’m sure that the science community (at least particle physics, where I come from) can use this. We used to build our own beowulf linux clusters for Monte Carlo simulation. Way easier to upload an image and run it for a finite amount of time.
Not sure how the economics work out, though ..
The interesting thing about the cost model is that they’re charging $0.10 per hour of time – not per hour of compute time. I think it works out nicely for CPU-bound computing since you can acquire as many cycles as you need, but you don’t have to pay for idle cycles.
It’s less cost effective if you’re running a web site that’s largely I/O bound.
Rich – I think that startup companies will be all over EC2. It gets you out of the infrastructure game and lets you focus all of your attention on building your app. It will be interesting to see how easy Amazon can make scale-out of your cluster (or what 3rd parties will step in to fill in that void).