Webistrano and EC2
Posted by on November 11, 2009 in Technical DevelopmentWhen we first deployed our system we had a very simple Web->Database server architecture: we had one web server talking to a single database server deployed with Webistrano. As the site grew we quickly outgrew this setup. We decided to build a dynamically scaling system on top of Amazon EC2 to make sure we would never have to worry about adding or removing hardware.
Our EC2 system will spin up and spin down web servers based on a set of metrics that we collect on a continuous basis. This presents some fairly interesting challenges with Webistrano. While Webistrano provides a great set of features it’s obviously designed for deploying to a fixed set of servers.
We looked at a variety of options for deployment but we wanted to stick with Webistrano for the following reasons:
- Convenient central store for recipes
- Easy integration into our source control system
- Tracked and repeatable deployments
We are also firm believers in not reinventing the wheel, so how did we work around the static nature of Webistrano? We leveraged the power of open source :) We added a RESTful API call to Webistrano that allows a remote host to add or remove an ip address to a Webistrano deployment stage.
Each instance of our application has a backgroundrb task that hits this web service every minute. Consequently the list of active servers is always up to date. Not only that, but when our images start up they will deploy the latest released version of the code. How do we deal with servers that die? We added a Webistrano backgroundrb job that that removes hosts that have not been updated within the last five minutes.
What does all this mean? Our deployments always go to our currently active servers and our system scales on demand.
You can follow any responses to this entry through the RSS 2.0 You can leave a response, or trackback.
I’m pretty much in the same situation you are but about 6 months late. I was just going to write the API to webistrano connecting it to EC2 (or build the EC2 functionality into Webistrano) but just noticed your post. Any chance your code can be released as open source so that others can benefit from it?
Hey Cybertoast, we totally agree with you and have always thought about giving back to the open source community. Besides our changes to webistrano, we have made many changes to our infrastructure so that it’s very streamlined for a cloud environment. We have also done a lot of work in the area of monitoring. Like all things the biggest issue right now is resource allocation. We have so many projects to implement so its tough to prioritize this.
If you would like, how about you PM us at pingtre@mskynet and mention this blog. I can put you in touch with the dev who implemented this change and maybe we can share some of our code if you can share with us a little about what you intend to do with it. Looking forward to hearing from you.