Next phase: Improvements
After running for a few weeks, we started to really feel some pain points.
Chef roles take effect immediately on save.
This makes phased dev/prod/qa rollouts tough. Say we have a new recipe we want to add to the "portal" runlist. It would be good to test that in dev first.
We started putting feature flags into recipes, so that they would only run if an environment attribute was set. But really, what we wanted was versioning of roles.
A speaker at ChefConf had the answer for us: Roles cookbook. Rather than having your runlists in the role, have a Roles cookbook. Each role has a recipe that includes anything that role should run.
That lets you pin your environments to a particular version of the Roles cookbook, effectively giving you
Windows is slow. Always.
Bootstrapping a new Windows instance from scratch takes about 35 minutes... so long that you really just can't adapt to dynamic processing needs.
We knew that ultimately we needed to start building fully configured AMIs to scale off of, but we didn't have the devops resources to do that. So, for now, we settled on launching a ton of instances after a release, then stopping most of them.
When we needed more power we could start the instances, which would have them online in 5 - 10 minutes.
Each release we'd start up all the stopped instances, deploy code, then stop a bunch again.
No comments:
Post a Comment