Engineering a New World

Something Can Be Done About It

Cowie from Etsy – Cooking with Chef

Cowie from Etsy - Cooking with Chef by tadnkat
Cowie from Etsy – Cooking with Chef, a photo by tadnkat on Flickr.

John Cowie from Etsy gave insight on how they use Chef at Etsy. This is a turbo-overview of his talk and my impressions.

First, as a note, Etsy prefers running on bare metal as opposed to the cloud.  There are cases where working in cloud is better, but not EVERY case as some techno-pundits seem to evangelize.   Not that some managers I’ve dealt with have this idea, but just know that the ENTIRE WORLD IS NOT MOVING TO AMAZON.  They’ve got around 800 servers and they’re real servers – not VMs and not AMIs.

Some rules of thumb for dealing with Chef they’ve got:

  • Never test chef in production
  • Keep things as simple as possible
  • For metrics – they use Chef handler to send data to graphite (in github.com/etsy/chef-handlers.git
  • They push their chef failures to IRC.

With respect to handling of Failures:

  • Use “knife node lastrun <<hostname>> to get what happened on the last run
  • gem install knife-lastrun, then install on client.rb to get this data.
  • Try to keep conditions simple if possible – not huge regexes, try to keep to simplicities if possible to help your readability at 3am.

Standards in Chef:

  • Foodcritic: This is a tool for enforcing rules & standards on Chef, and it sounds FULLY RAD.  http://acrmp.github.com/foodcritic/
    • Foodcritic integrates with Jenkins (!!)
    • Supports custom rules
  • Etsy standards (guidelines)
    • Never have chef auto-upgrade packages
    • If you want to send an action:restart — RELOAD instead of restart
    • (foodcritic can enforce)
  • Some more rules of thumb:
    • Don’t take Opscode’s word for it – if it doesn’t work for you, change it.
    • 41 people have Chef access, most have keys to push to prod
    • There is an unconstrained Test env.  NEVER test in prod
    • They tweaked the “environments” workflow with some tooling
      • SPORK!  (knife-spork)
      • knife-spork is a wrapper around environments
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Information

This entry was posted on June 25, 2012 by in technology and tagged , , , , , .
%d bloggers like this: