My name is Ryan Faerman

I make & break things – usually code and sometimes wood

Becoming a Carpenter

When my wife and I were looking for our first home, she joked with our realtor, “I’m looking for a house, my husband is just looking for a garage”. While not entirely true, it definitely had a source in truth. I grew up helping my Dad in the garage shop and I even had my own work bench that I never could keep organized.

I’ve always felt that a home has two hearts Just like a Timelord : the kitchen and the garage. When we bought our home, one of the first things I did was turn the garage into my shop. I started collecting tools for both home repair and to start building things from wood.

As a developer, I’ve learned to be pragmatic in regards to my skillset. Before having my own shop, I never really had a constant opportunity to practice woodcraft. I had built a few of my desks From scratch, not from Ikea.
One of my later desks was actually good enough to give to my Sister when she needed a table.
and a bookshelf or two, but nothing created at the quality level I wanted. I knew then that I could do better.

Mother’s day was only a few weeks after we moved into our home, we were having a family brunch and we needed a table for the patio.

Still feeling bruised from making the down payment, I threw together a rough table from some wood I reclaimed I even reused most of the screws too! How’s that for recycling? from a rickety shelving unit that came with the house.

Despite being thrown together one evening after work in the week before the party, the table served us well for several years and many patio meals.

I was never truly satisfied with the quick and dirty table. The joinery was rough, the edges were harsh, and it always required a table cloth to cover its less than beautiful tabletop.

Every year in the fall, we celebrate Succot by building a Succah. My Dad and I always put lots of thought into our Succah. Sometimes even planning it right after Passover, roughly six months beforehand.

It’s like an excuse to build a fort every year.
I tend to use regular 2x4s and then, after the holiday, I end up with a pile of lumber begging for a project.

The first year, I used the lumber to create a pair of wooden patio swings. These are still hanging in our patio and are one of my favorite places to take a nap. I modified some simple plans I found and banged them out pretty quick.

The second year, I stepped up my game when I ditched the Ikea table I was using as a desk and made a new one built into my closet. I hadn’t really learned how to joint boards together properly so I focused on how to get a really nice lacquer finish. The desk won’t win any awards, but I’m pleased with how it turned out.

With the finishing skills from the desk, a recently purchased planer, and another piece of disposable Ikea furniture to replace, I had my next project – a new end table. This was the first project that I consider actual furniture.

It’s nothing fancy.

end table

Just a simple table with simple angles. But it proved to me that I could actually make something nice.

I continued making random things. I made a few dice, a shelf, a couple frames. But I wanted something bigger. I wanted to make a full table. With summer coming to a close, and coming off a high of binge-watching some Wood Whisperer The Wood Whisperer has lots of great content. With most of his videos, even though I haven’t made what he has, he makes it seem attainable. videos, I decided that I was going to make a cedar table Specifically, the Rustic Outdoor Table from The Wood Whisperer. for the patio.

With Succos being just a couple months away, I gave myself a deadline. The first use of the table would be in our Succah.

The first trip to Home Depot I know there are better and cheaper ways to get lumber, but I can’t usually get to the one or two lumber yards in the county. They have strange hours and I wasn’t going to take a day from work just to make the trip. for rough cedar proved to be a challenge – they didn’t have enough in the dimensions I wanted that also was in the required quality. I ended up going to a few different stores throughout the project to get everything and I had to modify the plans slightly to compensate for the lumber I could get.

With the lumber purchased and taking up precious space in the garage, I got to work.

Each night, after my daughter went to sleep, I’d change into my overalls and spend a few hours in the garage. Each night I’d make a little progress milling and preparing the pieces for a sub-assembly. I’d put these assemblies aside to work on the next piece.

On each Sunday, after my daughter’s swimming lessons and after some family time, I’d spent 6 or 8 hours in the garage. These days let me make large strides, completing tasks like jointing the boards for the tabletop or leveling the tabletop and benches.

This continued for about three weeks and in the end, I had a completed table, delivered on time. I’d enjoyed every minute of the project and in the end, I created a piece of furniture that will last a lifetime.

outdoor table

After Succos, I dragged the old table out to the street corner. Within a few hours I got a knock on the door. A man wanted to take the table and to make sure I was really throwing it away. Things with a purpose tend to survive, even when we don’t intend them to.

With the patio cleared, my Dad helped me move the new table and benches to the patio where they have been serving us well.

We use the table frequently – breakfast on weekends and other meals when it’s cool enough. It feels like we use it more than the previous table. I’m not sure if that’s because of its novelty or because it’s a table that works better for us.

It’s a source of pride too. I’ve had many people ask me where I bought it; when they find out I made it they are amazed. I didn’t make it to get compliments, but that’s a nice bonus.

So far, I’ve used most of the lumber from Succos to make toys. I’ve made a handful of wooden cars for my daughter and a set of giant jenga set for us. There are a few boards left that aren’t earmarked for anything in particular.

I’m sure I’ll find a use for them.

Migrating to Hugo

I like to play with different technology to build websites. And, like a mad scientist, I generally experiment on myself and my own site.

My website was fairly recently powered by Middleman Middleman is great and all but I’ve become a gopher.
a happy gopher
and now it isn’t. I’ve moved everything over to use Hugo instead.

Hugo is pretty neat. Other than being written in Go, it’s also distributed as a single binary without any external dependency. I don’t need to get a ruby environment setup Don’t get me wrong, I like ruby but if you’ve ever had to resolve dependency conflicts, you know what I’m talking about. or fight with any dependencies.

Hugo, like the Middleman, Jekyll, and nanoc I used before it, is a static site generator. Despite being a software engineer and working daily operating an internal cloud, I’ve come to really appreciate having a pile of files. I can serve this with just about any web server with little or no configuration.

As an added bonus, since there isn’t any server-side generation, just about any server could handle an incredible amount of traffic.

I’m a realist, I know that no one is really looking at my content. Except you. You’re a somebody… yea! I don’t create it for most people, I write it mostly for myself. Knowing that, I want to spend as little as possible to host my personal site. The less it costs the better. To that end, I’m hosting this site on Amazon S3 and fronting it with CloudFront.

Yes. Using CloudFront for my little site is overkill but it’s the best way I could come up with to serve everything over SSL. Come on! Let’s Encrypt already!

Built on the Shoulders of Giants

First, we have the design of the site. I was greatly inspired by Tufte-css; when I first saw that page I was blown away by the typography and especially the side notes. I felt that the design supported the type of writing that I enjoy, where I can interject Saving literally dozens of dollars on parentheses and dashes. Punctuation doesn’t grow on trees you know! into my own writing.

A pretty face only gets you so far.

With the design and content generation figured out, it was time to get this up to Amazon S3. In line with the style of Hugo, I pulled two more tools off the shelf: s3cmd and awscli.

Sprinkle in a bit of bash Based off the work by Nicholas Whitter. , I now had a git post-commit hook done that:

  1. Compiles the site with hugo
  2. Uploads the site to Amazon S3
  3. Purges the index.html file from CloudFront

Done for now

Just like this wasn’t the first iteration of my site, this won’t be the last. After all the journey is most of the fun with this stuff.

A Fractured Community

I’ve been working on this post for a few weeks now – even longer in my head. This is a difficult topic for me to post but I feel compelled. There are too many rumors and half-truths floating about in the my community; I can’t sit by while a problem festers.

I’ve written this so there is at least a record of where I stand on things and maybe even a little bit of the why too.

Some Background

I live in Coral Springs and have been part of the Chabad community for many years. I’ve lived in the city for longer, but my family and I really started getting close to Judaism about 15 years ago. Before going to the Chabad, we attended a local reform temple.

The story, as I understand it, started when my Dad found his Father’s tefillin and wanted to put them on. He asked the Rabbi of the reform temple and the Rabbi didn’t know how.

Coral Springs, like most cities, doesn’t have much competition in the way of observant jewry. It’s either Chabad or nothing. That’s a subject of another discussion – it’s neither good nor bad, it just is.

My Dad went over the the Chabad (it was just down the street) and they not only knew how, they helped him. The Rabbis were open and warm, the community close, and we got hooked.

The rest as they say was history. We grew closer to judaism. I went to a jewish day school. We made our home kosher. And now I’m now a Torah-observant Jew with my own kosher home.

My family helped establish this community and we helped fund the construction of the building.

Growing up and becoming closer to Judaism in this community gives me a unique perspective to criticize it.

Noticing the Change

The community transformed from one of outreach and warmth to one of severity and closed-mindedness. I know this didn’t happen overnight. It has been a gradual process. I was just too involved with my own life and self-delusions (we all have them) to notice until it has gotten so far gone that I couldn’t help but notice.

Instead of an environment of acceptance and growth where whatever you’re doing is better than what you were, new-comers are insulted and demeaned. I know this is true because I invited a friend of mine to come join the minyan during Succot.

My friend arrived before I did and walked in. He was welcomed by someone saying “What’s wrong with you, why are you wearing jeans?” followed by a “How dare you have a cell phone in your pocket!”.

Instead of being happy that a jew walked in the door without being dragged, he was belittled and demeaned. That’s entirely unacceptable – it’s just not how you treat a fellow jew; it’s not how you treat a person.

My friend left, went for a walk, and stopped by later to share his experience. I was upset and ashamed. These were the people that I called myself a part of; I trusted my friend to them.

A Breakdown of Respect

There’s a general lack of respect in the community. A perfect example is the division during prayer. The sanctuary is generally divided into two groups, classes really.

The upper-class are in the front. They sit at tables, they usually are black hatters, and they generally wear the standard Chabad uniform. The lower-class sit at regular chairs in the back. They come from all walks of life and are at varying degrees of observance.

The tone at the shul is that if you don’t keep the arbitrary standards of the front-dwellers, you’re somehow a lesser Jew.

There is a general lack of respect for the building and its facilities by the community as a whole. There is even less respect from the people that use it the most.

Children push the adults out of the way for food, fill their entire plates, and then just throw it on the floor when they are done. Not every child does this, and not every kid is as bad – but it’s an example and not far from the norm.

There’s not really a single part of the building that hasn’t received some attention from either my Dad or one of the few other community members that actually care about the building and its upkeep.

We’ve patched walls that people have kicked, cabinet doors that kids ripped off the hinges, repainted walls after food stained them, replaced door hardware, and so much more. We’ve had to replace the same things many times, not because they wore out, but because the same kid thought it’d be fun to break again. The parents are never held responsible.

Even with all the work, the sanctuary still smells of mold and body odor. I think it absorbed into the drywall.

Along these lines, the mikvah is a disgrace. The water is not clean, everything smells like a terrible combination of mildew and bleach. There will be bugs, hair, and other things floating in the water.

My wife absolutely refuses to use it, and we live less than a mile away. Instead, she drives 25 minutes away to Plantation where there is a dedicated women’s mikvah and a group of people that actually care about it.

Mob Rule

Around the beginning of the year a friend of mine (we’ll call him Frank) suspected that his child was abused at the day school. I can’t say if abuse took place or not, only that it isn’t impossible. I don’t know if the claim had/has merit or not. The claim has nothing to do with me and I’ve had to fight not to get pulled directly into it.

The only reason I mention this issue at all is because of the leadership (or lack thereof) and the reaction of the community and its members.

People in the community would come talk to me, demanding I stop talking with Frank. My wife would receive random calls from people trying to convince her that Frank and his family were bad people. They’d also tell me that abuse was “impossible” because the teacher was above reproach.

Frank and his family had at least one community member come to their house and start screaming at them from the street. They would receive calls telling them to leave.

If anyone was a friend to Frank and they had kids in the school, they would be threatened with a raise in their tuition. Even worse, they’d be threatened that their kids would be unable to get a shidduch.

If they taught at the school (or nearby schools) they were threatened with termination for not shunning Frank and his family.

Keep in mind that this entire time there was an on-going case in front of a Beis Din that hadn’t ruled yet. The community attacked Frank and his family for weeks (months really) and this whole time, not a single Rabbi in the community thought it worth even addressing.

In fact, they (and more often their children) would instigate – I even had to stop a physical fight from breaking out!

The last piece of straw

With all that happened, I started to really pay attention to all that was transpiring. I’d sit in shul and just watch the interactions between people.

It was second day of Shavuous and I had just arrived. I walked through the side gate and entered the court yard. I walked up to the doors to the sanctuary and started opening them.

I froze.

I couldn’t bring myself to enter.

I ended up sitting in the succah (there was a tiny one that the kiddush club made in the back that stood year-round) for a few hours, thinking.

I had decided then and there, that I was done. It was time to just start a minyan at my house of people that feel the same as I did.

How is this all relevant?

After my second shabbos minyan, I received a phone call from Rabbi Denburg (the head Rabbi of Coral Springs). He wanted to come over to “explain some things to me”. He made no mention of what that meant, but he wanted to meet in person.

After consulting with a Rav, I made sure to record the entire meeting (with permission of course). I also invited my Dad to join the meeting, and my wife was there too.

The meeting, as it turned out, was about the minyan I had started.

The meeting lasted for over an hour. He dedicated the first 15-20 minutes or so to bad-mouthing Frank and his family. Rabbi Denburg offered to show us the reports from abuse investigators (I’m pretty sure that’s not really allowed), from the police, and he even had partial screen shots of text message threads between third parties.

He even told us that he discussed the issue at length with his advisor (who also later happened to be a judge on the Beis Din) and shared Beis Din’s ruling. I thought this was more than a little strange that a long time advisor would be a judge in his case, but I didn’t and still don’t know the exact rules on the permissibility of judges.

Rabbi Denburg claimed all of this proved that there was no abuse and that he and the school were entirely innocent of any wrongdoing; that there could “not possibly be any abuse”.

Rabbi Denburg refused to stop until my wife, my father, and I had all pushed back asking him the relevance and demanding that he stop.

Rabbi Denburg claimed that I was trying to “break the community”, that what I was doing was “machlokes”, and that he even threatened to “talk to the community” about me and my minyan. He spoke like the stereo-typical gangster too! He actually said “it’s not a threat, it’s a promise”.

And here I was thinking people only said that in the movies.

He went on to claim members of the community as “his members” and “his donors” and that I was trying to “steal them” from him.

Most of everything I’ve written here was brought up to Rabbi Denburg as our reasons for starting our own minyan. That our minyan wasn’t the cause of the fracturing but a symptom.

We were, and still are, searching for meaning – it’s an ongoing process. We’re just trying to find a more meaningful way to relate with our Creator. That apparently doesn’t matter to Rabbi Denburg.

We asked him directly what he wanted from us. His words: “I don’t want anything”.

Honestly, if he had just asked us to stop we would have. We had/have no intentions of harming the community and we would rather have peace. If he had really thought it would hurt the community and just asked us without threats, we really would have stopped.

We would have found another way to find meaning.


The Chabad of Coral Springs and it’s associated institutions (Beis Chaya, Hebrew Academy, and the Mesifta) are run by a wanna-be gangster that threatens his opponents and thinks he can do whatever he wants to whomever he wants.

This community and it’s leaders have lost touch with the ideals of Chabad as I understand them: warmth, love, and joy. I’m ashamed that it took me this long to realize it.

I’ve been hosting the minyan at my house ever since Shavuous and I’d like to continue doing it. I’ve considered formalizing the minyan a little bit, maybe forming a Shteibl and raising money for kiddushes, a building, or a mikvah.

I don’t know if having a completely separate minyan is the solution to the issues within the Coral Springs Jewish community.

I don’t know if there is a solution, but I have to at least try something.

Branch Specific Databases in Rails

If you’ve worked on a Rails project for any length of time and you use feature branches (you really should be), you quickly run into migrations “leaking” across your git branches.

Sometimes it isn’t a big deal, a missing field here, an extra field there – you learn to ignore errors. This is a bad idea, since then you’ll just dismiss the important errors too.

A technique I’ve started using (and I’m not the only one) is to use branch specific database tables. It’s exactly what it sounds like: every feature branch has its own database.

What’s different about the way I’ve solved this problem from others, is that branching the database is entirely optional. If you want to start working on a branched database, you just rake db:branch and when you’re done, you can rake db:unbranch to clean it up.

Plus, if you’re not really working with migrations or just don’t want the hassle of branching your database, just don’t run the rake commands and the default rails pattern of APPLICATION_ENVIRONMENT gets used.

Installing Dependencies

The class below doesn’t have any external dependencies beyond Rails. Even if you don’t use Rails, it’d be pretty simple to modify it to suit your needs.

I throw this into /lib since it isn’t really project specific and I already have my lib directory auto-loading.

This only works for MySQL. If you’re using another ActiveRecord-supported database server, you’ll have to change #existing_databases to something that will spit out the list of databases.

class BranchedDatabase

  def current_branch
    `git branch | grep "*"`.chomp.split.last.parameterize

  def existing_databases
    if ENV['DB_PASSWORD'].present?
      password = "-p#{ENV['DB_PASSWORD']}"
    `mysql -u #{ENV['DB_USER']} #{password} -h #{ENV['DB_HOST']} -e \"show databases\"`.chomp.split

  def name(prefix: application_name, separator: '_')
    branch_database = [prefix, current_branch].join(separator)
    if existing_databases.include?(branch_database)
      [prefix, Rails.env].join(separator)

  def application_name'::').first.downcase

Updating database.yml

With the prerequisites in place, it’s time to modify the database.yml to use the detected branch name.

  database:  <%= %>
  <<: *defaults
  database: myApplication_test
  <<: *defaults
  database:  myApplication_production

Even though I could use the branched naming scheme for the production environment, I prefer to be explicit here.

Rake to the Rescue

Since the branching feature is optional it opens an interesting can of worms. The standard rake tasks for the database won’t work because returns the branched name only if the database exists. It doesn’t yet… because we haven’t created the database yet.

It’s your classic Catch 22.

Instead of relying on the supplied database rake tasks, I reach into what these tasks are doing behind the scenes (sadly, this ties it to Rails 4.1 or so) and use the task classes directly.

namespace :db do
  desc 'Create branch specific database and load db/schema.rb'
  task :branch => :environment do
    [:development, :test].each do |env|
      config = ActiveRecord::Base.configurations.values_at(env.to_s).first

      if env == :development
        bdb =
        config['database'] = [bdb.application_name, bdb.current_branch].join('_')


      if env == :test
        ActiveRecord::Schema.verbose = false

  desc 'Remove the branch specific database'
  task :unbranch => 'branch:remove'

  namespace :branch do
    task :remove => :drop

    desc 'Remove then Create the branch specific database'
    task :reset => [:remove, :branch]

It’s important to note what the rake tasks do not do: any sort of error checking. If you’re in a different branch (like develop) and execute rake db:unbranch you will drop your development database.

You’re a big boy developer, you don’t need your hand held – don’t be stupid and you’ll be okay.

Getting to Work

With that, we’re done - just a matter of using it.

git checkout -b feature/new-feature-branch
bin/rake db:branch
git push -u origin feature/new-feature-branch
bin/rake db:unbranch
git checkout develop
git branch -d feature/new-feature-branch

Happy Hacking!

Gate (In)security

About a month ago, my wife and I moved to a new apartment (technically, it’s a condominium that we’re renting). It’s a gated community complete with RFID access cards and one of those boxes where visitors dial a code to ask us to let them through the gate. Thankfully, the gate isn’t tied to a landline and can be associated with a cell phone - it’s very modern.

There are a couple shortcomings with the system: you only can connect one number and you can’t (or rather, the management office won’t) setup a generic access code to give to friends and family. The former is a minor, manageable annoyance; it has been manageable thus far but it is far from ideal. The latter issue is more (but not much more) of a challenge.

My wife and I are observant jews, and as such, keep shabbat and don’t answer the phone. We often have guests (some do and some don’t keep shabbat) for the shabbat meals and this gate situation was really hampering our style.

Enter Twilio.

Twilio provides an API to programmatically handle phone calls and text messages; after a quick signup and a few lines of TwiML, I had a phone number that picks up a call and “dials” the code to open the gate. If I so desired, with a few lines of ruby code, I can have an access control system that I manage.

I can understand the position of the HOA office. I wouldn’t want to open the pandora’s box of supporting an access code management system beyond what is currently in place and I certainly wouldn’t want to deal with supporting a self-service system either. That being said, I wish they would have given us a general access code (even if they told us not to share it), obviating my need to bypass the security system entirely. My wife requested such a code and, only after a decent amount of convincing, they would only give us a 24 hour code as a rare exception.

The ease that I can bypass the “security” of our gated community just goes to show how little security it provided in the first place. Despite giving the appearance and (false) sense of security, I suppose that the true purpose of the gate traffic control, not access control and that true security, as always, is up to everyone using the system.

Building a Boxing Gym

For a handful of reasons, my Dad decided that a punching bag in the garage wouldn’t be such a bad idea. Since I have a pretty decent background in martial arts, I acted as the consultant for this project.

My martial arts background is mostly with Muay Thai and Krav Maga over standard boxing, so the setup we went with reflects that. We went with an MMA sized heavy bag since Ayelet and I both enjoy kicking the bag (we’ll be moving closer to my parent’s house in June, so this saves us gym membership). I find leg strikes to be a great workout that’s unlike anything else and wanted the equipment to allow for it.

Movement is very important so we didn’t get a steel tube bag stand, since it’s difficult (or impossible) to move all the way around the bag. Using a stand also prevents a partner from getting behind the bag and holding it which is pretty helpful when working certain techniques.

By the end of the day, we had an MMA sized heavy bag hanging from the ceiling, with MMA mats forming a nice workable area around it, and a clean, open garage to work in. We went with Title equipment all the way around since their stuff is some of the best available and their customer service is pretty good too.

Phase 1: Clear the Pinball Machines

My dad had five pinball machines in the garage. His favorite machines were in the game room, inside the house, and these garage machines didn’t get played as much. We needed the space, so these needed to be packed up and moved to storage.

If you’ve never moved or packed a pinball machine, let me tell you, it’s heavy work. Each machine weighs 300-400 pounds and, when using a regular truck, you have to get that weight up into the bed; the bed is easily 4.5 feet off the ground. You also have to get it off that truck bed gently, otherwise you’re likely to break something.

This was done two at a time - pretty much all that could fit comfortably in the bed.

With the pinball machines out of the way, we could pull all the extra wood and other junk that had accumulated out of the garage. This mostly came from the leftovers from projects since the last time the garage was cleaned. With the freshly collected garage junk and some junk we pulled from the storage unit, we could finally make a run to the dump.

The trash was out of the way and space was reclaimed from the pinball machines, so we could move into Phase 2.

Phase 2: Organize & Prepare

To maximize space in the garage while the pinball machines were there, my Dad built what my Mom nicknamed coffins. These were just large black boxes on casters made of plywood that were sized to fit directly under the machines. Since the machines were gone, the coffins would need to go too.

We put together a shelving unit and re-organized the garage aiming to clear as much floor space as possible. The contents of two coffins barely used three shelves after some minor purging of anything no longer needed.

With everything cleaned up and organized, we could unpack the heavy bag and its hardware to double check any parts we’d need from the hardware store. We planned on picking up some lumber to fabricate the mount which would be spanning several ceiling joists and it never hurts to double check everything. Other than the lumber, we just needed some quick links to shorten the top chains and connect the floor D-ring.

Dinner time was rapidly approaching, so after a quick jaunt to the hardware store, we took a break to grill up some burgers; we could move on to Phase 3 afterwards.

Phase 3: Mount & Hang

Finally, the long awaited time was upon us - we could fabricate the mount and hang the heavy bag. The mount is a pretty simply design, it’s just a 2x10 extending from a 2x6 which would span several joists. 100 pounds hanging over a single joist isn’t too much a big deal; but hanging over more is better.

After spending what a pretty decent amount of time getting the mount into place and through a hole in the ceiling - it was done. The mount was installed. The only evidence (after cleaning all the insulation and sheetrock bits) that we actually did anything being an eye-bolt through the middle of the ceiling.

Despite being called a “heavy bag”, it isn’t really very heavy. I was able to easily pick it up while my Dad attached it to the ceiling mount and got the chains handled. We used some quick links to “short-circuit” the hanging chains to bring it a little higher off the floor.

With the top mount complete, it was time for the floor mount. A hammer drill, masonry bit, and a few tapcons made quick work of the floor mount which then was quickly attached to the bag.

After a long day’s work, the bag was fully mounted and it was time for a few test blows. Thankfully, the ceiling mount was fairly solid and it all worked just fine. As a finishing touch, it was time to put down some MMA mats to make things a little more comfortable as I like training bare foot. These went down fairly quickly, and it was straight-forward to get it all looking great.

Ryan Faerman standing next to the heavy bag


The bag is up and pretty solid. My Dad purchased a fairly nice interval timer which will help with some of the training I have planned. There’s also some focus mitts for working technique 1-on-1 when the bag isn’t appropriate

All said and done, we have the start to a pretty solid Boxing/Muay Thai gym that we should get plenty of enjoyment for a pretty good time.

Upgrading my CentOS Server

My main server has been growing rather long in the tooth (CentOS 5.3) and until recently, I’ve been putting off actually doing something about it, since I’m just lazy. There were always a few “essential” domains and services that I couldn’t risk going down and until recently, I just accepted that – Until I couldn’t anymore; A few things forced my hands.

  1. Nginx doesn’t really operate well with CentOS 5.3; I couldn’t get it to run at all.
  2. Node.js started giving me issues with all the various cruft and versions of it that I’ve experimented with over the years.
  3. I kinda, sorta deleted my entire sites directory (I had backups!) which made me notice all the junk I’ve had in my trunk as I restored things.

Despite having less time than ever to spend on my server, I actually have less worries about things now than before, due, in part, to moving some of my critical DNS and such to Amazon Web Services. My website, for example, is hosted on S3 and generated on my end with Jekyll; I’ve moved the DNS for my primary domains to Amazon’s Route 53 (it has a 100% uptime SLA!). The only downside of hosting my domain on S3, is that you can’t point a naked domain to your S3 bucket; which means I still need my server to do a simple redirect from to I also can’t go without the dub-dubs, but the benefits far outweight the downsides here.

Enough introduction, on to the meat and potatoes.

Backup all the things!

Backing up is probably the most important step.

The things that I backed up before doing this potentially catastrophic system upgrade were:

I created tarballs or dumps where appropriate, downloaded them locally, then uploaded a copy to S3. I’ll be keeping these backups there for a month or two - basically until I know things are stable enough.

Upgrade all the things!

With backups out of the way and my mind mostly at ease, its time to run the monster command that starts this whole adventure rolling:

yum update

Of course that hit a quick snag. Yum failed with a bunch of nasty error messages as soon as it hit updating some perl stuff. A quick search later, and I discovered that this is a fairly common error that is easily resolved.

yum clean metadata

After that, I ran yum update again and after a few minutes, I was asked to confirm a 400+ megabyte download. Once that finishes its thing, it is time for one of the scariest commands: reboot. Giving the server 5 minutes or so to reboot (and being just slightly nervous), I give it a quick ping and it’s back.

Installing all the things!

While I’ve gotten everything setup by manually building & configuring each service (I did take notes for this post, but they are verbose, boring, and soon to be outdated), I’m learning to use Chef to take care of it all in the future. Chef can also be used with Vagrant, a really slick VM provisioning tool that I’m looking to use as a development server. Setting up my mac isn’t too much of a challenge, but it will let me switch machines a little faster and make collaboration easier.

Chef should make everything much simpler and easier to replicate when required and gives me much more flexibility should I ever decide to use something like Amazon EC2 (which I am considering).

Even though git-deploy to push my various projects live, I’ve been using Capistrano at work, and have really grown to enjoy using it. I’ll likely move my projects to use it after dealing with the server stuff.

Older Posts