How to make the ultimate Heroku/Trello/Github/Salesforce/Hipchat agile development integration - Part 1

About two years ago I was asked to assume the role of "scrum master" for the company where I work, Samanage. I didn't really know what to expect and how to approach this new role. At first I thought that my main objective was to just make sure everyone in the engineering department knew their current task, how long it will take, and what their next task will be. As time passed, I realized that my mission should be problem solving and making sure the development process runs smoothly in all aspects.

Master Status and the birth of SCRUManage

Like most SaaS companies we have three environments: Development (local machines), Stage (a similar environment to the actual live production environment for testing purposes before a deploy, usually deployed from our 'Master' git branch) and Production (the actual live product, deployed from our 'Production' branch, usually after having the Master branch merged into it). As we moved from deploying once a week to continuous deployment, and as our team grew significantly, we needed a tool to keep track of our 'master' git branch. Mainly, we needed to know at any given point if the Master branch was ok to be merged into the Production branch and then deployed to the production environment. If there was an issue with the Master branch and a quick fix had to be applied to the production environment, we could just apply the patch to the Production branch and deploy it without merging Master into it.
I decided to take this project on and find a solution. I needed a simple server that everyone on the team could have access to and would be very simple to update and get data from. I had recently played around with Heroku and decided to deploy a very small Rails project which will keep track of the Master branch. The project started out simple. A developer sends a PUT http request with a predefined token, the developer's name, the message and the color to use when printing the result in the terminal. To get the status, all that is needed is a GET request with the token.

Here is the code that I use to set the status.

#!/usr/bin/env ruby
require 'net/http'

host = "name_of_app.herokuapp.com"
username = `git config --get user.name`.strip.gsub(" ", "_")
path = "/master_statuses/1?app_token=123456789&current_status=Master_is_dirty!_DO_NOT_DEPLOY!&color=red&username=#{username}"

req = Net::HTTP::Put.new(path, initheader = { 'Content-Type' => 'text/plain'})
req.body = "whatever"
response = Net::HTTP.new(host).start {|http| http.request(req) }
puts response.body

In order to keep everyone informed and make their lives easier, I wrote a quick script which added a git hook to automatically get the status of the master branch after every pull. If someone wanted to mark the master branch as dirty all they had to do was run "dirty_master" or "clean_master" in order to mark it as clean. Here is how it looks:

My team really liked this feature and I decided to expand it further. The project, SCRUManage, was born.

My stage!

A lot of times a developer needs to test his work on the stage environment before merging his branch into master or for any other testing purposes. This meant that somebody either had to go around the office, or ask in our Hipchat room if they could use the stage. This was not only annoying but it was also not very effective. A lot of times someone would not hear/see that someone else needs to use the stage and would deploy over their work.

I decided to expand the Master Status idea and added the 'my_stage' and 'not_my_stage' commands. The result looks like this:

Again, these messages are automatically pulled from the SCRUManage servers after every pull from github. I decided to automate things just a bit more. When someone began a deploy to stage, the stage would be marked as his automatically. I did this by adding the following lines to our Capistrano recipe:

before "deploy", "scrum:my_stage"

namespace(:scrum) do
  desc "Tasks related to Scrumanage"
  task :my_stage do
    run_locally('my_stage') if ARGV[0] == "staging"
  end
end

After a while I decided to harness Hipchat to my cause and now whenever someone marks the master branch as dirty/clean or the stage as his/not his, a Hipchat room message is sent:

In part II of this post, I will explain how we synched our Trello 'Engineering' board with Github's pull requests and our deploy process. I will also explain how our sales team uses SCRUManage to sync their Salesforce account with their Trello 'Sales Opportunities' board.

How to keep track of Rails rake tasks

Using rake tasks to manipulate your database is a very common practice. A lot of times your code will change along with the database, making the local development environment fail for others until they run the corresponding rake task.

I work at Samanage with about 15 other developers. This means that each time someone creates a new rake task he has to tell others to run this task or email/IM them. Not only is this annoying, it's also error prone. Last month, a colleague of mine went on a month and half long vacation. When he returned, he couldn't load any page in his local environment. As it turns out, he didn't get the messages with which rakes to run and we had to 'hunt' these tasks down. It wasted a lot of time for both of us and was pretty frustrating. This ordeal made me think that we really needed a solution here.

Another problem we had with the rake tasks was that sometimes problems can occur which can cause pretty nasty things to happen. A colleague created a rake task which was designed to instantize a model, update some data and save the model. Unfortunately, that model had a callback which caused a lot of emails to be accidentally sent. In order to prevent this from happening in the future, we thought of seven points to be considered when creating rake tasks. We agreed that the best place to have this 'checklist' is at the top of every rake task file so the person who is code reviewing that task will also see the checklist and have it in mind while reviewing the code.

I remembered that there is a ruby gem that I love called 'O RLY?' by Yon Bergman which let's you know if you need to run 'rake db:migrate' or 'bundle install'. That's when I realized that if I could mimic the way migrations work, I could solve the rake tasks problem. Enter - "Rake Migrations".

So here's how I solved our 2 problems. I created a model called RakeMigration and a table with just a 'version' string (just like with the schema_migrations table). Next, I needed a way to automatically generate the rake tasks with a timestamp in the filename, the checklist at the top of the file, and the code to update the rake_migrations table with the timestamp of the file. I looked around a bit and quickly stumbled upon Rails Generators. Rails generators let you create custom generators to generate any kind of file, model, etc. I googled a bit and quickly found a way to override the default "rails generate task" command, and created my own template.

Now whenever somebody wanted to create a new rake task he simply typed the same default rails generate task command:

rails generate task <namespace> <task>

And that would generate the correct template. Here's an example (you can use 'g' as shortcut):

As you can see, a file was added with a timestamp to a (new) directory called rake_migrations. All your new rake files will go there.

This is the content of this file:

As you can see, we have the checklist at the top and line 16 updates the rake_migrations table.

The final part is to let my colleagues know that there is a new rake task. I wrote a script which was invoked by the git hook 'post-merge'. That means that if somebody merged a branch with this file in it, the hook would be invoked and the script to check the rake files would run. If a rake task needs to be run, the following message would print in the terminal:

After running the rake task, that message will not appear anymore.

I let my colleagues try this feature out for a little while and they loved it! I was encouraged to make this into a gem and share it with the world, so that's what I did. I googled some guides on how to create a gem and voilà, the 'rake_migrations' gem was born (my first gem!). Here is the link to it's Github page.

Feel free to let me know what you think and post some suggestions.

Enjoy,
Eyal.

3 Alfred workflows to make your Rails developing experience easier

If you are developing on a Mac there is a good chance you already know Alfred. If you don't, you should really head over to Alfred's website and check it out. Basically it's an app launcher for OS X but one that can do a lot more! You can extend it's functionality with "workflows". Workflows are very powerful "macros" which let you do, well, basically just about anything. I use Alfred all the time and I keep adding and creating workflows for it regularly. In this post I want to tell you about 3 workflows which I have created which really help me in my day to day Rails programming.

The first workflow is a simple one called 'Insert pry'. I use pry for debugging my rails projects. It has it's pluses and minuses but overall I find it to be the most comfortable for me. In order to add a breakpoint in a ruby/rails for pry, you need to type "binding.pry". With this workflow, all I have to do is press CMD + ALT + P and it will automatically insert a breakpoint to the app I am currently using. Yeah, I know, it's not really life changing, but it will save you a little time throughout the day. :)

The second workflow is called "Insert DOOM". A lot of times when designing new features, the text for these features is only ready towards the end of the development process. I used to just write a few words like "this is the text" but I found out that adding a paragraph helps visualize how the page will look when it's done. So I created a workflow which inserts the first paragraph from Lorem Ipsum. This worked well for me for a while until I stumbled upon this blog post. The post suggests that by using a text like Lorem Ipsum, you are reducing the text element to a visual design element (like a shape). Sure this was better than just writing "this is the text", but I knew I could do something better. So I decided to take my workflow and change the text to the first paragraph from wikipedia about the game DOOM (best game ever). This way I had a few lines of real text and whenever somebody sees this text in a form or paragraph they read it and smile, and not just skip it. I use CMD + ALT + L to insert the text.

The last workflow is "Terminal Rails". I use ZSH for all my terminal uses. I noticed that many people have at least 4 tabs open when developing a rails project:

1. Rails server
2. Rails console
3. A prompt for git commands, ssh, etc.
4. Guard rspec

I noticed that this configuration of multiple tabs caused a 'big' waste of time when having to switch through them. That's when I decided to make this workflow. This workflow has 2 functions:

1. If you launch Alfred (usually alt + space) and start typing 'rails' you will get an option to launch this 4 tab configuration automatically. I programmed the workflow to start the rails server and console but not to start the guard (I don't like guard running when I don't need it). The 4 tabs will also have the corresponding name in their tab title.

2.After launching the 4 tabs, you will be able to switch to them from anywhere (not just while the terminal is focused). I have set the following shortcuts for this:

• rs - Will switch to the rails server tab
• rc - Will switch to the rails console tab
• prompt - Will switch to the prompt tab
• guard - Will switch to the guard tab

I use these workflows daily and they save me a little time everyday. I think the reason I like them so much is that I don't have to take my hands off the keyboard, grab the mouse, look with my eyes for the terminal, click on the tab, etc.

I encourage you to use Alfred workflows and create new ones yourself. Here are a few resources that will help get you started:

• Alfred forums - The main forum for sharing workflows and getting tips and ideas for making new ones.

This github page has a nice collection of workflows.

How to create your first Alfred workflow.

Have fun!
Eyal.