Implementing Agile - When User Stories Meet Ticketing Systems

There's lots of "Introduction to Agile principles"-type articles around the blogosphere, but not many I've seen that actually give practical advice down to the level of the real mundanities like ticket-logging. But these things are important - if you're not careful, your ticketing system can end up like an elephants graveyard, a confusing, mournful desolate place full of orphaned tickets for which no-one can remember what they're doing there, whether they're still needed, or even why they were logged in the first place.

So I thought I'd share the way we've been doing it. I'm going to concentrate here on feature requests only, as bug-reporting is a whole separate issue. The system we've been trying recently at iTrigga has helped us keep on top of our estimates and project mgmt, as well as our daily tasks. So here goes:

1) We agree a formal user story with the requesting person

e.g. don't just record "Ability to get a second opinion on an article", go the extra small step of wording in the form

"As a (type of user) I want to (feature) so that (benefit)"

in this example, it would be "As an editor, I want to get a second opinion on an article so that when I'm editing an article that requires specialist technical knowledge I don't have, I can get it checked by a domain expert"

Getting the level of detail right can take a little bit of getting used to, but is well worth it in the long run. The "...so that (benefit)" may sound trivial or obvious, but it's absolutely vital when you refer back to lists of tickets long after the initial conversation has faded from memory. 

We've also found it very useful to agree this with the requesting person. It can be quite trying to get the rest of the business to adopt a standard format for this - the push-back is almost inevitable ("You put it into your system however you like, I don't have time for this!") but you can use it as a double-check ("..so let me just make sure I've got this straight - is it fair to say As a sales manager, you want a report of job progress so that you can estimate your booked revenue for this month?") and if you keep repeating the same format back to them, gradually they'll start using it themselves :)

2) We log a ticket for each user story

The ticket name is the "As a (type of user) I want to (feature)", the first line of the description is the "..so that (benefit)." 

After that, we can add however much more context and supporting information we need, but the title and first line always follow that format so that we can always see:

• what was requested
• who it was for, and
• why they wanted it

3) We log sub-tasks for each change required to fulfill the user story

Let's take a quick whizz through the example above - what do we need to add or change in order to satisfy the "ability to get a second opinion on an article" ? Well, we'll need to:

1. add a button marked "Get a second opinion" on the editing page
2. add a way of specifying (or automatically choosing) who to get the second opinion from 
3. send an email to the second opinion guy telling them that their input is needed
4. record their opinion in some way (another form? comments?)
5. an email sending their opinion back to the editor who originally requested the second opinion 

OK, at that level, we can log sub-tasks and probably start estimating time requirements for these tasks. 

( Or can we? Well, there's a couple there that need more detail - 2 and 4.  So there'll be some discussion around those, and maybe even another level of sub-task below those two, or maybe a separate user story. But let's assume for the purposes of this post that we can move on.)

4) Each sub-task is logged in the form of RSpec examples

For instance, sub-task number 2 above should be logged as something like this:

"when a second opinion is requested, it should send a SecondOpinionRequested email to the chosen domain expert"

- which translates nicely into a functional spec:

describe "when a second opinion is requested" do
  before do
    # set up your situation
  end

  it "generates a message" do
    # e.g. something like this:
    expect{ subject.save }.to change( Message, :count ).by(1)
  end

  describe "the generated message" do
    let(:msg){ Message.last }

    it "is a SecondOpinionRequested mail" do
      # your test here
    end 

    it "goes to the chosen recipient" do
      # your test here
    end
  end
end

So this gives us a top-level task for the user story, and a set of RSpec examples for the changes needed to fulfil the user story. This is really handy when you come back to a half-complete job later on and want to check how much of it is still left to do.

We use Redmine for our ticketing, and there are a couple of extra plusses when you do it this way in Redmine:

• estimates for the sub-tasks propagate up to the top-level task (the user story)
• you can't complete a parent task until the sub-tasks are complete
• a parent task can't be higher-priority than its lowest-priority sub-task
• if you complete each sub-task as you get the spec to pass, you can easily see from the parent task how much is left to do (see figure)

5) Each top-level user story gets its own git branch

...and the branch name is (ticket #)-brief-feature-description.

It's tempting to just dive in and start coding, but a little discipline here pays off big dividends in the long run. I've lost count of the number of times I've had to break off even ostensibly five-minute jobs for any of the following:

• You're halfway through the ticket when the CEO tells you to drop everything and work on X instead
• You realise that you can't do this until ticket Y is done, and that guy's away 
• You have to break off and do an urgent fix 
• You have to look at an issue with someone else's unreleased code

etc etc etc. Whatever - remember this:

in Git, branches are cheap - there's no reason NOT to use them!

Putting the ticket number at the start of the branch name helps keep things in a sensible order when you list the branches, and keeping the feature description in there means you don't have to keep referring back to your ticketing system to remember which is which.

It also helps when trying to trace exactly which feature got merged into what, when.


That's probably enough for now - I'm sure there are many alternative systems, this is just one that works for us. All suggestions and comments welcome!

Getting Accurate Count on Grouped Arel Relations with Squeel

Let's say you have a scope on a model that joins onto potentially several related models:

class Post < ActiveRecord::Base
  has_many :comments

  scope :commented_on_by, lambda{ |user|
    .joins(:comments).where( :comments=>{:author_id => user.id} )
  } 
end

( OK, so this is a contrived artificial example, there are better ways of doing this - but the real example which triggered this post is way too complex to go into in detail, and we're all familiar with blogs, posts and comments, right? )

So if you call Post.commented_on_by(user) you'll get all the posts which the given user has commented on. All well and good. BUT - what if the same user comments multiple times on the same post? You'll then get the same post repeated as many times in your resultset.

So the logical thing to do here is to introduce a group by clause, to make sure you only get each post once, right?

class Post < ActiveRecord::Base
  has_many :comments

  scope :commented_on_by, lambda{ |user|
    .joins(:comments).where( :comments=>{:author_id => user.id} ).group(:id)
  } 
end

OK, that's that problem solved. Except.... what if you now want to run an aggregate query over that result set? Say, you want to count the number of posts the user has commented on?

> Post.commented_on_by(user).count
> {1=>2, 2=>1, 3=>2}

Er... whut? We get a Hash?

Well yes - because we've grouped the query, ActiveRecord will group the count results as well. It's saying "post id 1 has two comments by this user, post id 2 has one, and post id 3 has two". All of which is entirely correct, but a bit annoying if you just want to get the number of posts. OK, you could do Post.commented_on_by(user).count.size, but that kind of defeats the intended purpose of a count call, right?

The fundamental problem is the underlying structure of your query. You're essentially collapsing multiple rows into one with the group clause, and your SQL parser interprets it as "get me the count of rows for each row BEFORE it's collapsed" rather than after.

So, how do we get it to treat these multiple rows per-post as if they were one? IN() is your friend!

The good news is, you can get round this using Ernie Miller's rather excellent gem Squeel. We've been using Squeel a lot at iTrigga, as it helps us keep on top of syntax for some pretty complex queries.

One of the many nice things about Squeel is that it lets you supply ActiveRecord::Relation objects as a value for a predicate - so if you can restructure your query to use an IN() clause, you can supply your already-built Relation object as the value to it.

In this case, you can rewrite:

> Post.commented_on_by(user).count

as

> Post.where{ id.in(Post.commented_on_by(user).select(:id)) }.count

and the result should be correct