[1+1=2]

OneAndOneIs2

« Your own elephantDark »

Tue, Mar 27, 2012

[Icon][Icon]Git to fogbugz with perl

• Post categories: Omni, FOSS, Technology, Programming, Helpful

(It's titles like ^ that lead to comics like this.. sigh)

We use Fogbugz for all our bug tracking needs at work. Well, I say *all* - we also have RT and bugzilla knocking around. But mostly we use FB.

We also use git as our VCS. So far so good, except we wanted to get some sort of integration going between git and FB, such that any commit made for a specific FB case would be viewable from FB.

Now, FB does have source control integration. But not for git. For CVS, yes. For SVN, yes. For Mercurial, definitely. But not out-of-the-box support for git.

It DOES have "generic other VCS" functionality, though, so all was not lost. And there were a few promising google results for "fogbugz git integration". But sadly, they were all GitHub-specific, or didn't really do what we wanted. So we decided to roll our own.

When you tell FB about a commit, it displays a "Checkins" section in the case sidebar, which opens a popup showing the name of the file changed in the commit, and the commit identifier - in the case of git, the SHA. Because some of our commits unavoidably have a lot of files, we didn't want to have all the files in there. So we decided to lie about the filename, and send in "date time user" instead.

And then also, we wanted the commit message to be added as a comment on the task itself, so you had more to go on than just a meaningless 40-character SHA when looking at a task without having to go to GitWeb.

So.. nothing too tricky there, surely?

So, before you can write the script, you have to do a bit of setting up in FB. Specifically, you need to tell it about the repos you want to track.
Admin->Source Control->Create New Repository
and add the repo name(s) you want. This will assign a number to each repo - you'll need to know this for your script. Having GitWeb or some other web frontend to Git is also useful if you want the SHAs to link to something more in-depth, which you can set in the "diff url" so long as you can work out the link format - something like
https://yoursite.com/gitweb/?p=repo.git;a=commitdiff;h=^R2

With that done, you now need to setup a git hook that'll run the necessary script for each commit. Hint: Hooks are not repo-specific, they're based on the files in /usr/share/git-core/templates/hooks/. If you set these up to be links, then all future repos created will also have their hooks set as links to those files, so will get any future updates automatically.

So, create the "post-commit" git hook and then write a script that'll do what you want. In our case, we needed to make two requests to FB - one to the source control; and one to the XML interface to edit the case, adding the commit message. If you've never used git hooks before, you may be expecting some kind of tricksy voodoo syntax. There's none - any executable script can be a hook: The name controls when it gets run, the contents of the file control what gets run. It's absurdly simple - you expect it to need some specific language or have some defined syntax for getting data from git, or something, But there isn't. I get the data I need about the commit using "git show", it's that easy.

We wanted to get a couple of things from git that are slightly outside of normal usage: the timestamp in "yyyy-mm-dd hh:mm:ss" format rather than the default "Tue Mar 27 11:39:00" because it's so much easier to parse, which we get by passing git "--date=iso"; and what repo this commit is for, which we find from the remote 'origin' - "git remote -v"

Obviously, FB has to do some kind of authentication, but you can create immortal auth tokens for the XML interface, so I opted to create one of those and save it to a file for future use. This still requires you to enter your password, but only once, and it gets masked out.

So, using only Term::ReadKey, LWP::UserAgent, HTTP::Request, XML::LibXML, and the excellent FB documentation, I was able to come up with a bit of perl that did everything we needed it to in terms of putting git information onto Fogbugz tasks.

First off it does a bit of generic data sorting - working out who you are; what your FB name must be; where to find FB anyway; what you just committed to git; and so on. It's very simple in its handling of what goes into FB and what doesn't - a commit that begins with "(FB\d+" gets submitted, anything else doesn't. Then it checks if it has an auth token or needs to generate a new one. Then it makes two XML requests. Hopefully nothing goes wrong, because I've done sod-all error handling beyond one or two dies. In the event that it fails due to, say, FB being temporarily unavailable, there's nothing to stop you just re-running the post-commit script manually when FB comes back.

Someday I may do something more "official" with it, like package it up or make it good enough to put on CPAN. In the meantime, since it seems like quite a lot of people have been looking for ways to get git and FB talking to each other, I thought it might be worthwhile posting our solution, for reference if nothing else.

So here 'tis:

...

#!/usr/bin/perl

use strict;
use warnings;

use Data::Dumper;
use Term::ReadKey;

use v5.10;

use LWP::UserAgent;
use HTTP::Request;
use XML::LibXML;

# The repo numbers used by FB
my %repos = (
    foo                 => 1,
    bar                 => 2,
    );

# Get the username automatically.
my $user = (getpwuid($<))[0];
# Assume FB username is always their email address
my $email = $user.'@yoursite.com';
my $url = "https://www.yoursite.com/fogbugz/api.asp";

# Get the summary of the commit we just made
my @commit = `git show --stat --date=iso`;

# Get the FB data from the commit
# Assume there'll only be one FB no. for now, update to handle multiples if we get a use case?
my ($fb, $sha, $time_and_author, $comment) = parse_commit(\@commit);

# Can't do anything if it doesn't have a FB number - so we're done!
exit unless $fb;

# Apparently we DO have something to send to FB. Is it for a repo we want to track?
my $repo = get_repo();
exit unless exists $repos{$repo};

# We do want to add this, so make sure we're able to log in.
my $token = login();

# Now, use our login to add a commit to the relevant FB
add_commit();

# We're done - communicate success to the user
say "Commit successfully added to Fogbugz case $fb";



sub login {
    # We need to have a token in order to send updates. This can be had in one of two ways:
    # 1) Using a pre-existing token saved in the user's home directory
    # 2) Asking for their password, generating a new token, saving it for future use
    #    Apparently, tokens only expire if we specifically log them out, so passwords should
    #    only ever need to be asked for once.
    # So, first check for a pre-existing token
    my $token_file = "/home/$user/.fb_auth_token";
    if (-r $token_file) {
        open (my $file, '<', $token_file);
        chomp ($token = <$file>);
        close $file;
        }

    # If we successfully got a token, we're done. If not, time to ask for a name & password
    return $token if $token;

    say 'No available authentication token.';
    say 'Please supply your LDAP password.';
    # We're asking for a password, let's not echo it for the whole world!
    ReadMode('noecho');
    my $password = ReadLine(0);
    ReadMode('normal');

    my $dom = get_url(logon => {
        email       => $email,
        password    => $password,
        });
    $token = $dom->findvalue('//token');

    # We've done our best to get a valid token - if we still don't have one, die
    die "No login from ".$dom->toString(1) unless $token;

    # If we DO have one, save it
    unlink $token_file;

    umask 0077;
    open(my $file, '>', $token_file);
    print $file $token;
    close $file;

    return $token;
    }

sub get_url {
    my ($cmd, $args) = @_;

    my $ua = LWP::UserAgent->new();

    $args->{token} = $token;

    my $get_url = "$url?cmd=$cmd&".join "&", map {$_."=".$args->{$_}} keys %$args;

    my $req = HTTP::Request->new(GET => $get_url);

    my $resp = $ua->request($req);

    unless ($resp->is_success){
        `clear`;
        print STDERR 'Error talking to FB\n'.$resp->_content;
        }

    my $dom = XML::LibXML->load_xml(string => $resp->content);

    return $dom->documentElement;;
    }

sub add_commit {
    # We'll want to do TWO additions - one to the FB source control functionality
    # and one to add the commit message as a comment

    # Add the comment
    my $add_stat = get_url(edit => {
        # Case to edit  = ixBug
        ixBug           => $fb,
        # The message we want to append
        sEvent          => "Commit: ".$repo." $sha\n$comment",
        });

    # Add to SC
    my $add_sc = get_url(newCheckin => {
        ixBug           => $fb,
        sFile           => $time_and_author,
        sPrev           => $sha,
        sNew            => $sha,
        ixRepository    => $repos{$repo},
        });
    # TODO: Handle errors
    }

sub parse_commit {
    my ($commit) = @_;

    # Get the SHA
    my ($sha) = $commit->[0] =~ m#commit (\S+).*#;

    # Get the committer
    my ($author) = map { m#^Author: (.*)# ? $1 : () } @$commit;
    my ($name, $email)  = $author =~ /(.*?)\s<(.*?)>/;
    # If possible, just strip down to a username
    $email =~ s/@yoursite.*//;

    # Get timestamp
    my ($time) = map { m#^Date:\s*([\d-]+\s+[\d:]+)# ? $1 : () } @$commit;
    $time_and_author = "$time $email";

    # Get FB case number
    my ($fb) = grep { m#^    \(FB\d+# } @$commit;
    $fb =~ s#^\s+\(FB(\d+).*#$1# if $fb;

    # Get the commit message
    my (@comments) = map { m#^    (.*)# ? $1 : () } @$commit;
    my $comment = join ' ', @comments;
    $comment =~ s/  / /g;

    return ($fb, $sha, $time_and_author, $comment);
    }

sub get_repo {
    chomp (my $repo = `git remote -v | grep origin.*fetch`);
    $repo =~ s#.*:(\S+?)(:?\.git)?\s+\(fetch.*#$1#;
    return $repo;
    }

No feedback yet

 

[Links][icon] My links

[Icon][Icon]About Me

[Icon][Icon]About this blog

[Icon][Icon]My /. profile

[Icon][Icon]My Wishlist

[Icon]MyCommerce

[FSF Associate Member]


July 2014
Mon Tue Wed Thu Fri Sat Sun
 << <   > >>
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      

Search

User tools

XML Feeds

eXTReMe Tracker

Valid XHTML 1.0 Transitional

Valid CSS!

[Valid RSS feed]

blogging tool