« Quick & easy ROT13 for the command lineOccasionally helpful stuff »

Wed, Mar 27, 2013

[Icon][Icon]I made a module!

• Post categories: Omni, FOSS, Technology, My Life, Programming, Helpful

Like the functional languages it stole from was inspired by, Perl has some very helpful list-processing functionality.

The two most important built-ins are 'map' and 'grep'.

Map runs over every item in a list and modifies it. Grep runs over every item and filters out any that don't match the given criteria.

So, for example

perl -E '$, = " "; say map { $_ * 2 } 1..5'

This takes the list of numbers from one to five, and doubles them: The returned list is "2 4 6 8 10"

perl -E '$, = " "; say grep { $_ % 2 } 1..5'

This takes the numbers from one to five, and removes those that are divisible by 2: Returned list is "1 3 5"

Helpfully, since each function accepts a list and returns a list, you can chain them together:

perl -E '$, = " "; say map { $_ * 2 } grep { $_ % 2 } 1..5'

2 6 10

First we filter down to the odd numbers, then we double them. Yay.

Another helpful facet of Perl is the flexible way that evaluation and assignment work. I remember from my days of playing with C the standard way of swapping two values between two variables. i.e.

a = 1;
b = 2;
// Swap values via 'c'
c = a;
a = b;
b = c;
//a == 2; b == 1

In Perl, it's far simpler:

perl -E '$a = 1; $b = 2; ($a, $b) = ($b, $a); say "a = $a, b = $b"'

a = 2, b = 1

($a, $b) = ($b, $a); is perfectly valid and works nicely. And similarly

perl -E 'my @numbers = 1..5; @numbers = grep { $_ % 2 } @numbers; say "@numbers"'

1 3 5

You can use map and grep to (effectively) change an array in place by using the array as both the subject of the map/grep and the recipient of the returned list.

All very nice. But yesterday I hit a slightly different problem: Although I wanted to remove elements from an array, I wanted to do so by putting them into another array.

I could have done this easily with two lines of code: Use grep to go over the existing array and assign the desired elements I wanted to remove to it; and then use grep a second time to remove the elements I had just copied and assign the resulting list to the original array.

This would have worked. But it annoyed me that I had to process a list twice. It was like having to move a file by copying it to the new place and then deleting the original, rather than just saying "move from here to there".

CPAN has modules that will do most things, but the closest I could find was List::MoreUtils::part which would have split the list into two arrayrefs. Which works, but then I'd have had the original list and the two new ones. I didn't want that, I wanted the original, filtered list and a new list with the filtered-out elements.

Didn't seem to exist. So I decided to write it.

The code was simple enough:

sub kgrep (&@) {
    my $filter = shift;
    my $filtered = [];
    my @keep;
    local $_;
    for (@_) {
        if ( $filter->() ) {
            push @keep, $_;
        else {
            push @$filtered, $_;
    return ($filtered, @keep);

It simply puts the filtered-out elements into an arrayref, and then returns that ref as the first item of the list. This meant I could rewrite my original code as:

# Oh no! The good list has been contaminated!
my @good = qw/good bad good evil good wicked good/;
my $bad;
($bad, @good) = kgrep { $_ =~ /good/ } @good;
say "@$bad | @good";   # bad evil wicked | good good good good

And now the original 'good' array contains only 'good' entries; the others have been shunted into $bad

Simple enough to write, and it did what I wanted.

It was just possible that this might be useful to other people, too. So I figured it might be time to take the leap and get onto CPAN.

First step, get a PAUSE account. That was pretty quick & easy, actually.

Next step: Make a CPAN-standard module. Ah. Tricksy.

As little as a week ago, I might have used something like Module::Starter to help with the task of making a proper module out of my little bundle of code. But as it happened, last week I saw on twitter that pjf had released a module & command-line app to manage HabitRPG

And I had a go with it (and submitted a bug report whilst at it :) and couldn't help but notice how tiny the git checkout was and how clever Dist::Zilla was at generating all the boilerplate stuff...

So, learning dzil as I went along (it has a great choose-your-own-adventure style tutorial) I cobbled together a proper module. And it was amazingly simple. A bit of trial and error here & there, but in no time at all I had generated a decent tarball that was installable.

Perl has a very strong testing culture, so I added a few simple tests, which was also made beautifully easy by Test::More, which makes it trivial to compare the contents of two arrays:

is_deeply($bad, [qw/bad evil wicked/], 'Correct bad values');

And suddenly I had a module with tests and makefile and automated versioning.. and I'd barely had to write any of it!

Since it seems to be the place to put things these days, I figured I'd shunt the repo up to my github account whilst I was at it. The first time I've put a proper repo on there, as it happens. And again, it was amazing just how easy it was: Create the repo, tell git about its new remote, push. Done.

Create a Markdown readme file, which is the simplest markup I've ever used, and you could almost believe it was a repo made by somebody who's bothered to learn how to make Github work.

And here we are, less than 24 hours after I decided to try writing an official module. And it's on CPAN and it's on GitHub and as of the end of our current sprint it'll be in use in enterprise software.

I've already had some helpful feedback about how to pick a better namespace for my modules in future (strictly speaking, it handles LISTS not ARRAYS) and other such book-keeping.

It's kinda nice to have started putting my own modules up instead of just submitting the occasional bug report & patch. But mostly what I've taken away from the whole thing is a sense of amazement over how good the tools are out there for creating & publishing code.

Dist::Zilla made it not just easy, but actually fun, to create the module. Test::More relieved me of any need to write any array comparison code. GitHub made it trivial to put the dev. history where anyone who wants it can grab it. Everything online looks shiny & like it was put together by somebody who knew exactly what they were doing, when 9/10s of it was just muddling through until it did what I wanted.

A lot of people have put in a lot of work to make it easy for me to look good.

Thanks, all of you! :)


David Golden
Comment from: David Golden [Visitor] · http://www.dagolden.com/
Well done! Thank you for showing people how easy it is!
28/03/13 @ 00:18
Paul Evans
Comment from: Paul Evans [Visitor]
You are aware, however, that this is basically List::UtilsBy::extract_by() ?
30/03/13 @ 14:42
Comment from: oneandoneis2 [Member] · http://geekblog.oneandoneis2.org/
I had no doubt that somewhere there was a module that would give me the functionality I needed. But after a quick look failed to find it, it was less of a yak to write my own than keep trawling through the vastness of CPAN in the hopes of coming across the one I needed..
01/04/13 @ 19:43
Doug Bell
Comment from: Doug Bell [Visitor] · http://blogs.perl.org/users/preaction
List::MoreUtils has a part() function that does this too, in a slightly different way. But CPAN is the marketplace of ideas, and more ideas is a good thing!
01/04/13 @ 20:45

[Links][icon] My links

[Icon][Icon]About Me

[Icon][Icon]About this blog

[Icon][Icon]My /. profile

[Icon][Icon]My Wishlist


[FSF Associate Member]

February 2018
Mon Tue Wed Thu Fri Sat Sun
 << <   > >>
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28        


User tools

XML Feeds

eXTReMe Tracker

Valid XHTML 1.0 Transitional

Valid CSS!

[Valid RSS feed]

powered by b2evolution