The OSCON Vote on Perl Certification

Although The Perl Journal was generous in allowing me lots of room to argue my case for Perl Certification in my October, 2003 article, they were unable to accommodate my request for footnotes, so this page is serving that function. If you've read the article, and come here in connection with the appropriate passage in the text, the context for the following elaboration will be clear to you. If not, you may wish to read the article first.

How the Votes were Counted

But the audience numbered almost 200, so what about those who abstained from the vote?

One plausible interpretation is that they didn't feel comfortable voting for either question, due to the way they were worded.

Another possibility is that at least some of the abstainers may have been people who were uninvolved with Perl, and who therefore felt ineligible to express their opinions. Such people would include Python, Apache, Ruby, and Linux specialists, media people, and other non-Perlish conventioners who chose to attend this session.

Another issue is that some people were already packing up their laptops and getting ready to leave by the time of the vote, and therefore already preoccupied with other matters.

It's also worth noting that the proportion of non-voters, approx. 50%, is in keeping with that commonly seen in political elections. This could mean the non-voters don't have strong leanings of their own, and are willing to go with the decision of the majority.

Of course, there are other possibilities as well, and there's no way to ever know where those people really came from -- although many certainly looked familiar to me!

Update on Voting Questions and Interpretations

First of all, I had hoped to delay this update on the voting results until a review of the videotape could be consulted, and we could be 100% sure of the facts -- to avoid the possibility of having to release yet another correction later! Unfortunately, that definitive evidence has not been forthcoming. However, in consideration of the import of this matter to the community, I have decided to make this interim announcement at this time (1/17/04).

The TPJ article characterizes the questions for the show-of-hands vote as being simply how many were for development of a certification program for Perl programmers and how many against it. I came to this understanding by examining my own fuzzy* recollections of the questions that Damian asked of the crowd, and by asking others who were present to share their memories. Moreover, that simple, sensible, and appealing characterization of the voting questions passed the scrutiny of all the reviewers for the TPJ article. (In case you're wondering why I as the leader of the Panel Discussion would have any uncertainty about the voting questions, please read the Explanatory Notes.)

Despite these attempts on my part and those of others to get the story right, it appears that the questions were actually more complicated than that. Autrijus Tang, who had the foresight to actually write down one of the questions on which the attendees voted, reports that the "for" question was:

How many of you are for continued discussion, and possible development, of a certification program by the Perl community?
(Presumably, the other question was worded identically but with against replacing for.)

This phrasing is a little tricky to work with, but nevertheless I'll venture some ideas as to what the associated votes might be trying to tell us.

The question expresses a dependency of development (via the logical and) on the results of a tested outcome of a discussion. These elements can be represented in pseudo-Perl-code as follows:

outcome_of_discussion() > $threshold_for_action &&
     do_development() ;

Or, for those who like to type more:

if (outcome_of_discussion() > $threshold_for_action) {
     do_development() ;
}

So one could argue that those voting for this position "might be" for a Perl certification program, in the sense that they're willing to see one developed if continued discussions favor that action. In fact, given the phrasing of the question, some (or all) of them may have only been willing to have further discussions because there was a distinct possibility of a program actually being created. Others may have been currently undecided about certification, but in favor of further discussions, regardless of which outcome might be forthcoming.

On the other hand, anyone paying attention who was "for continued discussion" but "against possible development" (i.e., in favor of "wasting time"), or "against continued discussion" but "for possible development" (i.e., in favor of "forging ahead"), would have had to abstain from the vote (as many did*).

So how can we characterize this group of voters? We don't know if they're currently for certification, against it (but willing to have further discussions, unlike the against voters), or undecided about it. However, because we know that they're amenable to the development of a certification program if further discussions would lead to that, I think it's best to identify them as "conditionally for certification".

Conclusions

100 of the approx. 200 attendees at the OSCON 2003 Panel Discussion on Perl Certification voted for continued discussions about the Perl community developing a Certification Program for Perl Programmers, with the understanding that those discussions could possibly lead to the development of such a program. In contrast, approx. 7 people voted against the idea of further discussions and possible development. That gives a majority of about 14 to 1 for those voting for.

Based on the specific phrasing of the voting question cited above , I submit that the most accurate interpretation of the results is that the majority was "conditionally for" the development of a Perl Certification program -- meaning they would be willing to go forward with its development if further discussions favored that action. Any other interpretation (that I can think of) either discounts the impacts of the "continued discussion" or "and possible development" clauses, or makes assumptions about the current leanings of the voters.

This conclusion is slightly different from my earlier reports of the results, including those in the TPJ article , which represented the majority as being simply "for certification". In consequence, the statements in the article referring to a majority being in favor of certification need to be reinterpreted in light of the "conditionality" -- on the results of further discussion -- of their support for the idea.

As the leader of the Panel Discussion and the one reporting on its results, it was my responsibility to get the story straight, and despite my best efforts, I fell short in that goal. I take full responsibility for that outcome, and I humbly apologize for the confusion!

A Note on Statistics

How seriously? Specifically, I contend we should be prepared to tentatively assume, unless/until better data become available, that the opinions of Perl programmers at large on Certification resemble those of the sample that voted at OSCON. I have explained my reasoning in a travel allegory (I'm partial to those!). It comes down to this: when you need to make a decision, you're wise to make use of whatever credible evidence you've got, rather than worrying about whether you could "prove" that your decision is the correct one to a statistician.

We in the Perl Community aren't currently facing any crisis that we can scientifically prove is due to the lack of a certification program, but there are reasons to believe that having one could help our standing in the enterprise. I'm all for making attempts to collect better evidence on JAPHly opinions and corporate needs, but in the meantime, I think it's prudent to make educated guesses about what's happening and prepare for the possibility of future action.

By the way, I took (and aced) several graduate courses in Statistics, covering both Inferential and Descriptive techniques and the Design of Experiments, at the University of Toronto. Later, as a Professor of CS, I wrote and taught an advanced graduate course on a hot new technique called "Multidimensional Scaling" at the University of Utah.

While studying at the U of T, I vividly recall my thesis supervisor, Dr. D. E. Berlyne, arguing the same points I raise in The Riddle of Snake Island with our department's head statistician, Dr. J. C. Ogilvie. The "snakebite" angle is entirely mine, but I credit them with enlightening me on the important principles it illustrates.

Of course, just because I've studied statistics doesn't mean you have to agree with me; you may have studied yourself and come to different conclusions. Even experts like Drs. Ogilvie and Berlyne often disagree. YMMV!

About the author

Dr. Tim Maher has trouble accepting pronouncements like Perl is too hard to parse to be beautifiable and Perl is too complex for its programmers to be certifiable. Accordingly, he was the first to demonstrate that automatic beautification of Perl source code was indeed possible, and now he's working on overturning the second pronouncement.

He has three degrees in Experimental Psychology, where his studies included techniques for computer assisted learning and testing, and he's a former professor of CS, which gave him lots of experience in constructing academic examinations.

While working as a Software Instructor for Sun Microsystems, Inc., Tim was required to pass all the Solaris System Administrator certification exams, which led to him finding mistakes in them, and recommending systemic changes to make errors easier to identify and correct.

In addition to being the founder of the Seattle Perl Users Group (aka SPUG and Seattle.pm), which is one of the world's oldest, largest, and most active, Tim is also the CEO of Consultix, which offers Perl, UNIX, and Linux training (see http://teachmeperl.com ).

He's currently writing a book entitled Minimal Perl: For Shell Users and Programmers, which will be published in 2004 by Manning Publications.

He'd like to hear your views, and can be reached via email as tim(AT)teachmeperl.com.

For those who weren't there, you should know how the vote came about. While Andy Lester was telling us his views on Certification, and I was straining to hear him from more than 100 feet away, with only seconds left in the session, Nat Torkington encouraged panelist Damian Conway to call for a show-of-hands vote, and then Damian improvised the questions on the fly and posed them to the audience. My attention was concentrated on estimating the number of votes for each question, rather than trying to memorize the questions themselves.