Daily Life in an Ivory Basement

This last term I facilitated the participation of five MSU students in the Undergraduate Capstone Open Source Projects (UCOSP) program, in which students do distributed open source software development and receive home institution credit. UCOSP was managed out of U Toronto by Greg Wilson, and I was (and am) enthusiastic to participate as it's clearly a good way bring open source into education.

However, I was less thrilled to see that the majority of the MSU students received, ahem, "less than passing" grades from their project leaders. I knew about the problems in one particular project from having met with the students on a regular basis, but the other results caught me by surprise. I would love to kick and scream and complain that I should have been made more aware of what was going on -- and where I had constructive things to suggest, I did -- but the more important failure may have been a mismatch between the MSU students' approach to these projects, and project expectations.

The students variously had a number of problems, ranging from team miscommunication & poor conduct to an inability to get the software to compile. This meant that for several students, no visible work got done -- for example, in one project, it regularly happened that person X was working on a patch, and person Y committed an overlapping patch first. Or on another project, person Z spent two months trying to get the basic project infrastructure compiled, and was reduced (at the very end) to submitting code fixes without testing them in the full project context. Or several times, person A spent a week working out how to refactor a test into something reliable, and resulted in what looked like (and maybe was) a trivial code change.

All of these situations may result (and did result) in low evaluations. This is understandable: no visible work got done, so how is an evaluator supposed to grade them!? Yet, all of the situations are legitimate issues that block progress. What is a student to do?

The answer won't be too hard to guess for anyone who has worked on real-world team projects: make your struggles visible.

Someone steps on your patch? Fine -- submit your patch too, and explain why it's better (or worse) than the first patch. Code review the other patch, while you're at it: who better to do the review than someone who really understands the issues? Then when you get poor marks for not having contributed code, point at your patch. (You are using version control, right?)

Can't compile the software? Fine -- write down what's going wrong, and post it publicly. Document your fix attempts. Ask for help. Bash your head against the wall repeatedly. Either fix the problem, or document the problem thoroughly. Either someone will help you, or you'll figure it out, or you'll leave an audit trail so that others won't have to do all that fail work. Then when you get poor marks for not having contributed any patches, point out that the project has technical issues and either no one could help you (project FAIL) or you spent all your time fixing them.

Trying to debug niggling details that turn out (in the end) not to involve big impressive code changes? Submitting too many unimpressive patches that no one seems to value? Write down why your contributions are valuable. At the end of the day the evaluation may (rightly or wrongly) be "not too smart, but sure did work hard" -- but that's better than "no evidence of any work having been done".

Note how a lot of this seems to involve communication? Right -- that. For team projects, being an effective communicator is more important than being a kick-ass programmer.

At the end of the day, there are things you can control, and things you can't control. You can't control what other people think of you, and you can't control how other people (including project leaders and professors) evaluate you. But you can visibly work hard, and defend yourself based upon that evidence.

I call the general approach of throwing energy at a project "aggressive competence", and I think it's a necessary component of effective team software development. Everyone has days, or weeks, or even months where they look incompetent or ineffective; often that's because outsiders don't understand or appreciate the work that you've done. Tough on you, but I don't think it's reasonable to expect your boss, or colleagues, to look hard at your work to find reasons to praise you. Fundamentally, it's your responsibility to "manage up" and communicate your progress to others effectively.

In open source projects (and elective college courses) the immediate ramifications of a poor evaluation may not be clear -- I'll leave you, dear reader, to figure out the longer term consequences. But I think the ramifications of a poor evaluation are immediately obvious in the context of a capstone course, or a paying job.

Incidentally, this illuminates one of the reasons why I'm such a big fan of UCOSP: it is reality. You're working on an existing project, with other developers, at a distance; and it's not anyone else's responsibility to frame the problem for you. It's your responsibility to make progress.

This is where I think there were mismatched expectations. The students expected that they were going to be managed, helped, and given clear expectations. They weren't. So they got bad evaluations.

What do I plan to do? Well, assuming that UCOSP + MSU goes forward next term, I will be communicating my expectations quite clearly to the students. And I will be asking for regular progress reports, sent to me and CCed to the project leaders. And I'll be sending them this blog post. And I'll be failing the ones that don't listen.

I'll end with a paraphrase of one of my favorite sci-fi authors: "every new developer has problems on a new project. The extent of our sympathy for those problems, however, will be dictated by the efforts made to overcome them."

--titus

p.s. It's also a good way to figure what projects you don't want to work on: I once got dinged for working too hard in a company; I was told that I was "rowing too fast and the boat was going around in circles." My response (that perhaps others might consider rowing faster) was not received well. That's the kind of job situation you can leave without guilt (as I did).

p.p.s. Code reviews can be an extraordinarily effective passive-aggressive way to correctively interact with jerks on a project, too.

I'm nominally involved in co-mentoring or cheerleading 5 Google Summer of Code projects this summer, and several of the students have the same problem: they send me one big e-mail (or post one big blog entry), every few weeks, asking for input.

This imposes a big energetic barrier to me. In order to answer their question(s) or look at their code, I need to come up to speed with all the changes they've made to their project since the last time they asked for help. (And because I'm busy, this often means I don't get to it at all. Mea culpa.)

I think this might be a consequence of the learning process the good students learn in school: "try to figure everything out yourself, and only come ask for help if you're really stuck." That, or they just don't want to bother me with "trivia", even though that's what I actually want -- to be bother with small-sized stuff, not big chunks!

Anyway, the message is: ask me to look at small things every now and then, and make them easy to look at (VCS rather than attachments, some form of docs, etc.). I'll rub the corners off of your code base more regularly and I can be much more helpful on that basis than on the big chunk/every month basis.

Paranthetically, one of the joys of this GSoC has been watching several students plow through the work without really any input or oversight from me. Eden Elos, Eric Pruitt, Shuaib Khan, Zach Riggle, and Yang Yang (and... who am I missing?) have all really stepped up to the plate, or at least allowed themselves to be inexorably moved towards the plate and assumed plate-occupying responsibility.

--titus

I'd like to invite you to attend the last of the Michigan State University CSE colloquia for the 2008-2009 academic year: jointly sponsored as an AT&T Visiting Lecturer by the MSU LCT, and the CSE department, Sam Ramji will speak about

Open Source at Microsoft: The Past, Present and Future

in CommArts room 147, Friday May 1, at 11:00am. I encourage you all to attend and to forward this on to others who might be interested! As you know, open source software is playing an increasingly big part in education, academia, science, and business, and so I expect this to be a very interesting talk.

Contact me at ctb@msu.edu for further information.

--

Abstract:

Since Microsoft established its Open Source Lab in Redmond more than five years ago, it has worked with many open source players to make Windows the best platform for all applications to run on. But this has not been without its challenges and there is a lot more work to be done on this front. This talk will cover the thinking behind Microsoft's current open source strategy and what this means for the software engineers of the future. It will also spotlight some innovative Open Source projects the company is supporting at universities across the world.

Biography:

Sam Ramji is the Senior Director of Platform Strategy leading Microsoft's platform strategy efforts across the company, including long-term strategic planning in the Windows Server and Tools organization. Sam's primary focus is to drive Microsoft's Linux and Open Source Strategy, working together with Microsoft technology development teams and open source communities to build interoperable solutions.

Prior to his current role at Microsoft, Sam was a Director of Emerging Business working on the Silicon Valley Campus where he managed relationships with Venture Capitalists and entrepreneurs. Prior to joining Microsoft, Sam led technical product strategy at BEA Systems, engineering teams building large-scale applications on Open Source software (at Ofoto.com) as well as hands-on development of client, client-server, and distributed applications on Unix, Windows, and Macintosh at prior companies.

Sam holds a Bachelor of Science degree in Cognitive Science from the University of California at San Diego, and is a member of the Institute for Generative Leadership.

Over at OLPC News, Wayan Vota asks: "Do you get better FOSS code ... if the developers are paid or unpaid?"

Interesting question, especially as considered in light of the OLPC code base.

As of about a year ago, virtually everybody I talked to was shocked and stunned at the poor quality of the OLPC code base. My personal encounters with the code left me frustrated at the poor Python style, angry at the simultaneously over- and under-engineered build process, livid at the poor quality control, and blue about the long-term prospects of the OLPC software. I was not alone, although most people were polite enough not to be a jerk about it like me. (Looking back, perhaps if more people had been jerks earlier, things could have improved. Or not.)

At the time I felt that the OLPC code, which might have been a triumph for the OLPC project, a gem of Python code, and a win for the open source community, had turned into a cesspool of code and a potentially infinite time sink for people working on it.

I haven't revisited the project since then, and I know that community enthusiasm has waned for the OLPC. I don't think the poor quality of the code was a huge factor in this decline, but it certainly decreased the interest that serious hackers had in getting involved in the project -- at least, it did for everyone I talked with.

So, let's get back to the original question: in an OSS project, is it better to let the community write your code for you, or should you have a core development team?

Let me rephrase that another way, actually: if your business practices drive you to write crappy, hurried code, with no attention to any form of reasonable software development processes (agile or otherwise), no automated testing, and very little QA of any kind, are you better off turning software development over to the community?

The answer becomes less difficult then: yes, if it's an option, you should jettison your software development efforts. If you can't do things competently, you should remove yourself from the responsibility of doing it. Abdicating responsibility is the responsible thing to do in these circumstances, if you can make it work.

But... what went wrong, anyway? I don't know. I'd guess that the OLPC software development team was overmatched from the beginning: responsibility of developing new software for a novel mix of hardware, together with a novel GUI interface, in an environment of much enthusiasm and relatively little funding, is hard. Add to this the enthusiastic but not very disciplined-sounding roadmap, as well as the unrealistic deadlines set due to PR and political considerations, and it's a recipe for disaster. When you also throw in an overwhelmed OLPC management team, I don't think you can expect anything but disaster.

So, looking back, I think the OLPC was probably headed into a mess from day 1, and while the fubar'ed software development process didn't help, it was at least partly driven by circumstances outside the control of the software developers. The software developers were hamstrung from the get go. And in that environment, turning the majority of your software development over to someone else -- someone not under your direct control -- may in fact be the best option.

I think that's actually a much more interesting question: at what point should an organization realize that they are so incompetent at managing and doing "in-house" software development that they should try to outsource it? What metrics should they look at, and how should the decision be made?

--titus

p.s. The general question of whether to go for paid or unpaid development is silly, of course. The answer is, "it depends." Duh. The interesting question is what questions you need to ask to figure it out!

p.p.s. Disclaimer: the individual OLPC people I know, up to and including lower management, are smart, enthusiastic, and good people. I place the majority of the blame squarely at the top.