OK, I'm going to try out twitter for the first time, in order to see if it works out at PyCon for keeping track of what's going on and letting people know what I'm up to. I guess you have to e-mail me to get in touch with me, though, as I'm not following anyone (and I don't expect to keep using twitter after PyCon).

Anyway, my twitterifick name (twitter handle?) is ctitusbrown. Boring as you'd expect.

--titus

I'm 3/4 of the way through my first ground-up code review for pygr, and I want to gripe about something that pygr does a fair bit of: use Python's **kwargs. I don't want to escale it into a policy argument & policy decision for pygr, so I'm posting it here; hopefully I can sway minds with, well, suasion, rather than dictat.

What am I griping about, exactly? Check out this code:

class SomethingExtensible(object):
   def __init__(self, foo, bar, **kwargs):
       ...
       baz(x, y, **kwargs)

def baz(a, b, **kwargs):
   ...

Now think about how you'd read code like this in order to answer the following four questions:

1. What keyword arguments does __init__ take?
2. What does __init__ do with these arguments?
3. What keyword arguments does baz take?
4. What does baz do with these arguments?

(These would among the first questions I'd ask of the code; you too, right?)

Well, one thing you can immediately tell is that there's a big black box of arguments passed into 'baz', so there's no point in trying to completely understand the arguments to __init__ without also completely understanding the arguments to baz.

Another thing you can immediately tell is that without a pretty good docstring or a detailed examination of both __init__ and baz, you're not going to be able to begin to understand either function. And since docstrings, documentation, and comments are always wrong or incomplete anyway, you're going to have to grok the entire function.

So, basically, you're going to be lost in this code without a lot of work. Also, because there's no concise way to validate that we only received the set of kwargs we were expecting -- __init__ may not even know what those are, and people rarely check anyway because it's a bit of ugly code to do so -- you're subject to errors from misspellings: arguments that look almost right, but lack an 's' at the end, for example, when the code expects that 's' right there.

On the flip side, you do have an opaque "box" of arguments that you're passing around, and this can come in very handy if you're doing something like this:

class SomethingExtensible(object):
   def __init__(self, foo, bar, **kwargs):
      ...
      baz(x, y, **kwargs)

   def baz(a, b, **kwargs):
      ...

(note indentation of baz -- it's now a method in the class, rather than an independent function). Why is this handy? Because now you can subclass SomethingExtensible and potentially redefine 'baz' to take new arguments without having to change the constructor at all -- you just change what keyword arguments you pass into it.

So it's readability vs extensibility. I tend to argue for readability over extensibility, and that's what I'll do here: for the love of whatever deity/ies you believe in, use **kwargs sparingly and document their use when you do.

--titus

p.s. It may be that I'm missing some syntax tricks here, where I can unpack **kwargs and demand that it nicely conforms to my expectations without doing serial gets and removals:

x = kwargs.get('x', some_default)
if 'x' in kwargs: del kwargs['x']

y = kwargs.get('y', some_default)
if 'y' in kwargs: del kwargs['y']

...

assert len(kwargs) == 0, "error, unexpected kwargs"

or some such. Any thoughts?

'tis the season, and so it's time for me to post my list of accumulated project ideas. I'll transfer these over to the wiki tomorrow, after I track down some references. I'm willing to mentor any or many of these but I'd prefer to find someone to be the primary mentor for most of 'em. The ideas are not isolated to a single project; there are some potential overlaps in the testing stuf, especially.

1. Improve subprocess

I and others (mainly others) have talked about cleaning up & improving the subprocess module and associated documentation. This project would involve gathering a bunch of features, integrating them, testing them out on multiple platforms (most especially Windows), documenting them, submitting them to the Python core, and working through at least the first round of critiques.

2. Core Python testing infrastructure/nose compatibility

Work on the Python 2.x/3.x test running infrastructure to build a nose compatibility layer, so that developers can run the Python tests with nose. This would then enable tag-based execution, code coverage analysis, and all sorts of other nice features. The goal would be to produce a nose plugin that was core-Python-specific, so no changes would need to be made to the core Python code.

3. Analyze code coverage and improve test coverage, for Python core.

The CPython and stdlib code coverage is not terribly great, ranging from 99% to ~50% for some stdlib modules. Measure the C code coverage, integrate it with the Python code coverage, and provide a convenient integrated report. Make it easy to run & generate code coverage. Do so for Mac OS X, Windows, and Linux. Improve code coverage by adding tests.

4. Integrate of Pyrex and C code coverage into figleaf reporting

Right now, figleaf (my code coverage analysis tool) measures and reports on Python code coverage. Add integration hooks to allow it to import and generate reports on C/C++ and Pyrex/Cython code coverage.

5. Port figleaf to Python 3.0.

Code coverage analysis is an important component of testing, especially when refactoring legacy projects. Make figleaf 3.0 compatible, probably in a separate branch.

6. Implement branch coverage measurement and reporting for CPython.

'nuff said. (This is a tough project that would be more research than implementation, I think.)

7. Extend a simple continuous build system for Python projects.

Work on a simple buildbot replacement that allows simple, flexible reporting and remote push of results. (This is also part of the snakebite project.) More in a bit.

8. GridRepublic/BOINC Python

Distributed Python: Borrow or invent a notation for master/slave execution in Python. Develop a system that implements this on BOINC, i.e., creates WUs and applications, and harvests the results. See the BOINC dev projects generally and the Python app design document specifically.

--titus

p.s. Oh, yeah, and a pygr-related project would be good, too.

i asked two other friends of mine, [ ... ], for recommendations about model code and about their work environment. their feedback was extremely helpful and i thought you'd be interested to hear the opinions of other good programmers. i would also love to hear your thoughts on these (extracted and scattered) comments:

I recently had the pleasure of being the technical reviewer for a new Apress offering, Beginning Python Visualization, by Shai Vaingast.

To quote from the apress page,

What you'll learn:

• Write ten lines of code and present visual information instead of data soup.
• Set up an open source environment ready for data visualization.
• Forget Excel: use Python.
• Learn numerical and textual processing.
• Draw graphs and plots based on textual and numerical data.
• Learn how to deal with images.

I can't recommend this book highly enough for people who need to do basic data processing, manipulation, and viewing in Python. My role as TR was largely limited to occasional style issues and some serious cheerleading: "more figures! more examples! great stuff!"

So, in sum: buy this book.

Note that apress didn't ask me to review this, and they only paid me for the TR role. This blog post is gratis and I was slow enough on the reviews that I suspect they won't ask me to TR again no matter what I say ;).

--titus