Daily Life in an Ivory Basement

Grig and I will be reprising our Testing Tutorial at PyCon '07! This year I think we have a better idea of what people found useful, and in consequence we should be a bit better organized, too.

Our outline is here: http://us.pycon.org/TX2007/Tutorials

More anon.

--titus

So, it looks like I will be giving a talk on twill, scotch, and figleaf at PyCon! One of the reviewers said something very nice, albeit probably unjustified; to quote:

...twill is more or less Py web-developer 101 at this point, so a talk
about it should be enlightening.

Hmm, guess I'd better get twill 0.9 out then, eh? Note to Ian Bicking: I'm planning on talking about some of the Paste/WSGI testing fixtures as well.

Something else that's particularly nice is that I'll be part of a whole talk session on testing. Hint, there's a fun twist to it that I'm sure you'll all appreciate if you come to the session. (If you don't come, then I'm not going to tell you what the twist is -- and you'll be missing out!)

The PyCon Web Panel will also be happening, but I'm a bit more apprehensive about this because it's a big responsibility: if it sucks, I may very well be stoned to death (and rightly so!). Oh, and I need to find people interested in representing web.py, Quixote and twisted.web. (I already have Jim Fulton (Zope), Ben Bangert (Pylons), Adrian Holovaty (Django), Robert Brewer (CherryPy), and probably Kevin Dangoor (TurboGears).) I asked Aaron Swartz and Neil Schemenauer about web.py and Quixote respectively; neither one is coming. Any ideas who to ask about twisted.web?

Please drop me a line if you have thoughts on questions to be asked, people to recruit, or moral support to offer ;).

--titus

p.s. No word on the Testing Tutorial yet...

Here's a (slightly edited) message from Bill Punch, a professor at MSU who is hoping to transition their Intro CS class over to Python. I would appreciate your thoughts and comments, both on his questions and my responses! (I do have permission to post it, in case you're wondering. ;)

(Just in case you're thinking about being unfriendly or snarky, please do keep in mind that the friendliness of the Python community is legendary, and we really want Python to be taught to CS students... so don't be unfriendly or snarky. Or else the PSF will hunt you down and ... well, let's just say that you may end up programming in Perl for the rest of your life.)

Bill sez:

"""

So I'm pushing ahead with the transition to Python and trying to see how it fits with the courses we have here. I've hit a couple of snags (as you might guess) and I wondered what the general opinion is.

1) The concept of class in python is not what I expected. You can do some things there that would be hard for C++ people (or other OO approaches, smalltalk, clos, java, ...) to take. You create instance variables by assignment, say in an __init__. They are then local to the instance namespace/dict and always public (never private). After that, an instances is just another data structure, nothing special. You can just assign new instance variables to that instance, independent of the class definition and of any other instance of that class. You can also just assign a new "class" variable to a class, independent of the def'n, and it will show up in all instances.

I know this is a result of the namespace lookup, which is how classes are implemented, but this is really different. Is the use of __slots__ popular to prevent this sort of thing? What do other "language people" think of this approach?

2) We have typically had trouble teaching pointers to students. They seem to relate better to standard variables (that have both an address and a value). However, everything here is a pointer. Whether an assignment change is local or not depends on the type being assigned (mutable or non-mutable), since everything potentially is a reference. Is that a tripping point for people? Do new students pick that up easily?

3) No real overloaded functions in python, at least not in the standard C++ sense. That is, there is no overloading a function on argument types (since there are no typed variables) or on number of arguments. Each def is a statement and the last one run is the one that holds (no multiple functions with the same name but different argument lists). However, they do overload in that the operation performed by the function depends on the type of the arguments. That is, a function "plus" that adds two arguments does integer summation on two ints, but concatenation on two strings. Thus you have to know what you are passing in (and what is done to it) to have an expectation of knowing the result. Is that clear to you? Seems confusing to me. I expect to know (in a C++ sense) what is going in and what is coming out, and if I don't get that right then the program/complier complains. Here, you pass the wrong args (you meant ints, you passed strings) and it works fine, just not what you expect. Seems less simple to me, is that right?

"""

My response:

"""

You actually can make variables private by naming them with a __ in front. (Don't do this.)

And remember that inheritance etc. all works properly in this, so it's not quite as simple as "oh, it's all just a namespace" -- it's actually a chained namespace with multiple inheritance ;).

(I can blow your mind with metaclasses later.)

__slots__ is intended only for optimization; see

http://www.dalkescientific.com/writings/diary/archive/2006/03/19/class_instantiation_performance.html

(search for 'The fastest approach uses ')

I focus on the design in OO programming more than on the constraints. Once I let go of the idea that I could dictate what code is publicly vs privately vs "protected"ly accessible to subclasses or external code, I don't think I ever looked back. After all, it turns out that most people aren't very good at designing class hierarchies in reality, so why overly constrain what people can do with them by designing the constraints in "up front"? (There's some good writing about this approach in Richard Gabriel's book on Patterns.)

If it's really critical for a particular problem, there are a couple of approaches. For one, you can use properties to make read-only variables:

class A(object):
   def get_p(self):
      return 5
   p = property(get_p)

a = A()
print a.p   # calls 'get_p'

a.p = 6     # raises error

You can also override __getattr__ and __setattr__, but properties are the recommended way to do it. See

http://www.python.org/download/releases/2.2/descrintro/

for waaaaaay too much information.

For (2),

Yes, I have a tough time explaining this in my head, but then again I learned to program in C and I unashamedly use pointers ;).

First, I'd go with 'pass by reference' instead of pointer, although they mean roughly the same thing.

Second, I may be missing something, but aren't assignments ALWAYS local?

def f(a, b, c, d):
   a = 1
   b = SomethingElse()
   c = 'str'

a = 2
assert a == 2
f(a, b, c, d)
assert a == 2

I would distinguish arguments by whether or not the argument is mutable. For example, in

def f(a, b, c):
   a = a + 1
   b += "hello"
   c['x'] = 'test'

d = dict(x='yo')
f(5, "world", d)

you wouldn't expect '5' to suddenly become '6' simply because you added one to it, right? Nor does "world" become "worldhello". That's because they are immutable types that are passed by reference but any operation on them returns an entirely new immutable type. However, d['x'] does become 'test', and doesn't stay as 'yo', because you're passing the dict as a mutable type.

I haven't really thought this through, tho, so let me know if you run across counterexamples...

My experience has been that new programmers don't get too confused by this, but that's probably because they don't know C++ ;).

For (3),

I think the concept is known as "duck typing", as in "if it looks like a duck, and acts like a duck, we should probably treat it like a duck."

Bruce Eckel has written a lot on it; here's one google'd post:

http://www.mindview.net/WebLog/log-0053

You might also be interested in reading Guido van Rossum's thoughts,

http://www.artima.com/weblogs/viewpost.jsp?thread=85551

and esp the comments which can at least give you a flavor of the discussion in the larger python community:

http://www.artima.com/forums/flat.jsp?forum=106&thread=85551

In practice, the way it's implemented is that each type has its own __radd__ etc. operators that "know" how to deal with its type. If you want to confuse the heck out of people (but make it consistent with your own C++ knowledge) point out that

a = a + b

actually can be written

a = a.__radd(b)__

So as long as you choose your operators wisely, then it all makes sense.

On all of these issues, I would say that it's simply a matter of habit. I find C++ frustrating at this point, because it doesn't easily let me do things with lists and strings that I'd like to be able to do, like unpack them intelligently, e.g.

x, y = y, x

to swap two variables.

HTH!

"""

Thoughts?

If you're posting comments, please note that I need to approve you if you're a first time poster -- so your comments may take a bit to show up. Sorry 'bout that.

--titus

This nice Guardian review of Shortbus has a few good quotes. (When reading these, bear in mind that the film shows a certain amount of explicit sexual activity.)

One quote, recounting a discussion with his (conservative Christian) parents about the film:

Have they watched Shortbus?

'No, they said, "Well, you didn't go to see The Passion of the Christ..." I
pointed out that they did not make The Passion of the Christ. And anyhow, why
would I prefer to go and see some three-hour film in which a man is flayed
alive than two people having it off? That kind of says all you need to know
about this stupid Western culture we live in.

Another quote from the article, talking about the lack of furor:

He was a little disappointed about the fact his film had not excited picket
lines of shocked Middle Americans, or fulminating editorials, but he put it
down not to a new open-mindedness but to Republicans being 'busy losing the
midterm elections and with their own sexual scandals'. He felt he had done his
level best to stir up a little patriotic debate - in one memorable scene, the
hindmost participant in a gay threesome sings a necessarily muffled version of
'The Star-Spangled Banner' - but there was nothing.

Heh.

I saw the film a few weeks ago, and it was pretty good. Surprisingly low key, in fact, given the trailers and the subject matter.

--titus

I'm continuing to build a testing infrastructure for Cartwheel. One thing to test is the XML-RPC API for controlling the server remotely; this lets users upload sequences, create analyses, and download results programmatically.

Now, doing functional testing of an XML-RPC server is just as annoying as testing any Web app: you have to run the Web server, bind the port, etc. Wouldn't it be great if I could just test the application directly without all that trouble!?

Luckily for me, I have a passing familiarity with wsgi_intercept, a tool that lets you bypass the external network by routing HTTP calls directly to a WSGI application. And, as it turns out, XML-RPC works via HTTP! So I can test my XML-RPC API without touching the network.

Here's the code:

wsgi_app = MyWSGI_ApplicationObject()

class _InprocessHTTP(httplib.HTTP):
    _connection_class = wsgi_intercept.WSGI_HTTPConnection

class _InprocessTransport(xmlrpclib.Transport):
    def make_connection(self, host):
        host, extra_headers, x509 = self.get_host_info(host)
        return _InprocessHTTP(host)

wsgi_intercept.add_wsgi_intercept('localhost', 80, lambda : wsgi_app)

# this is the "money line of code"
server = xlmrpclib.Server(server_url, transport=_InprocessTransport())

So, all calls to 'server', e.g. 'server.remote_function()', will work through an in-process call directly to 'wsgi_app'. Nifty!

The great thing about this is that for a minimal expenditure of code -- basically, the above -- you are actually testing every part of the XML-RPC connection without going through the hassle of running the actual server. So you can catch errors in any part of your stack with this technique.

--titus