Daily Life in an Ivory Basement

I finally got sick of manually schlepping BLAST files around, so I wrote something to do it for me. 'zounds' is a very simple server/client system for coordinating a bunch of 'worker' nodes through a central server; it does everything in Python with objects and pickling, so it's easy to do extra Python-based processing on the worker nodes. See 'filters' for more info.

You can read a bit more about zounds here:

http://iorich.caltech.edu/~t/zounds/README.html

It's freely available, open-source, etc. etc.

Comments and thoughts welcome; send them to the bip list.

--titus

We've been talking about how to manage pygr resources remotely via the existing XML-RPC interface, and for that HTTPS is a requirement. I offered to track down the code necessary for running an XML-RPC server over HTTPS. Here's what I found:

It turns out that while the Python stdlib supports HTTPS client
connections (connecting to https:// URLs), it does not directly support
HTTPS serving.  To do that, you need to use pyOpenSSL.  However, once
that's installed it's a breeze: it's as simple as this,

  server = SecureXMLRPCServer(server_address, KEYFILE, CERTFILE)

You can download the SecureXMLRPCServer code and an example here:

      http://iorich.caltech.edu/~t/transfer/xmlrpc-https.tar.gz

To run it just install pyOpenSSL ('python-openssl' under Debian),
and then execute 'python serve-ssl.py' in one shell and 'python
test-conn-ssl.py' in another.

Thanks to Laszlo Nagy for his Python Cookbook recipe which only needed a bit of fixing (for Python 2.5) and refactoring (for reusability).

The example .tar.gz above contains a private key and certification so that the code Just Works.

--titus

p.s. Ping me at titus@idyll.org if the .tar.gz file isn't accessible and I'll repost it.

On Thursday, May 15th, I finished my post-doc position at Caltech.

On Friday, May 16th, I officially started as an Assistant Professor split between Computer Science & Engineering and Microbiology & Molecular Genetics at Michigan State University.

On Friday evening and Saturday, we hung out down at the Caltech Marine Lab and partied.

As I type, I'm on a plane flying from California to Michigan, where my cat and I will spend our first night in our new house.

My wife and daughter will join me on Wednesday.

All of our stuff is en route and will arrive later in the week.

On Monday, I will start "directing" my new lab: I already have several local summer students, as well as a part-time research assistant. Two graduate students and a postdoctoral fellow will be starting with me later in the summer.

Hooray!

--titus

I've been hit by a few different e-mail-related problems over the last few months, and it's becoming intensely frustrating.

Some servers seem to randomly drop messages from me, for no obvious reason; at least, people don't get one message and then do get another, a day later. gmail may be at the center of this but it's not clear. (Changing my public From: address to ctb@msu.edu may have contributed to the lossage rate, but cannot be the only cause.)

Yahoo at first decided to mark messages as spam, then drop messages, and now likes me again.

As I write, caltech.edu seems to be bitch-slapping a select subset of my messages. Either they're grey-listing and waiting to forward some messages on, or they've dropped some messages entirely.

My daily e-mail load has grown to the point where I appear to be hand-deleting real messages from my inbox when my eye gets them confused with spam or automated notices. I know of at least two messages that I've deleted in the last month -- I found them in my received-not-spam folder (where I save all incoming messages) but have no recollection of having read them. Before this month, I can't recall having accidentally deleted two e-mails in the last 5 years. And it's not like my spam filter has a high false negative rate: I barely get any spam at all in my inbox.

Now I'm trying to start sending messages to people in my lab, and some of them are not responding or acknowledging the messages. Maybe they're not getting them. Maybe they're out to lunch. Maybe they don't like me, or authority, or something. I don't want to hassle people who don't want to respond, but I also want to make sure they got the $!%#$#$ message! Ah well, I will be physically there soon...

All of this goes to say that for a variety of reasons -- increasing amounts of e-mail, increasing amount of SPAM e-mail, increasingly random and annoying anti-spam measures implemented by the big inbox providers -- e-mail has become unreliable enough that I have to think about it. Even worse, the number and variety of anti-spam measures in play mean that neither I nor the receiver may have any ability to affect the spam filter that is dropping e-mail (translation: I don't know what to do to make things more reliable).

Grr.

I think the time is coming where a reputable SMTP forwarder could make some $$; I'd be willing to pay a $ or $$ for a bonded SMTP provider! Anything like that out there that actually works?

--titus

p.s. In a largely unrelated side note, the number of blog comment spammers attempting to post to my blog continues to hit record highs on a daily basis. I've never approved a spam comment -- yet they continue to try. It's mind-bogglingly stupid and it just goes to show that stupid behavior will continue indefinitely if it's approximately zero-cost to the commit-ee. Grr x 2.

So, err, drop me a line if you wrote a witty comment that didn't get posted, and if I don't accidentally delete your e-mail I will approve your comment.

(pygr is a neat bioinformatics framework in Python.)

After some commenters on my last post seemed happy to hear that pygr was the focus of some summer work, I realized I had only discussed the pygr summer work in a post to the biology-in-python list.

Whoops.

So, here's the scoop: not only is pygr the focus of Rachel McCreary's Google Summer of Code project, but Jenny Qian will be using pygr to build an ENSEMBL interface, also as part of the Google Summer of Code.

That's not all!

In addition to Rachel and Jenny (under the sterling mentorship of Chris Lee, Robert Kirkpatrick, Namshin Kim, and myself) I have two MSU students working with me over the summer, Alex Nolley and Marie Buckner. They'll both be working with pygr-related things, although like Jenny their efforts may end up being more on ways to use pygr than on pygr's code itself.

I also have a grad student or two that may drop in on pygr, if only to use it for something research-y.

So all in all, pygr will get a lot of love this summer. Hopefully we can polish the code and documentation and tutorials to the point where the learning curve is as minimal as it can get, and this fabulous package will become readily available to many others...

Why am I personally putting so much effort into pygr? Well, I've been using it more and more over the last few months, and (somewhat like scipy) it's transformed my work by turning annoyingly difficult data organization problems into trivial Python transformations. I can literally throw together a custom genome browser in a matter of hours -- I've implemented two or three already, for different projects -- and it has enabled several new research program. pygr seems to be one of those rare packages (kind of like Python itself) that is not only functional and effective but presents a unified and coherent intellectual interface. pygr is the only good middleware layer I've seen for sequence intertwingling in bioinformatics. It's not that mature yet, but it has serious promise, and I'm hoping to get in on the ground floor, so to speak :).

cheers,

--titus