snakemake checkpoints r awesome
read moreThere are comments.
Here are talk notes and links for my PyCon 2015 talk.
The talk slides are up on SlideShare.
You should definitely check out Mike Lin's great blog posts on "Blogging my genome".
I found SNPedia through this wonderful blog post on how to use 23andMe irresponsibly, on Slate …
There are comments.
Note: Turns out Nick Loman is a C programmer. Well, that's what happens when I make assumptions, folks ;).
Jared Simpson just posted a great blog entry on nanopolish, an HMM-based consensus caller for Oxford Nanopore data. In it he describes how he moved from a Python prototype to a standalone …
read moreThere are comments.
The apocalypse is nigh. Soon, binary executables and containers in object stores will join the many Web-based pipelines and the several virtual machine images on the dystopic wasteland of "reproducible science."
Anyway.
I had a conversation a few weeks back with a senior colleague about container-based approaches (like Docker) wherein …
read moreThere are comments.
Brian O'Shea (a physics prof at Michigan State) asked me the following, and I thought I'd post it on my blog to get a broader set of responses. I know the answer is "Python 3", but I would appreciate specific thoughts from people with experience either with the specific packages …
read moreThere are comments.
Here's a draft PyCon '14 proposal. Comments and suggestions welcome!
Title: Data intensive biology in the cloud: instrumenting ALL the things
Description: (400 ch)
Cloud computing offers some great opportunities for science, but most cloud computing platforms are both I/O and memory limited, and hence are poor matches for …
read moreThere are comments.
Note: Yarden Katz (the author of MISO) sent me the e-mail below, and I asked him if I could post it as a guest-post on my blog. He said yes - so here it is! Feedback solicited.
---
Hi Titus,
Hope all is well. A recent tweet you had about Ben Bolker's …
read moreThere are comments.
(This blog post was mightily helped by Qingpeng Zhang, the first author of the paper; he wrote the pipeline. I just ran it a bunch :)
We have been benchmarking k-mer counters in a variety of ways, in preparation for an upcoming paper. As with the diginorm paper we are automating …
read moreThere are comments.
I've been reading Peter Seibel's excellent book, Coders at Work, which is a transcription of interviews with a dozen or so very well known and impactful programmers. After the first two interviews, I found myself itching to highlight certain sections, and then I thought, heck, why not post some of …
read moreThere are comments.
I've been reading Peter Seibel's excellent book, Coders at Work, which is a transcription of interviews with a dozen or so very well known and impactful programmers. After the first two interviews, I found myself itching to highlight certain sections, and then I thought, heck, why not post some of …
read moreThere are comments.
Software installation is a real problem.
I'm writing this as I return from my fourth Software Carpentry workshop, or -- if you count the one I ran at LLNL almost a decade ago -- my fifth one. This workshop was taught with Karen Cranston and Rich Enbody, both of them very experienced …
read moreThere are comments.
The tech community is messed up in da head, yo.
Several times since Steve Holden's I'm Sorry post I've written long blog posts about my own views on codes of conduct and professional behavior, including the views informed by some of my own extraordinarily embarrassing transgressions. I never felt that …
read moreThere are comments.
A week or two ago, I posted a crazy idea about crowdsourcing a bioinformatics analysis pipeline. I may still try to do that. But in the meantime, here's another crazy idea.
First, some background.
I'm writing this as I fly back from …
There are comments.
Description
Random algorithms and probabilistic data structures are algorithmically efficient and can provide shockingly good practical results. I will give a practical introduction, with live demos and bad jokes, to this fascinating algorithmic niche. I will conclude with some discussions of how our group has applied this to …
read moreThere are comments.
One of the things that I have struggled with over the years is how to teach people how to actually program -- by this I mean the minute-to-minute process and techniques of generating code, more so than syntax and data structures and algorithms. This is generally not taught explicitly in college …
read moreThere are comments.
We just posted yet another pre-submission paper to arXiv.org:
Assembling large, complex environmental metagenomes
Authors: Adina Chuang Howe, Janet Jansson, Stephanie A. Malfatti, Susannah Tringe, James M. Tiedje, and C. Titus Brown
Abstract:
The large volumes of sequencing data required to deeply sample …read more
There are comments.
We just posted another pre-submission paper to arXiv.org:
Illumina Sequencing Artifacts Revealed by Connectivity Analysis of Metagenomic Datasets
Authors: Adina Chuang Howe, Jason Pell, Rosangela Canino-Koning, Rachel Mackelprang, Susannah Tringe, Janet Jansson, James M. Tiedje, and C. Titus Brown
Abstract:
Sequencing errors and …read more
There are comments.
I am just returning from a trip to Southern California that included, among other things, the teaching of a two day Software Carpentry workshop at The Scripps Research Institute. There were two instructors, myself and Tracy Teal, a research scientist at MSU; and two external TAs, Qingpeng Zhang (one of …
read moreThere are comments.
Inspired by the awesomeness of disqus on my other sites, I wanted to make it possible to enable disqus on my sites on ReadTheDocs. A bit of googling led me to Mikko Ohtamaa's excellent work on the Plone documentation, where a blinding flash of awesomeness hit me and I realized …
read moreThere are comments.
The IPython Notebook (or 'ipynb' for short) is one of the most exciting technologies for teaching and research that I've seen in recent years. It is a completely open source, well architected, and fairly stable system for scientific computing and data exploration.
I've now been using it for teaching for …
read moreThere are comments.
There's been a lot of discussion about PyCon talks that we do want to see. Here's a brief list of those I don't want to see, for those of you considering a submission -- in no particular order.
There are comments.
I've just posted my 2nd try at the NSF CAREER award to the lab Web site, where it joins my recent NSF BIGDATA proposal, my Moore Foundation proposal, last year's (rejected) NSF CAREER proposal, my NGS course grant, and my one big funded grant, my USDA proposal from 2009. The …
read moreThere are comments.
I've just moved my blog over to Pelican, a static blog generator that takes in reStructuredText and spits out, well, this! I'm now using Disqus for commenting, too.
The main motivations for the move (apart from slightly better theming) were to escape dynamic-blog-land in favor of static-blog-land, while enabling a …
read moreThere are comments.
As part of the 2012 Analyzing Next-Generation Sequencing Data course, I've been trying out ipython notebook for the tutorials.
In previous years, our tutorials all looked like this: Short read assembly with Velvet -- basically, reStructuredText files integrated with Sphinx. This had a lot of advantages, including Googleability and simplicity; but …
read moreThere are comments.
I'm a pretty big advocate of anything open -- open source, open access, and open science, in particular. I always have been. And now that I'm a professor, I've been trying to figure out how to actually practice open science effectively
What is open science? Well, I think of it as …
read moreThere are comments.
I'm going to pick on Mick Watson today. (It's OK. He's just a foil for this discussion, and I hope he doesn't take it too personally.)
Mick made the following comment on my earlier Big Data Biology blog post:
read moreI do wonder whether there is just a bit too much …
There are comments.
I'm out at a Cloud Computing for the Human Microbiome Workshop and I've been trying to convince people of the importance of digital normalization. When I posted the paper the reaction was reasonably positive, but I haven't had much luck explaining why it's so awesome.
At the workshop, people were …
read moreThere are comments.
I'm pretty proud of our most recently posted paper, which is on a sequence analysis concept we call digital normalization. I think the paper is pretty kick-ass, but so is the way in which we're approaching replication. This blog post is about the latter.
(Quick note re "replication" vs "reproduction …
read moreThere are comments.
We just posted a pre-submission paper to arXiv.org:
A single pass approach to reducing sampling variation, removing errors, and scaling de novo assembly of shotgun sequences
Authors: C. Titus Brown, Adina Howe, Qingpeng Zhang, Alexis B. Pyrkosz, and Timothy H. Brom
Paper Web site, with source code …
read moreThere are comments.
I'm writing this on my way back from Stockholm, where I attended a workshop on the 4th Paradigm. This is the idea (so named by Jim Gray, I gather?) that data-intensive science is a distinct paradigm from the first three paradigms of scientific investigation -- theory, experiment, and simulation. I was …
read moreThere are comments.
I desperately need something to run and test things at the command line, both for course documentation (think "doctest" but with shell prompts) and for script testing (as part of scientific pipelines). At the 2011 testing-in-python BoF, Augie showed us cram, which is the mercurial project's internal test code ripped …
read moreThere are comments.
First, I write a recipe file, 'metagenome.recipe', laying out my job description for, say, sequence trimming and assembly with Velvet:
fasta_file soil-data.fa qc_filter min_length=50 remove_Ns=true graph_filter min_length=400 velvet_assemble k=33 min_length=1000 scaffolding=True
Then I specify …
read moreThere are comments.
I just parachuted in on (and heli'd out of?) the Beyond the Genome conference in Boston. I gave a very brief workshop on using EC2 for sequence analysis, which seemed well received. (Mind you, virtually everything possible went wrong, from lack of good network access to lack of attendee computers …
read moreThere are comments.
I've spent the last few weeks working on a simple solution to a challenging problem in DNA sequence assembly, and I think we've got a nice simple theoretical solution with an actual implementation. I'd be interested in comments!
Briefly, the algorithmic challenge is this:
We have a bunch of …
There are comments.
After my recent next-gen sequencing course, which was supposed to tie into the whole software carpentry (SWC) effort but didn't really succeed in doing so the first time through, I started thinking about the Right Way to tie in the SWC material. In particular, how do you both motivate scientists …
read moreThere are comments.
Laurie Dillon just posted the SIGPLAN eduction board article on Why Undergraduates Should Learn the Principles of Programming Languages to our faculty mailing list at the MSU Computer Science department. One question that came up in the ensuing conversation was: what functional programming language(s) would/should we teach?
I …
read moreThere are comments.
Our sequencing analysis course ended last Friday, with an overwhelmingly positive response from the students. The few negative comments that I got were largely about organizational issues, and could be reshaped as suggestions for next time rather than as condemnations of this year's course.
The 23 students -- most with no …
read moreThere are comments.
So, I've been teaching a course on next-generation sequence analysis for the last week, and one of the issues I had to deal with before I proposed the course was how to deal with the volume of data and the required computation.
You see, next-generation sequence analysis involves analyzing not …
read moreThere are comments.
So, I'm running this summer course and I am trying to figure out how to organize the notes for students. I'd like to mix curriculum-specific notes ("here's what we're doing today, and here are some problems to work on") with tutorials (material independent of a single course, like "here's how …
read moreThere are comments.
Inspired by the brilliant mind(s) behind python-commandments.org, here's a list of ideas you can use to help newbies learn Python!
There are comments.
I've been doing some more focused bioinformatics programming recently, and as I'm thinking about how to teach biologists about data analysis, I realize more and more how much backstory goes into even relatively simple programming.
The problem: given a reference genome, and a very large set of short, error-prone, random …
read moreThere are comments.
These days, molecular biologists are dealing with lots and lots of sequences, largely due to next-gen sequencing technologies. For example, the Illumina GA2 is producing 100-200 million DNA sequences, each of 75-125 bases, per run; that works out to 20 gb of sequence data per run, not counting metadata such …
read moreThere are comments.
Analyzing Next-Generation Sequencing Data
May 31 - June 11th, 2010
Kellogg Biological Station, Michigan State University
CSE 891 s431 / MMG 890 s433, 2 cr
Applications are due by midnight EST, April 9th, 2010.
Course sponsor: Gene Expression in Disease and Development Focus Group at Michigan State University.
Instructors: Dr. C. Titus …
read moreThere are comments.
A new meme was born at PyCon 2010: The Testing Goat.
Or, "Be Stubborn. Obey the Goat."
The goat actually emerged from the Testing In Python Birds of a Feather session at PyCon, where Terry Peppers used slides full of goat in his introduction. This was apparently an overreaction to …
read moreThere are comments.
On the heels of my aggressive competence post, about (among other things) my failure to outline my expectations for students, I've started putting together a page to help manage student expectations for the pony-build project, which is participating in the Undergraduate Capstone Open-Source Projects (UCOSP) course this term.
(Please comment …
read moreThere are comments.
I just finished teaching Concepts in Database-Backed Web Development for the second time -- the post-mortem from the first course is here.
In the course, the students implement a reasonably complete HTTP server from the socket library on up, and integrate CSS, JavaScript (jQuery), and a little bit of databases into …
read moreThere are comments.
I've recently turned my basilisk eye from Web testing and code coverage analysis to continuous integration, as you can see from my PyCon '10 talk and my UCOSP proposal, not to mention everyone wants a pony.
There's some confusion about what "continuous integration" means (see Martin Fowler on CI) so …
read moreThere are comments.
This last term I facilitated the participation of five MSU students in the Undergraduate Capstone Open Source Projects (UCOSP) program, in which students do distributed open source software development and receive home institution credit. UCOSP was managed out of U Toronto by Greg Wilson, and I was (and am) enthusiastic …
read moreThere are comments.
or, "those python-dev people are awesome."
My experience with the Python bug tracker has been pretty sparse and largely limited to some of the eternaissues like "make HTMLParser deal with even more broken HTML" that never really get resolved because they're not very important and don't have a champion. So …
read moreThere are comments.
Does anyone have any experience with CloudStore, formerly known as KosmosFS? From http://en.wikipedia.org/wiki/CloudStore:
CloudStore (KFS, previously Kosmosfs) is Kosmix's C++ implementation of Google File System. ... CloudStore supports incremental scalability, replication, checksumming for data integrity, client side fail-over and access from C++, Java and Python.
The …
read moreThere are comments.
Since a few people have asked, here's a rough guide to the diversity discussion. No specifics allowed.
1. diversity list created to (among other things) ponder an official diversity statement for Python. List is closed-archive but open for general subscription.
2. Various diversity list discussions become heated. Some people (including …
read moreThere are comments.
As I wrote over the weekend, the Google Highly Open Participation contest (intended to get high-school students involved in open source work) may be run again this winter. I say "may", because quite a bit of work needs to be done on the GHOP hosting app, Melange.
We in the …
read moreThere are comments.
In the interests of social anthropology, I feel compelled to point Pythonistas at this fascinating discussion on the stdlib-sig on adding argparse to the Python stdlib. (Yeah, it's pretty much the only traffic that list got so far this month.)
Fascinating stuff. If there's a secret cabal out there masterminding …
read moreThere are comments.
I'm looking for examples of frustratingly simple-yet-wrong Python code, suitable for an undergrad class to debug. I'd prefer things that don't rely on tricky features of Python (like shared list references), but rather code where subtly bad logic or program flow leads to bad behavior.
Comment below, or e-mail me …
read moreThere are comments.
Very odd. I mean, it's nice to have my prayers spell-checked and all, but really, Apple? Cthulhu?
Also, jinja2 rocks. I think I'll be teaching it as a templating language this term...
And finally, people interested in using sqlite3 for shelve-like storage in Python 2.x can take a look …
read moreThere are comments.
Sarah Mei posts about teaching Ruby to high school girls. Good stuff.
While searching for some GHOP info from way back, I ran across this post asking "where are the girls among the GHOP winners?" (The statistics mentioned in the post may have been posted since, although I haven't seen …
read moreThere are comments.
So, it's nomination season for the Python Software Foundation again... and I have this niggling feeling that I'm forgetting about several people that have demonstrated significant commitment to the Python community, are good 'uns, and are otherwise people I would trust with some part of the future of Python and …
read moreThere are comments.
Courtesy of Rich Enbody, this blog post, How XML Threatens Big Data -- Dataspora, elicited a big "duh" from me.
You don't solve any of the semantic problems with data by elaborating on a textual format. You may bring them into the light, but along with the visibility comes "bureaucracy" -- technology …
read moreThere are comments.
Victoria Laidler just announced a Pandokia release. She gave a great lightning talk on Pandokia at the PyCon '09 testing BoF and I've been looking forward to this release.
Pandokia seems like a nice way to manage test running and results analysis & reporting, and it fits a fairly unique niche …
read moreThere are comments.
After being on the new Python diversity mailing list for a bit, I've just unsubscribed. While there was an unpleasant personal incident that catalyzed my decision, I also don't think I'm a good fit for the style of discussion taking place. (YMMV ;)
That having been said, I want to give …
read moreThere are comments.
I'm nominally involved in co-mentoring or cheerleading 5 Google Summer of Code projects this summer, and several of the students have the same problem: they send me one big e-mail (or post one big blog entry), every few weeks, asking for input.
This imposes a big energetic barrier to me …
read moreThere are comments.
The last two weeks were pretty miserable, for some scientific/collaboration reasons as well as some personal reasons (visiting sick parents != fun). Two things that weren't miserable -- that were in fact quite fun -- were PyOhio and the Science 2.0 talks in Toronto.
PyOhio was a nice little community-based conference …
read moreThere are comments.
This TDD anti-pattern catalogue is truly excellent!
--titus
Legacy Comments
Posted by J Klassen on 2009-06-11 at 02:41.
hey, i used that whilst studing TDD for a CMPT 376 (tech writing) paper.
Posted by Paul Hildebrandt on 2009-06-12 at 23:06.
Great link, thanks!read more
There are comments.
I'd like to find an MSU student to report semi-monthly on python-dev. The student would be responsible for monitoring the python-dev mailing list and active PEPs, summarizing substantive discussions in a public forum, and integrating feedback from the community. This would be a 1 credit CSE independent study course (CSE …
read moreThere are comments.
Apparently the ipaddr module in Python 3.1 is disliked by some, and there was a reasonably robust discussion on python-dev about how it's wrong, wrong, wrong. Guido finally ruled: ixnay on the addr-pay.
This is pretty relevant given the twitstorm caused by Zed Shaw's ludicrously self-confident rants about how …
read moreThere are comments.
I just submitted a Mellon Award for Tech Collaboration nomination for the Python Buildhaus. What's that, you ask?
read moreThe Python Buildhaus is a project to systematically build, test and release Open Source Python packages on Windows, Mac OS X, and a wide array of other UNIX architectures and operating systems …
There are comments.
Just submitted this on Thursday:
Next generation sequencers are beginning to impact agricultural biology. Over the next few years, next generation sequencing will produce incredibly large datasets that will address structural (e.g., SNPs, CNVs, indels, methylation, translocations) and functional (e.g., RNA expression, transcription factor binding sites) variation in …read more
There are comments.
I'm writing some proposals to expand support for Python infrastructure (think cross-platform build and test farms a la Snakebite) and for the Mellon Foundation application, I'd like to find out how Python is being used in the humanities. I found NLTK, the Natural Language Toolkit; what else is big?
thanks …
read moreThere are comments.
I'd like to invite you to attend the last of the Michigan State University CSE colloquia for the 2008-2009 academic year: jointly sponsored as an AT&T Visiting Lecturer by the MSU LCT, and the CSE department, Sam Ramji will speak about
Open Source at Microsoft: The Past, Present and …read more
There are comments.
As part of a CiSE submission I'm working on, I interviewed the lead developer on a scientific software package today. This software package is mainly used for evolutionary studies, and has a small but devoted following - ~6 developers and ~12 users locally, plus a few dozen users outside of MSU …
read moreThere are comments.
Open source coding is like a not-so-demanding mistress: I work on it at night, surreptitiously, after my wife and daughter are asleep. twill and figleaf are like bastard children, who only get attention when I can spare it from my "real" family (my teaching, research or my actual family, depending …
read moreThere are comments.
Anyone out there used disco (http://discoproject.org/)? Comments, good/bad/neutral?
From the page:
read moreDisco is an open-source implementation of the Map-Reduce framework for distributed computing. As the original framework, Disco supports parallel computations over large data sets on unreliable cluster of computers.
The Disco core is written in …
There are comments.
My talk this year kinda sucked -- more on that later -- and I am trying to come up with good and perhaps even non-testing talk ideas for next year.
One intriguing idea contributed by Brian Dorsey is that of giving 5 lightning talks in a 30 minute session. Since I like …
read moreThere are comments.
John Gall apparently said:
A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with …read more
There are comments.
Just a short note with characteristic inhumility (ahumility? abhumility?) -- for my Concepts in Database-Backed Web Programming course, I received the Withrow Award for Teaching Excellence from the students.
This means a lot to me, because I spent a huge amount of time on that course (and will have to do …
read moreThere are comments.
'tis the season, and so it's time for me to post my list of accumulated project ideas. I'll transfer these over to the wiki tomorrow, after I track down some references. I'm willing to mentor any or many of these but I'd prefer to find someone to be the primary …
read moreThere are comments.
I recently had the pleasure of being the technical reviewer for a new Apress offering, Beginning Python Visualization, by Shai Vaingast.
To quote from the apress page,
read moreWhat you'll learn:
- Write ten lines of code and present visual information instead of data soup.
- Set up an open source environment ready …
There are comments.
My last post, Good code coverage: Necessity vs Sufficiency, about how you should maintain high code coverage with your automated tests, seems to have really struck a nerve in a small group of people -- I got some fantastic comments, with some great pointers. Michael Foord's comment, 'Too often "testing is …
read moreThere are comments.
I get really frustrated with posts that claim your unit tests lie to you or 100% code coverage is fallacious or there are flaws in coverage measurement. These are sensationalist headlines that encourage bad behavior, by confusing new or inexperienced or argumentative or lazy developers: "well, we all know test …
read moreThere are comments.
Over at OLPC News, Wayan Vota asks: "Do you get better FOSS code ... if the developers are paid or unpaid?"
Interesting question, especially as considered in light of the OLPC code base.
As of about a year ago, virtually everybody I talked to was shocked and stunned at the poor …
read moreThere are comments.
Infoworld did a brief writeup of Snakebite; my only quibble is that MSU comes off as a more of a passive partner in the article than we actually are ;).
--titus
Legacy Comments
Posted by titus brown on 2009-02-14 at 10:46.
foo
Posted by titus brown on 2009-02-14 at 10 …
read moreThere are comments.
I'm switching several projects from darcs to either git on github, or svn on Google Code.
twill, a simple Web testing tool/HTTP driver in Python, was switched over to Google Code several months ago: see http://code.google.com/p/twill. I'll post more on twill development soon, I …
read moreThere are comments.
As Jesse writes, Trent revealed the existence of Snakebite yesterday. Snakebite is an "open network" of various machines that Trent and others (myself included) are making available to the Python community for build and debug purposes. I'm coordinating the MSU component, which basically means that I run interference for Trent …
read moreThere are comments.
A fellow prof here at MSU, Rich Enbody, whipped up the following cheat-sheet for new programmers transitioning from Python (CSE 231) to C++ (CSE 232). He welcomes comments. Here's the link:
http://web.cse.msu.edu/~cse231/python2Cpp.html
Paranthetically, he and his cohort in crime, Bill Punch, will be …
read moreThere are comments.
A friend asks,
i'm going to be recoding <x> from scratch starting next week, in python. what codebase would you recommend as good to model after?
Any thoughts on a well-formed, reasonably sized (yet not huge), and simple Python code base?
There have to be some examples somewhere! I'd suggest …
read moreThere are comments.
The decision of python-dev to deprecate bsddb has left us in a bit of a pickle (hah!) over in the pygr project. We're looking for a replacement for bsddb for default storage of infrequently- (or never-) changed pickled Python objects. Some of the parameters under consideration are:
read more
- Python version availability …
There are comments.
(This blog post is a long, rambling retrospective on my recent undergrad comp-sci course at Michigan State U., newly renamed to "Concepts in Database-backed Web Programming".)
I set out this term to teach a CS class in the way I would have wanted it taught when I was an undergrad …
read moreThere are comments.
Here's a summary of the e-mail responses I got to my lazyweb query re code review and git.
A few people (Paul Nasrat and Jeff Balogh) pointed out that Review Board supports git. So I may try that.
Charles McCreary has had good experience with Rietveld, but I don't think …
read moreThere are comments.
The pygr project is gearing up to do some code reviews, and we're not aware of too many (any?) mature (or even adolescent) tools that interact well with git. A Google search finds Gerrit and a blog post about Code Review -- anything else we should know about?
thanks, --titus
p …
read moreThere are comments.
We're going through the PyCon '09 review process, and participating in the process has been pretty interesting. (I joined the Program Committee in large part because I was told to put up or shut up after I critiqued PyCon '08. Ahh, the open source world... where you're encouraged to go …
read moreThere are comments.
The ongoing debate about doctests (here, and links therein) seems to me to be somewhat silly.
doctests should be assessed by their utility to you and your project, in whatever role you happen to be using them. I personally find them to be very useful in API documentation, where they …
read moreThere are comments.
As a new prof, I've been too busy to blog much. What am I doing?
Apart from all the normal academic crud (meeting with people, answering e-mail, doing paperwork, etc.) and parenting & home ownership stuff, I've been teaching my Intro to Database-Backed Web Programming course. This has been neither a …
read moreThere are comments.
This post on why academics should work to integrate their research ideas with open source software in order to actually push forward on the OSS side (and presumably vice versa) is really good. I think such things could be applied in software testing, too, where there's a gulf between how …
read moreThere are comments.
My last post initiated a discussion on the biology-in-python mailing list about BioPython, among other things. (Here is a link to the discussion, which is kind of long and unfocused.)
I'm happy that the bip list is serving as a place for people to interact with the BioPython maintainers to …
read moreThere are comments.
Chris Lasher wrote a nice blog post naming me as a rabble rouser in the area of "Python in bioinformatics". His post raised a number of interesting points, some of which I'd like to discuss here on my blog.
First, why is Python not more dominant in bioinformatics? I really …
read moreThere are comments.
I'm surprised I haven't seen this on planetpython yet...
...an emerging consensus in the scripting community holds that Python is the right solution for freshman programming. Ruby would also be a defensible choice.
(emphasis mine). Originally found via Lambda the Ultimate, and also passed onto me by Rich Enbody.
In …
read moreThere are comments.
In reply to elanthis's post on Advogato,
1. I agree that the documentation could be improved, and we've been working on it. The next release should add a whole bunch of examples. Google is your friend, as is the Python Cookbook.
There are comments.
Recently the question came up: suppose you wanted to give enthusiastic people some guidance on how to help work on Python. What suggestions do you have? Surely there's a Web page on this!
Well, no: a few quick Google searches led me to discover that "contributing to Python" was answered …
read moreThere are comments.
I finally got sick of manually schlepping BLAST files around, so I wrote something to do it for me. 'zounds' is a very simple server/client system for coordinating a bunch of 'worker' nodes through a central server; it does everything in Python with objects and pickling, so it's easy …
read moreThere are comments.
We've been talking about how to manage pygr resources remotely via the existing XML-RPC interface, and for that HTTPS is a requirement. I offered to track down the code necessary for running an XML-RPC server over HTTPS. Here's what I found:
It turns out that while the Python stdlib supports …read more
There are comments.
(pygr is a neat bioinformatics framework in Python.)
After some commenters on my last post seemed happy to hear that pygr was the focus of some summer work, I realized I had only discussed the pygr summer work in a post to the biology-in-python list.
Whoops.
So, here's the scoop …
read moreThere are comments.
Dear Lazyweb, help!
I'm embarking on a number of summer projects in my new lab at MSU, and several of them focus on using pygr to do cool genomic stuff. In particular, I'm planning to build a personal genome annotation system that will let people run their own full genome …
read moreThere are comments.
So I'm pretty bullish on testing for maintenance reasons. It was nice to see how well it worked out for me when a user recently reported a problem with Cartwheel.
This is what happened: third-party package (LAGAN) that the user was running through the Web interface depended on certain command-line …
read moreThere are comments.
I'm having a long-running discussion with some people about threading and why using threads with simple subprocess calls is almost certainly an overcomplicated (== BAD) use of threads. Everyone seems to think I'm wrong (at least, there's either deafening silence or straight out argument ;) and I think I finally figured out …
read moreThere are comments.
In some discussions with a moderately new Python programmer who seems to value complexity over simplicity, I may have coined a new term:
"Penis size" style of programming -- the (mistaken) belief that the more advanced programming language features you use, the more impressive your code will look.
I think it's …
read moreThere are comments.
Pavel Vinogradov <fastnix> has been keeping me updated on an issue he discovered while testing TCMalloc with Python as a Google Highly Open Participation (GHOP) task, task 105.
Briefly, Pavel discovered a situation in which replacing the Python memory allocator with TCMalloc resulted in really bad performance. The latest is …
read moreThere are comments.
At Google Campfire One, v 2.0 -- introducing AppEngine.
IT'S FREEZING. The cider ran out. Brr.
Deploying Web apps is annoyingly difficult. Technical hurdles, etc. Need machines. Blech. Costly.
AppEngine solves all these problems. Runs web apps, handles app lifecycles, apps are run on Google infrastructure can make use of …
read moreThere are comments.
Please send this on to anyone who might be interested...
Disney Animation has an opening for a summer intern to work on a testing project under the supervision of Paul Hildebrandt and Dr. C. Titus Brown. The ideal candidate will have experience with a dynamic language supporting introspection (Python preferred …
read moreThere are comments.
From Wages or Shortage, this comment
""" A-grade engineers are unfortunately similar to Welsh longbowmen: devastatingly potent compared to their peers, but you have to start their training at age 10 or so. Simply upping the salaries of A-grade engineers won't magically create more of them. We know this, as we …read more
There are comments.
On top of dreamhost dropping off the 'net just when I posted a bunch of screencasts... our socal-piggies meeting nearly got whacked because this month's organizer uses Yahoo, and most of the messages going through my mail server (which hosts the mailing list) were filed as "spam".
Now Yahoo is …
read moreThere are comments.
I put together an unofficial screencast about the Google Summer of Code based on a pitch I gave to the Michigan State undergrad CSE population. Enjoy. Please forward on to anyone who might be interested...
--titus
Legacy Comments
Posted by Leo Soto on 2008-03-27 at 09:16.
idyll …read more
There are comments.
The simple application I demoed at PyCon '08 during my talk on the OLPC and testing is now available: I call it "peekaboo".
peekaboo is a way to watch your code being executed in another process using sys.settrace, figleaf, and XML-RPC.
The two screencasts below should explain it.
The …
read moreThere are comments.
At PyCon '08, I gave a talk on testing and the OLPC project where I referred to the "Testing Death Spiral". My accompanying slide, which aimed to be simple rather than comprehensive, had this scenario:
Write a bunch of code & manually test it.
(Good so far.)
Start adding features over …
There are comments.
Here are some of the materials from my PyCon activities:
The tutorial source code (which is really just a cut down version of my PyCon '07 talk's source code; see the README and my blog post from back then).
(Sorry about …
read moreThere are comments.
Kumar MacMillan pointed me towards GUITAR, a framework built by Atif Memon and others. There's a YouTube video of him and Adam Porter talking at the Google Test Automation Conference (2007).
Looks and sounds interesting. Also nice to note that GUITAR is being open-sourced...
--titus
Legacy Comments
Posted by Kumar …
read moreThere are comments.
Just left PyCon yesterday; now I'm up in Michigan looking at some more houses, arranging lab stuff, talking with people, and getting ready to prosyletize the Google Summer of Code to a bunch of Michigan State CSE students as well as a few professors.
Some freeflow thoughts. Feel free to …
read moreThere are comments.
Steve Holden and Doug Napoleone both attended our testing tutorial, (as did AMK, which was a bit of a surprise!), and had fairly positive things to say about it. This was a relief, because Grig and I always wonder whether or not this stuff is useful to anyone.
Our hope …
read moreThere are comments.
I'm at PyCon, and I have a tip for people: don't stay at the same hotel as everyone else.
My hotel room Internet connection is great, perhaps in part because I'm not sharing it with the rest of the PyCon attendees! (Plus it's free -- the DoubleTree wireless Internet doesn't charge …
read moreThere are comments.
The blognet is full of people posting their own opinions, and that's a good thing. What is a little less supportable is flawed argumentation.
I recently spent some time discussion a post about software engineering; I was trying to figure out why the author thought what he did. The annoying …
read moreThere are comments.
I spent some time over the last week adding fairly simple motif searching to Cartwheel, my bioinformatics site for biologists doing cis-regulatory analysis of genomic sequence. The new features include the ability to define and search with IUPAC and position-weight matrix (PWM) motifs, as well as visualization of motif search …
read moreThere are comments.
I'm having trouble with some tests of a PostgreSQL-based system. Briefly, I have a set of functional tests that
- create a new database
- populate it with a data model
- run a Web server (in-process)
- test the integrated Web server - database functionality
The tests are now slow enough that I'm averse …
read moreThere are comments.
Noah and Grig have been CCing me on a conversation about JoelOnChecklists and Grig's post. Noah's writing a book chapter on this stuff, and asked for some tips.
Here are mine.
First, I have a bunch of individual twill scripts in a directory that are run every hour. These scripts …
read moreThere are comments.
Tracy recently asked me if there were any good guidelines about how to write configuration files -- not coding-level guidelines, but guidelines on structure and content.
I was unable to come up with anything: my Google-fu failed me, and my DevonThink database was silent (although it did have some nice testing …
read moreThere are comments.
Via http://www.nodalpoint.org/2008/01/18/one_thousand_databases_high_and_rising, on the Nucleic Acids Res "database" issue:
As we pass the one thousand databases mark (1kDB) I wonder, what proportion of the data in these databases will never be used?
This is an unsettling thought for …
read moreThere are comments.
As many people have doubtless read, PyCon '08 has announced the tutorial sessions. This year, Grig and I are doing a workshop-tutorial on testing rather than a teaching-tutorial; what this means is that our tutorial will focus on actually applying testing tools effectively to your source code.
We're billing this …
read moreThere are comments.
Steve Yegge recently wrote a long article, "Code's Worst Enemy", about how "many lines of code" causes problems in projects.
That's obviously pretty silly. To see why, let's examine a little project I've recently started; conservatively, I estimate that it incorporates well over a million lines of code:
print 'hello …read more
There are comments.
Brett Cannon notes that GHOP is working out well, and muses about the future of the CPython test infrastructure (among other things). This is something I'm interested in as well (guess where all those testing tasks in GHOP came from? ;) and I've been confused, if not frustrated, by the apparent …
read moreThere are comments.
Andrew Binstock hits the nail perfectly on the head with his post, Beautiful Code vs Readable Code.
--titus
read moreThere are comments.
Matt Harrison's post, Gnome devs too lazy for python?, and the linked post by Thomas Vander Stichele strongly typed, are both really entertaining and illuminating.
I preserve them here for my own reference.
--titus
read moreThere are comments.
So, the Google Highly Open Participation Contest is going quite well for Python, with about 25 additions/reworkings of core CPython documentation and tests and dozens of contributions to other projects.
One of the really unique opportunities that GHOP offers for the Python community is being underutilized, however, and that …
read moreThere are comments.
My future colleague, Bill Punch at MSU, is teaching Python to intro CS students. He asks (slightly edited):
In C++, you can write multiple constructors, each one taking a different type and/or number of arguments. Let's say we are writing a RationalNumber class. I could write 2 constructors:
class …read more
There are comments.
If anyone is having a Python Interest Group meeting this month, please consider devoting 15-30 minutes to coming up with random task ideas for the Google Highly Open Participation Contest.
Briefly,
read more
- tasks must involve Python and Open Source;
- non-core pet projects are welcome;
- building screencasts, updating documentation, and adding unit …
There are comments.
Hi folks,
just a quick note -- Kumar McMillan has offered to take over wsgi_intercept. You can see the new project over at code.google.com, http://code.google.com/p/wsgi-intercept/.
While I will miss the income from the project, I think that Kumar will treat it well.
--titus …
read moreThere are comments.
I'm going through some of my saved up e-mail from the last few months, and found these two gems.
Noah Gift on grokking threads, from the testing-in-python list:
Trying to understand what massive pools of threads that spawn other massive spools of threads, that spawn other massive pools of threads …read more
There are comments.
Here are some factoids about the GHOP/Python project, from the end of the first 30 hours.
Of 63 tasks, 32 remain unclaimed.
26 tasks have been claimed by students and are being worked on; some of those are nearing completion.
5 have already been completed: all three Rosetta Code …
read moreThere are comments.
In trying to pull together ~50 tasks for the initial Python part of the Google Highly Open Participation Contest, I ran into some interesting issues.
First: it is not easy to find "easy" or "intro" tasks for projects.
I sent out a lot of e-mails and posted a blog request …
read moreThere are comments.
Based on the current "burn" rate of Python's task list for the Google Highly Open Participation Contest, I would estimate that we will need more tasks within at most two weeks. The current mentors are certainly capable of coming up with more tasks, but we would welcome input and ideas …
read moreThere are comments.
I recently gave an informal talk on Software Carpentry for the Caltech e-Science 101 course. Since even "Intro Software Carpentry" is a whole course of study, I obviously couldn't cover much, but I tried to motivate people to get interested. And, of course, I pushed testing. TESTING, DAMNIT!
Anyway, here …
read moreThere are comments.
So, next May I'm starting as an assistant professor split between the Computer Science and Microbiology and Molecular Genetics departments at Michigan State U., and I'm interested in attracting as many good CS grad applicants as I can from the open source and bioinformatics communities. (I would also like to …
read moreThere are comments.
Here's a small collection of links to take the edge off my general grumpiness at how much software really does suck. Presumably more will follow.
Firefox will be released with many known bugs.
I switched away from Firefox months ago, because it was so unstable and broken. I'm now using …
read moreThere are comments.
Calling all Pythonistas,
What is your personal list of "modules you love and hate" -- stdlib modules that you use all the time, but that have weak documentation, poor examples, or are otherwise difficult to use the first time?
Here are five modules that I think could use some documentation help …
read moreThere are comments.
Hi all,
I need to find some useful projects for new, young contributors, especially in the area of 3rd party packages; we've been thinking about things like porting 3rd party packages to Py3K, adding tests to existing projects, and providing Windows binary eggs for various packages. Everything would be open …
read moreThere are comments.
Some books make me want to leap up out of my chair and go change the world, or die trying.
Charles Stross' Halting State is not one of them.
However, it is a darn good read, and -- for those of you who are into the Internet, Python, MMORPGs, and/or …
read moreThere are comments.
Hey everyone,
Grig and I are thinking of doing another tutorial at PyCon '08, but we'd like to break out of the mold of "intro testing" and do something more exciting for us. Here's a promotional blurb:
Practical Agile Web Testing --------------------------- Have Web site? Need testing? Bring your tired (code …read more
There are comments.
I'm happy to announce that the Python Software Foundation is part of a new Google Open Source program, the Highly Open Participation Contest. This contest is an effort by Google to engage pre-college students in open source programming: Google is offering prizes and awards for completing a variety of tasks …
read moreThere are comments.
I am in need of some test automation tools for a GTK/GNOME GUI, and after scanning the Python Testing Tools Taxonomy and doing a bit of investigating, it looks like there are three possibilites.
dogtail and the Linux Desktop Testing Project both use accessibility libraries to drive GNOME applications …
read moreThere are comments.
I spent a fairly satisfying two days setting up unit and functional tests for figleaf, my code coverage package. I implemented the tests using a technique that I gather is called vertical slicing -- I just called it "starting a new phase of a project" before ;). The idea is that you …
read moreThere are comments.
Every few days, a new hysterical screed against formal education, undergrad studies, or grad studies comes across my screen. As a result I've been mulling over the hot-button issue of academic study, both undergrad and grad, for the last few months. Why are there so many loud people decrying the …
read moreThere are comments.
If you use sys.settrace to set a tracing function, and that function prints to sys.stdout`, then don't ever trash ``sys.stdout, even briefly. You will raise an invisible exception and your trace function will be removed.
(I don't know precisely what is supposed to happen when a trace …
read moreThere are comments.
I haven't seen Kyle Wilson's article, Software is Hard, making the rounds yet... worth reading!
--titus
Legacy Comments
Posted by Noah Gift on 2007-10-03 at 08:15.
Great article, at least it is a comfort that some people admit it is hard and that "software is hard" is becoming more …read more
There are comments.
There's been lots of discussion recently about the global interpreter lock, and how evil it is. (I personally don't think it's evil. More on that some day when I write code again, if ever.)
Then I read this article, on parrot and Threading Building Blocks, and somethign clicked. I quote …
read moreThere are comments.
After our long software licensing discussion on the biology-in-python list, I realized that I wanted something different in a license for scientific software.
Specifically, I would like to attach the following clause to either a BSD or L/GPL style license:
Publications relying on derivative works of this software must …read more
There are comments.
This month the newly minted biology-in-python mailing list erupted into a discussion of licenses. There was some confusion about the goal of the discussion, for which I'm largely responsible: we didn't make it clear that we were talking about licenses for code and content posted on the bio.scipy.org …
read moreThere are comments.
In the spirit of cleaning up my desktop... here's a PDF of my talk on Cartwheel at SciPy 2007.
--titus
read moreThere are comments.
I'm now listed on the Gene Expression in Disease and Development page, as well as on the CompSci faculty page, MicroMolecularGenetics faculty page, QuantBio page, and SysBio page.
It was quite a shock to log into the CompSci cluster at MSU and see my group set as "faculty". As a …
read moreThere are comments.
So, a few people commented on my how to write Python code that doesn't suck post, and I thought I'd respond here rather than in the comments.
First, John Camara suggests adding the MIT license as an option. I chose the BSD because it's essentially equivalent to the MIT license …
read moreThere are comments.
My last post on this subject got a number of good comments, both here and on the biology-in-python mailing list, so I've amended and updated it. (Note that Brandon King is now listed as a contributor.)
I would particularly appreciate comments on the licensing section and the conclusions. Also, I'll …
read moreThere are comments.
Note: this is ultimately intended for the biology-in-python Wiki at http://bio.scipy.org/. I will release it under a CC license, so please feel free to use it for your own site! --titus
Here are some prescriptions for writing Python code that other Python programmers will find more usable …
read moreThere are comments.
I've been on the run for well over a year -- I started writing my PhD thesis in July '06, just after I got back from teaching at Woods Hole. At the time I was also interviewing for a faculty position at MSU (since offered & accepted). Since then I've defended my …
read moreThere are comments.
It's been a busy few weeks, in part because I've been writing a grant. Last Thursday, I submitted a grant proposal to NIH for their program announcement, Continued Development and Maintenance of Software. The proposal was to continue maintaining Cartwheel, while integrating a new visualization frontend (MUSSA) and a fast …
read moreThere are comments.
I was the official mentor for a Google Summer of Code student this year -- Martin van Loewis was "technical mentor" -- and I found it to be a disappointing experience. At the beginning, I felt guilty about not being more on the ball about pushing the student to do more work …
read moreThere are comments.
I'm in the process of writing up a "when and how to test" screed, and I discovered this:
Karl Fogel's book, Producing Open Source Software, has precisely two keyword hits to testing in the ToC.
WTF?
--titus
Legacy Comments
Posted by Ricardo Niederberger Cabral on 2007-08-22 at 17:00.
I …read more
There are comments.
So, I "organized" a Biology Birds of a Feather at SciPy 2007. This mainly consisted of posting about it and then trying to write stuff on a white board while keeping abreast of the conversation. About 15 people attended.
I didn't get everyone's name and in any case I don't …
read moreThere are comments.
Thanks to a kind invitation by Fernando Perez, I was alerted to a BoF on Python/testing at SciPy. He made the mistake of introducing me as "the resident expert" so I felt even less inhibited than normal, which was hopefully not too problematic...
Gael Varoquaux took notes.
Basically, this …
read moreThere are comments.
I wanted to write a comment on timing unittests, but that blog does not allow anonymous comments and there is no obvious place to e-mail the author.
Bummer.
(The short version of my comment is that getting the basic data out with something like nose is trivial; see my pinocchio …
read moreThere are comments.
There was another mildly amusing incident during the recent SoCal Piggies meeting.
Michael Carter was showing us his incredibly neat Web 3.0 / HTTP PUSH software, orbited, by demoing an interactive IRC client on the Web. He signed onto the ruby-on-rails IRC channel and (this being a Python meeting) asked …
read moreThere are comments.
To get people talking, I've created a "biology-in-python" mailing list. You can subscribe here: http://lists.idyll.org/listinfo/biology-in-python, and you can post to it at bip@lists.idyll.org once you're a member.
This list is a tool/package/library-agnostic list, for people who use Python to work …
read moreThere are comments.
Chris Lee and I would like to set up a Birds-of-a-Feather gathering at SciPy '07. We'll probably have an initial meeting on Thursday, August 16th, and then maybe work into sprint mode for that Saturday.
Contact me if you're interested. No reservations needed, but we should probably all plan to …
read moreThere are comments.
While I'm thinking about SciPy '07, here are a few other notes:
- Chris Lasher and I are thinking about doing a Software Carpentry sprint of some kind. Interested?
- I'd be up for doing a half-day tutorial on "Testing for Scientists" or "Idiomatic Python". Interested?
--titus
Legacy Comments
Posted by Chris …
read moreThere are comments.
It's nice to see Python come out on top for threading.
--titus
Legacy Comments
Posted by Scott Lamb on 2007-06-28 at 12:32.
Hmm, I wouldn't call "better than Perl" validation. "Easier to read the Perl, better threading support than COBOL, more productive than TriMedia VLIW assembler, faster than MUMPS …read more
There are comments.
On Tuesday (June 12), Wednesday, and Thursday I taught the course "Intermediate and Advanced Software Carpentry in Python" at Lawrence Livermore National Labs. This was intended to be an extension of some of the ideas from the Software Carpentry course.
The pre-course course advert, the handouts distributed at the course …
read moreThere are comments.
Does anyone know if there are any faculty programming contests out there?
It'd be fun, and I can't imagine that the competition would be as tough as the student programming contests probably are ;).
--titus
Legacy Comments
Posted by jt on 2007-06-22 at 19:56.
Um... faculty members... programming? You lost …read more
There are comments.
Spoke yesterday with Bill Punch, a CS professor at MSU who is using Python for intro CS. He told me his tale of tracking down what an '@' sign meant (it's a decorator, http://docs.python.org/ref/function.html). Needless to say it took him quite a while to google …
read moreThere are comments.
The O'Reilly bookstore didn't actually let you buy our Introduction to Testing Web Applications with twill and Selenium for the first 24 hours -- some bug on the Web site, apparently...
(See? That's why you need to buy it!)
--titus
p.s. Jason wants to make sure you know that the …
read moreThere are comments.
Check out our e-book, An Introduction to Testing Web Applications with twill and Selenium! And please, let us know what you think...
--titus
read moreThere are comments.
Recently seen on a list (with a nasty archival system that I don't want to link to):
"Don't write a test plan. Instead, test." -- Bret Pettichord
Need I say more?
Probably.
One of the most important tenets of agility (<-- little 'a' ;), in my opinion, is to not overthink. Unless you're …
read moreThere are comments.
So, I've finally put together a patch for subprocess; see replacing-commands-with-subprocess for background, and this python-dev thread for discussion.
You can go grab my patch (or the entire subprocess module) here. I'm interested in comments, either here or via e-mail.
Note that I am putting together a separate docs patch …
read moreThere are comments.
I spent part of Saturday and Sunday working on Cartwheel, and then I spent part of Monday morning nailing down a bug that I thought I'd introduced. The process of tracking down and fixing this bug led me to give serious thanks for version control and automated tests, and at …
read moreThere are comments.
This morning, I described my recent work-related activities to Tracy as "Whack-a-Mole". For those of you who aren't familiar with the Hasbro Whac-a-Mole game, it's a fun game for four year olds where you, err, hit moles on the head with a hammer as they pop up out of the …
read moreThere are comments.
I've just finished the last rough draft of a short e-book, An Intro to Testing Web apps with twill and Selenium. The book has four sections:
- Introduction
- twill intro
- Selenium intro
- Testing an app with twill and Selenium
The "app" being tested in #4 is just the very simple poll …
read moreThere are comments.
According to the silly language quiz, I'm Python!
"You are Python. You are slower than others, but easier to understand. You are a minimalist, who doesn't like clutter."
I don't like to think of myself as slow, but there you are -- that's what I must be. (Perhaps I have a …
read moreThere are comments.
I spent the weekend hacking out a BLAST parsing package with pyparsing.
BLAST is a really common bioinformatics tool used to search large-ish sequence databases, and the NCBI BLAST program is probably the single most heavily used program in bioinformatics by a long shot. Unfortunately, the NCBI folk have a …
read moreThere are comments.
I haven't seen anyone announce this, so I guess I should: there's now a Planet Python/Summer of Code site, http://soc.python.org/, hosted by yours truly.
Enjoy!
--titus
p.s. Regular blogging may resume shortly.
read moreThere are comments.
I'm in the early throes of building tests into my Cartwheel project. Cartwheel was one of the two projects that inspired my Web testing project, twill, so naturally I'm happy to finally be putting twill to good use in my own projects. Naturally the transition from building tools for building …
read moreThere are comments.
After an innocent question was answered positively, I am putting together a patch to deprecate the commands module in favor of a slightly expanded subprocess module (for 2.6).
Briefly, the idea is to add three new functions to subprocess:
output = get_output(cmd, input=None, cwd=None, env=None …read more
There are comments.
From reading Collin Winter's blog he's designing a new unittest module first, and then he's going to ask c.l.p and presumably python-dev about adding it to py3k. So it's not quite the fait accompli I thought it was, which reduces my complaints to mild grumbling.
And, dear lazyweb …
read moreThere are comments.
For my own future reference, as well as to attract people to the fairly high signal testing-in-python mailing list, here are some particularly interesting posts to the TIP list.
Raphael Marvie implements a simple textual specification -> test suite generator.
Kumar McMillan makes some nice suggestions for Michal's "spec" nose plugin …
read moreThere are comments.
I've put together a brief discussion, with links to the source code, surrounding the demos I did at PyCon '07 during my testing tools talk. Here's a brief TOC:
# Demo 1: Testing CherryPy # Demo 2: Testing CherryPy without exec'ing a process # Demo 3: Basic code coverage analysis with figleaf # Demo …read more
There are comments.
Michal Kwiatkowski and I just released pinocchio 0.1, a collection of plugins for nose. It's got my figleaf section recording plugin, figleafsections, and it's also got Michal's spec plugin, which lets you write tests like this:
class TestFoobar: def test_is_a_singleton(self): pass def test_can_be …read more
There are comments.
OK, this is one thing I love about Python -- in my entry on "five things I hate about Python" (which achieved "entry worth bashing" status on reddit; go provocative titles!), I complained about the difficulty in getting patches submitted.
AMK read this and posted about it to the python-dev mailing …
read moreThere are comments.
Brian D. Foy claims that if you want to be an effective advocate for a language, you need to be able to answer the question, "What are five things you hate about it?"
Hmm, lessee. I actually have a tough time coming up with five things off the top of …
read moreThere are comments.
Voidspace makes a mildly provocative post about how testing seems to be more emphasized in the Ruby community.
I'm not entirely sure I agree, but then again I'm firmly embedded in the Python testing community and I might have a slightly biased view ;).
What I am sure about is that …
read moreThere are comments.
I have been asked to submit an outline for a three day course on Python, for ~20 scientists who already know basic Python. On fairly short notice, I came up with the following; what am I missing? (I plan to make the course materials publicly available, of course.)
(Note that …
read moreThere are comments.
Of all the fever-induced hallucinatory things I said at PyCon '07, I'm proudest of this: "I don't do test-driven development; I do stupidity-driven testing. When I do something stupid, I write a test to make sure I don't do it again."
So true.
For readers that don't get it, my …
read moreThere are comments.
Last year, Grig and I flew down to Dallas together to attend PyCon. I don't recall the exact problem, but for some reason the plane didn't go and we ended up flying through Houston and missing our original flight to Dallas. (We did have some unreasonably yummy BBQ in Houston …
read moreThere are comments.
PyCon07 (the biggest PyCon ever!) is just around the corner, and I'm really looking forward to (pardon the term) geeking out for a few days! Python is my hobby, although it leaks over into my work pretty regularly, and I expect to have a great time forgetting about work and …
read moreThere are comments.
Ran across this diary entry from wainstead, and just wanted to mention that this sort of thing (replaying HTTP traffic) is something that I do with scotch quite successfully. It hasn't proven to be particularly useful in my own projects yet, but it has been nice when debugging JavaScript-y web …
read moreThere are comments.
Yesterday for our SoCal Piggies meeting I whipped up something I'd been thinking about doing for a while: sectioning in figleaf recording. (figleaf is my package for Python code-coverage analysis.)
It's easier to show than to explain, but briefly, I added two new functions to figleaf:
figleaf.start_section(name …read more
There are comments.
First, let me mention that I've recruited Duncan McGreggor to the panel to talk about twisted.web and Nevow. That makes it a real party! In random order, we now have:
read more
- Zope: Jim Fulton
- Pylons: Ben Bangert
- Django: Adrian Holovaty
- CherryPy: Robert Brewer
- TurboGears: Kevin Dangoor
- pyjamas: James Tauber
- twisted …
There are comments.
So, I'm the organizer/moderator of the PyCon Web Panel. This will be a forum for maintainers of many of the main Python Web frameworks -- Jim Fulton/Zope, Ben Bangert/Pylons, Adrian Holovaty/Django, Robert Brewer/CherryPy, and Kevin Dangoor/TurboGears -- to discuss their take on Python, the Web, and …
read moreThere are comments.
I don't feel like I need to "defend" twill -- it's successful beyond both my expectations and my cognizance (I have no idea who's actually using it, but it's apparently a lot of people!), but I may need to promote it better. I ran across this post earlier today. It shows …
read moreThere are comments.
After a fantastic presentation on subprocess at the SoCal Python Interest Group last June, I bought into it completely. That recently translated into making the twill unit tests work properly on Windows (I switched from fork() to subprocess) and also building some of the functional tests for Cartwheel. For the …
read moreThere are comments.
I've finally got a buildbot that compiles FamilyRelationsII on Linux and Windows/cygwin, and I've got another buildbot running tests for twill, scotch, and figleaf on Linux and Windows.
The convenience of this cannot be overestimated: I no longer need to boot Windows in order to compile or test my …
read moreThere are comments.
I took a further look at CMake and discovered that contrary to my earlier understanding it does in fact generate XCode project files quite nicely. Huzzah!
I'm still curious about the differences between DART2 and buildbot. I may have to put some time into installing DART2 to figure this out …
read moreThere are comments.
I just put the finishing touches on some automated regression tests for figleaf, my simple code coverage analysis program. In the process I found a nice use for nose's yield test constructor.
Briefly, nose lets you write code like this:
def test() for i in range(0, 10): yield check …read more
There are comments.
In the spirit of Greg's Not on the Shelves post about books he'd like to see, here's one class I'd like to see taught:
Test-Driven Web Development
This class will introduce students to test-driven software engineering through the development of a database-backed Web site. Student development will be driven by …
read moreThere are comments.
Dear lazyweb,
does anyone know of any comparisons of DART with buildbot?
thanks!
--titus
Legacy Comments
Posted by Grig Gheorghiu on 2007-02-02 at 17:36.
Never heard of DART, but I used something called STAF/STAX in the past, and I had to struggle with tests defined in XML syntax …read more
There are comments.
I spent an hour or so coding on Quixote, in response to Neil Schemenauer's call for community help. Result: automated twill tests, WSGI, and qpy "integration". It helped that this was all stuff Mike Orr and I already had sitting on the shelf ;).
--titus
read moreThere are comments.
I'm midway through Scott Rosenberg's Dreaming in Code, and I can unabashedly recommend it to anyone who likes a good yarn. Yes, it's about software development, and you'll need a fair bit of technical exposure -- not experience, just exposure -- to navigate the references. But anyone who is reading this, including …
read moreThere are comments.
Just a short note to say that I've taken a faculty position at Michigan State University, in Lansing, Michigan. The position is split 65%/35% between the Computer Science and Microbiology & Molecular Genetics departments, and I expect to be working on a fairly wide range of problems. My computational "focus …
read moreThere are comments.
I spent part of the last few days figuring out how to get my cross-platform GUI, FamilyRelations II (a.k.a. FRII), to build on Windows. It used to compile on my old desktop machine, which we then converted over to MythTV with a Windows partition for compilation, but that …
read moreThere are comments.
Over the last few days, I put a bunch of time into my ongoing refit of Cartwheel.
Cartwheel is a sizeable toolkit for doing bioinformatic sequence analysis; it's got a Web interface, a batching and queuing system built on PostgreSQL, and a client library for manipulating things over XML-RPC. So …
read moreThere are comments.
I just wrote a small GalCon strategy guide, available here. Comments welcome.
--titus
Legacy Comments
Posted by Brian Dorsey on 2007-01-05 at 13:24.
I think we should play some GalCon at PyCon this year! (Your earlier post about GalCon got me completely addicted.) Take care, -Brian
Posted by Titus …
read moreThere are comments.
Done. DONE. done. Done done done done done.
I defended Sep 5th. That went well.
Unfortunately I had lots of revisions to do.
That is finally finished: everyone signed off on the revisions.
If I don't officially have my nigh-irrevocable PhD by the new year, it will be because of …
read moreThere are comments.
Over the last week, I've spent far too much time playing Galcon, an addictive game developed by Phil Hassey using Pygame and other tools.
In Galcon, you are a general out for conquest. You start with one or a few planets that produce fighters, and you're pitted against "independent" planets …
read moreThere are comments.
Grig and I will be reprising our Testing Tutorial at PyCon '07! This year I think we have a better idea of what people found useful, and in consequence we should be a bit better organized, too.
Our outline is here: http://us.pycon.org/TX2007/Tutorials
More anon.
--titus …
read moreThere are comments.
So, it looks like I will be giving a talk on twill, scotch, and figleaf at PyCon! One of the reviewers said something very nice, albeit probably unjustified; to quote:
...twill is more or less Py web-developer 101 at this point, so a talk about it should be enlightening.
Hmm …
read moreThere are comments.
Here's a (slightly edited) message from Bill Punch, a professor at MSU who is hoping to transition their Intro CS class over to Python. I would appreciate your thoughts and comments, both on his questions and my responses! (I do have permission to post it, in case you're wondering. ;)
(Just …
read moreThere are comments.
Here's my dirty little secret: I read PlanetPerl.
I do so for a couple of reasons. First, there are a number of good programmers that post through it. Second, I'm easily amused by any discussion of Perl 6. And third, occasionally my mind gets boggled.
Take, for example, Ovid's post …
read moreThere are comments.
I'm continuing to build a testing infrastructure for Cartwheel. One thing to test is the XML-RPC API for controlling the server remotely; this lets users upload sequences, create analyses, and download results programmatically.
Now, doing functional testing of an XML-RPC server is just as annoying as testing any Web app …
read moreThere are comments.
I've been posting under Andrew Kuchling's name on planetpython.org for a while now, and today it appeared that all was fixed! Sadly, it turns out that my new posts are the result of me posting via my new blog, which is then syndicated to my old blog, which in …
read moreThere are comments.
I spent the better part of today refitting Cartwheel for testing. Cartwheel is a bioinformatics framework that is used by a few hundred people; it's mostly a database-backed Web site, with a compute server queueing system tacked on.
Cartwheel is one of the two sites that got me interested in …
read moreThere are comments.
Not only is MIT teaching Python, but it appears that MSU is going to try it out for their intro course. Hooray!
--titus
read moreThere are comments.
Dear lazyweb, what's the situation with Java and C++ OODBs? I seem to recall hearing vaguely good things about some Java OODBs from real people (i.e. not marketing flacks) but I haven't kept that information anywhere. (I did find this amusing article on how O/R mapping is the …
read moreThere are comments.
A local friend asked for advice on her OSS project, because her boss is questioning the value of making things OSS. Here's my rambling reply, preserved for posterity:
There are several de facto models of open source at this level (the small niche projects level, that is). The first is …read more
There are comments.
I'll edit http://us.pycon.org/TX2007/WebFrameworksPanel as I update things, but here's the proposal I submitted:
There are comments.
After Mark Eichin's comment on my Jabber e-mail notification post, I realized I had to whip up a twill module for sending Jabber messages. Easily done; here's an example twill script:
extend_with twill_xmpp jabber_login titus2@xmpp.us XXXXX send_jabber_message titus@xmpp.us message
Since I …
read moreThere are comments.
So, I wrote a custom plugin called 'ocomments' that uses an SQLAlchemy- based database API to assign cookies to users who make comments. That way I can control who has automatic posting access (anyone who posts a sensible message, basically) and who doesn't. I can also toggle comment visibility on …
read moreThere are comments.
A simple program to get pinged whenever you get new e-mail:
#! /usr/bin/env python import sys, email from pyxmpp.jid import JID from pyxmpp.jabber.simple import send_message inp = sys.stdin.read() message = email.message_from_string(inp) jid = 'someid@xmpp.us' password, recpt = 'XXXXX', 'otherid@xmpp.us' …read more
There are comments.
After learning from Jeff Rush that no one had volunteered (yet) to organize a panel on Web technologies, I stepped up to the plate. I've also submitted a talk proposal about twill/scotch/figleaf and (with Grig) I'm going to submit a proposal for a testing tutorial, so attendees may …
read moreThere are comments.
I noticed this morning (and Grig just reminded me of it) that on http://www.planetpython.org/ I'm listed as Andrew Kuchling. Heh.
--titus
read moreThere are comments.
While in Lansing, Michigan last week, I used the MichiPUG meeting as an excuse to visit Ann Arbor. (Ann Arbor is nice!) I presented twill, scotch, and wsgi_intercept. I also met Kevin Dangoor and Jason Pellerin, people that I'd previously only known online.
It was especially gratifying to be …
read moreThere are comments.
I was out at Michigan State University discussing a faculty appointment last week, and we continued to talk about using Python for an introductory Computer Science class.
Two questions came up and I'm hoping to get an answer to both via the LazyWeb:
read more
Who else is doing it? What other …
There are comments.
corebio, the joint effort by a junta of California bioinformaticians to replace BioPython with something we like better, is proceeding interestingly. So far we have discussed the following issues:
read more
- what license? (BSD)
- what focus? (sequence manipulation & parsing)
- what about binary extensions? (focus on API, provide fast implementations where appropriate, but …
There are comments.
A bevy of Erlang links has been appearing on my radar screen. In part this is probably because I've become moderately addicted to Reddit, which is Erlang Cheerleading Central these days, but I'm also just plain interested in Erlang.
Dijkstra discusses CS education but I really disagree with him. He …
read moreThere are comments.
Grig and the agile-testing list both sent me some interesting testing links today.
foo gui writes in Integrating Selenium IDE/RC and FitNesse about a neat trick where UI tests can be run via Selenium from FitNesse. (The subtitle is "Automated Web Testing Bliss" ;)
Jeremy Miller discusses …
There are comments.
I'm going to be visiting Michigan for a conference (on Mackinaw Island) at the end of September, where I'll be talking about my thesis research. Following said conference, I'll be hanging out at Michigan State U. for the week of Oct 2-6; I'll be giving a talk on Agile Testing …
read moreThere are comments.
I've said some mean things about BioPython in the past -- that it's broken, that it's crufty, etc. One prominent former BioPython developer responded with the very reasonable question of why I wasn't fixing it, if it was so broken. The answer, of course, is that I've been working on my …
read moreThere are comments.
Ok, I've just enabled my ocomments plugin for pyblosxom on this blog.
ocomments is a hacked version of the comments plugin that keeps track of users and lets me approve users and posts individually. Hopefully it will let me keep the blog spam free while letting regular posters comment freely …
read moreThere are comments.
Page 1 / 38 »