What are the options?

One option for the future is the status quo, adjusted in small bits -- finding funding to continue teaching a two-week course past 2013, maybe rotating course instructors to reduce exhaustion. This is what courses like the Embryology Course at Woods Hole do. If we could find administrators to handle some of the work, and a bunch of like-minded instructors, this could work.

Another option is to move away from grant subsidies and run the course as a workshop that pays for itself. While good in some senses -- it ensures that the course will go away when it's not needed any more, and frees us from having to apply for grants (which takes a fair bit of energy and is uncertain to boot) -- it has some downsides. The two dominant downsides for me are these: first, the expense will have to increase dramatically, to $1000-2000 in tuition per person,which limits access substantially; and second, I dread the consumer mentality in education. Right now everything is very collegial and academic, and I like it that way.

We could also try to "flip" the course, which is a term in vogue these days -- it refers to moving courses online and reserving classroom time for homework. Basically we could put a lot of effort into developing good online materials with instructional videos, and then organize courses where we send TAs to help out locally while offering teleconf support at the professorial level. This would be a more scalable alternative to flying professors around, and should maximize impact.

One reason for thinking this could work out is that in practice, the two week course goes like this:

1. three days getting people up and running on our tutorials, overcoming the inevitable technical fear.
2. five days of lectures followed by impatient students wanting to play with the tutorials, with some occasional tech-support questions.
3. three days of more intense "but how would I do..." questions that are research specific..

So, if we could figure out how to put the lectures online, we'd eliminate the middle five days of not-so-required human presence and just deal with the tech support issues.

You could also imagine a "firebase" model in which we establish local workshops at regional hubs and then have people attend. This minimizes travel cost and maximizes the utility of flying instructors around.

But I'm not sure there's any way to compensate for the absence of trained, thoughtful people focused on you that I think works so well for students in the two-week workshop as it's currently run. And if you don't drag students out of their daily life, will they be sufficiently immersed in the content to learn well?

The Software Carpentry folk have claimed good success with one-to-few video- or teleconferencing. Does this scale? Not sure... but:

As a fourth option, we could also join forces with the fine folk at Software Carpentry, who are working on implementing this kind of flipped approach for aspects of computational science training. I really like what Software Carpentry is trying to do (in fact, I like it so much I'm not just a customer, I'm involved! :) and I think the Hacker Within model is an awesome way for students to self-organize. The only possible downside I see here is a dilution of focus, which is why Way Back When I proposed a hub & spoke model; since the alternative may be a cessation of focus, I'm willing to risk dilution :).

I am leaning towards some mixture of the last two, if I do continue the course.

To go in this direction, I'll probably try to marshal both federal funding and internal support at MSU. Frankly, I think MSU should be begging us to continue running the course, but so far the reaction from above has been "meh"; that's the reason for needing the federal funding, with which no one can argue.

Assessment

The main other thing I want to be able to do is to figure out how to assess the quality of our teaching and suggest ways to improve it.

Driven by this course, as well as another set of courses I teach for my day job at MSU, as well as the expectation that higher ed is going to collapse within 10-15 years, I have been reading a fair bit about teaching. And I've come to a personally startling conclusion:

Most professors have no idea if they're a good teacher. They have not measured it, assessed it, or even really asked anyone else about it, ever. The most assessment they do is in the form of a scantron sheet with handwritten comments, which amounts to a popularity contest (and, note, students are apparently very bad at self-evaluating learning).

This isn't for a lack of caring: many professors (most of the ones I interact with at MSU) do care, quite a bit, about teaching. But they also care, more deeply, about research (as do I). And when you're squeezed at both ends by the demands of teaching, research, and paper/grant writing, not to mention having a personal life, you mold yourself to what your employer hired you for -- and that's not teaching, at least not at an R1 like MSU. (Much has been written about this issue and I will simply say that MSU cares more about quality of teaching than Caltech, but not as much as Reed. :) So it's a matter of where you choose to apply effort, and most people apply their intelligence and effort to research -- which is impossibly difficult and challenging enough to occupy me, at least, 24/7 for the rest of my life.

It turns out that there's a great literature on what works in teaching that most professors are unaware of. But, when I looked into it, adapting those techniques to my own courses was anything but obvious, largely because I neither had a mentor nor a local expert nor an exemplar that fit my subject areas.

When I talk to other professors, there is an interesting kind of willful ignorance of this literature. There are usually a lot of casual opinions thrown around ("we need smaller classrooms!"; "more TAs!"; "better students!") but they are generally backed up by hand-waving arguments that are suspiciously self-serving: yeah, it'd be easier to teach fewer & better students with more support, but would it improve your teaching? Dunno.

I also think we have entered into a kind of Stockholm syndrome situation where we've convinced our students that sitting there, upright, with minimal drool, is sufficient -- I've even heard claims that people actually learn better in lecture classes than in others. I have seen little to support this in my reading, and I suspect that those students would also do fine if you just locked them in a library for a year. But our lifelong training in sitting through lectures does seem to affect the way students behave when confronted with things other than lecture: they wibble. You have to train them in a different way of doings, which is quadratically difficult when you don't know what you're doing yourself, either.

Overall, I really, really want to figure out if I'm making a positive difference on the students, beyond merely entertaining them for a brief few hours on a regular basis. And I want to use this kind of assessment to guide and tune my teaching, and that of others.

I also think it is going to be much, much easier to make the argument to a grant panel that they should give us moolah if we can point to rigorous, well-researched and published assessment that demonstrates that yes, we are in fact teaching something to the students. Right now I can cavail all I want about the lack of grant money for teaching, but I can't actually prove they're not flushing it down the toilet when they give it to me. Kinda weakens the argument...

And do you have any goals?

Thus I present my goals for myself for the next year:

1. Work with an assessment expert to build good assessment instruments for my workshop, my courses, and for Software Carpentry workshops (if possible) -- and then perform assessments.
2. Dip my toes further into the water of "flipping" via video.
3. Push technology a bit further to make the summer workshop more sustainable (via automated testing of tutorials) and to do new & better things with some teaching (iPads and ipython notebook are my current targets for fall 2012).
4. Explore Coaching as a way to do a better job in the classroom.

--titus

p.s. Parenthetically, I love this work on active learning in big Astronomy 101 courses.

p.p.s. Greg Wilson has pointed me at a few other places: "Eric Mazur's stuff on peer instruction (Beth Simon is applying those ideas to CS at UCSD with great success), and Mark Guzdial's "media first" approach (working very well at Georgia Tech), and Steve Joordens' work on peer assessment (lots of other people got there before he did, but Joordens is the one I ran into first), etc."

Legacy Comments

Posted by Karen Cranston on 2012-03-14 at 16:06.

Great post! We have a very similar experience here at NESCent. Our NGS
course (which runs the same week as yours this year) was massively
over-subscribed. Too much need, and not enough spaces. We charge
tuition and costs, but still get a huge number of applications. I've
been wondering about online solutions (have you heard any feedback
about last year's online courses at Stanford?)

Posted by Titus Brown on 2012-03-15 at 12:00.

Hi Karen,    "glad" to hear you're having the same problem :).  I'm
pretty skeptical about a pure online solution, but I've been following
the courses at Stanford and elsewhere with interest.    We should do a
cross-lecture or something between our courses this year...    I'm
curious -- do you have any assessment results you could share with me,
publicly or privately?  Really want to figure out if I'm doing a good
job!    --titus

Posted by Scott Handley on 2012-03-21 at 17:51.

Hi Titus,    We run similar Workshops (one of Genomics, one of
Molecular Evolution/PopGen) in both the US and Europe. Similar to what
you have seen, every time we organize a Genomics Workshop (5 so far)
we receive more and more applicants. The applicants are from every
discipline of biology, and with varying levels of expertise. This
makes for a rewarding, but very challenging group to design a
curriculum for. So we try to focus on fundamental principles and
techniques that will be applicable to just about anyone working in the
genomic sciences (UNIX, NGS quality control, study design ...). In
addition we pepper in several speakers and laboratories on key topics
in genomics (RNA-seq, metagenomics, variant detection ...). Overall I
think this works well, and while we will iterate for future Workshops
the fundamental structure will remain the same.    The Workshop on
Genomics is based on the Workshop on Molecular Evolution which
originated at Woods Hole. So your reference to the Woods Hole
Embryology course is relevant to what we try to do. Immersive training
opportunities with lots and lots of interaction with faculty, staff
and other students. Being isolated from daily life is also important.
We value these interactions equally with the actual training material,
so a move to an all on-line course is not in the cards for us. Yes it
is more expensive and yes it is difficult to coordinate, but to me
this is where the real value is.    I also ask myself what
professional benefits I have experience over the course of my career
from on-line training experiences. The answer is always one of two
things, "I have never done any on-line training." or "Oh yes I have, I
had to do on-line biosafety training, but I barely remember it and
basically just surfed Reddit in the background while doing it.". I
joke, but I think that the value of on-line training is still suspect,
even given the success (e.g. large enrollment numbers) of the Stanford
courses. But the Workshop model is tried and true, so that is where we
are spending our time.    That isn't to say we aren't interested in
using the web to aid the training process. We have a lot of training
exercises up on our site and are working to convert them to versions
that can be more easily complete by people that don't actually attend
the Workshop. There is a lot of interest in our faculty from this.
They like to point people to the exercises as good learning examples
throughout the year, but in their current state many of them require
files or an environment specific to the Workshop.    We have applied
for some grant funding (unsuccesfully ... :( ), but this has not
stopped us. Our basic algorithm is as follows: 1) invite a bunch of
high-quality faculty to cover a spectrum of interesting concepts in
genomics/evolution, 2) add up total Workshop costs, 3) advertise, 4)
cancel Workshop if we can't cover the costs. So far we have always
been in the black and have only cancelled one Workshop.    As you
point out, this is more expensive than having a grant to subsidize
some of the expenses. The trade-off is unlimited amounts of freedom on
how to run the Workshop. There are also dozens of other of advantages
and disadvantages to this, we can discuss that elsewhere if you wish,
but this is already becoming long in the tooth ...    At the end of
the day this is something that myself and most of the faculty we have
worked with are committed to. There is a huge need for this training,
and it is important for the advancement of science. I know that sounds
a bit high-brow, but having experienced the rewards that come from
this training and seeing how much can be done across a broad-range of
scientific disciplines (ecology, genetics, evolution, popgen,
microbiology, botany ... you get the picture) I am a true believer in
what this training can do to help support the advancement of science.
It's just part of what being a scientist should be about.    I would
love to discuss this more with you either on this blog or some other
forum. If you are interested we should coordinate training. I like
your hub idea. This is something we have set-up at our location in
Europe. In addition to the genomics Workshop, last year we added
special sessions on R/Bioconductor, Python/BioPython and were
interested in adding GMOD and Galaxy, but didn't get to that in time.
We have a great infrastructure set-up and the location is more than
isolated and idyllic.    Send me an email or call and we can discuss!
Or better yet, come visit Washington University and give a talk!