Fri, 30 Mar 2007
Strangling Your Code and Growing Your Test Harness: The 9 Phases of Building Automated Tests Into Legacy Code
I'm in the early throes of building tests into my Cartwheel project. Cartwheel was one of the two projects that inspired my Web testing project, twill, so naturally I'm happy to finally be putting twill to good use in my own projects. Naturally the transition from building tools for building tools, to actually using the tools to build a tool, is a bit painful: I can't be general any more, now I have to be specific.
The process I'm going through right now is appropriately referred to as Strangling Legacy Code, in this case by Growing A Test Harness (both great articles). Leveraging the power of nose directory hierarchies, I'm slowly growing my setup/teardown code to cover more and more functional testing scenarios, which in turn exposes more scenarios to test. While it's an endless-seeming process, I think I'm at the inflection point where I've now automated more than half of the testing tasks for the Cartwheel Web server. Note that I'm by no means testing even 50 percent of the functionality, but what remains is relatively specific and accessible to testing by very small increments in my testing code. Given my general time constraints, I'm going to switch my focus to testing newly written code and writing automated tests for reported bugs; down the road I'll probably use coverage analysis to figure out what large masses of untestedness lie hidden in my codebase.
Looking back and prognosticating forward, I'd divvy the process up into N steps:
- Shock. (How the heck do I start testing this sprawling mass of code??)
- Hello, world. (Hey, look at that, I've got a basic import working in my automated tests!)
- Fixture code sucks. (Oh, gawd, I've got to automate setting that up, and that, and that...)
- Fixtures rock. (Wow, look at what I can test now!)
- Over the hump. (Where I am now.)
- What lurks beneath? (Using coverage analysis to find large areas of untested code.)
- Relaxing. (I've got XX% of my code covered with some kind of automated test! Hooray!)
- Reaction. (Hmm, guess I'm not actually testing for that specific bug...)
- Goto 7, increment XX.
- Asymptotically approach perfection.
Regardless of the steps ahead, it feels good to be at stage 5...
--titus
posted at: 12:03 | path: /mar-07 | 3 comments
An e-mail to the Xerces c-users mailing list
If anyone knows someone actually on the Xerces c-users mailing list or development team, could you please forward this on?
(I sent it directly to the list mentioned on http://xml.apache.org/xerces-c/feedback.html, but it hasn't shown up in the archives and I don't seem to have received a bounceback. I'm guessing that I can't post to the list without moderation because I'm not a member of the list, but I can't see any obvious way to sign up for the mailing list. At this point I'm worried it has simply vanished into the ether; hence this post.)
Hi folks,
I thought you might find my discussion of how to compile Xerces-C++ into
Mac OS X universal binaries useful:
http://ivory.idyll.org/blog/mar-07/compiling-x-platform-on-macs.html
The two pertinent sections are #2 (proper runConfigure incantations) and
3(a) (linking the installed libxerces libraries into Mac OS X's
development hierarchy). 3(c) (distributing dylibs with your app) is
also extremely useful.
Someone should obviously verify all of this before putting it into the
docs; if someone else can verify it, I'd be happy to write it up in the
appropriate format and contribute it as a patch to Xerces-C. Just ask
;).
cheers,
--titus
p.s. The reason I'd like someone *else* to verify it is that I've
fiddled with various paths on my laptop and no longer have a clean
environment within which to test instructions. I'm pretty sure I got
all of the steps, but it'd be nice to have someone else check.
I continually have problems figuring out how to get e-mail through to projects. grumble Ahh well.
--titus
posted at: 11:03 | path: /mar-07 | 1 comments
Wed, 28 Mar 2007
Compiling Universal Binaries under Mac OS X -- My Experience
I've spent a few months (on and mostly off) trying to get my C++/FLTK program, FamilyRelationsII, to build on my MacBook for both old and new Macs.
I was helped immensely by Mando Rodriguez and Diane Trout, both of whom contributed various snippets. Getting it all to play nice together was still painful enough that I think it's time to contribute something back to the lazyweb/googleplex.
Problem 1: Compiling FLTK cross-platform
I didn't actually keep notes for this, but basically I generated an X code project for FLTK and then selected "build cross-platform".
- In your fltk-1.1.x directory, generate an Xcode project with cmake -G Xcode .
- open FLTK.xcodeproj
- Select "FLTK" in the "Groups and Files" pane (left)
- Double-click, select 'Build'. Double-click on Architectures. Pick 'em both.
- Assuming the build works, you should see something like 'bin/Debug/libfltk.a'. These are the libraries you want.
I haven't figured out how to have them installed correctly yet; presumably that's yet another click away. At this point I just copy them into /usr/local/lib ;).
Problem 2: Compiling Xerces C++ cross-platform
Substitute
./runConfigure -p macosx -n native -t native \
-z -arch -z i386 -z -arch -z ppc \
-l -arch -l i386 -l -arch -l ppc \
-l -Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk
for the default runConfigure command in the Xerces C documentation. Then make, make install.
(I guess I'll contact the Xerces C people about adding this to their docs...)
Problem 3(a): Linking /usr/local in properly
Because FLTK and Xerces-C++ are installed into /usr/local/lib by default, the -isysroot /Developer/SDKS/MacOSX10.4u.sdk stuff will not work unless you also do this:
% ln -fs /usr/local /Developer/SDKS/MacOSX10.4u.sdk/usr
Apparently you need to do this because -isysroot adds the /Developer/SDSKS/MacOSX10.4u.sdk/ prefix on to all library and header filename lookups; this lets Mac OS X know that universal libraries etc can be found under /usr/local as well.
Problem 3(b): Compiling your own code properly
The following CMake snippet (courtesy of Diane Trout) does the job well:
if(APPLE)
set(APPLE_COMPILE
"-isysroot /Developer/SDKS/MacOSX10.4u.sdk -arch i386 -arch ppc")
set(APPLE_LINK
"-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk -arch ppc -arch i386")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${APPLE_COMPILE}")
set(LINK_FLAGS "${LINK_FLAGS} ${APPLE_LINK}")
endif(APPLE)
Then a standard 'cmake .' will put the right magic into your compilation flags.
Problem 3(c): Distributing dylibs with your Mac app
If you're planning to distribute your universal binary, and it depends on Xerces-C++ libraries, read this very helpful Qt doc; look esp at the Shared Libraries section. This worked beautifully for me.
Briefly, you need to use install_name_tool on both the Xerces-C++ dylib files and your compiled applications, to change the location in which they will be looking for libxerces-c.27.dylib. See my build-dist script diff for exactly what I do.
Conclusion: Double-checking stuff
If at any point you run into trouble compiling with both the '-arch ppc' and '-arch i386' flags, run 'file' on required libraries and other binaries to make sure they're universal:
% file FRII/app/FRII FRII/app/FRII: Mach-O universal binary with 2 architectures FRII/app/FRII (for architecture i386): Mach-O executable i386 FRII/app/FRII (for architecture ppc): Mach-O executable ppc
Well, I hope this helps someone! It was painful to learn, and AFAIK the correct Xerces-C++ incantations are not available elsewhere on the Web ;).
cheers, --titus
posted at: 12:14 | path: /mar-07 | 0 comments
Wed, 21 Mar 2007
Replacing ``commands`` with ``subprocess``
After an innocent question was answered positively, I am putting together a patch to deprecate the commands module in favor of a slightly expanded subprocess module (for 2.6).
Briefly, the idea is to add three new functions to subprocess:
output = get_output(cmd, input=None, cwd=None, env=None): (status, output) = get_status_output(cmd, input=None, cwd=None, env=None) (status, output, errout) = get_status_output_errors(cmd, input=None, cwd=None, env=None)
with the goal of replacing commands.getstatusoutput and commands.getoutput. (commands.getstatus has already been removed from 2.6.)
This will provide a simple set of functions for some very common subprocess use-cases, as well as providing for a cross-platform alternative to commands, with better post-fork behavior and error trapping, adhering to PEP 8coding standards. A win-win-win, I hope ;).
In addition to writing the basic code & some tests, I would like to:
- reorganize, correct, and expand the subprocess documentation: right now it's not as useful as it could be.
- put some warnings/error reporting into subprocess for bad class parameters; e.g. Popen.communicate should check to be sure both subprocess.stdout and stderr are PIPEs.
Questions:
- anything else I should think about doing to subprocess?
- right now the functions take only the input, cwd, and env arguments to pass through to the Popen constructor. Any other favorite arguments out there?
- should language be added to the popen2 module pointing people at subprocess, and should popen2 be deprecated?
- GvR suggested that I reimplement commands in terms of these subprocess functions for 2.6, even though the commands module could be deprecated in 2.6 and probably removed in 2.7. I would rather simply amend the documentation to point people at subprocess.
Thoughts?
--titus
p.s. The implementation of the above functions is dead simple:
def get_status_output(cmd, input=None, cwd=None, env=None):
pipe = Popen(cmd, shell=True, cwd=cwd, env=env, stdout=PIPE, stderr=STDOUT)
(output, errout) = pipe.communicate(input=input)
assert not errout
status = pipe.returncode
return (status, output)
def get_status_output_errors(cmd, input=None, cwd=None, env=None):
pipe = Popen(cmd, shell=True, cwd=cwd, env=env, stdout=PIPE, stderr=PIPE)
(output, errout) = pipe.communicate(input=input)
status = pipe.returncode
return (status, output, errout)
def get_output(cmd, input=None, cwd=None, env=None):
return get_status_output()[1]
posted at: 12:32 | path: /mar-07 | 19 comments
``unittest`` bitching: premature; lazyweb request
From reading Collin Winter's blog he's designing a new unittest module first, and then he's going to ask c.l.p and presumably python-dev about adding it to py3k. So it's not quite the fait accompli I thought it was, which reduces my complaints to mild grumbling.
And, dear lazyweb... is there a good way to find out when a particular line of code was introduced (or last touched) through subversion?
thanks, --titus
posted at: 11:03 | path: /mar-07 | 8 comments