Software Developer, Chicago, IL: Python and C++ for Engineering and Finance

Regression Troubleshooting

Take a shot at fixing broken regression tests.

Your goal: help turn the trunk regression test matrix pure green by troubleshooting the failures you find there.

Choose your platform: Operating system and compiler. You'll need 5GB disk space.

Get/update your working copy of the trunk via subversion check-out/update. Instructions here. See below for links to help orient you its contents. Get to know subversion--it's an important tool.

Find the yellow/fail on the trunk regression test matrix -- the unexplained failures. Click through to see specific tests and failures. Get to know the matrix. Keep a bird's-eye view of it, peeking at failures across other platforms, too.

Pick a failure for your platform and investigate it.

What's the pattern? Does it fail across (a) all platforms, (b) some like yours, or (c) just yours? Note this carefully so you can recall it later if need be. The same pattern--multiple tests failing--across two or more platforms could mean that they both cascade from a single cause. If the pattern seems particularly baffling, it could be revision-related more than platform-related.

Reproduce it locally. Run bjam in the lib/<library>/test directory to build and run the test on your machine. Redirect bjam's output to a file for easier navigation. Try bjam --help.

Try to fix it. Make changes to your working copy and test them via bjam.

If at first you don't succeed, revert and try again. Don't clutter your working copy with spurious changes.

Less is more. A minimal change, particularly for a platform-specific fix, is optimal. That will be the most clear to reviewers.

Stumped? Can't figure it out?

Could it be the Standard C++ Library itself be failing? Use BOOST_TEST_CHECKPOINT() to vindicate boost. (Example here.)
Be self-study. Put it down and come back to it later. A couple times. There's no shame asking for help, but doing it yourself is most satisfying.
Collaborate with other volunteers in your position.
Ask a mailing list.
Skip it and move on. Come back in a month, or leave it for someone else.

Make a patch to show exactly what you changed to make it work, in a machine-readable format. The Tortoise SVN tool has a patch function built into its menus. Or use the gnu patch utility. A single patch can span multiple files.

Target an author specifically. Look at the SVN log of the file(s) you changed. (TortoiseSVN: right click on the file.) Find the most recent author and revision. Assign your ticket to him/her. You can view the entire changeset via https://svn.boost.org/trac/boost/changeset/<revision> (e.g. https://svn.boost.org/trac/boost/changeset/66578).

Submit a problem ticket with the patch, to the boost Trac system. Attach your e-mail address so a reviewer can interact with you. You're creating low-hanging fruit for the reviewers to incorporate. Set each attribute carefully, assigning to the author you found above.

Revert your working copy when you're through.

Update often: update your trunk working copy often.

Follow up your Trac tickets.

Keep a list of what you've fixed, for input into the next version's release notes.

Adapt/adopt: adopt (or adapt to) the author's style. Reach to grasp what they're trying to accomplish. Suspend judgment. (It is peer reviewed.) Make your patch friendly to their way of thinking, pleasing to their eye.

Be patient with everyone: maintainers, release managers, submitters, trunk-patchers. They're all volunteering their time. Things always move more slowly than you want.

Your inputs: