A Syntax First

I’ve seen a lot of C and C++ source-code in my career.

I’ve never seen anyone write a do while loop without using curly braces. Until now:

do cin.ignore(1024); while (cin.gcount() == 1024);

It’s one of those unconscious things, I guess. If you asked me if you could write it that way, I would have supposed so.

I don’t mean any criticism: it’s fine. It’s clear. Just amused (at myself) that I’ve never seen it.

Save Windows XP

It seems Windows XP is being phased out:

Fans of the six-year-old operating system set to be pulled off store shelves in June have papered the Internet with blog posts, cartoons and petitions recently. They trumpet its superiority to Windows Vista, Microsoft’s latest PC operating system, whose consumer launch last January was greeted with lukewarm reviews.

No matter how hard Microsoft works to persuade people to embrace Vista, some just can’t be wowed. They complain about Vista’s hefty hardware requirements, its less-than-peppy performance, occasional incompatibility with other programs and devices and frequent, irritating security pop-up windows.
[…]
“You really can’t make 69 percent of your installed base unhappy with you,” he said.
[…]

This troubles me, too. My wife has Vista on her laptop, and I’m amazed at how it makes a brand new processor feel slow. And how often Internet Explorer 7 has problems.

I don’t feel like I missed a thing when I skipped over Windows ME and 2000. I kind of have that same feeling towards Vista.

Maybe I should be happy. After all, it creates more work for us software folks. More work, but less real progress.

Many Windows apps run under Linux via Wine, or ported to Linux via a variety of tools. Is it time to (re)evaluate?

It’s a good time to think about your criteria: What do you need? What would tempt you to switch?

Stepping into Linux can’t be done casually. You may be trading one set of unknowns for another. Steep new learning curves. You can buy support, but how much will you need? How much will you spend in the end?

I’m amazed at how far Linux has come, but I still don’t think it has caught up as a full-featured desktop. And what flavor? Ubuntu? OpenSUSE? Xandros? Red Hat? CentOS?

The Problem With Threads

Multithreaded (concurrent) programming is a valuable tool in my toolbox. I don’t hesitate to use it when I need it. I can’t imagine getting by without it.*

So the SQLite project got my attention when they said, “Threads are evil. Avoid them.”

They referred me to a very interesting paper by Edward A. Lee of UC Berkeley: The Problem With Threads. From the abstract:

Although threads seem to be a small step from sequential computation, in fact, they represent a huge step. They discard the most essential and appealing properties of sequential computation: understandability, predictability, and determinism.
Threads, as a model of computation, are wildly nondeterministic, and the job of the programmer becomes one of pruning that nondeterminism. Although many research techniques improve the model by offering more effective pruning, I argue that this is approaching the problem backwards. Rather than pruning nondeterminism, we should build from essentially deterministic, composable components. Nondeterminism should be explicitly and judiciously introduced where needed, rather than removed where not needed. The consequences of this principle are profound …

Well put. Threads can create intractable problems, bugs can be incredibly subtle, and there’s no way to prove you’ve thought things through well enough.

Section 3 gets into some heavy theory to say, essentially, that we can’t compare two multithreaded programs for equivalence (i.e., if two programs will yield the same result). He concludes with this interesting analogy:

To offer a third analogy, a folk definition of insanity is to do the same thing over and over again and to expect the results to be different. By this definition, we in fact require that programmers of multithreaded systems be insane. Were they sane, they could not understand their programs.

I’ll take that as a compliment. I see his point, though.

Here’s a very interesting piece:

An anecdote from the Ptolemy Project is telling (and alarming). In the early part of the year 2000, my group began developing the kernel of Ptolemy II [20], a modeling environment supporting concurrent models of computation. …
The portion of the kernel that ensured a consistent view of the program structure was written in early 2000, design reviewed …, and code reviewed … The reviewers included concurrency experts, not just inexperienced graduate students … We wrote regression tests that achieved 100 percent code coverage. … The Ptolemy II system itself began to be widely used, and every use of the system exercised this code. No problems were observed until the code deadlocked on April 26, 2004, four years later.

It is certainly true that our relatively rigorous software engineering practice identified and fixed many concurrency bugs. But the fact that a problem as serious as a deadlock that locked up the system could go undetected for four years despite this practice is alarming. How many more such problems remain? How long do we need test before we can be sure to have discovered all such problems? Regrettably, I have to conclude that testing may never reveal all the problems in nontrivial multithreaded code.

Very sobering. I have to say their threading is much more ambitious than anything I do.

The more I think about it, the more I realize that I do “prune” my nondeterminism very aggressively, and introduce it only when needed. For me, concurrency is usually a private attribute of the classes I write (though C++ can’t enforce anything in the time domain). It needs to be warranted for the complexity it introduces, and must avoid (non-private) things like interactions with other classes that might cause deadlocks.

Another typical application is a user interface and a core process. The user interface runs concurrently, allowing us to watch the arbitrarily complicated core process, and poke at it only in well-defined ways (like cancelling it). The process notifies the interface of significant events through a straightforward mechanism like a message queue. (I’ve never needed to implement a full-blown MVC.)

The author makes excellent points. As multi-core computers become more and more common, we would do well to harness their power. But with caution.

* Ok: I do get by without multithreading when it’s not available: like 8-bit microcontrollers that can’t support it. A polling, event-driven system can do a lot in parallel and emulate a multithreaded system pretty well, though its sequencing can be difficult to track.

Rvalue References

Here’s a new C++ feature: rvalue references.

An rvalue reference behaves just like an lvalue reference except that it can bind to a temporary (an rvalue), whereas you can not bind a (non const) lvalue reference to an rvalue.

A& a_ref3 = A(); // Error!
A&& a_ref4 = A(); // Ok

Question: Why on Earth would we want to do this?!

It turns out that the combination of rvalue references and lvalue references is just what is needed to easily code move semantics. The rvalue reference can also be used to achieve perfect forwarding, a heretofore unsolved problem in C++. From a casual programmer’s perspective, what we get from rvalue references is more general and better performing libraries.

I’m sure it will be a while before we see a compiler supporting it, but it’s a good concept to know.

NY Subway Office

Despite our best efforts, we can’t always reproduce phenomena we see in the field. So to the field we go.

Here are a few shots of my makeshift office in the New York Subway system. Not the most productive or ergonomically correct environment.
NY Subway Office
The chair is a life-saver. Under $6 from Target, it collapses into a bag slung across the shoulder.

Here’s a view from the seat. Fifteen feet from the track.
NY Subway Office

Yes, a very active track…
NY Subway Office

NY Subway Office

30 Years from Today

30 years from today–that’s January 19th, 2038–at 3:14am, the 32-bit unsigned clocks of legacy Unix systems will roll over. They’ll read January 1, 1970.

I wrote about it here.

More serious than Y2K, since all such clocks will roll over. If you recall, with Y2K it was primarily whether the programmer used two-digit years or accounted for the 1999-2000 transition some other way.

Similarly to Y2K, only those computers making decisions based on the clock/calendar will be affected. For instance, your car won’t since it doesn’t care (or even know) what day it is.

Another similarity: we can easily categorize calendar problems from severe (loss of life) to nuisance to trivial. For instance, if some cash register receipts (or even bank statements) read 1970, that’s unlikely to cause mass insanity/hysteria.

Our best bet is that all such legacy systems will have retired by then. And that’s not too bad a bet, either.

Python beats Perl

I like this news:

The TIOBE Programming Community Index has declared Python as the Programming Language of 2007 due to a 58% surge in its popularity rating during the year, making it now the sixth most popular programming language and finally surpassing Perl. They also assert that Python has become the “defacto glue language,” being “especially beloved by system administrators and build managers.”

One of my goals in life is to never learn Perl. I had a few close calls over the years, but now I think I just might make it.

When I need to write a script, I use bash for the simpler stuff and Python for the more complicated. I used to be able to write stupefying sed/awk scripts, but now I’ll do that in Python, too.

Btw, the TIOBE link above tries to gage programming languages’ popularity, though carefully qualifying it: “Observe that the TIOBE index is not about the best programming language…”

Cast in Concrete

From RFC 3551 (RTP Profile for Audio/Video), for G722:

Even though the actual sampling rate for G.722 audio is 16,000 Hz, the RTP clock rate for the G722 payload format is 8,000 Hz because that value was erroneously assigned in RFC 1890 and must remain unchanged for backward compatibility.

Ouch. Even Internet RFC’s aren’t immune to errors, and backward compatibility will always be with us.

The Mythical 5%

Bruce Eckel, author of Thinking in C++, relays a commencement address he gave.

First, some bad news:

The statistics are sobering: 50-80% of programming projects fail. These numbers are so broad because people don’t brag about their failures, so we have to guess. In any event, this makes the world sound pretty unreliable. …

Many projects that do somehow “succeed” still resemble sausage: you don’t want to see how it was made, or what’s in it.

Now, some good news:

An even more fascinating metric is this: 5% of programmers are 20x more productive than the other 95%. If this were a science, like it claims, we could figure out how to get everyone to the same level.

Differences this drastic do exist. Talent, experience and education provide raw material. But it must be cultivated.

So how do you become one of these mythical 5%?

These people are not those who can remember all the moves and have fingers that fly over the keyboard erupting system commands. In my experience those in the 5% must struggle to get there, and struggle to stay there, and it’s the process of continuous learning that makes the difference.

Because of what I do, I’ve met more than my share of these people. They read a lot, and are always ready to tackle a new concept if it looks worthwhile. I think if they do go to conferences they’re very selective about it. Most of their time is spent being productive, figuring things out.

The big issue is knowing that you’re going after that 20x productivity increase. Which means getting leverage on everything you do. Never just “bashing something out,” but using the best tools, techniques, and ideas at your disposal. Always doing your best.

…Being able to analyze and understand a situation and discover the hinge points of a problem is essential; this takes a clear mind and detached perspective. For example, sometimes the choice of programming language makes a huge difference, but often, it’s relatively unimportant. Regardless, people will still spend all their time on one decision while something else might actually have a far greater influence. Architectural decisions, for example.

Well put. Understand the problem well. Avoid the temptation to shoe-horn what seemed to work last time. Think, too: what would you do differently next time? How would you solve the problem in a more clear manner? How would you make the source code–every line–more clear?

Often, you need to “bash[] something out.” Do it in the smartest way possible, despite the pressures to deliver. Hesitating to dive in at the appropriate time is called analysis paralysis: you might need a very imperfect first cut to get past that.

One factor of the 20x difference is learning what you can live with, and fixing what you can’t. The cost of rewriting (let’s say “refactoring”) a piece of code can be painful, but there’s also a cost of living with it. It can be an agonizing call.

Gerald Weinberg … is most famous for saying “no matter what they tell you, it’s always a people problem.”

Usually the things that make or break a project are process and people issues. The way that you work on a day-to-day basis. Who your architects are, who your managers are, and who you are working with on the programming team.

An appropriate finish:

You’ll need to make a lot of mistakes in order to figure things out. So be humble, and keep asking questions.

CastleCops Program List

I get a strange message about a program crashing. I don’t recognize its name (swdsvc.exe). Is it legit or not? When I google it, I don’t know if I should trust the web-sites that come up any more than the mysterious executable. What to do?

CastleCops keeps very valuable lists of programs and things you’d expect to find on your PC, and those you don’t.

Hint: Use the search box. There are a lot of entries.

(As it turns out, swdsvc.exe is SpyWareDoctor.)