HFT [High Frequency Trading] Breaks Speed-of-Light Barrier, Sets Trading Speed World Record.
Adds a new unit of time measurement to the lexicon: fantasecond.

On September 15, 2011, beginning at 12:48:54.600, there was a time warp in the trading of Yahoo! (YHOO) stock. HFT has reached speeds faster than the speed-of-light, allowing time travel into the future. Up to 190 milliseconds into the future, or 0.19 fantaseconds is the record so far. It all happened in just over one second of trading, the evidence buried under an avalanche of about 19,000 quotes and 3,000 individual trade executions. The facts of the matter are indisputable. Based on official UQDF/UTDF exchange timestamps, there is unmistakable proof that YHOO trades were executed on quotes that didn’t exist until 190 milliseconds later!

Millions of traders depend on the accuracy of exchange timestamps — especially after bad timestamps were found to be a key factor in the disastrous market crash known as the flash crash of May 2010. …

The lowly time-stamp. Not rocket science. Not crucial like price, right? What’s to get wrong? An exchange with unlimited resources couldn’t mess up like that, could it? (Assuming the analysis is correct.)

As a software builder, I wonder how did it happen?

It probably wasn’t buggy code: unit tests are good, but more of them probably won’t catch this one.

Was it sheer load? It wasn’t the highest overall traffic they experienced:

The chart below shows quote message rates for UQDF and multicast line #6 which is the line that carries YHOO quotes. … Although traffic from YHOO was a significant percentage of all traffic on UQDF, it was not high enough to indicate any problems. Note the much higher surge on the right side of the chart; there weren’t any known problems at that time in YHOO.

So the system would pass a generic stress test. Yet still allow this.

Maybe the system was stressed in some unexpected way. This stress’ profile wasn’t predicted, characterized, mitigated. The system design didn’t account for it.

A multi-tasking operating system is likely part of the picture. Maybe another task was started that tied up crucial resources, even just CPU. Maybe not even a lot of resources, or not for very long. A multi-tasking operating system is geared to never say “enough.” Another task in the CPU’s ready queue? No problem. Out of RAM? Swap some to disk: user programs can’t even tell!

Information hiding gets us off the “honors system” with memory usage. But in the time domain, you’re still very much on the honors system: no one can make you complete your task in a given time. In fact, any innocuous-looking call can have any amount of code hiding deep inside it.

You can make a user wait for a tenth of a second or more and still seem really responsive. (Think “garbage collection.”) But a real-time data feed won’t wait. It’s a different ball-game.

Maybe we’re just seeing a hardware limitation hit. Maybe the best machines money can buy just can’t keep up with certain kinds of volume. There’s no guarantee they can.

Interesting stuff. Hat’s off to the author for such a detailed analysis.

Via ZeroHedge.

If you’re a developer, consider screen height a limited, precious resource. Conserve it. Trade width for height. Kill height cruft. (Can I coin that?)

Button bars: traditionally horizontal at the top of the app. Can you make it vertical, to the left or right?

Even the traditional menu bar (“File Edit View … Tools Window Help”) crowds you down by its height.

Case study: Amazon’s Instant Video in Internet Explorer. IE adds menu items and multiple button bars at the top. Then amazon’s web-site adds its headers across the top. The video winds up a tiny fraction of its potential size, squeezed by all the height cruft. Not to blame Amazon: there’s a “full screen” button, and F11 in IE takes away a bunch of stuff (if you know about it).

The lesson: when every layer adds its own, it compounds.

Software developers take heed.

As a user, there’s lots you can do to conserve height.

Did you know you can put the Windows task bar to the left or right? I put mine to the left, as narrow as possible. You lose a little of the icons’ descriptions, but mousing over gives them back. It flushes out some bugs, too: some programs start in the upper left corner, winding up under the task bar. (To move the task bar, put the mouse on the middle top edge and drag it to the middle left.)

Most software today lets you move menu bars, and the best ones let you dock them to the left or right, making them vertical. Very nice, though they can truncate text menus. Your word processor has several: don’t live with them crowding you. It’s worth retraining your eye to work with them vertically. (Convince your complaining co-workers, too.)

For working with source code, height is particularly valuable. Source-code flows down, and seeing more at once is better. I have a two-monitor set-up on my desk-top, and my cheap, last-generation NVidia video card lets me turn one monitor sideways. A $40 monitor stand and you’re in business. This has been an indispensable productivity boost for me. Don’t work with all horizontal monitors.

When looking at code, can you bump your font-size down one? You’ll see a lot more.

Update. I promise I’m not making this up: one day after this pontification a client passes along a request to do exactly this: rearrange things to conserve screen height in a program I wrote a couple years ago. File this under eerily prescient or physician heal thyself?

Here’s another to add to the mix: Cake. And this brief intro on the Boost-interest digest makes the pitch.

No makefile at all. You put compiler flags right in the source code, and it walks your source, looking for special comments and using some naming conventions.

I like the idea of compiler flags in the source. Source-code attributes like dllimport and pragma comment(lib…) prove that’s useful.

You need g++ and make (behind the scenes), so that constrains its user base. Will the naming conventions be too restrictive? It uses a global configuration (in /etc/), but I can envision wanting a project-specific common configuration file, too, being the control freak I am. Will it accommodate?

Put it on your watch list.

import serial	# http://pyserial.sf.net
import SerialPortScanWin32

pList = [x for x in SerialPortScanWin32.comports()]
port = pList[0][0] - 1 # single/first serial port
ser = serial.Serial(port, baudrate=1200, parity='E', timeout=0.2) # opens, too.
print "Monitoring serial port " + ser.name
data = []
while True:
	ch = ser.read(1)
	if len(ch) == 0:
		# rec'd nothing print all
		if len(data) > 0:
			s = ''
			for x in data:
				s += ' %02X' % ord(x)
			print '%s [len = %d]' % (s, len(data))
		data = []
	else:
		data.append(ch)

Python rocks!

PS: No, serial comm hasn’t gone the way of the dodo, particularly for many industries.

The module “msflxgrd.ocx” was loaded but the call to DllRegisterServer failed with error code 0x8002801c.

One person says get and run SubInACL.msi, and use it to make sweeping recursive permissions changes to the registry and system directories.

Yikes! Really?!?!?

This much-less-intrusive solution worked for me:

Right-click on C:\WINDOWS\System32\cmd.exe, and choose “Run as Adminisstrator”. From that command-line, run the registration:

regsvr32 msflxgrd.ocx

The registration now passes, and my program now comes up (not needing to be run as administrator).

Note that we thought the account we were running under had full Administrator privileges, but still failed to run this app as such. (It didn’t ask for the password when running cmd as administrator.)

The install popped up the familiar “Microsoft Visual C++ 2005 Redistributable” installer as part of its installation.

Microsoft Visual C++ 8 is still solid and widely used.

Microsoft is doing away with the manifest scheme in Visual Studio 2010. From Deployment in Visual C++ 2010:

Differences between Visual C++ 2008 and Visual C++ 2010

The most significant changes between Visual C++ 2008 and Visual C++ 2010 are:

* Visual C++ libraries no longer depend on manifests and are no longer installed in the WinSxS folder.
* Dependent applications and libraries no longer require manifest information.
* Local deployment no longer requires a satellite manifest.
…

To me, the manifest scheme just traded one problem for another. And the problem traded away (DLL issues) was one I never had.

Software engineers hard to find
Shortage of trained IT talent challenges Chicago companies

Good to know in these troubled times.

And to keep in mind:

“One good developer can do the work of three or four guys,”

And this…

… finding qualified job candidates is “the bane of my existence.”

Wally gets it (in his own Wally way).

The art of this job is binding the rare moments of inspiration to knowledge and machines.

That’s what I mean when I say, “…good ideas…”.

But we can’t afford the moments to be rare.