Software, Technology, Business & Life

I’m doing some low-level Ethernet work (TCP/IP), with a device connected through two plain vanilla network switches. (I’m testing breaking the connection with neither side detecting the link-lost condition.)

I made a change to my software, and suddenly the Ethernet is dead. It seems to send and receive nothing.

I back out my change, and it still doesn’t work.

I back out a few more: no effect.

As it turns out, one of the Ethernet switches got confused and quit routing correctly (even through this simple configuration).

You can tell that the switch has some smarts:

• When you look at the traffic with WireShark, you don’t see what’s exchanged between the switch’s other nodes.
• When you have a mix of GB devices and 10MB devices on the same switch, the gigabit devices transfer data at full speed, so you know the switch isn’t exposing every packet to every (10MB) connection.

Still, I was astonished that this could happen.

For the record, the switches were:

• Netgear ProSafe 5 Port Gigabit Switch (Model GS105)
• Netgear 4-Port 10/100 Dual Speed Hub (Model DS104)

The older DS104 was the prime suspect. I simplified the route to just one switch (the GS105), which got it working. So I cycled power on the DS104, reconnected the original configuration, and everything worked again.

Update 4/8: They say you never learn anything from a mule’s second kick. I unceremoniously decommissioned the DS104 after it did the same thing again, this time with no provocation.

Symptoms: Windows machine with two network interfaces (NIC), 10.x.x.x and 192.168.x.x. I send a UDP broadcast message (to destination address 255.255.255.255), and see the same message go out on both networks. The problem is, the source address that goes out with each packet is the IP address for my first network interface, so it’s sending an unreachable address out my second network. Responding devices start doing ARP requests to try (in vain) to find that unreachable address instead of responding. (Watch it with WireShark.)

Cause: 255.255.255.255 is too general. TangentSoft has the general solution (implemented in boost). I craft the broadcast address more carefully, and it works.

	10.x.x.x (subnet 255.0.0.0) => 10.255.255.255
	192.168.1.x (subnet 255.255.255.0) => 192.168.1.255

It’s not the language or library. It’s seen for C#, C, and C++, etc, on Windows Vista, and here.

The broadcast address 255.255.255.255 is also known as INADDR_BROADCAST or ‘‘ in Python.

You only need one socket, which you can bind to all interfaces.

Enumerating your network interfaces requires some work: find it under Win32_NetworkAdapterConfiguration, fields IPAddress and IPSubnet. Those fields are arrays: you want element zero.

If you’re using python, you need the WMI package, which is built on the PyWin32 package.

import wmi
nicsList = wmi.WMI().Win32_NetworkAdapterConfiguration() # your network adapters

If you’re getting WinSock error 10013, you need to set SO_BROADCAST.

Hope that starts you down the right path.

Also, if your devices aren’t configured for the same subnet, crafting a sub-net-specific broadcast message means you’re not going to find them. For example, if you have a 10.x.x.x device off your 192.168.x.x interface, it’s going to ignore your 192.168.255.255 broadcast. You’ll have to fall back to disabling a network interface and using 255.255.255.255 to get to it.

OpenSSL is a particularly difficult one to build on MinGW, and it’s not clear even what approach will work. But I got it to build.

My MinGW/MSYS configuration is with the full suite of tools, including bash. (Follow these FFmpeg build instructions for setup.)

As well, I’m using GCC 4.3.3, from the TDM recent builds (http://www.tdragon.net/recentgcc/) setup.

I’m building OpenSSL 0.9.8k.

You need ActivePerl. MinGW has a /bin/perl, but it’s not up to the task. See OpenSSL’s INSTALL.WIN32 file for details.

From a Win32 command prompt (not bash), set up the your paths:

• First in the path: ActivePerl’s bin directory.
• Then MinGW’s paths (before Win32 native).

Fix the file e_os2.h, per BUG 1685. Remove the “static” modifier in the OPENSSL_IMPLEMENT_GLOBAL macro. (Note that it’s a 3-line macro.) This gets you by the “crypto/des/set_key.c line 72″ compile error.

Run ms\mingw32.bat. It will fail with “/bin/sh: copy: command not found”.

Now, edit the file ms\mingw32a.mak. This gets generated by every run of the ms\mingw32.bat file. Search for “exist” and you’ll see two DOSish commands:

if exist $(O_CRYPTO) ] $(RM) $(O_CRYPTO)

Change each to its bash equivalent:

if [ -e $(O_CRYPTO) ]; then $(RM) $(O_CRYPTO); fi

Finally, run the make, and specify the MinGW copy command:

make -f ms/mingw32a.mak “CP=cp”

The output files are in the ‘out’ subdirectory. Intermediate files are in ‘tmp’.

Include files are in ‘outinc’: the include/openssl directory has zero-byte place-holders for each file.

Make’s “clean” and “test” targets are broken. To clean, remove tmp and out. To test, run the various test programs in the ‘out’ directory.

All tests pass.

If you get missing symbols like DeleteDC and send when linking against libcrypto, you’ll need to include some win32 libraries. This typically does it:

make LIBS=”-lgdi32 -lws2_32″

xcopy vs. rsync, rsync vs. xcopy

Though rsync has capabilities that Win32’s xcopy only dreams of, how do the two stack up when compared apples to apples?

My test: synchronize a large collection of files, between two different local disks. 8.19 GB of data in 11,072 files across 182 directories.

My platform: a Dell Optiplex 740, AMD Athlon 64 X2 Dual Core 5200+, 2.61 GHz, 4GB RAM, Windows XP SP 2, latest updates applied.

I’m running rsync under CygWin, version 2.6.9. That rsync was written for Unix I don’t think handicaps it. But being forced to work through the CygWin DLLs just might. No networking or data compression, as that’s unnecessary here and would only slow it down.

Command lines:
    xcopy C:\TestSrc F:\tmp\xctest1 /D /E /C /I /Q /H /R /K /O /Y
    C:\Software\Open\CygWin\bin\rsync -q -a -r /cygdrive/c/TestSrc /cygdrive/f/tmp/rstest1

Results:

Building the directory from scratch:
	xcopy:	4:59.42
	rsync:	6:11.95
Updating one file somewhere in the directories:
	xcopy:	1.70 sec
	rsync:	2.98 sec
No files to update:
	xcopy:	1.33 sec
	rsync:	2.22 sec
Updating three files somewhere in the directories:
	xcopy:	1.25 sec
	rsync:	2.78 sec

These numbers don’t take into account the fact that XP caches the directory entries off the disk the first time they’re referenced. That operation penalizes the first operation (xcopy or rsync) by 10 seconds.

Conclusion: Though xcopy moves data 25% faster than rsync on its native Win32, rsync keeps up in all other respects. In the typical case (for me) where some small subset of files has changed, they’re neck and neck. So using rsync instead of xcopy wouldn’t put me at a performance disadvantage.

Keep in mind that dealing with CygWin’s paths aren’t for the faint of heart.

This is a quick and dirty benchmark. No averaging or further exploration than what you see above, though the numbers seemed consistent across a few runs.

An impressive performance by both rsync and CygWin.

Hope it helps.

[8/18/2008] P.S.: Important: Does your xcopy seem slow? Having it output each file’s name slows it down by orders of magnitude. Use /Q to make it run silently, once you’re convinced it’s doing the right thing.

See also the comments below.

Frank Wiles discusses hiring programmers.

• Finding good programmers is hard in any language. And that a good programmer can be as effective as 5-10 average programmers.

…

• You don’t need to hire an expert in language X, you can and should look for expert programmers that are willing to learn language X. An expert can easily cross over from being a novice in any language in a matter of a few weeks.

More:

What is an expert programmer?

Experience is key, but not necessarily in ways you might imagine. Time in the saddle, with a particular language is not as important as diversity of experience. Someone who has worked in several disparate industries, a generalist, is often a much better developer than one who has spent years in the same industry. There are exceptions to this, but in general I have found this to be the case.

If hiring and managing software developers is something you do, the article is well worth the read.

Via Slashdot