ten billion butterfly sneezes

One simultaneous curse and blessing of having worked with Unix systems for lots of years is that I encounter features and programs that I’d heard of, but never explored in any fashion. These encounters are uniformly all fun, after I realize what not-used feature had bitten me in the butt.

And so it was last week, I was compiling openssl, to get the latest libraries to link with a new version of openssh. And I was cruising; ssh RPMS were done for RHEL4 and RHEL5 in both 32 bit and 64 bit. Then I moved to the Itanium version, where the openssl piece simply refused to compile. A look at the debug output and Makefile, and I was able to reduce my problem to a one-line shell script; the referenced crypto directory existed in my current working directory. Observe the bold parts of the following:

#!/bin/bash
( cd crypto && echo “current directory is $(pwd)” )

Worked as expected.
#!/bin/sh
( cd crypto && echo “current directory is $(pwd)” )

failed with:
./x: line 2: cd: crypto: No such file or directory

WTF?? I could cd into crypto, see the files therein, and couldn’t begin to fathom what the problem could be. And, /bin/sh was a symlink to /bin/bash. I knew bash behaves differently depending on how it’s invoked. A look at the bash manpage provided no enlightenment.

This situation had to be solved; there could potentially be a number of third-party scripts which could fail in bizarre ways. I concluded, erroneously as it turned out, that the problem was Itanium-specific, and opened a ticket with Red Hat. The little script behaved properly on their Itaniums. A few straces later, and RH and I concluded that the environment on the offending system was at the heart of the problem.

I then began unsetting environment variables by hand, one at a time, and trying my little script again¹. And, at long last, I unset CDPATH, ran the script again, and Voila!, this little fracas was over. Setting CDPATH on systems where “it worked” before caused an identical failure.

A little CDPATH research revealed, from the bash manpage:

CDPATH The search path for the cd command. This is a colon-separated
list of directories in which the shell looks for destination
directories specified by the cd command. A sample value is
“.:~:/usr”.

And, from the bash in POSIX mode docs, which happens when #!/bin/sh is line 1 of your script:

19. If `CDPATH’ is set, the `cd’ builtin will not implicitly append
the current directory to it. This means that `cd’ will fail if no
valid directory name can be constructed from any of the entries in
`$CDPATH’, even if the a directory with the same name as the name
given as an argument to `cd’ exists in the current directory.

I’ve known that CDPATH exists for close to 20 years. Never used it. A coworker had set it globally in some profile. He’d been working with Oracle RAC, and there are a boatload of uniquely named directories under /usr/local, or wherever. This artifice made his life easier, but at the cost of sacrificing an out of the box system, that behaves as one is used to. I can’t fault him. Had he included a “:.” in CDPATH, everything would have worked as expected. Or have set CDPATH for his sessions only.

I’m an old timer; I run as close to a default vanilla system as prudence allows. I have two aliases defined in my profile, which pass different options to ssh. I learned this via the school of hard knocks, back in the day when I’d aliased rm to rm -i, and then got on a system where it wasn’t so aliased, and watched an hour’s worth of work go down the tubes, as I typed rm *, and expected to be prompted a file at a time.

Once burned, twice shy. Nevertheless, episodes like this are what makes being a sysadmin an absolute hoot.

-k-