I’ve been pushing buttons, clicking mice, and typing in arcane crap from command lines for a good many years now. And today carried on that tradition. I was on my way to the bank to deposit some checks 1, with a trip to my franchise haircut place planned for afterwards. A ring of the cellphone derailed the tonsorial plans, as an admin buddy of mine found himself in undeserved distress from system users.
He needed some help; no problem, I’ll be home when I’m done at the bank. The haircut can wait.
Some of our filesystems had gone out to lunch due to failure of external systems which we don’t manage.
First step was to assess the damage. I logged in from home, my buddy was ensconced in his shabby cubicle at work, and we did the evaluation of the problem. No smoking gun. Damn. So, iff’en you cant fix it, you monitor the problem.
Over the next few hours, we crafted a simple-minded little script to watch our stuff, put that script in cron, and wired it up to the Tivoli monitors we already use. Tested it, and by damn, it catches the problem, without solving same.
This is what sysadmins do:
- Help a brother out.
- Stay until the job is done.
- Proceed one step at a time.
- Drink celebratory beers.
- Be thankful, that at least for this weekend, they aren’t on call.
Not a lot of work-specific stuff, but admins help each other out. That’s rule #1.
-k-
1