[SGVLUG] rsnapshot: the good and the bad - recovery after patching that goes very very wrong

[David Lawyer]

On Tue, May 08, 2007 at 08:42:53AM -0700, Claude Felizardo wrote:
> First, what's the best way to defer fsck of a large filesystem when
> rebooting?  I have a rather large filesystem with thousands of hard
> links that takes forever to run fsck, especially when there are
> several errors and I have to reboot over and over.

I seldom run fsck on my Pentium-1 PC since I've set it to seldom run
using tune2fs.  Use the -l option to list the current status.  I've
set mine to a max. mount count of 300, and a "check interval" of
infinity.  Also, I use ext3 so if someone (like my wife) pulls the
plug, it fixes the resulting problem on reboot without running fsck.

> 
> ------------------------------------------------------------------------
> 
> Here's the story of a patch job going horribly wrong and how rsnapshot
> came to my rescue but stalling a full startup...

Do you mean that you are patching source code and then compiling?  If
the "patch" fails, shouldn't it not install anything so that it will
work as before?
			David Lawyer

> Okay, stupid me.  Before completing the installation of my replacement
> fileserver, I got carried away playing with iptstate on various
> machines and wanted to install it on my fileserver at home.  I needed
> to resolve some dependencies but the machine hadn't been patched
> since, oh probably Feb when the mirror I was pointing to wasn't
> available at the time.  I found a different mirror so feeling lucky, I
> told it to patch everything and went back to work.  Checked back on it
> a little later and a few screens of errors had scrolled by about
> various patches failing.  oh oh.  I tried to type ls, coredump.  even
> ps caused a coredump.  I tried to open up a new ssh session and got no
> response.  big oh oh.  I could access some web services but I couldn't
> do anything new.  Port mapped to the new server on the port and tried
> to hop over to the old server, no response.  damn.
> 
[snip]