BackupPC-users

Re: [BackupPC-users] Restore via rsync gets stuck

2010-04-22 10:55:46
Subject: Re: [BackupPC-users] Restore via rsync gets stuck
From: Tim Connors <tconnors AT rather.puzzling DOT org>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 23 Apr 2010 00:37:58 +1000 (EST)
On Wed, 21 Apr 2010, Les Mikesell wrote:

> On 4/21/2010 7:47 AM, Les Mikesell wrote:
> > Philippe Bruhat (BooK) wrote:
> >> On Wed, Apr 21, 2010 at 08:45:29AM +0200, Philippe Bruhat (BooK) wrote:
> >>> On Wed, Apr 21, 2010 at 08:35:03AM +0200, Philippe Bruhat (BooK) wrote:
> >>>> I understand this as rsync currently restoring the file above. Using ls 
> >>>> -l
> >>>> to see the progress of the restore, I get this:
> >>>>
> >>>> -rw-r--r-- 1 esouche esouche 107368448 2010-04-05 22:47 places.sqlite
> >>>> -rw------- 1 root    root    107216896 2010-04-21 00:00 
> >>>> .places.sqlite.UL48SN
> >>> I tried restoring only this file, and watched the progress.
> >>> The restore got stuck at the same point:
> >>> -rw-r--r-- 1 esouche esouche 107368448 2010-04-05 22:47 places.sqlite
> >>> -rw------- 1 root    root    107216896 2010-04-21 08:43 
> >>> .places.sqlite.uVfJCE
> >>>
> >>
> >> So I tried to restore that specific file using a zip archive, only to
> >> discover that the old file on the target and the file being restored
> >> by backuppc had the same md5 and sha1 sums.
> >>
> >> Is that a known bug in rsync?
> >
> > No - or at least I haven't heard of it. Could you have filesystem 
> > corruption on
> > either side?
>
>
> One other thought here: was anything else running that could have been
> writing to or locking the file during the restore?

I've seen this problem too, and it's not filesystem issues.

It's either rsync or the perl library (and I very strongly suspect the
latter) that is deadlocking.  Trace it with strace - it's always the same
place (er, sorry, can't remember offhand!).  I'm restoring with rsync
version 3.0.6 and 3.0.7.

I've found that rsync restore to a filesystem that is already populated is
so unreliable I just avoid it now.  I restore to an empty path, then
rsync from that location back to the final destination.

I *once* had a backup lockup in the same fashion (only to get interuppted
with SIGARLM about 26 hours later from memory).  Repeatedly, with
incremental backups, always locking up on the same file (this was an
incremental of about level 3 or so, with 1 and 2 obviously having
succeeded).  I think I had it configured with full backups every 60 days,
incrementals every day with levels 1,2,3,4,5,2,3,4,5,3,4,5,4,5 repeating.

If I removed the problematic partition from the scheme, then readded it a
few days later when it was back to a lower level (and hence doing an
incremental against a previous level that *did* work), it *still* locked
up.

If I deleted back to previous levels, then I still had it lock up on the
same file.  I can't remember whether I had to delete all the way back to a
full, or whether I had to go back several incrementals.  But I had to go
back further than I was expecting given the layout (sorry, too long ago to
remember the details) and what had previously succeeded.  I think I even
tried forcing a full, which still locked up at the same place.  I can't
remember whether I deleted the file from the incremental backup to force
it to pull it in from scratch.

Once I got it all the way through a successful backup, I've never had a
problem with that host again.  The rsync version on that was 2.6.8 on
solaris.  It's a strange enough machine with weird ssh behaviour that I
wasn't overly concerned about that backup locking up, particularly as it
hasn't happened since.

But the restores failing in this way is scary.  And it makes me nervous to
replace our existing backup infrastructure with backuppc if it can't
reliably do a restore!

-- 
TimC
It is amazing how much "mature wisdom" resembles being too tired.
                                     --Robert A. Heinlein

------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/