On Mon, Apr 14, 2008 at 11:21:02AM -0400, Raman Gupta wrote:
> > I have three hosts configured to backup to my PC. Here are the speeds
> > from the host summary:
> >
> > host 1: 24.77 GB, 14,000 files, 18.78 MB/s (slower WAN link)
> > host 2: 1.27 GB, 4,000 files, 1.89 MB/s (faster WAN link)
> > host 3: 4.82 GB, 190,000 files, 0.66 MB/s (fast LAN link)
> >
> > They all use rsync with the same setup, other than the exclude list.
> > Backups are configured to run one at a time so there is no overlap
> > between them.
> >
> > The speed of host 3 concerns me. Host 3 is by far the beefiest
> > machine, and on the fastest network link of all the hosts, but yet
> > backs up at only 0.66 MB/s (incrementals are even slower).
>
> Ok, it seems that the number of files has a large non-linear affect on
> the performance of BackupPC. I excluded a bunch of stuff from my host
> 3 backup, and the new stats are:
>
> host 3: 4.2 GB, 85,000 files, 2.19 MB/s
>
> For a file count reduction factor of 2.2, there was a speed increase
> factor of 3.3.
I suppose, BackupPC's speed is mainly affected by random access speed of
the server's pool storage. I've got hosts with lots of files as well
(small ones, mostly) and they take pretty long to back up. Look at I/O
utilization of the client during backup - it might be a bottleneck as
well.
Reading a file linearly is a quite cheap operation: The OS will
read-ahead (the RAID probably as well), disk heads don't need to move a
lot, metadata fits nicely into the OS' disk cache (and stays there) etc.
But if you've got a file system with several millions of files (like the
pool) distributed across tens of thousands of directories (like the
backup directories below the pc/ directory), things get worse: Lots of
random seeking across the disk, cache trashing, I/O waiting etc.. I'm
thinking about getting another 2 GB of RAM for my BackupPC server and
see whether it improves things.
This is an iostat -x -k 60 output during backup runs (2 backups in
parallel, 1 pretty fast client, 1 pretty slow with lots of files):
avg-cpu: %user %nice %system %iowait %steal %idle
1.89 0.00 2.53 9.49 0.00 86.09
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz
avgqu-sz await svctm %util
sdb 0.00 0.73 91.64 139.54 1208.53 2105.88 604.27 1052.94 14.34
2.75 11.89 4.31 99.57
(Server is quad-core with 2 GB, 3x500GB RAID5 on Dell PERC5/i -
switching to RAID10 would probably improve things a lot as well)
Bye,
Tino.
--
„What we resist, persists.” (Zen saying)
www.craniosacralzentrum.de
www.forteego.de
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
