It's not really just a number of files issue, it's the size of the files
too. The best way to back up a large filesystem with lots of little
files is an image based backup. If you have Clariion, Symmetrix, FAStT,
NAS, STK D, StorEdge AVS, or StorEdge 6920 as your storage you can use
PowerSnap. Another option would be the SnapImage module. Since the image
will be backed up as one large bit of data, it will stream to the tape
much more quickly.
-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Allan Nelson
Sent: Wednesday, December 05, 2007 9:09 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] NetWare throughput
Thanks for all the suggestions.
I'm pretty much convinced it's 'number of files' that has the huge
effect.
We're trialling a Data Domain de-dup device at the moment (so backing up
to disk) and to tell you the truth, for this particualr Netware volume
it's no faster which again tends to confirm my suspicions. We get good
throughput while all the other clients (mix of Linux, Netware, Solaris
and Windows) are in the mix streaming data to it, but we always end up
with this one lonely server chugging along when all the rest have
finished. Certtainly CPU umph isn't a problem - plenty of CPU left on
the client when the backups in progress.
I'm sure I saw a thread some time ago re 'number of files' and some sort
of plateau after which performance plummets, but can't seem to find it.
As a file server it fair zips along with no issues.
I was thinking of splitting the volume in two to try and get around
600,000 filers in each to see if that improved things. A project for a
rainy weekend perhaps.
Thanks... Allan.
>>> ajs AT STOLAF DOT EDU 04/12/07 16:56 >>>
Somewhat better here, though we've never been impressed with the speed
of the NetWare client. Our biggest volume (~770GB, 1.2 million files)
backs up to disk in about 13 hours (1GB/min, 16.67MB/s).
This is pretty much what we get from all of our NetWare volumes. The
only exception is the volume that hosts our workstation images (maybe a
dozen 2GB files). Here we see ~25MB/s.
We used to tweak the TCP/IP stack a tad (to increase buffers, receive
windows, etc.) but recent kernels do a good job of autotuning and this
is generally frowned upon.
No magic bullet here, unfortunately...
ajs
Allan Nelson wrote:
> Hi
> I'd be interested in the asnwer to this too.
> We basically have the same issue - a 380GB volume which takes about
15-16 hours to backup.
> My hunch is that it's the number of files (1.2 million) or the poor
Netware client (7.2).
>
> I've also setup load balancing on 2 GBe NICs but it doesn't really
help. The bottleneck seems to be trawling through the file system.
>
> Be interesting if anyone is getting much better throughput than this
on Netware with similar sized volumes/number of files.
>
> Cheeers... allan.
>
>>>> LPamintuan AT REGINA DOT CA 04/12/07 15:11 >>>
> Fellow admins,
>
> Just need to find out if there is something like 'bigasm' on Windows
and Unix servers which I use on my NetWare clients to test backup
throughput. Somehow to backup 500 GB of data, it is taking almost 24
hours to complete. This NetWare server is a Dell PE 2850 with quad
processor and 8 GB of RAM and connected to a GB network.
>
> My environment is:
> Networker server v7.3.3 on Red Hat Linux ES 4 backing up to disk
devices.
> NetWare client/server v7.2 on NetWare v6.5 support pack 6.
>
> Any help or advise is greatly appreciated.
>
> regards,
>
>
> Librado Pamintuan
> Technical Support Analyst II
> I.T.S
> City of Regina
>
> Phone: (306) 777-7573
> General Fax: (306) 777-6804
> eFax: (306) 546-6002
> eMail: lpamintuan AT regina DOT ca
>
>
> DISCLAIMER: The information transmitted is intended only for the
addressee and may contain confidential, proprietary and/or privileged
material. Any unauthorized review, distribution or other use of or the
taking of any action in reliance upon this information is prohibited. If
you received this in error, please contact the sender and delete or
destroy this message and any copies.
>
> To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>
--
Tony Skalski
Systems Administrator
507-786-3227
ajs AT stolaf DOT edu
-------------------------------------------------------
St. Olaf College 1510 St. Olaf Avenue
Information and Northfield, MN
Instructional Technologies 55057-1097
=======================================================
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
--
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
