rsync drive to drive speed?

This forum is for Marvell Kirkwood devices such as the GoFlex Home/Net, PogoPlug v1/v2, SheevaPlug, and ZyXEL devices.

rsync drive to drive speed?

Postby BinaryJay » Sat Mar 26, 2011 3:43 pm

Looking for any advice on how I might be using rsync wrong here...

Every so often I plug a secondary drive in and rsync the contents for backup. Problem is, it only copies at around 3MB/s... now, the main drive is EXT4 and the backup drive is NTFS but I double checked top while copy is in progress and it is not being CPU bottlenecked at all.

I've tried doing it forcing whole file mode, etc. but it made no difference. I can't figure out why rsync is going so slow when both of the drives are physically connected. As such, it takes freaking forever to complete the backups.
BinaryJay
 
Posts: 87
Joined: Mon Dec 06, 2010 10:27 pm

Re: rsync drive to drive speed?

Postby WarheadsSE » Sun Mar 27, 2011 7:16 pm

These are both over USB, correct? and which format is the source drive?
Core Developer
Remember: Arch Linux ARM is entirely community donation supported!
WarheadsSE
Developer
 
Posts: 6807
Joined: Mon Oct 18, 2010 2:12 pm

Re: rsync drive to drive speed?

Postby BinaryJay » Mon Mar 28, 2011 1:59 pm

$this->bbcode_second_pass_quote('WarheadsSE', 'T')hese are both over USB, correct? and which format is the source drive?


Yes, both over USB directly connected to the Pogo. The EXT4 is the source drive so the poor write performance of NTFS is a factor though I thought that it shouldn't be that bad considering that the CPU overhead is not getting anywhere close to 100% during the writes. Perhaps there is more to it than just that, though.
BinaryJay
 
Posts: 87
Joined: Mon Dec 06, 2010 10:27 pm

Re: rsync drive to drive speed?

Postby kmihelich » Mon Mar 28, 2011 5:01 pm

I have seen 20-30 mbyte/sec between two plugs using gigabit, 5400rpm USB drives, ext4 formatted both sides, unencrypted transfer. Encryption shoots speeds down to around 7mbyte/sec.
Arch Linux ARM exists and continues to grow through community support, please donate today!
kmihelich
Developer
 
Posts: 1133
Joined: Tue Jul 20, 2010 6:55 am
Location: aka leming #archlinuxarm

Re: rsync drive to drive speed?

Postby BinaryJay » Mon Apr 11, 2011 4:08 pm

With both sides EXT4 the speed is now a still disappointing 9.5MB/s.

I know, at least, that I can read over the network from Drive A at close to 30MB/s... I guess it's plausible that the Drive B write speed just sucks but it's a WD Elite 1.5TB drive that I don't recall having any write performance issues with in the past.

Any other clues?
BinaryJay
 
Posts: 87
Joined: Mon Dec 06, 2010 10:27 pm

Re: rsync drive to drive speed?

Postby kmihelich » Mon Apr 11, 2011 4:15 pm

Be sure you're actually networked gigabit. 9.5MB/s is an awfully familiar number when hitting the top end of 100mbit.
Arch Linux ARM exists and continues to grow through community support, please donate today!
kmihelich
Developer
 
Posts: 1133
Joined: Tue Jul 20, 2010 6:55 am
Location: aka leming #archlinuxarm

Re: rsync drive to drive speed?

Postby BinaryJay » Mon Apr 11, 2011 5:26 pm

$this->bbcode_second_pass_quote('kmihelich', 'B')e sure you're actually networked gigabit. 9.5MB/s is an awfully familiar number when hitting the top end of 100mbit.


I'm not using the network at all, both drives are plugged directly into the plug via USB... so I'd be surprised if the network was the problem. :P

Do all USB ports on the pogoplug share the same bus? Even so, I wouldn't expect that to matter in this kind of situation. I verified the plug is not being CPU limited during the rsync operation as well.
BinaryJay
 
Posts: 87
Joined: Mon Dec 06, 2010 10:27 pm

Re: rsync drive to drive speed?

Postby kmihelich » Mon Apr 11, 2011 6:38 pm

Yes they do. USB to USB transfers on a plug will not reach full speed.
Arch Linux ARM exists and continues to grow through community support, please donate today!
kmihelich
Developer
 
Posts: 1133
Joined: Tue Jul 20, 2010 6:55 am
Location: aka leming #archlinuxarm

Re: rsync drive to drive speed?

Postby BinaryJay » Thu Apr 14, 2011 3:56 am

You know, my first instinct was that connecting everything directly to the plug would result in the fastest transfer but I guess I'll have to pull out the pocketbook and start up my HTPC which is a gigabit capable desktop linux machine and rsync over the network because I don't have any issues saturating USB2 for unidirectional transfers over the network with the plug.
BinaryJay
 
Posts: 87
Joined: Mon Dec 06, 2010 10:27 pm

Re: rsync drive to drive speed?

Postby kmihelich » Thu Apr 14, 2011 4:07 am

Regular computers hit the same wall. When you transfer between two or more USB devices on the same hub the overall speed decreases significantly. When I go between the USB drives on two plugs over gigabit, things happen at full speed.
Arch Linux ARM exists and continues to grow through community support, please donate today!
kmihelich
Developer
 
Posts: 1133
Joined: Tue Jul 20, 2010 6:55 am
Location: aka leming #archlinuxarm


Return to Marvell Kirkwood

Who is online

Users browsing this forum: No registered users and 14 guests