[Solved] Memory efficient, friendly & cross plat' sync tool?

Ask questions about Arch Linux ARM. Please search before making a new topic.

[Solved] Memory efficient, friendly & cross plat' sync tool?

Postby dhead666 » Sat Mar 02, 2013 4:57 am

I'm looking for a file synchronization that could handle 300GB+ folders to be used with PogoPlug 4.
It should be memory efficient and cross platform (windows & linux), and of course friendly (aka gui).

I've tried Unison but the memory consumption was too high and it crashed even with a 512MB swap file.
I haven't tried yet Rsync.
Last edited by dhead666 on Tue Mar 05, 2013 12:34 pm, edited 2 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: Memory efficient, friendly and cross plat' sync tool ?

Postby dhead666 » Sat Mar 02, 2013 11:18 am

From another look at Unison I'm getting the feeling that something is wrong on Linux.
On initial scan Unison for Win7 is steady on 30MB but on Arch Linux it can get to 140MB (with swap).
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: Memory efficient, friendly and cross plat' sync tool ?

Postby dhead666 » Sat Mar 02, 2013 1:53 pm

I made more testing with Unison, now on a SheevaPlug (mounting a nfs share served by my PogoPlug).
Seems like Unison is hitting 180~200MB on Arch Linux while on Win7 it's 40~60MB.
It also looks much more stable, but extremely slow (remote nfs share).

Now I'm not sure why Unison crashed on PogoPlug 4 with 512MB swap,
I may need to test it again with swap on sata (and not on usb 3.0) and also separated drives for data and swap,
but my target setup will only use one usb 3.0 hdd.

Another option that I'm considering is to use a separate usb flash drive for swap (maybe a fast usb 3.0) but currently all my flash drives are pretty occupied.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: Memory efficient, friendly and cross plat' sync tool ?

Postby dhead666 » Tue Mar 05, 2013 12:33 pm

I went and gotten myself some brand new class 10 sd cards and tried again running Unison on PogoPlug, now with swap partition on SD card.
To my surprise Unison worked well and didn't crash even with large folders.
So swap file on the same usb 3.0 ext4 drive don't work, swap partition on sd card do work.
Only initial syncing requires the extra swap memory so I'm not really worries with wearing the sd card and these are darn cheap.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm


Return to User Questions

Who is online

Users browsing this forum: No registered users and 3 guests