Encrypted 2-drive mirrored RAID setup

[mastashake57]

After an EXT4 upgrade:

$this->bbcode_second_pass_code('', '[root@pogoweb ~]# dd count=100 bs=1M if=/dev/zero of=/srv/test oflag=sync
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 6.91609 s, 15.2 MB/s
[root@pogoweb ~]# rm /srv/test

[root@pogoweb ~]# dd count=1000 bs=1M if=/dev/zero of=/srv/test oflag=sync
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 72.8226 s, 14.4 MB/s
[root@pogoweb ~]# rm /srv/test ')

[mastashake57]

So, putting my first Samba stats here. Downloading a 600+ MB file from my Windows 7 Server:

$this->bbcode_second_pass_code('', 'smb: \> get SW_CD_Windows_Svr_Std_2003_R2_32-BIT_X64_English_ISO_32bit_1_MLF_X13-73742.ISO
getting file \SW_CD_Windows_Svr_Std_2003_R2_32-BIT_X64_English_ISO_32bit_1_MLF_X13-73742.ISO of size 623075328 as SW_CD_Windows_Svr_Std_2003_R2_32-BIT_X64_English_ISO_32bit_1_MLF_X13-73742.ISO (24259.3 KiloBytes/sec) (average 24259.3 KiloBytes/sec)')

Throughput is around 23.690722656 MBytes. Let me try on bigger files.

[mastashake57]

Gigabit backbone, no JUMBO frames:

$this->bbcode_second_pass_code('', 'smb: \> get bigfile.iso
getting file \bigfile.iso of size 1246150656 as bigfile.iso (19998.4 KiloBytes/sec) (average 21242.1 KiloBytes/sec)')

htop showed that the smbclient varied from 20% CPU to as high as 71%, averaged around 45~46%

Second run:

$this->bbcode_second_pass_code('', 'smb: \> get bigfile.iso
getting file \bigfile.iso of size 1246150656 as bigfile.iso (24499.6 KiloBytes/sec) (average 22435.3 KiloBytes/sec)')
Average htop is 67.2% CPU usage

Third run, turned off cherokee, mysqld and subsonic, nothing else running:

$this->bbcode_second_pass_code('', 'smb: \> get bigfile.iso
getting file \bigfile.iso of size 1246150656 as bigfile.iso (25745.6 KiloBytes/sec) (average 23290.9 KiloBytes/sec)')
Average htop is 87.8% CPU usage

[sambul13]

I assume you're writing to the RAID. Not sure, why you're not hitting 30 MB/s if its not 100% CPU load. Unless your enclosure chipset controls writing to the RAID and is not a bottleneck, RAID should increase Plug proc load thus limiting copy speed, but it seems the proc is not maxed out here, so it may be the enclosure.

[mastashake57]

$this->bbcode_second_pass_quote('sambul13', 'I') assume you're writing to the RAID. Not sure, why you're not hitting 30 MB/s if its not 100% CPU load. Unless your enclosure chipset controls writing to the RAID and is not a bottleneck, RAID should increase Plug proc load thus limiting copy speed, but it seems the proc is not maxed out here, so it may be the enclosure.

[sambul13]

It doesn't look from your data that the enclosure adds overhead to the Plug's CPU. Its USB 3.0 chipset may underperform in 2.0 mode, or its internal bus is slow as Greg_E reported: "RAID 0 is not faster than any other mode on USB 2.0, only getting maximum 20MBps writes and 30MBps reads, same for all modes. ATTO 2.46 used for benchmark." But its nice to see, HW RAID-enabled enclosures work fine with Pogo, and can deliver comparable read & write speeds without overloading the plug.

There might be a modded FW available for the enclosure. Try also hooking it directly to a PC via USB 2.0 and check the speeds. Despite Samba is not a limiting factor here, I wonder if ISCSI over LAN can be faster? If you run ArchLinux 3.5.3, you should be able to install ISCSI Target (targetcli and lio-utils) to the plug without issues. ArchLinux Wiki (not ARM) has articles on ISCI Target and Initiator setup.

[WarheadsSE]

sambul13: the likely limit here is the USB on the device itself, plus the CPU overhead of samba.

[mastashake57]

$this->bbcode_second_pass_quote('sambul13', 'T')here might be a modded FW available for the enclosure. Try also hooking it directly to a PC via USB 2.0 and check the speeds. Despite Samba is not a limiting factor here, I wonder if ISCSI over LAN can be faster? If you run ArchLinux 3.5.3, you should be able to install ISCSI Target (targetcli and lio-utils) to the plug without issues. ArchLinux Wiki (not ARM) has articles on ISCI Target and Initiator setup.

[WarheadsSE]

I wouldn't go messing with the firmware just yet!!

[sambul13]

$this->bbcode_second_pass_quote('mastashake57', 'I')'ll try to perform some exhaustive disk tests on a desktop machine and provide feedback.