Misterious LVM2 slowness ... [SOLVED]

[erm67]

Hi to everybody, I recently acquired a Seagate GoFlex Net to upgrade my old IA32 mini-itx nas, I had no problem installing archlinux, using a serial cable, and I am satisfied so far. Today I installed the second disk, both disks were formatted on the old nas and use lvm, and I noticed it was considerably slower than the first disk, but supposedly it is a much faster and recent disk.
This are my results, I am using hdparm here to test but the results are the same using iscsi or aoe:
$this->bbcode_second_pass_code('', ' LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data fedora -wi-ao---- 131.55g
root fedora -wi-ao---- 15.00g
swap fedora -wi-a----- 2.00g
home sl -wi-a----- 647.35g
root sl -wi-a----- 16.00g')

the fedora VG is on the first disk /dev/sdb and the sl VG is on /dev/sda
$this->bbcode_second_pass_code('', '[root@alarm vblade]# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 314 MB in 3.02 seconds = 104.09 MB/sec
[root@alarm vblade]# hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 166 MB in 3.01 seconds = 59.13 MB/sec
[root@alarm vblade]#
')

Accessing the physical disk sda is a lot faster than sdb as expected, and both disks are as fast as they were on the old system.

$this->bbcode_second_pass_code('', '[root@alarm vblade]# hdparm -t /dev/mapper/fedora-data

/dev/mapper/fedora-data:
Timing buffered disk reads: 178 MB in 3.00 seconds = 59.28 MB/sec
')

Accessing a logical volume on sdb is as fast as accessing the physical device, as expected.

$this->bbcode_second_pass_code('', '/dev/mapper/sl-home:
Timing buffered disk reads: 56 MB in 3.09 seconds = 18.13 MB/sec
')

Accessing logical volumes on sda instead is terribly slow 5 times slower than the physical disk ... what is going on? I had 40Mb/s write speed over Gbit network on exactly that logical volume when it was mounted on the old nas ...
Is it possible that it has something to do with the ARM architecture?

[erm67]

Update:
mistery at least in part solved:
$this->bbcode_second_pass_code('', ' --- Logical volume ---
LV Path /dev/fedora/data
LV Name data
VG Name fedora
LV UUID oUJgSV-PbEo-7Sfq-r0ht-tR0U-d9Oy-kGB2tB
LV Write Access read/write
LV Creation host, time erm67.lan, 2016-07-26 21:33:06 +0000
LV Status available
# open 0
LV Size 131.55 GiB
Current LE 33678
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 1024
Block device 254:2

--- Logical volume ---
LV Path /dev/sl/root
LV Name root
VG Name sl
LV UUID t22NU7-0m3A-9eAH-FRRo-Lp0K-RKFF-psGgup
LV Write Access read/write
LV Creation host, time localhost, 2014-12-01 19:28:04 +0000
LV Status available
# open 0
LV Size 16.00 GiB
Current LE 4096
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 254:3
')
lvdisplay -v shows that LVs in the fedora VG are using 1024 readahead while LVs in the sl VG use 0 sectors

$this->bbcode_second_pass_code('', '[root@alarm ~]# blockdev --report
RO RA SSZ BSZ StartSec Size Device
rw 256 512 4096 0 8388608 /dev/ram0
rw 256 512 4096 0 8388608 /dev/ram1
rw 256 512 4096 0 8388608 /dev/ram2
rw 256 512 4096 0 8388608 /dev/ram3
rw 256 512 4096 0 8388608 /dev/ram4
rw 256 512 4096 0 8388608 /dev/ram5
rw 256 512 4096 0 8388608 /dev/ram6
rw 256 512 4096 0 8388608 /dev/ram7
rw 256 512 4096 0 8388608 /dev/ram8
rw 256 512 4096 0 8388608 /dev/ram9
rw 256 512 4096 0 8388608 /dev/ram10
rw 256 512 4096 0 8388608 /dev/ram11
rw 256 512 4096 0 8388608 /dev/ram12
rw 256 512 4096 0 8388608 /dev/ram13
rw 256 512 4096 0 8388608 /dev/ram14
rw 256 512 4096 0 8388608 /dev/ram15
rw 1024 512 4096 0 1000204886016 /dev/sda
rw 1024 512 4096 2048 999678803968 /dev/sda2
rw 1024 512 4096 0 160041885696 /dev/sdb
rw 1024 512 4096 2048 524288000 /dev/sdb1
rw 1024 512 4096 1026048 159515672576 /dev/sdb2
rw 1024 512 4096 0 15733161984 /dev/sdc
rw 1024 512 4096 23 15732959744 /dev/sdc1
rw 1024 512 4096 0 2151677952 /dev/dm-0
rw 1024 512 4096 0 16106127360 /dev/dm-1
rw 1024 512 4096 0 141255770112 /dev/dm-2
rw 0 512 4096 0 17179869184 /dev/dm-3
rw 0 512 4096 0 695088447488 /dev/dm-4
')
blockdev confirms it, and after a
$this->bbcode_second_pass_code('', '[root@alarm ~]# blockdev --setra 1024 /dev/dm-4
')
the problem is solved.
$this->bbcode_second_pass_code('', '[root@alarm ~]# hdparm -t /dev/sl/home

/dev/sl/home:
Timing buffered disk reads: 322 MB in 3.01 seconds = 106.85 MB/sec
')

Now the mistery is only in part solved since this is a fresh install, I have not made any customization yet, why dm-3 and dm-4 use by default 0 readahead is still a mistery ..... lvm? systemd?