ix2-200 no 1000baseT-FD [Updated]

This forum is for Marvell Kirkwood devices such as the GoFlex Home/Net, PogoPlug v1/v2, SheevaPlug, and ZyXEL devices.

ix2-200 no 1000baseT-FD [Updated]

Postby kriztioan » Sat Sep 01, 2018 1:41 am

I own an Iomega Storcenter ix2-200 and installed Archlinuxarm following the instructions posted by Sven Kiljan at https://www.kiljan.org/2016/11/15/installing-arch-linux-on-an-iomega-ix2-200-2/; notably those on the headless preparation (i.e., not via TTL). I'm on the latest kirkwood-dt kernel using the kirkwood-iomega_ix2_200.dts overlay. Installing Archlinuxarm has really put the life back into this aging NAS-box. However, one thing has been bothering me for quite a while now; I am unable to connect the Storcenter at 1Gb/s to my R7000 (running dd-wrt).

I was hoping that somebody with more know-how (@WarheadsSE? @moonman? @summers?) could help me with this issue.

I tried different cables, confirmed the cables connecting at 1 Gb/s with the R7000 using another machine, and tried to force the NIC into a 1 Gb/s connection using both ethtool and mii-tool; all to no avail. Unfortunately the only other router I own is a Airport Express, which only has a 100 Mb/s Ethernet port. [update] I was able to enable the DHCP-server (bootpd) on my 2011 MacBook Pro and use it to connect to the Storcenter via a single Ethernet cable. However, again the connection could only be made when forcing the Mac's NIC to 100 Mb/s. Dmesg showed the link going up at 1000 Mb/s but going down again quickly. Lastly, the two leds on the Storcenter's Ethernet port are solid green (bottom) and flashing orange (top).[/update]

Below is a snippet from dmesg that shows that actually a 1 Gb/s link goes up but goes down a few times and eventually the connection stays up at 100 Mb/s.

$this->bbcode_second_pass_code('', '[ 1.723965] mv643xx_eth: MV-643xx 10/100/1000 ethernet driver version 1.4
[ 1.732633] mv643xx_eth_port mv643xx_eth_port.0 eth0: port 0 with MAC address aa:00:00:00:00:01
[ 2.602918] mv643xx_eth_port mv643xx_eth_port.1 eth1: port 0 with MAC address xx:xx:xx:xx:xx:xx
[ 27.186010] mv643xx_eth_port mv643xx_eth_port.1 eth1: link up, 1000 Mb/s, full duplex, flow control disabled
[ 28.309665] mv643xx_eth_port mv643xx_eth_port.1 eth1: link down
[ 34.940450] mv643xx_eth_port mv643xx_eth_port.1 eth1: link up, 1000 Mb/s, full duplex, flow control disabled
[ 36.047826] mv643xx_eth_port mv643xx_eth_port.1 eth1: link down
[ 42.757901] mv643xx_eth_port mv643xx_eth_port.1 eth1: link up, 1000 Mb/s, full duplex, flow control disabled
[ 43.872502] mv643xx_eth_port mv643xx_eth_port.1 eth1: link down
[ 46.361503] mv643xx_eth_port mv643xx_eth_port.1 eth1: link up, 100 Mb/s, full duplex, flow control disabled
[ 46.371426] mv643xx_eth_port mv643xx_eth_port.1 eth1: link up, 100 Mb/s, full duplex, flow control disabled')

Scouring the internet did reveal that there have been issues with the mv643xx_eth driver in the past. Notably with TSO corruption and unaligned packages. I am unable to say for sure if any of the patches I found addressing these issues have made it to main-line (https://git.kernel.org/pub/scm/linux/ke ... ?h=v4.18.5), as the driver's version number is stoically kept at 1.4. However, there is a patch at https://github.com/archlinuxarm/PKGBUIL ... irkwood-dt that explicitly disables TSO, but it is not included in the PKGBUILD anymore. Nonetheless, I have the following udev rule set up:

$this->bbcode_second_pass_code('', 'ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth*", RUN+="/usr/bin/ethtool -K %k tx off sg off tso off"')

I took note of the discussions at https://archlinuxarm.org/forum/viewtopic.php?f=53&t=7723 and https://archlinuxarm.org/forum/viewtopic.php?f=27&t=3851 and wonder if there is a possible issue with auto-negotiation or something not properly being initialized by the driver. My suspicion is that the R7000 gives up after a few times and then stops advertising 1000baseT-FD.

Below the information reported by ethtool

$this->bbcode_second_pass_code('', 'driver: mv643xx_eth
version: 1.4
firmware-version: N/A
expansion-rom-version:
bus-info: platform
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

Settings for eth1:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Link partner advertised pause frame use: Symmetric Receive-only
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 11
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: d
Wake-on: d
Link detected: yes')

and here the those by mii-tool:

$this->bbcode_second_pass_code('', 'Using SIOCGMIIPHY=0x8947
eth1: negotiated 100baseTx-FD, link ok
registers for MII PHY 11:
1000 796d 0141 0e40 01e1 cde1 000d 0000
0000 0200 4000 0000 0000 0000 0000 3000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0040 0000 0000 0000 0000 0000
product info: vendor 00:50:43, model 36 rev 0
basic mode: autonegotiation enabled
basic status: autonegotiation complete, link ok
capabilities: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control')

Thanks in advance!
kriztioan
 
Posts: 51
Joined: Sat Apr 29, 2017 1:32 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby kriztioan » Sat Sep 08, 2018 4:38 pm

After confirming that someone who also installed Arch Linux Arm on their ix2-200 is actually able to connect at 1000baseT-FD leads me to believe I have a hardware issue. Disassembling the device and inspecting the circuitry does show some corrosion around the diodes leading to each of the three USB ports. Nonetheless, the USB ports are working fine. Further investigation by directly connecting to the ix2-200 via TTL and having U-Boot pull up the link also results only in a 100 Mb/s connection, giving further corroboration of a hardware issue. NB U-Boot has a built-in NIC driver that it uses to handle TFTP.

I purchased a 1 Gb/s USB-Ethernet adapter that can be driven by the ax88179_178a driver that is part of the Linux kernel (loaded as a kernel module). Unfortunately the NAS only has USB 2, but a maximum throughput of 480 Mb/s would be a significant improvement on 100 Mb/s and more than the drives can handle anyway. The USB-Ethernet adapter worked out-of-the-box. However, it turns out that there is a significant overhead that maxes the CPU out at around 150 Mb/s. Compare this to the built-in Ethernet saturation the 100 Mb/s link with the CPU hardly showing any activity. Of course this is not entirely unexpected, but still somewhat disappointing; moving from 13 MB/s to 19 MB/s at such a CPU cost.

Currently I'm contemplating replacing my NAS. I would welcome and appreciate any insights/tips/tricks/suggestions etc. anyone might have.

Thanks in advance!
kriztioan
 
Posts: 51
Joined: Sat Apr 29, 2017 1:32 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby moonman » Mon Sep 10, 2018 5:34 am

What did you to measure speed? Samba? What is the filesystem you are accessing? These 2 things will already impose a big overhead for the CPU and 20MB/s sounds reasonable.

Test your speed with iperf and see if the CPU usage spikes. Somehow I doubt that the NIC was the problem here. Then you research how to optimize your device by using ext4 and samba config tweaks for example.
Pogoplug V4 | GoFlex Home | Raspberry Pi 4 4GB | CuBox-i4 Pro | ClearFog | BeagleBone Black | Odroid U2 | Odroid C1 | Odroid XU4
-----------------------------------------------------------------------------------------------------------------------
[armv5] Updated U-Boot | [armv5] NAND Rescue System
moonman
Developer
 
Posts: 3387
Joined: Sat Jan 15, 2011 3:36 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby kriztioan » Wed Sep 19, 2018 6:42 pm

Thanks for responding moonman, much appreciated!

I used NFS (v3) to do speed tests from the NAS (HDD; RAID1; ext4 filesystem; lazy_itable_init=0 and lazy_journal_init=0) to my MacBook (SSD; APFS) connected through the router. I measured the speeds on both ends of the connection. On the NAS I simply probed /proc/net/dev and on the Mac I wrote a tiny C-program that makes a sysctl-call to probe the RTM_IFINFO2-structure. Benchmarks posted in some old reviews of the NAS list read speeds (using the internal NIC) hitting around 40 MB/s for the ix2-200 (e.g., https://www.smallnetbuilder.com/nas/nas-reviews/30976-new-to-the-charts-iomega-ix2-200-storcenter-network-storage).

I wasn't aware of iperf and will give that a go. I agree the (USB 3) NIC itself is not to blame. However, I did see mentioned somewhere (...) that the polling-mechanism used by the USB 2 standard (vs. interrupt-based for USB 3) can have a significant impact on performance; hence my suspicion that the overhead is from the USB side of things, not the NIC side.

If iperf does show ~480 Mb/s I would consider optimizing my filesystem and NFS---any suggestions?

update:
Doing several iperf runs with the USB 1Gb connection I get the following results:

UDP
160-170 Mb/s
CPU load 100%

TCP
107-140 Mb/s
CPU load ~25-50%

Definitely not the hoped for ~480 Mb/s ...

Also, their is a lot of variation between the runs that is not seen when I use the internal NIC.

Thanks again!
kriztioan
 
Posts: 51
Joined: Sat Apr 29, 2017 1:32 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby moonman » Thu Sep 20, 2018 4:45 pm

Interesting. I'll test mine when I get home as mine has a different chip (M$ Surface adapter, IIRC it's realtek ).
Pogoplug V4 | GoFlex Home | Raspberry Pi 4 4GB | CuBox-i4 Pro | ClearFog | BeagleBone Black | Odroid U2 | Odroid C1 | Odroid XU4
-----------------------------------------------------------------------------------------------------------------------
[armv5] Updated U-Boot | [armv5] NAND Rescue System
moonman
Developer
 
Posts: 3387
Joined: Sat Jan 15, 2011 3:36 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby summers » Sat Sep 22, 2018 1:22 pm

Apols - hadn't seen that this thread was asking questions in my way.

Yes that your original ethernet port was switching between 1Gbps and 100Mbps, suggests that the kernel was working correctly, so it suggests a hardware fault. What I would guess is that 1Gbps connections need 4 twisted pairs, but iirc 100/10 Mbps only uses one - so probably a problem in one of the twisted paired, either in the cable or on the board.

Now with the usb-ethernet overloading the cpu. Only machine I have that I think has a usb to hardware ethernet convertor is my RPi, and that I guess is only 10/100 - been over a year since I plugged it in.

My beagle network, has usb otg on them, which I've brought up a virtual ethernet port. So although it connects via a usb cable, in the kernel it looks like ethernet. Am not sure what speed, I never worried about Gbps - as my internet is way below that. Anyway I don't have a load problem on the beagles.

Back a decade or so, USB used to load the kernel, for device detection (at least used to cause lost interrupt requests on high end audio cards) - but progress was made on the kernel - not that I checked this.

Anyway let me see if I can hammer my beagle network - and see what speed I can get ...

... hammered the beagle otg connection - its maxing out at 47Mbps so this won't help much ...

So alas no hardware here to look into the problem, sorry ...
Last edited by summers on Sun Sep 23, 2018 10:02 am, edited 1 time in total.
summers
 
Posts: 984
Joined: Sat Sep 06, 2014 12:56 pm

Re: ix2-200 no 1000baseT-FD [Updated]

Postby moonman » Sun Sep 23, 2018 12:14 am

Well, I checked out mine. It is on par with the spec which is 270 Mb/s.
Tried samba, and same thing.

Keep in mind you will not get 480Mb/s since USB2 is half duplex (vs USB3 full duplex). That's why USB 2.0 hard drives max out at around 30MB/s

So to summarize:

USB 3.0 MS Surface adapter on USB 2.0 port with samba to sata hdd with ext4:
Down: 27MB/s.
UP: 21:MB/s

Native Marvell interface with samba to sata hdd with ext4:
Down: 50MB/s
UP: 30MB/s

If you have USB hard drives, it will make it worse. This is exactly why RPi is horrible for this usecase.

This was tested after a fresh install with nothing else running on my GoFlex Home, I even disabled Journald.
Pogoplug V4 | GoFlex Home | Raspberry Pi 4 4GB | CuBox-i4 Pro | ClearFog | BeagleBone Black | Odroid U2 | Odroid C1 | Odroid XU4
-----------------------------------------------------------------------------------------------------------------------
[armv5] Updated U-Boot | [armv5] NAND Rescue System
moonman
Developer
 
Posts: 3387
Joined: Sat Jan 15, 2011 3:36 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby kriztioan » Sun Sep 23, 2018 7:24 pm

Thanks summer and moonman, I very much appreciate you looking into this!

My internet connection is around 60/6 Mb/s up/down, so no real need for 1 Gb/s there. My intent was to use the NAS as a backup-hub. That is, have my files (photo's, documents, etc.) (r)sync to the NAS and then have it, albeit it much slower, overnight sync to the 'Cloud'. Since my daily driver (MacBook Pro) would also backup there, having it complete the (incremental) backup as quickly as possible is highly desirable.

I didn't realize USB 2 is half-duplex, I presume that is on the USB-side of things as the NIC is reporting full-duplex.

Your USB NIC is definitely outperforming mine. I should note that I boot the system from a USB stick to avoid unnecessary disk spin up, so it might well be that the NIC and stick are competing for the USB's bandwidth. Also, the USB NIC is one of the cheaper ones by plugable on Amazon. Lastly, my disks are Baracuda Greens 5900 RPM drives.

For completeness, I attach two photos. One of the Ethernet port, the other of the main board. The latter circles the corrosion around the diodes(?) on the traces connecting the USB ports to the chip. The NIC is soldered to the board 5x2 pins above the circled area. Nothing obvious (beyond the corrosion around diodes) stands out to me.

Thanks again!

Ethernet Poort.JPG
NAS ethernet port
Ethernet Poort.JPG (224.01 KiB) Viewed 8527 times
kriztioan
 
Posts: 51
Joined: Sat Apr 29, 2017 1:32 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby kriztioan » Sun Sep 23, 2018 7:26 pm

Mainboard.JPG
NAS main board
Mainboard.JPG (196.99 KiB) Viewed 8526 times
kriztioan
 
Posts: 51
Joined: Sat Apr 29, 2017 1:32 am

Re: ix2-200 no 1000baseT-FD [Updated]

Postby summers » Sun Sep 23, 2018 7:43 pm

Thats actually exactly how I use my NAS, ie rsync back up every night. In my case across an ssh connection, and the NAS is running btrfs with compression enabled on IIRC 5400rpm disks. Cron job runs every night at midnight - and it backs up quickly.

OK- just ran it by hand. Across WiFi my NAS is loaded to about 30% cpu, peaking at 40%. Its the rsync that is using the cpu. Now in your case, you'll get more bandwidth than I get on wifi - so guess you could hit 100%. So another question is what is it that is using your cpu time?

So guess you must generate a huge amount of data!

I never checked what was limiting my speed - I suspect its ssh + the compression on the btrfs. The NAS only has an ARMv5 processor (kirkwood) - so suspect this stresses it ...

That corrosion looks odd, I'll ask some pcb people at work if they have any ideas what it is. e.g. doesn't look to me like a device has burnt out - but I can't see what would cause corrosion like that ...
summers
 
Posts: 984
Joined: Sat Sep 06, 2014 12:56 pm


Return to Marvell Kirkwood

Who is online

Users browsing this forum: No registered users and 8 guests