Hardware accelerated encryption not working on Dockstar

[beecher]

I have a problem with cryptodev on my Dockstar. It looks like it is not being used even though everything is installed to take advantage of it.

$this->bbcode_second_pass_code('', '[bofphile@alarm ~]$ pacman -Q openssl-cryptodev
openssl-cryptodev 1.0.2.h-1')

$this->bbcode_second_pass_code('', '[bofphile@alarm ~]$ openssl engine
(cryptodev) cryptodev engine
(dynamic) Dynamic engine loading support')

$this->bbcode_second_pass_code('', '[bofphile@alarm ~]$ lsmod
Module Size Used by
cfg80211 454495 0
rfkill 16116 2 cfg80211
blowfish_generic 3649 0
blowfish_common 6549 1 blowfish_generic
ses 6642 0
enclosure 6577 1 ses
marvell_cesa 26781 0
cryptodev 33639 5
ip_tables 10778 0
x_tables 13461 1 ip_tables
ipv6 347478 24')

$this->bbcode_second_pass_code('', '[bofphile@alarm ~]$ dmesg | grep crypto
[ 10.127590] cryptodev: loading out-of-tree module taints kernel.
[ 10.164218] cryptodev: driver 1.8 loaded.
[ 12.359250] marvell-cesa f1030000.crypto: CESA device successfully registered')

I'm also using the latest kernel:
$this->bbcode_second_pass_code('', '[bofphile@alarm ~]$ uname -a
Linux alarm 4.10.4-1-ARCH #1 PREEMPT Tue Mar 21 19:16:28 MDT 2017 armv5tel GNU/Linux')

It doesn't look like the hardware crypto engine is being used because I have almost the same result with or without the crypto engine in openssl benchs:
Without crypto engine:
$this->bbcode_second_pass_code('', 'openssl speed -evp aes-128-cbc
Doing aes-128-cbc for 3s on 16 size blocks: 2503986 aes-128-cbc's in 2.94s
Doing aes-128-cbc for 3s on 64 size blocks: 761407 aes-128-cbc's in 2.96s
Doing aes-128-cbc for 3s on 256 size blocks: 201732 aes-128-cbc's in 2.94s
Doing aes-128-cbc for 3s on 1024 size blocks: 51169 aes-128-cbc's in 2.97s
Doing aes-128-cbc for 3s on 8192 size blocks: 6406 aes-128-cbc's in 2.96s
OpenSSL 1.0.2h 3 May 2016
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)
compiler: gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DHAVE_CRYPTODEV -DHASH_MAX_LEN=64 -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -march=armv5te -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -Wl,-O1,--sort-common,--as-needed,-z,relro -O3 -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 13627.13k 16462.85k 17565.78k 17642.11k 17729.04k
')

With crypto engine:
$this->bbcode_second_pass_code('', 'openssl speed -evp aes-128-cbc -engine cryptodev
engine "cryptodev" set.
Doing aes-128-cbc for 3s on 16 size blocks: 2503000 aes-128-cbc's in 2.96s
Doing aes-128-cbc for 3s on 64 size blocks: 761204 aes-128-cbc's in 2.96s
Doing aes-128-cbc for 3s on 256 size blocks: 201597 aes-128-cbc's in 2.96s
Doing aes-128-cbc for 3s on 1024 size blocks: 51132 aes-128-cbc's in 2.96s
Doing aes-128-cbc for 3s on 8192 size blocks: 6403 aes-128-cbc's in 2.96s
OpenSSL 1.0.2h 3 May 2016
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)
compiler: gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DHAVE_CRYPTODEV -DHASH_MAX_LEN=64 -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -march=armv5te -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -Wl,-O1,--sort-common,--as-needed,-z,relro -O3 -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 13529.73k 16458.46k 17435.42k 17688.91k 17720.74k')

Furthermore, when doing the openssl benchmark with the crypto engine, the CPU is still being used at 100% and there are no increase in interrupts on f1030000.crypto :
$this->bbcode_second_pass_code('', 'cat /proc/interrupts
CPU0
17: 89213534 bridge-interrupt-ctrl 2 Edge orion_event
25: 622 main-interrupt-ctrl 33 Edge serial
27: 0 bridge-interrupt-ctrl 3 Edge f1020300.watchdog-timer
28: 0 main-interrupt-ctrl 22 Edge f1030000.crypto
29: 6833214 main-interrupt-ctrl 19 Edge ehci_hcd:usb1
30: 1385397 main-interrupt-ctrl 46 Edge f1072004.mdio-bus
32: 2 main-interrupt-ctrl 5 Edge f1060800.xor
33: 2 main-interrupt-ctrl 7 Edge f1060900.xor
34: 110816927 main-interrupt-ctrl 11 Edge eth0')

Any idea why the crypto engine isn't being used ?

Thanks.

[Sergeanter]

I also get identical benchmark results with and without cryptodev on Pogoplug V4. I guess it is broken again.

[j000]

Crypto api changes in kernel 4.8 broke it: https://mail.gna.org/public/cryptodev-l ... 00000.html
There seams to be some patches on mailing list (http://cryptodev-linux.org/lists.html) but I haven't tried them. Maybe someone with more experience will see this, because I have no idea.
I will check out https://github.com/cryptodev-linux/cryptodev-linux. It looks like there is one commit more there than in cryptodev-dkms package, but there were some changes in 4.10 too.

[j000]

It looks like it's working. I will open pull request on github.
$this->bbcode_second_pass_code('', '~$ fgrep crypto /proc/interrupts; openssl speed -evp aes-128-cbc; fgrep crypto /proc/interrupts; uname -a
28: 520256 main-interrupt-ctrl 22 Edge f1030000.crypto
Doing aes-128-cbc for 3s on 16 size blocks: 27480 aes-128-cbc's in 0.13s
Doing aes-128-cbc for 3s on 64 size blocks: 28800 aes-128-cbc's in 0.10s
Doing aes-128-cbc for 3s on 256 size blocks: 27078 aes-128-cbc's in 0.08s
Doing aes-128-cbc for 3s on 1024 size blocks: 22267 aes-128-cbc's in 0.07s
Doing aes-128-cbc for 3s on 8192 size blocks: 10965 aes-128-cbc's in 0.03s
OpenSSL 1.0.2h 3 May 2016
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)
compiler: gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DHAVE_CRYPTODEV -DHASH_MAX_LEN=64 -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -march=armv5te -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -Wl,-O1,--sort-common,--as-needed,-z,relro -O3 -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 3382.15k 18432.00k 86649.60k 325734.40k 2994176.00k
28: 636846 main-interrupt-ctrl 22 Edge f1030000.crypto
Linux redqueen 4.10.8-1-ARCH #1 PREEMPT Sun Apr 2 00:32:23 MDT 2017 armv5tel GNU/Linux')

[Sergeanter]

sshd cannot use cryptodev (again). It happened about half a year ago and got fixed by itself when kernel went to 4.8.1-1. Nobody was looking into this issue.
Then it was working for quite a while until recently. Now the same problem popped up again.Accelerated ciphers aes-256-cbc and aes-128-cbc are advertised by sshd but connection gets immediately dropped.
Symptoms are exactly the same as half a year ago.
https://archlinuxarm.org/forum/viewtopic.php?f=53&t=10704#p53144

[moonman]

Openssl refused (or did not notice) patches submitted by cryptodev developer for years. We were patching it manually until the patches would no longer apply on top of one of the later 1.0.1.X versions. There was some work done recently in this respect so it is possible that Openssl 1.1.0 will work without any patching. It is on my list

[Sergeanter]

It all seems to be working properly. Problems with sshd and cryptodev were related to sshd configuration.
Benchmarks also suggest cryptodev is doing what it is supposed to do.
Please see
https://archlinuxarm.org/forum/viewtopic.php?f=53&t=11505#p55595