Hardware Cryptography for Kirkwood

[moonman]

For anyone who wants to take advantage of dedicated cryptographic resources in Kirkwood platform (AES-128/256, SHA1).

EDIT (2012/02/11): thanks to Arch Linux ARM community developers all packages necessary to use hardware cryptography accelerators are available through pacman, updated instructions for systemd (thanks to pklaus):

Code: Select all

pacman -Syyu
pacman -S openssl-cryptodev
echo 'KERNEL=="crypto", MODE="0666"' > /etc/udev/rules.d/99-cryptodev.rules
# Load the kernel module manually after each new boot:
modprobe cryptodev
# or let systemd load it automatically on boot
echo "cryptodev" > /etc/modules-load.d/cryptodev.conf


I'm leaving old instructions just in case.
------------------------------- old instructions below -------------------------------
While searching the web for information on this topic I've found an article describing use of cryptodev-linux driver for openssl. Since I own a Pogoplug Pink (v2) based on the Kirkwood platform I've decided to give it a try.

1) Download current cryptodev-linux source from http://download.gna.org/cryptodev-linux/
2) compile and install and generate module.dep

Code: Select all

# make
# make install
# depmod -a


3) download PKGBUILD for openssl from ArchLinuxArm git repositories
4) edit PKGBUILD and add configuration options for cryptodev:

Code: Select all

./Configure --prefix=/usr --openssldir=/etc/ssl --libdir=lib \
      -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -DHASH_MAX_LEN=64 shared threads zlib enable-md2 \
      "${openssltarget}" \
      -Wa,--noexecstack "${CFLAGS}"


5) compile openssl into a package

Code: Select all

# makepkg


6) install package

Code: Select all

# pacman -U openssl-*.pkg.tar.gz


7) load cryptodev module (you can check that it's loaded with lsmod)

Code: Select all

# modprobe cryptodev


if everything worked and there were no errors you should have new openssl that is using cryptodev to offload cryptographic tasks from CPU to dedicated resources (and therefore every other program that depends on openssl like ssh, scp, transmission, etc.)

8) make cryptodev load at boot -> edit /etc/rc.conf

Code: Select all

MODULES=(cryptodev)


you can view my benchmarks in this topic: viewtopic.php?f=7&t=2451#p13657

edit:
9) because udev creates /dev/crypto with only root permitted to read/write to it, normal users can't use the device, this should be handled by apropriate udev rule (unfortunately I don't know how to create one), but in the meantime you can add this to /etc/rc.local

Code: Select all

chmod go+rw /dev/crypto


[kmihelich]

Nicely done figuring this out. Later today I'll roll cryptodev into a package so it's easily installed, and get an update to openssl pushed out including the changes you mentioned.

[WarheadsSE]

Will this be problematic for non-kirkwoods if we made this a v5 wide? Or would cryptodev be rolled, plus an openssl-kirkwood, since these hardware bits are platform dependant?

[firefoxPL]

One thing I've just discovered is that (at least on my pogo, basically freshly installed few days ago, and updated through pacman) /dev/crypto can be only used by root, so when I execute openssl speed ... as normal user, then cryptodev is not used at all, changing permission for /dev/crypto makes it work but only until boot, I think this could be changed with udev, but I don't know where to look, at least not yet

[firefoxPL]

Actually from what I understand cryptodev works with 2.6.36+ kernels so if there would be a stable update for ARM v6 family of Pogo devices, You could enable hardware cryptography for whole lineup.

[devr]

Just got done building, looking forward to testing. Word of advice, do not do makepkg with the cryptodev module loaded (yeah I followed your guide out of order). When I did, the openssl test failed when generating a cert.

Code: Select all

Generate and verify a certificate request
generating certificate request
rsa
There should be a 2 sequences of .'s and some +'s.
There should not be more that at most 80 per line
This could take some time.
Generating a 512 bit RSA private key
...++++++++++++
.........++++++++++++
writing new private key to 'testkey.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:AU
State or Province Name (full name) [Queensland]:
Locality Name (eg, city) []:Brisbane
Organization Name (eg, company) []:CryptSoft Pty Ltd
Organizational Unit Name (eg, section) []:.
Common Name (eg, YOUR name) []:Eric Young
Email Address []:eay@mincom.oz.au
1074709712:error:0606B06E:digital envelope routines:EVP_SignFinal:wrong public key type:p_sign.c:125:
1074709712:error:0D0C3006:asn1 encoding routines:ASN1_item_sign:EVP lib:a_sign.c:279:
problems creating request
make[1]: *** [test_gen] Error 1
make[1]: Leaving directory `/media/intData/work/cryptodev-openssl/src/openssl-1.0.0g/test'
make: *** [tests] Error 2
==> ERROR: A failure occurred in check().
    Aborting...

EDIT1: Adding more info

Also, cryptodev makes nginx rather unhappy. When the module is loaded, nginx is unable to fulfill https requests. In the nginx error.log you will just see tons of:

Code: Select all

2012/02/17 16:04:38 [notice] 27151#0: signal 17 (SIGCHLD) received
2012/02/17 16:04:38 [alert] 27151#0: worker process 27152 exited on signal 11
2012/02/17 16:04:38 [notice] 27151#0: start worker process 27155
2012/02/17 16:04:38 [notice] 27151#0: signal 17 (SIGCHLD) received
2012/02/17 16:04:38 [alert] 27151#0: worker process 27155 exited on signal 11


Since I normally have to compile nginx to get it to work on my device.. I recompiled just on the off chance... but did not help at all.

EDIT2: adding yet more info

Doesn't seem like it is nginx's fault, but that openssl is segfaulting

Code: Select all

# openssl s_client -showcerts -connect localhost:443
CONNECTED(00000004)
depth=0 C = US, ST = CT, O = None, CN = test
verify error:num=18:self signed certificate
verify return:1
depth=0 C = US, ST = CT, O = None, CN = test
verify return:1
Segmentation fault


[firefoxPL]

HI, I also get similar errors while trying to build OpenVPN CA, since I'm not author of neither cryptodev nor openssl I cannot offer any assistance, I can only propose to reboot with cryptodev disabled, generate the certificates and reboot with cryptodev enabled, then check if normal operation (like certificate based authentication) would work with cryptodev

[firefoxPL]

I've tried to debug this and there is definitely something wrong during EVP_SignFinal() call during certification signing, for now you could recompile openssl without -DUSE_CRYPTODEV_DIGESTS which will prevent it from using cryptodev md5/sha1 engine (still you will have hardware support for AES), I've tried that and certificates are created without errors,
I will try to get more into what happens in EVP_SignFinal, maybe there is a bug in openssl itself - I don't know

[devr]

Want to encourage you on your debug work I successfully compiled without the "-DUSE_CRYPTODEV_DIGESTS" flag. But afterwards I did a performance comparison and saw that with nginx/https/cryptodev there was a performance hit without that flag.

I spent some time today trying to eval this and see if there really was any noticeable performance benefit. Also, somehow nginx started working when serving https, not sure why.

The way I am testing performance is that from my laptop I run a bash loop making curl calls that hit my server. The curl command is as follows:

Code: Select all

curl -kso /dev/null -w "tcp:%{time_connect}, ssldone:%{time_appconnect}\n" https://11.22.33.44

Without cryptodev & distro supplied openssl: CPU: 45%-55% & ssldone: 0.200-0.400
With cryptodev & compiled openssl w/ DUSE_CRYPTODEV_DIGEST flag: CPU: 1%-4% & ssldone: 0.000
With cryptodev & compiled openssl w/out DUSE_CRYPTODEV_DIGEST flag: CPU: 43%-50% & ssldone 0.000

So if nginx is getting lots of https hits, then yeah cryptodev + flag really helps system performance. Problem is this is not stable (i.e. causes openssl to segfault), When the USE_CRYPTODEV_DIGESTS flag is removed, openssl does not segfault, but I still had issues when using pam_ldap & ldaps.

I did not dig into the cause, was more trying to collect some perf data to see if this was really worth the effort. with the #s I was seeing, it seems very beneficial to me.

[firefoxPL]

Looks like for your setup hardware hashing (md5/sha1) is more profitable and important than encryption/decryption, I'm trying to compile openssl with some debug and safe guarding options, so hopefully this will give me some insight on what's going on there - I don't think I'll be able to fix it though, more probable is submitting a bug to cryptodev or openssl and then wait on developers to check it out

edit: I just saw that yesterday there was an update to cryptodev to version 1.1, I'll try that as well

edit2: unfortunately updated cryptodev does nothing to improve on certificate related failures, and debuging takes long time, since strace don't show much, and recompiling of openssl takes some time on Pogo. I'll try also AF_ALG which is another way to use hardware crypto accelerators through openssl but without recompiling (it is supposed to be slower though)