CrashPlan with JDK 8

[dhead666]

As part of my (slow) move to Linux I installed today CrashPlan on my RPi so my laptop could be free of it.
After a bit of googling I decided to go with Oracle’s Java 8 preview which supposed to be much faster than OpenJDK.
The following mainly based on Bion & Jon Rogers great posts.

I'm not a developer nor system admin so any suggestion is welcome.
I only tested it briefly and backed up about 100mb, but it seems very fast and stable.

Everything was compiled on the RPi against Oracle’s Java 8.


Install some dependencies

Code: Select all

# pacman -S cpio nss nspr ca-certificates-java libffi

Create a temporary working folder

Code: Select all

# mkdir /tmp/cpins
# cd /tmp/cpins

Install Java
Get & install java, use the latest from http://jdk8.java.net/fxarmpreview/

Code: Select all

# wget http://www.java.net/download/JavaFXarm/jdk-8-ea-b36e-linux-arm-hflt-29_nov_2012.tar.gz
# tar -xvf jdk-8-*.tar.gz
# mv jdk1.8.0/ /opt/jdk1.8.0/

Temporarily add the Java binaries location for path.

Code: Select all

# export PATH=$PATH:/opt/jdk1.8.0/bin

The above won't be kept during reboots, but this doesn't matter for CrashPlan as the installation script store this path in Crashplan's configs.
If you want to permanently store Java location in your path read about Environment Variables on the ArchWiki.

A fix for the missing libffi.so.5

Option 1: Just link

Code: Select all

# ln -s /usr/lib/libffi.so.6.0.0 /usr/lib/libffi.so.5

Option 2: Install precompiled libffi5 package

Code: Select all

# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libffi5-3.0.10-1-armv6h.pkg.tar.xz
# pacman -U libffi5-*-armv6h.pkg.tar.xz

Option 3: Build & install libffi5 package

Code: Select all

# wget https://aur.archlinux.org/packages/li/libffi5/libffi5.tar.gz
# tar -xzvf libffi5.tar.gz
# cd libffi5
# makepkg -A
# pacman -U libffi5-*-armv6h.pkg.tar.xz
# cd ..

[dhead666]

libmd5

Option 1: Download precompiled binary (by me)

Code: Select all

# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libmd5.so

Option 2: Compile the binary

Code: Select all

# wget http://www.twmacinta.com/myjava/fast-md5-2.7.1.zip
# unzip fast-md5-2.7.1.zip
# cd fast-md5
# gcc -O3 -shared -fPIC -I. -I/opt/jdk1.8.0/include -I/opt/jdk1.8.0/include/linux \
  src/lib/arch/linux_x86/MD5.c -o libmd5.so
# cp libmd5.so ../
# cd ..


jtux

Option 1: Download precompiled binary (by me)

Code: Select all

# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libjtux.so

Option 2: Compile the binary

Code: Select all

# wget http://forum.excito.net/download/file.php?id=145 -O jtux.PS3-YDL6.1.patch.txt.gz
# gunzip jtux.PS3-YDL6.1.patch.txt.gz
# wget http://basepath.com/aup/jtux/jtux.tar.gz
# tar xzvf jtux.tar.gz
# cd jtux
# patch < ../jtux.PS3-YDL6.1.patch.txt

Edit the first line in the make file so it would point to the right place.

Code: Select all

# nano Makefile

JAVA_INCLUDE = /opt/jdk1.8.0/include

Edit jtux_util.c to include a missing header.

Code: Select all

# nano jtux_util.c

#include <sys/resource.h>

Code: Select all

# make
# cp libjtux.so ../
# cd ..

[dhead666]

libjna

Option 1: Download precompiled binary (by me)

Code: Select all

# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/jna.jar
# mkdir -p /usr/share/java
# cp jna.jar /usr/share/java/
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libjnidispatch.so
# mkdir /usr/lib/jni
# cp libjnidispatch.so /usr/lib/jni/

Option 2: Compile binary
Install ant which is necessary for compiling. ant got a large number of dependencies,
I highly recommend copying the list of the dependencies for later removal.

Code: Select all

# pacman -S apache-ant
# wget http://ftp.de.debian.org/debian/pool/main/libj/libjna-java/libjna-java_3.2.7.orig.tar.gz
# tar -xzvf libjna-java_3.2.7.orig.tar.gz
# cd libjna-java-3.2.7
# cd native
# wget https://github.com/atgreen/libffi/archive/master.zip
# unzip master.zip
# mv libffi-master libffi

Edit the makefile

Code: Select all

# nano Makefile

JAVA_INCLUDES=-I"/opt/jdk1.8.0/include" \
-I"/opt/jdk1.8.0/include/linux"

Code: Select all

# cd ..

Build

Code: Select all

# ant

The build will fail with the following error

../libjna-java-3.2.7/build/test-classes

BUILD FAILED

no worries our binaries already compiled

Code: Select all

# mkdir -p /usr/share/java
# cp build/jna.jar /usr/share/java/
# mkdir /usr/lib/jni
# cp build/native/libjnidispatch.so /usr/lib/jni/
# cd ..

Remove the unnecessary packages

Code: Select all

# pacman -R freetype2  hicolor-icon-theme  inputproto kbproto  libice libsm \
  libx11 libxau libxcb libxdmcp libxext libxi libxmu libxrender libxt libxtst openjdk6 \
  recordproto renderproto xcb-proto xdg-utils xextproto xorg-xset xproto  apache-ant

[dhead666]

Install CrashPlan

Download & Install CrashPlan, get the latest version from CrashPlan Linux client download page.

Code: Select all

# wget http://download.crashplan.com/installs/linux/install/CrashPlan/CrashPlan_3.5.2_Linux.tgz
# tar -xvf CrashPlan_3.*_Linux.tgz
# cd CrashPlan-install/

Add java 1.8 support

Code: Select all

# nano install.sh

OKJAVA="1.5 1.6 1.7 1.8"

Run the installation script

Code: Select all

# ./install.sh

Here the output of my installation

CrashPlan will install to: /opt/crashplan
And put links to binaries in: /usr/local/bin
And store datas in: /opt/crashplan/manifest
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc.d

We won't need the scripts on rc.d and init.d, better keep them in these folders so the uninstallation script would remove them easily.

Remove unwanted binaries, the libjniwrap.so & libjniwrap64.so are closed code and proper substitute is unavailable

Code: Select all

# ls /opt/crashplan/*.so
# rm /opt/crashplan/libmd*.so
# rm /opt/crashplan/libjt*.so
# rm /opt/crashplan/libjni*.so

Copy our new arm binaries

Code: Select all

# cd ..
# cp libmd5.so /opt/crashplan/
# cp libjtux.so /opt/crashplan/

[dhead666]

More configs

Allow remote desktop (thanks to Bryan Ross) Edit: You may need to run CrashPlan once if the file doesn't exist

Code: Select all

# nano /opt/crashplan/conf/my.service.xml

<serviceHost>0.0.0.0</serviceHost>

Point to our libjna

Code: Select all

# nano /opt/crashplan/bin/CrashPlanEngine

Look for the line starting at FULL_CP=" by pressing CTRL+W and add at the start of the string

/usr/share/java/jna.jar:

this is the line after the change

FULL_CP="/usr/share/java/jna.jar:$TARGETDIR/lib/com.backup42.desktop.jar:$TARGETDIR/lang"

Systemd
The systemd service is basically the same as the one on the aur, I only removed the java check.

Code: Select all

# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/service/crashplan
# cp crashplan /etc/rc.d/

Code: Select all

# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/service/crashplan.service
# cp crashplan.service /usr/lib/systemd/system/

Testing
Start the service

Code: Select all

# systemctl start crashplan

It will take a minute or two for CrashPlan to initialize, you can monitor it with # top (and press 1).
After CrashPlan initialized check if it still running with

Code: Select all

# systemctl status crashplan

Look for CrashPlan engine errors

Code: Select all

# /opt/crashplan/log/engine_error.log

Enable the service and remove the working folder

Code: Select all

# systemctl enable crashplan
# cd ..
# rm -r cpins

[dhead666]

Remote Desktop

Connect to a Headless CrashPlan Desktop

On Windows:
Install windows client, close CrashPlan, edit ui.properties at C:\Program Files\CrashPlan
find

#serviceHost=

and change to

serviceHost=server ip

Optional Configs

Enlarging inotify watch limit

For real-time backup CrashPlan recommends enlarging the inotify watch limit.
I'm not sure how much this affect performance, it may be wiser to set it after your initial syncing is completed.
You will probably want to set it if you're getting the "No space left on device" error.
To temporarily change watch limit and experiment with different values execute the following command, the general consensus is to use a product of 1024, like 16384, 32768 and etc..

Code: Select all

# echo 1048576 > /proc/sys/fs/inotify/max_user_watches

To permanently store the new watch limit value add the following line in the end of sysctl.conf.

Code: Select all

# nano /etc/sysctl.conf

fs.inotify.max_user_watches=1048576

Setting ram limit on the Java VM

The memory limit of the Java VM created by CrashPlan defaults to 512MB,
if the VM will reach this limit with no swap it will crash.
Open run.conf

Code: Select all

# nano /opt/crashplan/bin/run.conf

In the line started with "SRV_JAVA_OPTS" find "-Xmx512m" and change it to a lower value than the physical ram available on your board, 412MB worked well for me.
You can check the current available memory with

Code: Select all

# free -m

Be advised, on demanding tasks like backing up very large files and minimizing the backup archive (I made that mistake), the memory usage of CrashPlan could be high (my highest value was about 800MB) and setting a low memory limit for the VM would also end with a crash, so for the initial syncing if CrashPlan is crashing you may want to add a swap partition and to set 1024MB limit for the Java VM.
A swap partition on a SD card is o.k. for temporarily use on initial syncing, but not advised as permanent solution, it will surelly quickly die.

Resources

CrashPlan Documentation: Connect to a Headless CrashPlan Desktop
Some errors and solutions on Crashplan on an IOMEGA Storcenter Guide

Installing CrashPlan on a PogoPlug Pro
Installing CrashPlan on a BeagleBone

[dhead666]

I've found the error with libjtux, seems that it was missing sys/resource.h.
Compiled against Java 8 and tested it, it seems to be running with no issues.
I updated the the guide and added a link to my precompiled binary.


This guide can be used to install CrashPlan on different Arm devices than the Raspberry Pi.
Anyone who wants to do so will probably need to use openjdk6,
and of course to manually compile the different libs,
don't forget to point the location of openjdk6 @ /usr/lib/jvm/java-6-openjdk/ when editing the configs and Makefiles.

p.s.
When attempting to run CrashPlan on Arm device memory is the key factor,
128MB will not suffice, 256MB is the bare minimum and won't be enough for 100GB+ backup, 512MB probably will do.

[jeanlouiejam]

Hi dhead666, thanks for the updated tutorial. my crashplan was running very slowly so i decided to take the plunge and try your settings.

i followed everything twice, including the java portion but i still have trouble running the install script. even though i edit the file to include 1.8, added 1.8 to the path, and followed very closely your instructions on download/installing the 1.8 jdk, it still does not want to install.

This is one I get

Code: Select all

Welcome to the CrashPlan Installer.

Press enter to continue with installation.

Validating environment...
  detected root permissions
47745 blocks
./install.sh: line 154: /opt/jdk1.8.0/bin/java: No such file or directory
The current version of Java () is incompatible with CrashPlan.
Please install one of the following version of the Sun JRE or OpenJDK: 1.5 1.6 1.7 1.8

i assure you that when i browse over to /opt/jdk1.8.0/bin/java the file is there. i just cannot figure out why it will not continue.

thoughts/ideas?

[dhead666]

You're not in the CrashPlan installation folder
do:

Code: Select all

cd CrashPlan-install/

[dhead666]

I've change a dropbox link from https to http to remove wget's no certificate error.
Also I added a remark about the export command on the Java installation section.

@jeanlouiejam,
I've followed my guide today with the latest CrashPlan installation script and had no problems at all.

p.s.
I'm officially out off the testing phase, currently synching my 200GB+ backup back to CrashPlan servers.
The data stored on an usb 3.0 drive connected directly to a PogoPlug 4 shared via nfs to the Raspberry Pi.