CrashPlan with JDK 8

This forum is for topics specific to the Raspberry Pi and Arch Linux ARM

CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:41 pm

As part of my (slow) move to Linux I installed today CrashPlan on my RPi so my laptop could be free of it.
After a bit of googling I decided to go with Oracle’s Java 8 preview which supposed to be much faster than OpenJDK.
The following mainly based on Bion & Jon Rogers great posts.

I'm not a developer nor system admin so any suggestion is welcome.
I only tested it briefly and backed up about 100mb, but it seems very fast and stable.

Everything was compiled on the RPi against Oracle’s Java 8.


Install some dependencies
Code: Select all
# pacman -S cpio nss nspr ca-certificates-java libffi


Create a temporary working folder
Code: Select all
# mkdir /tmp/cpins
# cd /tmp/cpins



Install Java
Get & install java, use the latest from http://jdk8.java.net/fxarmpreview/
Code: Select all
# wget http://www.java.net/download/JavaFXarm/jdk-8-ea-b36e-linux-arm-hflt-29_nov_2012.tar.gz
# tar -xvf jdk-8-*.tar.gz
# mv jdk1.8.0/ /opt/jdk1.8.0/


Temporarily add the Java binaries location for path.
Code: Select all
# export PATH=$PATH:/opt/jdk1.8.0/bin

The above won't be kept during reboots, but this doesn't matter for CrashPlan as the installation script store this path in Crashplan's configs.
If you want to permanently store Java location in your path read about Environment Variables on the ArchWiki.

A fix for the missing libffi.so.5

Option 1: Just link
Code: Select all
# ln -s /usr/lib/libffi.so.6.0.0 /usr/lib/libffi.so.5


Option 2: Install precompiled libffi5 package
Code: Select all
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libffi5-3.0.10-1-armv6h.pkg.tar.xz
# pacman -U libffi5-*-armv6h.pkg.tar.xz


Option 3: Build & install libffi5 package
Code: Select all
# wget https://aur.archlinux.org/packages/li/libffi5/libffi5.tar.gz
# tar -xzvf libffi5.tar.gz
# cd libffi5
# makepkg -A
# pacman -U libffi5-*-armv6h.pkg.tar.xz
# cd ..
Last edited by dhead666 on Tue May 14, 2013 6:34 pm, edited 9 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:41 pm

libmd5

Option 1: Download precompiled binary (by me)
Code: Select all
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libmd5.so


Option 2: Compile the binary
Code: Select all
# wget http://www.twmacinta.com/myjava/fast-md5-2.7.1.zip
# unzip fast-md5-2.7.1.zip
# cd fast-md5
# gcc -O3 -shared -fPIC -I. -I/opt/jdk1.8.0/include -I/opt/jdk1.8.0/include/linux \
  src/lib/arch/linux_x86/MD5.c -o libmd5.so
# cp libmd5.so ../
# cd ..


jtux

Option 1: Download precompiled binary (by me)
Code: Select all
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libjtux.so


Option 2: Compile the binary
Code: Select all
# wget http://forum.excito.net/download/file.php?id=145 -O jtux.PS3-YDL6.1.patch.txt.gz
# gunzip jtux.PS3-YDL6.1.patch.txt.gz
# wget http://basepath.com/aup/jtux/jtux.tar.gz
# tar xzvf jtux.tar.gz
# cd jtux
# patch < ../jtux.PS3-YDL6.1.patch.txt

Edit the first line in the make file so it would point to the right place.
Code: Select all
# nano Makefile

JAVA_INCLUDE = /opt/jdk1.8.0/include

Edit jtux_util.c to include a missing header.
Code: Select all
# nano jtux_util.c

#include <sys/resource.h>

Code: Select all
# make
# cp libjtux.so ../
# cd ..
Last edited by dhead666 on Tue May 14, 2013 6:36 pm, edited 7 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:42 pm

libjna

Option 1: Download precompiled binary (by me)
Code: Select all
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/jna.jar
# mkdir -p /usr/share/java
# cp jna.jar /usr/share/java/
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libjnidispatch.so
# mkdir /usr/lib/jni
# cp libjnidispatch.so /usr/lib/jni/


Option 2: Compile binary
Install ant which is necessary for compiling. ant got a large number of dependencies,
I highly recommend copying the list of the dependencies for later removal.
Code: Select all
# pacman -S apache-ant
# wget http://ftp.de.debian.org/debian/pool/main/libj/libjna-java/libjna-java_3.2.7.orig.tar.gz
# tar -xzvf libjna-java_3.2.7.orig.tar.gz
# cd libjna-java-3.2.7
# cd native
# wget https://github.com/atgreen/libffi/archive/master.zip
# unzip master.zip
# mv libffi-master libffi


Edit the makefile
Code: Select all
# nano Makefile

JAVA_INCLUDES=-I"/opt/jdk1.8.0/include" \
-I"/opt/jdk1.8.0/include/linux"

Code: Select all
# cd ..


Build
Code: Select all
# ant

The build will fail with the following error
../libjna-java-3.2.7/build/test-classes

BUILD FAILED

no worries our binaries already compiled
Code: Select all
# mkdir -p /usr/share/java
# cp build/jna.jar /usr/share/java/
# mkdir /usr/lib/jni
# cp build/native/libjnidispatch.so /usr/lib/jni/
# cd ..


Remove the unnecessary packages
Code: Select all
# pacman -R freetype2  hicolor-icon-theme  inputproto kbproto  libice libsm \
  libx11 libxau libxcb libxdmcp libxext libxi libxmu libxrender libxt libxtst openjdk6 \
  recordproto renderproto xcb-proto xdg-utils xextproto xorg-xset xproto  apache-ant
Last edited by dhead666 on Tue May 14, 2013 6:37 pm, edited 4 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:53 pm

Install CrashPlan

Download & Install CrashPlan, get the latest version from CrashPlan Linux client download page.
Code: Select all
# wget http://download.crashplan.com/installs/linux/install/CrashPlan/CrashPlan_3.5.2_Linux.tgz
# tar -xvf CrashPlan_3.*_Linux.tgz
# cd CrashPlan-install/


Add java 1.8 support
Code: Select all
# nano install.sh

OKJAVA="1.5 1.6 1.7 1.8"


Run the installation script
Code: Select all
# ./install.sh


Here the output of my installation
CrashPlan will install to: /opt/crashplan
And put links to binaries in: /usr/local/bin
And store datas in: /opt/crashplan/manifest
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc.d

We won't need the scripts on rc.d and init.d, better keep them in these folders so the uninstallation script would remove them easily.

Remove unwanted binaries, the libjniwrap.so & libjniwrap64.so are closed code and proper substitute is unavailable
Code: Select all
# ls /opt/crashplan/*.so
# rm /opt/crashplan/libmd*.so
# rm /opt/crashplan/libjt*.so
# rm /opt/crashplan/libjni*.so


Copy our new arm binaries
Code: Select all
# cd ..
# cp libmd5.so /opt/crashplan/
# cp libjtux.so /opt/crashplan/
Last edited by dhead666 on Tue Mar 05, 2013 1:55 pm, edited 7 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 2:10 pm

More configs

Allow remote desktop (thanks to Bryan Ross) Edit: You may need to run CrashPlan once if the file doesn't exist
Code: Select all
# nano /opt/crashplan/conf/my.service.xml

<serviceHost>0.0.0.0</serviceHost>


Point to our libjna
Code: Select all
# nano /opt/crashplan/bin/CrashPlanEngine

Look for the line starting at FULL_CP=" by pressing CTRL+W and add at the start of the string
/usr/share/java/jna.jar:

this is the line after the change
FULL_CP="/usr/share/java/jna.jar:$TARGETDIR/lib/com.backup42.desktop.jar:$TARGETDIR/lang"



Systemd
The systemd service is basically the same as the one on the aur, I only removed the java check.
Code: Select all
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/service/crashplan
# cp crashplan /etc/rc.d/


Code: Select all
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/service/crashplan.service
# cp crashplan.service /usr/lib/systemd/system/


Testing
Start the service
Code: Select all
# systemctl start crashplan

It will take a minute or two for CrashPlan to initialize, you can monitor it with # top (and press 1).
After CrashPlan initialized check if it still running with
Code: Select all
# systemctl status crashplan

Look for CrashPlan engine errors
Code: Select all
# /opt/crashplan/log/engine_error.log


Enable the service and remove the working folder
Code: Select all
# systemctl enable crashplan
# cd ..
# rm -r cpins
Last edited by dhead666 on Tue May 14, 2013 8:18 pm, edited 10 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 2:17 pm

Remote Desktop

Connect to a Headless CrashPlan Desktop

On Windows:
Install windows client, close CrashPlan, edit ui.properties at C:\Program Files\CrashPlan
find
#serviceHost=

and change to
serviceHost=server ip


Optional Configs

Enlarging inotify watch limit

For real-time backup CrashPlan recommends enlarging the inotify watch limit.
I'm not sure how much this affect performance, it may be wiser to set it after your initial syncing is completed.
You will probably want to set it if you're getting the "No space left on device" error.
To temporarily change watch limit and experiment with different values execute the following command, the general consensus is to use a product of 1024, like 16384, 32768 and etc..
Code: Select all
# echo 1048576 > /proc/sys/fs/inotify/max_user_watches

To permanently store the new watch limit value add the following line in the end of sysctl.conf.
Code: Select all
# nano /etc/sysctl.conf

fs.inotify.max_user_watches=1048576


Setting ram limit on the Java VM

The memory limit of the Java VM created by CrashPlan defaults to 512MB,
if the VM will reach this limit with no swap it will crash.
Open run.conf
Code: Select all
# nano /opt/crashplan/bin/run.conf

In the line started with "SRV_JAVA_OPTS" find "-Xmx512m" and change it to a lower value than the physical ram available on your board, 412MB worked well for me.
You can check the current available memory with
Code: Select all
# free -m

Be advised, on demanding tasks like backing up very large files and minimizing the backup archive (I made that mistake), the memory usage of CrashPlan could be high (my highest value was about 800MB) and setting a low memory limit for the VM would also end with a crash, so for the initial syncing if CrashPlan is crashing you may want to add a swap partition and to set 1024MB limit for the Java VM.
A swap partition on a SD card is o.k. for temporarily use on initial syncing, but not advised as permanent solution, it will surelly quickly die.

Resources

CrashPlan Documentation: Connect to a Headless CrashPlan Desktop
Some errors and solutions on Crashplan on an IOMEGA Storcenter Guide

Installing CrashPlan on a PogoPlug Pro
Installing CrashPlan on a BeagleBone
Last edited by dhead666 on Thu Apr 18, 2013 12:33 pm, edited 10 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Sat Mar 02, 2013 4:16 am

I've found the error with libjtux, seems that it was missing sys/resource.h.
Compiled against Java 8 and tested it, it seems to be running with no issues.
I updated the the guide and added a link to my precompiled binary.


This guide can be used to install CrashPlan on different Arm devices than the Raspberry Pi.
Anyone who wants to do so will probably need to use openjdk6,
and of course to manually compile the different libs,
don't forget to point the location of openjdk6 @ /usr/lib/jvm/java-6-openjdk/ when editing the configs and Makefiles.

p.s.
When attempting to run CrashPlan on Arm device memory is the key factor,
128MB will not suffice, 256MB is the bare minimum and won't be enough for 100GB+ backup, 512MB probably will do.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby jeanlouiejam » Sat Mar 02, 2013 8:42 am

Hi dhead666, thanks for the updated tutorial. my crashplan was running very slowly so i decided to take the plunge and try your settings.

i followed everything twice, including the java portion but i still have trouble running the install script. even though i edit the file to include 1.8, added 1.8 to the path, and followed very closely your instructions on download/installing the 1.8 jdk, it still does not want to install.

This is one I get

Code: Select all
Welcome to the CrashPlan Installer.

Press enter to continue with installation.

Validating environment...
  detected root permissions
47745 blocks
./install.sh: line 154: /opt/jdk1.8.0/bin/java: No such file or directory
The current version of Java () is incompatible with CrashPlan.
Please install one of the following version of the Sun JRE or OpenJDK: 1.5 1.6 1.7 1.8


i assure you that when i browse over to /opt/jdk1.8.0/bin/java the file is there. i just cannot figure out why it will not continue.

thoughts/ideas?
jeanlouiejam
 
Posts: 1
Joined: Sat Mar 02, 2013 8:35 am

Re: CrashPlan with JDK 8

Postby dhead666 » Sat Mar 02, 2013 1:31 pm

You're not in the CrashPlan installation folder
do:
Code: Select all
cd CrashPlan-install/
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Tue Mar 05, 2013 1:53 pm

I've change a dropbox link from https to http to remove wget's no certificate error.
Also I added a remark about the export command on the Java installation section.

@jeanlouiejam,
I've followed my guide today with the latest CrashPlan installation script and had no problems at all.

p.s.
I'm officially out off the testing phase, currently synching my 200GB+ backup back to CrashPlan servers.
The data stored on an usb 3.0 drive connected directly to a PogoPlug 4 shared via nfs to the Raspberry Pi.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Next

Return to Raspberry Pi

Who is online

Users browsing this forum: No registered users and 1 guest