CrashPlan with JDK 8

This forum is for topics specific to the Raspberry Pi and Arch Linux ARM

CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:41 pm

As part of my (slow) move to Linux I installed today CrashPlan on my RPi so my laptop could be free of it.
After a bit of googling I decided to go with Oracle’s Java 8 preview which supposed to be much faster than OpenJDK.
The following mainly based on Bion & Jon Rogers great posts.

I'm not a developer nor system admin so any suggestion is welcome.
I only tested it briefly and backed up about 100mb, but it seems very fast and stable.

Everything was compiled on the RPi against Oracle’s Java 8.


Install some dependencies
$this->bbcode_second_pass_code('', '# pacman -S cpio nss nspr ca-certificates-java libffi')

Create a temporary working folder
$this->bbcode_second_pass_code('', '# mkdir /tmp/cpins
# cd /tmp/cpins')


Install Java
Get & install java, use the latest from http://jdk8.java.net/fxarmpreview/
$this->bbcode_second_pass_code('', '# wget http://www.java.net/download/JavaFXarm/jdk-8-ea-b36e-linux-arm-hflt-29_nov_2012.tar.gz
# tar -xvf jdk-8-*.tar.gz
# mv jdk1.8.0/ /opt/jdk1.8.0/')

Temporarily add the Java binaries location for path.
$this->bbcode_second_pass_code('', '# export PATH=$PATH:/opt/jdk1.8.0/bin')
The above won't be kept during reboots, but this doesn't matter for CrashPlan as the installation script store this path in Crashplan's configs.
If you want to permanently store Java location in your path read about Environment Variables on the ArchWiki.

A fix for the missing libffi.so.5

Option 1: Just link
$this->bbcode_second_pass_code('', '# ln -s /usr/lib/libffi.so.6.0.0 /usr/lib/libffi.so.5')

Option 2: Install precompiled libffi5 package
$this->bbcode_second_pass_code('', '# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libffi5-3.0.10-1-armv6h.pkg.tar.xz
# pacman -U libffi5-*-armv6h.pkg.tar.xz')

Option 3: Build & install libffi5 package
$this->bbcode_second_pass_code('', '# wget https://aur.archlinux.org/packages/li/libffi5/libffi5.tar.gz
# tar -xzvf libffi5.tar.gz
# cd libffi5
# makepkg -A
# pacman -U libffi5-*-armv6h.pkg.tar.xz
# cd ..')
Last edited by dhead666 on Tue May 14, 2013 6:34 pm, edited 9 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:41 pm

libmd5

Option 1: Download precompiled binary (by me)
$this->bbcode_second_pass_code('', '# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libmd5.so')

Option 2: Compile the binary
$this->bbcode_second_pass_code('', '# wget http://www.twmacinta.com/myjava/fast-md5-2.7.1.zip
# unzip fast-md5-2.7.1.zip
# cd fast-md5
# gcc -O3 -shared -fPIC -I. -I/opt/jdk1.8.0/include -I/opt/jdk1.8.0/include/linux \
src/lib/arch/linux_x86/MD5.c -o libmd5.so
# cp libmd5.so ../
# cd ..
')

jtux

Option 1: Download precompiled binary (by me)
$this->bbcode_second_pass_code('', '# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libjtux.so')

Option 2: Compile the binary
$this->bbcode_second_pass_code('', '# wget http://forum.excito.net/download/file.php?id=145 -O jtux.PS3-YDL6.1.patch.txt.gz
# gunzip jtux.PS3-YDL6.1.patch.txt.gz
# wget http://basepath.com/aup/jtux/jtux.tar.gz
# tar xzvf jtux.tar.gz
# cd jtux
# patch < ../jtux.PS3-YDL6.1.patch.txt')
Edit the first line in the make file so it would point to the right place.
$this->bbcode_second_pass_code('', '# nano Makefile')
$this->bbcode_second_pass_quote('', 'J')AVA_INCLUDE = /opt/jdk1.8.0/include

Edit jtux_util.c to include a missing header.
$this->bbcode_second_pass_code('', '# nano jtux_util.c')
$this->bbcode_second_pass_quote('', '#')include <sys/resource.h>

$this->bbcode_second_pass_code('', '# make
# cp libjtux.so ../
# cd ..')
Last edited by dhead666 on Tue May 14, 2013 6:36 pm, edited 7 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:42 pm

libjna

Option 1: Download precompiled binary (by me)
$this->bbcode_second_pass_code('', '# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/jna.jar
# mkdir -p /usr/share/java
# cp jna.jar /usr/share/java/
# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/arch_compiled/libjnidispatch.so
# mkdir /usr/lib/jni
# cp libjnidispatch.so /usr/lib/jni/')

Option 2: Compile binary
Install ant which is necessary for compiling. ant got a large number of dependencies,
I highly recommend copying the list of the dependencies for later removal.
$this->bbcode_second_pass_code('', '# pacman -S apache-ant
# wget http://ftp.de.debian.org/debian/pool/main/libj/libjna-java/libjna-java_3.2.7.orig.tar.gz
# tar -xzvf libjna-java_3.2.7.orig.tar.gz
# cd libjna-java-3.2.7
# cd native
# wget https://github.com/atgreen/libffi/archive/master.zip
# unzip master.zip
# mv libffi-master libffi')

Edit the makefile
$this->bbcode_second_pass_code('', '# nano Makefile')
$this->bbcode_second_pass_quote('', 'J')AVA_INCLUDES=-I"/opt/jdk1.8.0/include" \
-I"/opt/jdk1.8.0/include/linux"

$this->bbcode_second_pass_code('', '# cd ..')

Build
$this->bbcode_second_pass_code('', '# ant')
The build will fail with the following error
$this->bbcode_second_pass_quote('', '.')./libjna-java-3.2.7/build/test-classes

BUILD FAILED

no worries our binaries already compiled
$this->bbcode_second_pass_code('', '# mkdir -p /usr/share/java
# cp build/jna.jar /usr/share/java/
# mkdir /usr/lib/jni
# cp build/native/libjnidispatch.so /usr/lib/jni/
# cd ..')

Remove the unnecessary packages
$this->bbcode_second_pass_code('', '# pacman -R freetype2 hicolor-icon-theme inputproto kbproto libice libsm \
libx11 libxau libxcb libxdmcp libxext libxi libxmu libxrender libxt libxtst openjdk6 \
recordproto renderproto xcb-proto xdg-utils xextproto xorg-xset xproto apache-ant')
Last edited by dhead666 on Tue May 14, 2013 6:37 pm, edited 4 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm
Top

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 1:53 pm

Install CrashPlan

Download & Install CrashPlan, get the latest version from CrashPlan Linux client download page.
$this->bbcode_second_pass_code('', '# wget http://download.crashplan.com/installs/linux/install/CrashPlan/CrashPlan_3.5.2_Linux.tgz
# tar -xvf CrashPlan_3.*_Linux.tgz
# cd CrashPlan-install/')

Add java 1.8 support
$this->bbcode_second_pass_code('', '# nano install.sh')
$this->bbcode_second_pass_quote('', 'O')KJAVA="1.5 1.6 1.7 1.8"


Run the installation script
$this->bbcode_second_pass_code('', '# ./install.sh')

Here the output of my installation
$this->bbcode_second_pass_quote('', 'C')rashPlan will install to: /opt/crashplan
And put links to binaries in: /usr/local/bin
And store datas in: /opt/crashplan/manifest
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc.d

We won't need the scripts on rc.d and init.d, better keep them in these folders so the uninstallation script would remove them easily.

Remove unwanted binaries, the libjniwrap.so & libjniwrap64.so are closed code and proper substitute is unavailable
$this->bbcode_second_pass_code('', '# ls /opt/crashplan/*.so
# rm /opt/crashplan/libmd*.so
# rm /opt/crashplan/libjt*.so
# rm /opt/crashplan/libjni*.so')

Copy our new arm binaries
$this->bbcode_second_pass_code('', '# cd ..
# cp libmd5.so /opt/crashplan/
# cp libjtux.so /opt/crashplan/')
Last edited by dhead666 on Tue Mar 05, 2013 1:55 pm, edited 7 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm
Top

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 2:10 pm

More configs

Allow remote desktop (thanks to Bryan Ross) Edit: You may need to run CrashPlan once if the file doesn't exist
$this->bbcode_second_pass_code('', '# nano /opt/crashplan/conf/my.service.xml')
$this->bbcode_second_pass_quote('', '&')lt;serviceHost>0.0.0.0</serviceHost>


Point to our libjna
$this->bbcode_second_pass_code('', '# nano /opt/crashplan/bin/CrashPlanEngine')
Look for the line starting at FULL_CP=" by pressing CTRL+W and add at the start of the string$this->bbcode_second_pass_quote('', '/')usr/share/java/jna.jar:

this is the line after the change
$this->bbcode_second_pass_quote('', 'F')ULL_CP="/usr/share/java/jna.jar:$TARGETDIR/lib/com.backup42.desktop.jar:$TARGETDIR/lang"



Systemd
The systemd service is basically the same as the one on the aur, I only removed the java check.
$this->bbcode_second_pass_code('', '# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/service/crashplan
# cp crashplan /etc/rc.d/')

$this->bbcode_second_pass_code('', '# wget http://dl.dropbox.com/u/6902100/archlinux/rpi-crashplan/service/crashplan.service
# cp crashplan.service /usr/lib/systemd/system/')

Testing
Start the service
$this->bbcode_second_pass_code('', '# systemctl start crashplan')
It will take a minute or two for CrashPlan to initialize, you can monitor it with # top (and press 1).
After CrashPlan initialized check if it still running with
$this->bbcode_second_pass_code('', '# systemctl status crashplan')
Look for CrashPlan engine errors
$this->bbcode_second_pass_code('', '# /opt/crashplan/log/engine_error.log')

Enable the service and remove the working folder
$this->bbcode_second_pass_code('', '# systemctl enable crashplan
# cd ..
# rm -r cpins')
Last edited by dhead666 on Tue May 14, 2013 8:18 pm, edited 10 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm
Top

Re: CrashPlan with JDK 8

Postby dhead666 » Mon Feb 25, 2013 2:17 pm

Remote Desktop

Connect to a Headless CrashPlan Desktop

On Windows:
Install windows client, close CrashPlan, edit ui.properties at C:\Program Files\CrashPlan
find
$this->bbcode_second_pass_quote('', '#')serviceHost=

and change to
$this->bbcode_second_pass_quote('', 's')erviceHost=server ip


Optional Configs

Enlarging inotify watch limit

For real-time backup CrashPlan recommends enlarging the inotify watch limit.
I'm not sure how much this affect performance, it may be wiser to set it after your initial syncing is completed.
You will probably want to set it if you're getting the "No space left on device" error.
To temporarily change watch limit and experiment with different values execute the following command, the general consensus is to use a product of 1024, like 16384, 32768 and etc..
$this->bbcode_second_pass_code('', '# echo 1048576 > /proc/sys/fs/inotify/max_user_watches')
To permanently store the new watch limit value add the following line in the end of sysctl.conf.
$this->bbcode_second_pass_code('', '# nano /etc/sysctl.conf')
$this->bbcode_second_pass_quote('', 'f')s.inotify.max_user_watches=1048576


Setting ram limit on the Java VM

The memory limit of the Java VM created by CrashPlan defaults to 512MB,
if the VM will reach this limit with no swap it will crash.
Open run.conf
$this->bbcode_second_pass_code('', '# nano /opt/crashplan/bin/run.conf')
In the line started with "SRV_JAVA_OPTS" find "-Xmx512m" and change it to a lower value than the physical ram available on your board, 412MB worked well for me.
You can check the current available memory with
$this->bbcode_second_pass_code('', '# free -m')
Be advised, on demanding tasks like backing up very large files and minimizing the backup archive (I made that mistake), the memory usage of CrashPlan could be high (my highest value was about 800MB) and setting a low memory limit for the VM would also end with a crash, so for the initial syncing if CrashPlan is crashing you may want to add a swap partition and to set 1024MB limit for the Java VM.
A swap partition on a SD card is o.k. for temporarily use on initial syncing, but not advised as permanent solution, it will surelly quickly die.

Resources

CrashPlan Documentation: Connect to a Headless CrashPlan Desktop
Some errors and solutions on Crashplan on an IOMEGA Storcenter Guide

Installing CrashPlan on a PogoPlug Pro
Installing CrashPlan on a BeagleBone
Last edited by dhead666 on Thu Apr 18, 2013 12:33 pm, edited 10 times in total.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm
Top

Re: CrashPlan with JDK 8

Postby dhead666 » Sat Mar 02, 2013 4:16 am

I've found the error with libjtux, seems that it was missing sys/resource.h.
Compiled against Java 8 and tested it, it seems to be running with no issues.
I updated the the guide and added a link to my precompiled binary.


This guide can be used to install CrashPlan on different Arm devices than the Raspberry Pi.
Anyone who wants to do so will probably need to use openjdk6,
and of course to manually compile the different libs,
don't forget to point the location of openjdk6 @ /usr/lib/jvm/java-6-openjdk/ when editing the configs and Makefiles.

p.s.
When attempting to run CrashPlan on Arm device memory is the key factor,
128MB will not suffice, 256MB is the bare minimum and won't be enough for 100GB+ backup, 512MB probably will do.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby jeanlouiejam » Sat Mar 02, 2013 8:42 am

Hi dhead666, thanks for the updated tutorial. my crashplan was running very slowly so i decided to take the plunge and try your settings.

i followed everything twice, including the java portion but i still have trouble running the install script. even though i edit the file to include 1.8, added 1.8 to the path, and followed very closely your instructions on download/installing the 1.8 jdk, it still does not want to install.

This is one I get

$this->bbcode_second_pass_code('', 'Welcome to the CrashPlan Installer.

Press enter to continue with installation.

Validating environment...
detected root permissions
47745 blocks
./install.sh: line 154: /opt/jdk1.8.0/bin/java: No such file or directory
The current version of Java () is incompatible with CrashPlan.
Please install one of the following version of the Sun JRE or OpenJDK: 1.5 1.6 1.7 1.8')

i assure you that when i browse over to /opt/jdk1.8.0/bin/java the file is there. i just cannot figure out why it will not continue.

thoughts/ideas?
jeanlouiejam
 
Posts: 1
Joined: Sat Mar 02, 2013 8:35 am

Re: CrashPlan with JDK 8

Postby dhead666 » Sat Mar 02, 2013 1:31 pm

You're not in the CrashPlan installation folder
do:
$this->bbcode_second_pass_code('', 'cd CrashPlan-install/')
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Re: CrashPlan with JDK 8

Postby dhead666 » Tue Mar 05, 2013 1:53 pm

I've change a dropbox link from https to http to remove wget's no certificate error.
Also I added a remark about the export command on the Java installation section.

@jeanlouiejam,
I've followed my guide today with the latest CrashPlan installation script and had no problems at all.

p.s.
I'm officially out off the testing phase, currently synching my 200GB+ backup back to CrashPlan servers.
The data stored on an usb 3.0 drive connected directly to a PogoPlug 4 shared via nfs to the Raspberry Pi.
Pogoplug Series 4 - Network Storage and Music Server: NFS/TVHeadend
Cubox-i2 - Applications Server: Lighttpd/CherryMusic/HTPCManager/Transmission/Couchpotato/SickBeard/OpenVPN
Samsung Chromebook: Chroach in ChromeOS
dhead666
 
Posts: 116
Joined: Sat Aug 11, 2012 10:25 pm

Next

Return to Raspberry Pi

Who is online

Users browsing this forum: No registered users and 7 guests