Distcc Cross-Compiling failure [SOLVED]

[graysky]

Problem: Master device (Rasbperrypi running up-to-date ArchARM) does not distribute compile tasks through distccd.

Example error (from master device):

$this->bbcode_second_pass_code('', '% makepkg -s
...
g++ -DHAVE_CONFIG_H -I. -I../../src/include -I./../include -I/usr/include/p11-kit-1 -I/usr/lib/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -fpch-preprocess -march=armv6 -mfloat-abi=hard -mfpu=vfp -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -Wall -g -fexceptions -std=gnu++11 -MT libengine_a-FileZillaEngine.o -MD -MP -MF .deps/libengine_a-FileZillaEngine.Tpo -c -o libengine_a-FileZillaEngine.o `test -f 'FileZillaEngine.cpp' || echo './'`FileZillaEngine.cpp
distcc[10363] ERROR: compile commands.cpp on 10.20.40.101 failed
distcc[10363] (dcc_build_somewhere) Warning: remote compilation of 'commands.cpp' failed, retrying locally
distcc[10363] Warning: failed to distribute commands.cpp to 10.20.40.101, running locally instead
...')

*Both client and master are running distccd
*My distccd.log on the master device remains empty.
*My distccd.log log on the client:
$this->bbcode_second_pass_code('', '
distccd[27179] (dcc_job_summary) client: 10.20.40.115:39202 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:288ms cc /tmp/cgeIGPBS/dummy.c
distccd[27183] (dcc_job_summary) client: 10.20.40.115:39205 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:332ms g++ commands.cpp
distccd[27182] (dcc_job_summary) client: 10.20.40.115:39203 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:500ms g++ backend.cpp
distccd[27185] (dcc_job_summary) client: 10.20.40.115:39206 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1216ms g++ engineprivate.cpp
distccd[27184] (dcc_job_summary) client: 10.20.40.115:39204 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1592ms g++ ControlSocket.cpp
distccd[27187] (dcc_job_summary) client: 10.20.40.115:39208 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:693ms g++ directorycache.cpp
distccd[27186] (dcc_job_summary) client: 10.20.40.115:39207 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1547ms g++ directorylistingparser.cpp
distccd[27188] (dcc_job_summary) client: 10.20.40.115:39209 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:906ms g++ directorylisting.cpp
distccd[27190] (dcc_job_summary) client: 10.20.40.115:39211 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:886ms g++ FileZillaEngine.cpp
distccd[27189] (dcc_job_summary) client: 10.20.40.115:39210 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:1168ms g++ externalipresolver.cpp')

Client:
*Arch x86_64 and is up-to-date (gcc 4.8.0-1 if that is what's mucking this up).
*Using the pre-built crosstool-ng toolchain (http://archlinuxarm.org/builder/xtools/x-tools6h.tar.xz)
$this->bbcode_second_pass_code('', '% tar Jxf x-tools6h.tar.xz -C /mnt/data
% cd /mnt/data/x-tools6h/arm-unknown-linux-gnueabi/bin
% ./link')

*Edited my distccd on the client accordingly; relevant lines:
$this->bbcode_second_pass_code('', 'PATH=/mnt/data/x-tools6h/arm-unknown-linux-gnueabi/bin:$PATH
DISTCC_ARGS="--allow 10.20.40.0/24 --log-file /tmp/distccd.log"')
*Distccd is running on the client with no errors per systemctl.

Master:
*Relevant lines from /etc/makepkg.conf
$this->bbcode_second_pass_code('', 'MAKEFLAGS="-j9"
BUILDENV=(fakeroot distcc color !ccache check !sign)
DISTCC_HOSTS="10.20.40.101"')

[WarheadsSE]

The pi will have issues with 9 threads, back it down to something sane, 3 is pushing it.
Otherwise nothing is jumping at me immediately. Try my distcc-alarm package @ github.com/WarheadsSE/PKGs

[graysky]

$this->bbcode_second_pass_quote('WarheadsSE', 'T')he pi will have issues with 9 threads, back it down to something sane, 3 is pushing it.
Otherwise nothing is jumping at me immediately. Try my distcc-alarm package @ github.com/WarheadsSE/PKGs

[WarheadsSE]

Seriously:
https://github.com/WarheadsSE/PKGs/tree ... tccd-alarm

Solves all your issues, and then all you have to do it point distcc hosts to IP:PORT(my package makes it 3633)
Very simple, sets it all up for you.

[graysky]

Thanks. I actually got it to work by a simple reboot?!

Here is a test compiling a preconfigured pacman (RPi modest overclock)[1].

$this->bbcode_second_pass_code('', '
No distccd: make -j2 677.82s user 19.27s system 98% cpu 11:46.99 total

With distccd: make -j2 417.44s user 16.84s system 96% cpu 7:29.50 total
With distccd: make -j3 422.21s user 16.18s system 97% cpu 7:30.75 total
With distccd: make -j8 413.49s user 15.73s system 97% cpu 7:20.96 total
')

Note that these are just n=1 runs. I didn't want to see about the robustness of -j3 vs -j8.

1. /boot/config.txt:
$this->bbcode_second_pass_code('', 'arm_freq=800
core_freq=300
sdram_freq=400
over_voltage=0')

[dhead666]

$this->bbcode_second_pass_quote('WarheadsSE', 'S')eriously:
https://github.com/WarheadsSE/PKGs/tree ... tccd-alarm

Solves all your issues, and then all you have to do it point distcc hosts to IP:PORT(my package makes it 3633)
Very simple, sets it all up for you.