[solved] distccd no longer distributes

Ask questions about Arch Linux ARM. Please search before making a new topic.

[solved] distccd no longer distributes

Postby keithspg » Fri Mar 26, 2021 3:54 am

Recently I did something and now I can no longer get my distccd setup working. It used to work and I must've messed it up, somehow, as it no longer distributes.
I get tons of this in the log on the x86 machine:
Code: Select all
distccd[151255] /usr/bin/gcc drivers/net/usb/kaweth.c on localhost failed
distccd[151255] job complete
distccd[151255] (dcc_cleanup_tempfiles_inner) deleted 5 temporary files
distccd[151255] (dcc_job_summary) client: 192.168.2.199:42572 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:2935ms /usr/bin/gcc drivers/net/usb/kaweth.c

and on the master I get this:
Code: Select all
distcc[151255] (dcc_build_somewhere) Warning: failed to distribute, running locally instead

I grab the pre-built tool chains from here:
https://archlinuxarm.org/wiki/Distcc_Cross-Compiling
and I unpack them to: /usr/local/x-tools-armv7h/xtools7h/

and use this service file:
Code: Select all
[Unit]
Description=A distributed C/C++ compiler for armv7h
Documentation=man:distccd(1)

[Service]
EnvironmentFile=/etc/conf.d/distccd
Environment="PATH=/usr/local/x-tools-armv7h/x-tools7h/arm-unknown-linux-gnueabihf/bin:/usr/bin"
ExecStart=/usr/bin/distccd $DISTCC_ARGS --log-file=/var/log/distcc-armv7h.log --port 3635

[Install]
WantedBy=multi-user.target

and my Environment file is this:
Code: Select all
DISTCC_ARGS="--user nobody --no-detach --allow 192.168.2.0/24 --log-level error --daemon --make-me-a-botnet --verbose"

Like I said this worked for years and all I had to do was update the prebuilt binaries periodically. Now I get 'failed to distribute' and no distribution. Any help?
This example is for the armv7, but I get the same results with the armv6 and armv8. All used to work.
The x86 machine is running:
Code: Select all
distcc 3.3.5-2
gcc 10.2.0-6
gcc-libs 10.2.0-6

The arm machine is running:
Code: Select all
distcc 3.3.5-2
gcc 10.2.0-1
gcc-libs 10.2.0-1
Last edited by keithspg on Sun Mar 28, 2021 2:55 pm, edited 2 times in total.
keithspg
 
Posts: 141
Joined: Mon Feb 23, 2015 4:14 pm

Re: recent distccd failures

Postby keithspg » Fri Mar 26, 2021 5:30 am

I subsequently removed my 'custom distccd-armv7h install' and built and installed the AUR packages:
https://aur.archlinux.org/pkgbase/distccd-alarm/
and get the same result on multiple machines as with my 'custom install'. 'Failed to distribute' on the master (arm) and 'COMPILE_ERROR' on both x86 machines.
This is a snippet of the log on the x86 when I was trying to compile upmpdcli:
Code: Select all
distccd[160845] (dcc_r_file_timed) 1586273 bytes received in 0.013808s, rate 112188kB/s
distccd[160845] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[160845] /usr/bin/g++ src/ohinfo.cxx on localhost failed
distccd[160845] job complete
distccd[160845] (dcc_job_summary) client: 192.168.2.199:42938 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:977ms /usr/bin/g++ src/ohinfo.cxx


This is a kernel compile snippet (from the arm machine):
Code: Select all
distcc[23789] (dcc_build_somewhere) Warning: failed to distribute, running locally instead
distcc[23661] (dcc_please_send_email_after_investigation) Warning: remote compilation of 'fs/nfsd/nfsfh.c' failed, retried locally and got a different result.
  CC [M]  fs/nfsd/vfs.o
distcc[23669] (dcc_please_send_email_after_investigation) Warning: remote compilation of 'fs/ecryptfs/keystore.c' failed, retried locally and got a different result.
keithspg
 
Posts: 141
Joined: Mon Feb 23, 2015 4:14 pm

Re: distccd no longer distributes

Postby graysky » Fri Mar 26, 2021 7:25 am

What are you trying to build? If it's the linux kernel, did you see this: https://wiki.archlinux.org/index.php/Di ... el_package
graysky
 
Posts: 923
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: distccd no longer distributes

Postby keithspg » Fri Mar 26, 2021 11:44 am

@graysky I am trying to build the kernel in the one snippet, but I get it when compiling any package. I get the same result if I compile upmpdcli (above)
keithspg
 
Posts: 141
Joined: Mon Feb 23, 2015 4:14 pm

Re: distccd no longer distributes

Postby keithspg » Fri Mar 26, 2021 1:30 pm

I made that edit to the kernel makefile and get the same result.
Volunteer (x86 tower):
Code: Select all
distccd[182624] (dcc_check_client) connection from 192.168.2.199:44060
distccd[182624] compile from inet_timewait_sock.c to inet_timewait_sock.o
distccd[182624] (dcc_r_file_timed) 3711399 bytes received in 0.032215s, rate 112507kB/s
distccd[182624] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distccd[182624] /usr/bin/gcc net/ipv4/inet_timewait_sock.c on localhost failed
distccd[182624] job complete
distccd[182624] (dcc_job_summary) client: 192.168.2.199:44060 COMPILE_ERROR exit:1 sig:0 core:0 ret:0 time:2315ms /usr/bin/gcc net/ipv4/inet_timewait_sock.c

Master (Arm):
Code: Select all
distcc[45576] (dcc_build_somewhere) Warning: failed to distribute, running locally instead
keithspg
 
Posts: 141
Joined: Mon Feb 23, 2015 4:14 pm

Re: distccd no longer distributes

Postby keithspg » Sun Mar 28, 2021 1:16 pm

It is not yet 'solved' but I did figure something out. My BUILDENV was:
Code: Select all
BUILDENV=(distcc color ccache check !sign)

And I changed it to:
Code: Select all
BUILDENV=(distcc color !ccache check !sign)

Now it compiles. I enabled ccache because I am compiling some Rust packages and they do not use distccd, but can use ccache. These programs take a long time to build. Is ccache not possible across architectures?
keithspg
 
Posts: 141
Joined: Mon Feb 23, 2015 4:14 pm

Re: distccd no longer distributes

Postby graysky » Sun Mar 28, 2021 2:22 pm

I haven't found any real speed gains with ccache. Not sure about cross compiling. I think it's a got a niche use if you're building the same thing over and over, perhaps if bisecting a large project. I'd say just use distcc without it.
graysky
 
Posts: 923
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: [solved] distccd no longer distributes

Postby keithspg » Sun Mar 28, 2021 2:57 pm

Marked solved.
Based on my experience, here, for Arch Arm as master and x86 as slave, do not enable ccache in the BUILDENV or it will not distribute.
keithspg
 
Posts: 141
Joined: Mon Feb 23, 2015 4:14 pm


Return to User Questions

Who is online

Users browsing this forum: No registered users and 1 guest