Pi Zero kernel compilation issues

Ask questions about Arch Linux ARM. Please search before making a new topic.

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 8:29 am

aarch64 is only for Pi 3 or 4 I think. Zero is 32bit only. And I can't test for my 4 as I bricked it
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 10:54 am

Seems to be working. But doesn't seem it's using the full volunteer power. I mean it sends a job to the volunteer and it does it, but the issue is in how fast/many jobs it sends to the volunteer.
What would net in a faster compile time overall, 10 volunteers with 1 core or one volunteer with 10 cores? Assuming the same core power for each configuration.

edit: also in this case since the Zero is so weak, wouldn't it make sense to not compile at all and concentrate all available processing power to send jobs to different volunteers? Seems it's not prioritizing jobs for volunteers or something like that. Volunteer finishes fast and waits for next job while the pi is locked with current compiling file.
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

Re: Pi Zero kernel compilation issues

Postby graysky » Fri Nov 26, 2021 12:45 pm

$this->bbcode_second_pass_quote('quark', 'a')arch64 is only for Pi 3 or 4 I think. Zero is 32bit only. And I can't test for my 4 as I bricked it


I know, just sharing for general info.

$this->bbcode_second_pass_quote('quark', 'S')eems to be working. But doesn't seem it's using the full volunteer power. I mean it sends a job to the volunteer and it does it, but the issue is in how fast/many jobs it sends to the volunteer.
What would net in a faster compile time overall, 10 volunteers with 1 core or one volunteer with 10 cores? Assuming the same core power for each configuration.

edit: also in this case since the Zero is so weak, wouldn't it make sense to not compile at all and concentrate all available processing power to send jobs to different volunteers? Seems it's not prioritizing jobs for volunteers or something like that. Volunteer finishes fast and waits for next job while the pi is locked with current compiling file.


There is overhead associated with compiling, linking, sending/receiving jobs etc. on the client (the zero). You probably want just 1 volunteer. What does your DISTCC_HOSTS on the zero look like? What about MAKEFLAGS?
graysky
Developer
 
Posts: 1727
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 1:36 pm

In the meanwhile I completely dropped the distcc approach as it was too slow. At the moment I just chrooted into the pi's sd-card using this approach:
https://wiki.archlinux.org/title/QEMU#C ... rom_x86_64
seems to be working faster.
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

Re: Pi Zero kernel compilation issues

Postby graysky » Fri Nov 26, 2021 1:50 pm

$this->bbcode_second_pass_quote('quark', 'I')n the meanwhile I completely dropped the distcc approach as it was too slow.


You're not setup correctly. This is the gold standard method.

$this->bbcode_second_pass_quote('graysky', 'W')hat does your DISTCC_HOSTS on the zero look like? What about MAKEFLAGS?
My AUR packagesMy zsh configsStreamzap remote config
graysky
Developer
 
Posts: 1727
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000
Top

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 2:10 pm

I removed distcc from the pi but I think I had it like this:
DISTT_HOSTS="192.168.1.101/3" I think I also had the port :3634 after the "/3". I wanted to use 3 out of the 4 cores on my laptop.
It was working, I checked the logs on my laptop and was receiving jobs. Also if I turned the networking off for my laptop the pi process would complain that it cannot send jobs to the laptop and has to compile it locally, was saying so for each item, so I know it was working.
I was testing with an older i5 from my laptop, right now I moved on my main Ryzen and doing it on 10 cores. I expected it to be faster with chroot tbh, now that some time has passed.
Say I use the laptop and PC, all cores on all, would that be fastest? I expected it to be faster with chroot on my main PC but overall it's a simpler setup atm for me.
But I might give DISTCC another try with all at max load on all cores.
I tried to add localhost/0 in DISTT_HOSTS so the Pi doesn't do any local compilation but it didn't like it, spits out errors with bad divider or something.
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 2:21 pm

Yeah, looking at it now it's clearly faster on the ryzen on 10 cores than pi + 3 out of 4 cores from that i5, but clearly it's not very efficient in translation. It should have been way faster. When I cross-compiled on the ryzen for raspbian it was extremely fast.
Just that I got the feeling that with DISTCC it was moving pretty slow in sending jobs to the laptop, and watching the processes on the laptop it seemed that only 1 core is used at a time, dunno if because the pi wasn't sending them faster or there was some conflict somewhere where it would only send for one core.

edit: forgot to mention, I also had MAKEFLAGS="-j8", 3 from the laptop + 1 from pi doubled.
also don't know how much network speed matters. I have a wifi dongle on the Zero, and network speed seemed somewhere around 3-4MB/s up/down.
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

Re: Pi Zero kernel compilation issues

Postby graysky » Fri Nov 26, 2021 2:35 pm

Right the zero is shitty wifi.. forgot that. Not sure if that is also limiting you. Best was to answer is to

1) compile locally on the zero
2) compile with distcc using your laptop
3) compile with distcc using your other more powerful machine

DISTCC_HOSTS should be the n+1 and MAKEFLAGS don't really matter in my experience, n+1 or 2n are pretty close (n=all cores local and remote).

Anyway, not sure a realtime kernel will help much on a zero but I could be wrong. Have fun.
graysky
Developer
 
Posts: 1727
Joined: Sun Jun 26, 2011 6:56 am
Location: /run/user/1000

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 3:30 pm

I'm testing on the Zero as Raspberry 2 is busy atm can't test on it. I want to know if I move on Arch for all my Pi's, I'm using the Zero for testing atm. After I see what's what, how I compile the kernels for each etc then I'll make the move.
There's a serious issue tho, seems that the pi is not sending jobs to the volunteers. I mean it does, just that once in a while, and I don't understand why.
Here's what I see with distccmon-text:

Image

And I have to wait a while to catch one process on the volunteer. It doesn't really send much out. What could be the issue?
made the distcc setup again using my main PC now and all my cores are unused, barely some activity which explains my previous experience with the laptop. the pi doesn't send out jobs to the volunteer, maybe once in a while. volunteer logs also show some activity maybe once every 10 seconds or so. I also tried adding localhost/1 after the volunteer ip in the pi's config, no change.
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

Re: Pi Zero kernel compilation issues

Postby quark » Fri Nov 26, 2021 3:49 pm

Also the Zero has no networking, at all. I'm using a normal Zero, not Zero W. I added a usb wifi dongle on the normal Zero. But should be enough to send jobs. Not sure what the issue is, as it's not sending the jobs and failing along the way, that would have been observable in the logs or monitor. It just doesn't really send much to volunteers.
I'll try pump mode as well, don't really know what else to do. They're configured as per the wiki.

edit: also the make command from the PKGBUILD file is modified like this:
$this->bbcode_second_pass_code('', 'make CC=distcc CXX=distcc "$_image" modules dtbs')
so it should make use of distcc, which it does, just that not enough it seems.
also can pump mode be used in this configuration? by adding for example ",cpp,lzo:3634" to the DISTCC_HOSTS IP?
quark
 
Posts: 59
Joined: Thu Nov 25, 2021 8:26 am

PreviousNext

Return to User Questions

Who is online

Users browsing this forum: No registered users and 6 guests