Some mirrors are unhealthy

This forum is for discussion about general software issues.

Some mirrors are unhealthy

Postby connesc » Fri Apr 03, 2020 1:22 pm

Hello,

I'm located in France and I receive many errors when retrieving packages from the default mirror: mirror.archlinuxarm.org.
After a few days without improvement, I started to investigate and I discovered that a mirror is currently unhealthy: pt.mirror.archlinuxarm.org.

Would it be possible to temporarily exclude it from the GeoIP redirection system?

Thanks!

PS: raw command used to find unhealthy mirrors: $this->bbcode_second_pass_code('', 'for mirror in au.mirror.archlinuxarm.org br2.mirror.archlinuxarm.org dk.mirror.archlinuxarm.org de3.mirror.archlinuxarm.org de.mirror.archlinuxarm.org de4.mirror.archlinuxarm.org eu.mirror.archlinuxarm.org de5.mirror.archlinuxarm.org gr.mirror.archlinuxarm.org hu.mirror.archlinuxarm.org nl.mirror.archlinuxarm.org pt.mirror.archlinuxarm.org sg.mirror.archlinuxarm.org za.mirror.archlinuxarm.org tw.mirror.archlinuxarm.org il.us.mirror.archlinuxarm.org ca.us.mirror.archlinuxarm.org nj.us.mirror.archlinuxarm.org fl.us.mirror.archlinuxarm.org; do echo -n "$mirror: "; curl -sSf --max-time 10 -o /dev/null "$mirror" && echo OK; done')

EDIT: my post originally mentioned de3.mirror.archlinuxarm.org as well, but it has been fixed soon after. Only pt.mirror.archlinuxarm.org seems to be really unhealthy.
connesc
 
Posts: 3
Joined: Mon Feb 04, 2019 6:27 pm

Re: Some mirrors are unhealthy

Postby Kabbone » Fri Apr 03, 2020 5:17 pm

I also realized some days ago, that some mirrors are quite outdated. As you already mentioned I think it was de3, pt, sg, za. I just tried some manually.
Kabbone
 
Posts: 153
Joined: Thu Jul 25, 2013 9:20 am

Re: Some mirrors are unhealthy

Postby karog » Fri Apr 03, 2020 6:06 pm

$this->bbcode_second_pass_quote('connesc', 'W')ould it be possible to temporarily exclude it from the GeoIP redirection system?

Just comment out the offending server in /etc/pacman.d/mirrorlist
karog
 
Posts: 300
Joined: Thu Jan 05, 2012 7:55 pm

Re: Some mirrors are unhealthy

Postby ufo6000 » Fri Apr 03, 2020 6:33 pm

$this->bbcode_second_pass_quote('', 'J')ust comment out the offending server in /etc/pacman.d/mirrorlist


And enable servers near to you (France), e.g. Netherlands and Germany.
ufo6000
 
Posts: 117
Joined: Fri Jan 22, 2016 7:54 pm

Re: Some mirrors are unhealthy

Postby connesc » Fri Apr 03, 2020 6:51 pm

Thanks, that's what I'm doing currently.

But I guess it would be good to fix the officially recommended mirror, which is supposed to automatically load-balance across best mirrors.
connesc
 
Posts: 3
Joined: Mon Feb 04, 2019 6:27 pm

Re: Some mirrors are unhealthy

Postby pcfreak » Fri Jun 05, 2020 6:59 am

I ran into similar problems and came up with a quick and dirty solution. I created a script that measures the response time of the servers and then creates a new mirrorlist. I attached the script below. Store it as $this->bbcode_second_pass_code('', 'armreflector.bash'), create a copy of the actual mirrorlist $this->bbcode_second_pass_code('', 'cp /etc/pacman.d/mirrorlist /etc/pacman.d/mirrorlist.bak') then make it executable $this->bbcode_second_pass_code('', 'chmod +x armreflector.bash') and let it run it against the backup of the mirrorlist to create a new one

$this->bbcode_second_pass_code('', './armreflector.bash /etc/pacman.d/mirrorlist.bak')

It will then create a mirrorlist like this

$this->bbcode_second_pass_code('', '
##################################################
### quick and dirty speed optimized mirrorlist ###
##################################################
#speed 0.023398
Server = http://de5.mirror.archlinuxarm.org/$arch/$repo
#speed 0.032132
Server = http://eu.mirror.archlinuxarm.org/$arch/$repo
#speed 0.033872
Server = http://de3.mirror.archlinuxarm.org/$arch/$repo
#speed 0.043265
Server = http://de4.mirror.archlinuxarm.org/$arch/$repo
#speed 0.043954
Server = http://de.mirror.archlinuxarm.org/$arch/$repo
#speed 0.058741
Server = http://hu.mirror.archlinuxarm.org/$arch/$repo
#speed 0.068747
Server = http://dk.mirror.archlinuxarm.org/$arch/$repo
#speed 0.109516
Server = http://gr.mirror.archlinuxarm.org/$arch/$repo
#speed 0.184106
Server = http://nj.us.mirror.archlinuxarm.org/$arch/$repo
#speed 0.193738
Server = http://fl.us.mirror.archlinuxarm.org/$arch/$repo
#speed 0.261217
Server = http://mirror.archlinuxarm.org/$arch/$repo
#speed 0.527954
Server = http://tw.mirror.archlinuxarm.org/$arch/$repo
#speed 0.619921
Server = http://au.mirror.archlinuxarm.org/$arch/$repo
#speed 0.677908
Server = http://vn.mirror.archlinuxarm.org/$arch/$repo
')

See the script below. Please keep in mind, it is just quick and dirty :oops: and I needed it as part of my Digital Signage solution that includes it: https://github.com/boschkundendienst/raspisignage

armreflector.bash

$this->bbcode_second_pass_code('', '
#!/bin/bash
# This script is in a very early stage but does the job for now.
# Feel free to come up with a better solution.
# The purpose of this script is to measure the responsiveness
# of archlinux|ARM mirrors in a given mirrorlist and print
# out a speed sorted list of them to stdout.
if [ "$1" == "--help" ]
then
echo -e "\nUsage: armreflector <source mirrorlist>"
echo "Prints a speed optimized mirrorlist of <source mirrorlist>"
echo "to stdout. Can be used to optimizie your pacman mirrors."
echo -e "**!! Early Beta !!**\n"
exit 1
fi

mirrorlist=$1
if [ ! -f "$mirrorlist" ]; then echo "please specify a valid mirrorlist file as parameter 1. Aborting";exit 1;fi
if [ "$mirrorlist" == "/etc/pacman.d/mirrorlist" ]; then echo "You should not specify the 'live-list' as input, use a copy instead. Aborting";exit 1;fi
LANG=C
# create an array containing all urls from mirrorlist
declare -a lines # declare the indexed array for Servers
# put all lines that contained 'Server = ' in mirrorlist into the array 'lines'
for line in $(cat "$mirrorlist" | grep -o ".*Server = .*" | sed 's@.*Server = \(http.*\)@\1@g')
do
lines+=("$line")
done
#echo "lines has ${#lines[@]} elements"

# print new mirrorlist sorted by speed
echo '##################################################'
echo '### quick and dirty speed optimized mirrorlist ###'
echo '##################################################'
# measure the time_total for any url and put it into the speed array
for (( a=0; a<${#lines[@]}; a++ ))
do
#url=$(echo "${lines[a]}" | sed 's@\(http.*\)$arch.*@\1@g' | sed 's@\(http.*\)$(arch|repo).*@\1@g') # for all arch
url=$(echo "${lines[a]}" | sed 's@\(http.*\)$arch.*@\1@g') # for archlinuxarm
#url=$(echo ${lines[a]} | sed 's@\(http.*\)$rep.*@\1@g') # for arch
#echo "$url"
speedline=$(curl -L -s -o /dev/null -w '%{http_code} %{time_total}' --url "$url" --connect-timeout 5 --max-time 2)
http_code=$(echo "$speedline" | awk '{print $1}')
speed=$(echo "$speedline" | awk '{print $2}')
if [ "$http_code" == "200" ] # kann abgeändert werden
then
echo "$speed ${lines[a]}"
fi
done | sort -n -k 1 | awk '{print "#speed "$1"\nServer = "$2}'
')
pcfreak
 
Posts: 5
Joined: Fri Jun 05, 2020 6:40 am


Return to General

Who is online

Users browsing this forum: No registered users and 8 guests