Can't get internet working because of systemd-resolved

[Minding]

I've now switched completely to systemd and disabled netctl. I configured /etc/systemd/timesyncd.conf and added my NTP server to the hosts file. After my first reboot it didn't work, now it does. I hope this configuration doesn't include any race condition.

$this->bbcode_second_pass_code('', '
[root@alarm alarm]# timedatectl status
Local time: Fri 2020-01-03 14:59:26 UTC
Universal time: Fri 2020-01-03 14:59:26 UTC
RTC time: n/a
Time zone: UTC (UTC, +0000)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
')

To summarize, this fixed my issue (wlan0 is the name of my network interface):

$this->bbcode_second_pass_code('', '
systemctl disable netctl
nano /etc/systemd/network/wlan.network
+ [Match]
+ Name=wlan0
+
+ [Network]
+ DHCP=true
> C-x
nano /etc/wpa_supplicant/wpa_supplicant-wlan0.conf
+ ctrl_interface=/var/run/wpa_supplicant
+ network={
+ ssid="MyNetworkName"
+ scan_ssid=1
+ key_mgmt=WPA-PSK
+ psk="MyNetworkPassword"
+ }
> C-x
nano /etc/systemd/timesyncd.conf
m NTP=0.arch.pool.ntp.org
m FallbackNTP=1.arch.pool.ntp.org 2.arch.pool.ntp.org 3.arch.pool.ntp.org
> C-x
nano /etc/hosts
+ 129.70.132.36 0.arch.pool.ntp.org
> C-x
systemctl enable wpa_supplicant@wlan0.conf
reboot
')

I think Arch should at least disable one network manager by default.

Thanks, for the help! (and the corrections below)

[summers]

Great - I glad this its working so far. Do less us know how it goes, e.g. if this solves the problem in the long term, we'll know what to suggest to others.

Oh yes, in /etc/systemd/network/wlan.network I did:
$this->bbcode_second_pass_code('', ' [Network]
DHCP=true ')
Now my router can only do ipv4 - but I like "true" so if I change my router, and ipv6 gets switched on, then it will change to ipv6.

Anyway have a good weekend, and hope it keeps working.

Oh yes in:/etc/systemd/timesyncd.conf

$this->bbcode_second_pass_code('', '
NTP=0.arch.pool.ntp.org
FallbackNTP=1.arch.pool.ntp.org 2.arch.pool.ntp.org 3.arch.pool.ntp.org
')
in /etc/hosts I change:
$this->bbcode_second_pass_code('', '
129.70.132.36 0.arch.pool.ntp.org
')
So it has the same name (NTP and /etc/hosts). Or change both to 0.arch.pool.ntp.de if you want.

And yes, I agree about changing to one Network manager. Probably systemd as thats worked now on most computers for 5 years. Alas though this probably has to be done in the main arch set up, on arm here they just copy what is upline ...

[Sourav]

Facing this issue again!
The output from above commands:

$this->bbcode_second_pass_code('', '
┌┄┄[root::archlinux-arm]┈[/home/sourav]
└──╼⮚ networkctl status
● State: routable
Address: 192.168.2.5 on wlan0
fe80::c076:f97c:5bfb:4de5 on wlan0
Gateway: 192.168.2.1 (Smartlink Network Systems Limited) on wlan0
fe80::217:7cff:fe5b:c5df (Smartlink Network Systems Limited) on wlan0
')

$this->bbcode_second_pass_code('', '
┌┄┄[root::archlinux-arm]┈[/home/sourav]
└──╼⮚ resolvectl status
Global
LLMNR setting: yes
MulticastDNS setting: yes
DNSOverTLS setting: no
DNSSEC setting: allow-downgrade
DNSSEC supported: no
Fallback DNS Servers: 1.1.1.1
9.9.9.10
8.8.8.8
2606:4700:4700::1111
2620:fe::10
2001:4860:4860::8888
DNSSEC NTA: 10.in-addr.arpa
16.172.in-addr.arpa
168.192.in-addr.arpa
17.172.in-addr.arpa
18.172.in-addr.arpa
19.172.in-addr.arpa
20.172.in-addr.arpa
21.172.in-addr.arpa
22.172.in-addr.arpa
23.172.in-addr.arpa
24.172.in-addr.arpa
25.172.in-addr.arpa
26.172.in-addr.arpa
27.172.in-addr.arpa
28.172.in-addr.arpa
29.172.in-addr.arpa
30.172.in-addr.arpa
31.172.in-addr.arpa
corp
d.f.ip6.arpa
home
internal
intranet
lan
local
private
test

Link 3 (wlan0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
DefaultRoute setting: yes
LLMNR setting: yes
MulticastDNS setting: no
DNSOverTLS setting: no
DNSSEC setting: allow-downgrade
DNSSEC supported: yes
Current DNS Server: 8.8.8.8
DNS Servers: 203.147.91.2
8.8.8.8
203.147.88.2
DNS Domain: ~.
domain.name

Link 2 (eth0)
Current Scopes: none
DefaultRoute setting: no
LLMNR setting: yes
MulticastDNS setting: no
DNSOverTLS setting: no
DNSSEC setting: no
DNSSEC supported: no
')

$this->bbcode_second_pass_code('', '
┌┄┄[root::archlinux-arm]┈[/home/sourav]
└──╼⮚ systemctl --all list-units | egrep "net|resol"
sys-devices-platform-soc-3f300000.mmcnr-mmc_host-mmc1-mmc1:0001-mmc1:0001:1-net-wlan0.device loaded active plugged /sys/devices/platform/soc/3f300000.mmcnr/mmc_host/mmc1/mmc1:0001/mmc1:0001:1/net/wlan0
sys-devices-platform-soc-3f980000.usb-usb1-1\x2d1-1\x2d1.1-1\x2d1.1:1.0-net-eth0.device loaded active plugged SMSC9512/9514 Fast Ethernet Adapter
sys-subsystem-net-devices-eth0.device loaded active plugged SMSC9512/9514 Fast Ethernet Adapter
sys-subsystem-net-devices-wlan0.device loaded active plugged /sys/subsystem/net/devices/wlan0
systemd-networkd.service loaded active running Network Service
systemd-resolved.service loaded active running Network Name Resolution
systemd-networkd.socket loaded active running Network Service Netlink Socket
network-pre.target loaded inactive dead Network (Pre)
network.target loaded active active Network
')

$this->bbcode_second_pass_code('', '
┌┄┄[root::archlinux-arm]┈[/home/sourav]
└──╼⮚ timedatectl status
Local time: Sun 2020-01-05 01:12:05 IST
Universal time: Sat 2020-01-04 19:42:05 UTC
RTC time: n/a
Time zone: Asia/Kolkata (IST, +0530)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
')

I am connected to SSH from my laptop, and the LAN is working just fine. The internet doesn't work when the problem arises...

-------------------------------------------------------
Edit:

The internet seems to work now after
1. Creating /etc/systemd/network/wlan.network with the contents:
$this->bbcode_second_pass_code('', '
[Network]
DHCP=true
')

2. Editing /etc/systemd/timesyncd.conf
$this->bbcode_second_pass_code('', '
[Time]
NTP=0.arch.pool.ntp.org
FallbackNTP=1.arch.pool.ntp.org 2.arch.pool.ntp.org 3.arch.pool.ntp.org
')

I will update if it again stops working randomly.

[summers]

Hi Sourav, I'll go through your output - and what each bit means. And how it stands with the best guess that its a time msimatch that is causing DNSSEC to fail, and so loose all name lookup.

$this->bbcode_second_pass_code('', 'timedatectl status
System clock synchronized: yes
NTP service: active')
This says you are using NTP, and that the NTP has syncronised. This means your internal clock should be good to something like 1ms. This should be easily good enough for DNSSEC. So your time shouldn't be a problem.

$this->bbcode_second_pass_code('', 'resolvectl status
Link 3 (wlan0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
DefaultRoute setting: yes
LLMNR setting: yes
MulticastDNS setting: no
DNSOverTLS setting: no
DNSSEC setting: allow-downgrade
DNSSEC supported: yes
Current DNS Server: 8.8.8.8
DNS Servers: 203.147.91.2
8.8.8.8
203.147.88.2
DNS Domain: ~.
domain.name')
So you are using DNSSEC but in the allow-downgrade, which means if the far DNS server doesn't do DNSSEC (and few seem to these days) that you still use the DNS lookup.

Only odd bit is the DNS server, 8.8.8.8 is google - which is a good back up, but I prefer something local if possible. 203.147.* is Meghbela Broadband in India, is this your ISP? If so would be best.

Its a pity we can't say how systemd-resolved got the DNS servers, it usually give pripority to DHCP, so when that contains a DNS server that is used. E.g. my desktop has "Current DNS Server: 192.168.2.1" the address of my router. My router has: "DNS 1: 212.159.6.9 DNS 2: 212.159.6.10", which is what my local ISP gives me when my router connects, so good for me (but probably not for anyone else).

So this all says, from timing we would expect your connection to work, other than you need to go to google to get any names. So if you still had a problem with lookups, with those settings - its says we havn't got to the bottom of the problem. It says the next thing we probably should try is switching off DNSSEC totally, as changing that to allow-downgrade is I think the recent change where problems started.

Now /etc/systemd/timesyncd.conf
$this->bbcode_second_pass_code('', '
[Time]
NTP=0.arch.pool.ntp.org
FallbackNTP=1.arch.pool.ntp.org 2.arch.pool.ntp.org 3.arch.pool.ntp.org
')
Is good - it means we *know* that the default NTP server tried is 0.arch.ntp.org. To make sure we know its number you can do
$this->bbcode_second_pass_code('', 'drill 0.arch.pool.ntp.org') which will give you several possible IP numbers for the machine (and those numbers will vary!). This doesn't matter choose one, and add it /etc/hosts with the same name as the default NTP host 0.arch.pool.ntp.org. This means you can always attach to an NTP host to set the time on your machine.

Anyway how to totally switch off DNSSEC - its in a post from few months ago, in the file /etc/systemd/resolved.conf uncomment the DNSSEC line (remove the # at the sart) then change it to "DNSSEC=false". It means you won't do any DNSSEC, and the moment its not much of a problem, as its so rarely suported, so this change is worth trying to see if it helps.

https://archlinuxarm.org/forum/viewtopic.php?f=9&t=14056&hilit=DNSSEC

[summers]

Interesting! I just hit the same problem on my desktop computer. Had to switch to an old router (am working on my openwrt router, so needed to switch to the old plusnet one). Anyway couldn't look up any names.

You can see the source of the problem with:
$this->bbcode_second_pass_code('', 'resolvectl query www.google.com
www.google.com: resolve call failed: DNSSEC validation failed: failed-auxiliary
')

But a quick fix, that showed the source of the problem:
$this->bbcode_second_pass_code('', 'sudo resolvectl dnssec enp4s0 no')
where enp4s0 is the name of my ethernet interface. This just quickly switched off dnssec totally, and network started working at once.

So anyway, am posting this here - as it gives all the people with DNS problems a quick one line way of verifying if the problem is dnssec.

Whats also interesting is how this was router dependent, e.g. my 10 year old plus.net router can't handle dnssec ...