Context: Evan and Colin could not see our website in the last few days. They also may have had trouble with email from or to our lists. I don't know if this has started working again. I think that the problem is rooted in the Domain Name System. How the Domain Name System works: DNS is a distributed database that is used to "resolve" domain names (like gtalug.org, penguin.gtalug.org, www.gtalug.org). The distribution is hierarchical: - there are Name Servers designated to resolve "." (the root of the DNS tree). The IP addresses of these servers are wired into your resolver. - These servers will tell you the name servers responsible for "org" - those servers will tell you the name servers responsible for "gtalug" within "org" To improve caching, most folks use "recursive name servers", usually the one proposed by the ISP as part of the negotiation that assigns an IP address (DHCP) to your gateway. Your computer just asks this recursive name server "What's the IP address for gtalug.org?" and the recursive name server walks the DNS tree behind the scenes are returns the ultimate result. The recursive nameserver remembers results of previous queries in a cache, saving repetitive queries. Fine points of DNS: DNS serves "resource records" of different types. Some types are: A: an IPv4 address for the named domain AAAA: IPV6 address MX: the mail server NS: the Name Server DNSsec is a system where each answer from a server is authenticated by a digital signature. This is somewhat complex and many domains don't support it. How the DNS data modifications be reflected in caches: Each DNS record contains a TTL (time-to-live) in seconds. Any cache of the data must respect the TTL. So one can modify a record in the "authoritative name server" for a "zone" and be sure that only the new record will be found after the TTL of the old record. DNS can fail in may ways, some hard to diagnose. Now you know the architecture you can think about failures that you observe. Some command-line tools to figure out DNS: host(1): $ host gtalug.org gtalug.org has address 69.164.222.247 gtalug.org has IPv6 address 2600:3c03::f03c:91ff:fe50:ea0a gtalug.org mail is handled by 0 penguin.gtalug.org. dig(1): This will craft a DNS query and show you the almost-raw results. The raw results are almost overwhelming but they can be understood. $ dig gtalug.org A "A" means: ask for the A record (IPv4 address) ; <<>> DiG 9.16.31-RH <<>> gtalug.org A Leading ";" means comment. ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30332 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 1 This answer has three sections: the query, the actual answer to the query, the authorities: name servers that were the ultimate source of the information ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;gtalug.org. IN A Query: what is the A record for gtalug.org ;; ANSWER SECTION: gtalug.org. 1190 IN A 69.164.222.247 Answer: the IP address is 69.164.222.247 ;; AUTHORITY SECTION: gtalug.org. 1190 IN NS ns-128-c.gandi.net. gtalug.org. 1190 IN NS ns-165-b.gandi.net. gtalug.org. 1190 IN NS ns-136-a.gandi.net. Authorities: three distinct name servers that will serve the zone for gtalug. We (GTALUG) contract with gandi.net to supply name services. ;; Query time: 1 msec The result was quick. ;; SERVER: 127.0.0.53#53(127.0.0.53) The (recursive) name server that handled this query is on this very machine. Usint "sudo netstat -p -l" I find out that systemd-resolve is listening on this UDP port. ;; WHEN: Wed Sep 14 17:02:55 EDT 2022 ;; MSG SIZE rcvd: 133 Now, trickier: $ dig gtalug.org A +trace This will get dig to do the queries itself, asking ., then org., then gtalug.org. Ask Google's public name server: $ dig @8.8.8.8 gtalug.org A Ask CLoudflare's public name server: $ dig @1.1.1.1 gtalug.org A Note: one of Cloudflare's tricks is to give different answers to queries from different locations. This is likely true for Google too. Yesterday Google worked for gtalug.org and Cloudflare did not. Today they both work. Maybe not for everyone. Maybe the problem is solved.