Categories DevOps

Understanding Kubernetes DNS – Pod and Namespace Resolution

Introduction

In the dynamic world of container orchestration, DNS (Domain Name System) plays a pivotal role in enabling service discovery and communication within Kubernetes clusters. As applications become more distributed and microservices-based, understanding how Kubernetes handles DNS resolution becomes crucial for effective cluster management and troubleshooting.

Kubernetes DNS has evolved significantly, transitioning from kube-dns to CoreDNS as the default DNS solution. This evolution brought improved stability, flexibility, and performance to cluster networking. DNS in Kubernetes serves as the backbone for service discovery, enabling pods and services to locate and communicate with each other seamlessly across the cluster.

Using dnsutils for DNS Resolution Examples

Throughout this article, we’ll use the dnsutils pod to demonstrate DNS resolution in a Kubernetes environment. The dnsutils pod is a utility container that includes common DNS troubleshooting tools like dignslookup, and host. To deploy a dnsutils pod in your cluster, apply the following manifest:

apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: default
spec:
  containers:
  - name: dnsutils
    image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
  restartPolicy: Always

After deploying the pod, you can run DNS queries using:

kubectl exec -i -t dnsutils -- nslookup <name-to-resolve>
# or
kubectl exec -i -t dnsutils -- dig <name-to-resolve>

This pod will serve as our DNS testing utility throughout the examples in this article.

Core DNS Architecture in Kubernetes

CoreDNS Fundamentals

CoreDNS operates as a flexible, plugin-based DNS server designed specifically for container environments. Its architecture consists of a modular system where each plugin handles specific DNS functionality. The core architecture includes:

  • A central DNS server process
  • Plugin chain processing
  • Customizable configuration via Corefile
  • Kubernetes integration components

DNS Service Deployment

CoreDNS runs as a critical service within the kube-system namespace, typically deployed as a Deployment with multiple replicas for high availability. The deployment includes:

  • CoreDNS pods running the DNS server
  • Service account with necessary RBAC permissions
  • ConfigMap containing the Corefile configuration
  • Kubernetes Service exposing DNS functionality

Resource requirements are carefully managed through:

  • CPU and memory requests/limits
  • Horizontal pod autoscaling
  • Anti-affinity rules for high availability

Basic DNS Flow in Kubernetes

When a DNS query is initiated within the cluster, it follows a specific resolution path:

  1. Pod sends DNS query to cluster DNS service
  2. CoreDNS receives and processes the query through its plugin chain
  3. Resolution occurs based on the following hierarchy:
    • Pod DNS records
    • Service DNS records
    • External DNS (if configured)

The default domain structure follows the pattern: .cluster.local as the base domain, with services and pods organized in their respective subdomains.

Let’s see this in action by resolving the Kubernetes default service:

$ kubectl exec -it dnsutils -- nslookup kubernetes.default
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

Pod DNS Resolution Deep Dive

Pod DNS Naming Conventions

Pods follow a structured naming convention for DNS resolution:

pod-ip-address.namespace.pod.cluster.local

For example, a pod with IP 10.244.1.10 in the default namespace would have the DNS name:

10-244-1-10.default.pod.cluster.local

IP addresses in DNS names use dashes instead of dots for compatibility reasons.

Let’s verify this with our dnsutils pod:

$ POD_IP=$(kubectl get pod dnsutils -o jsonpath='{.status.podIP}' | sed 's/\./-/g')
$ kubectl exec -it dnsutils -- nslookup $POD_IP.default.pod.cluster.local
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      10-244-1-10.default.pod.cluster.local
Address 1: 10.244.1.10

DNS Record Types for Pods

Kubernetes creates various DNS records for pods:

  • A/AAAA records: Direct pod IP address mapping
  • SRV records: For named ports in headless services
  • PTR records: Reverse DNS lookup support

Pod DNS Policies

Kubernetes supports multiple DNS policies that can be configured through the dnsPolicy field:

  • Default: Inherits the DNS configuration from the node
  • ClusterFirst: Uses cluster DNS with fallback to upstream nameservers
  • None: Allows custom DNS configuration
  • Host: Uses the host’s DNS resolution configuration

Service Discovery and DNS

Service DNS Patterns

Services in Kubernetes receive DNS entries following the pattern:

service-name.namespace.svc.cluster.local

Different service types have specific DNS behaviors:

  • ClusterIP services: Single A/AAAA record
  • Headless services: Multiple A/AAAA records (one per pod)
  • ExternalName services: CNAME record

Let’s examine how a ClusterIP service is resolved using our dnsutils pod:

$ kubectl exec -it dnsutils -- dig kubernetes.default.svc.cluster.local

; <<>> DiG 9.10.3-P4-Debian <<>> kubernetes.default.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15912
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;kubernetes.default.svc.cluster.local. IN A

;; ANSWER SECTION:
kubernetes.default.svc.cluster.local. 30 IN A 10.96.0.1

;; Query time: 2 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Thu May 15 12:30:44 UTC 2025
;; MSG SIZE  rcvd: 89

Cross-namespace Communication

Cross-namespace communication requires proper DNS qualification:

  • Within same namespace: Use service name
  • Across namespaces: UseĀ service-name.namespace
  • External services: Use complete FQDN

To demonstrate cross-namespace resolution, let’s look up a service in the kube-system namespace:

$ kubectl exec -it dnsutils -- nslookup kube-dns.kube-system
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kube-dns.kube-system
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

DNS Search Paths

Default search paths include:

  • namespace.svc.cluster.local
  • svc.cluster.local
  • cluster.local

This allows for shorter DNS names when accessing services within the same namespace.

CoreDNS Corefile

The Corefile defines CoreDNS behavior through plugin configurations. In Kubernetes, this file is stored as a ConfigMap in the kube-system namespace.

You can view the current Corefile using:

$ kubectl get configmap coredns -n kube-system -o yaml

To edit the CoreDNS configuration:

  1. Extract the current configuration:
    $ kubectl get configmap coredns -n kube-system -o yaml > coredns-configmap.yaml
  2. Edit the file to modify the Corefile section:
    $ vi coredns-configmap.yaml
  3. Apply the changes:
    $ kubectl apply -f coredns-configmap.yaml
  4. Verify CoreDNS pods restart with the new configuration:
    $ kubectl get pods -n kube-system -l k8s-app=kube-dns -w

Let’s examine a typical Corefile structure:

.:53 {
    kubernetes cluster.local in-addr.arpa ip6.arpa {
        pods insecure
        fallthrough in-addr.arpa ip6.arpa
    }
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    health
}

We can verify the active configuration is being used by querying the dnsutils pod:

$ kubectl exec -it dnsutils -- dig @10.96.0.10 kubernetes.default +nocmd +noall +answer
kubernetes.default.	30	IN	A	10.96.0.1

Changes to the Corefile take effect automatically as CoreDNS includes the reload plugin, which monitors configuration changes and applies them without requiring manual restarts.

Essential Plugins

CoreDNS relies on several critical plugins:

  • kubernetes: Handles cluster DNS records
  • forward: Manages upstream DNS resolution
  • cache: Provides DNS caching functionality
  • health: Enables health checking
  • metrics: Exposes Prometheus metrics
  • log: Provides query logging capabilities

Advanced Plugin Configurations

Custom plugin configurations can enhance DNS functionality:

.:53 {
    errors
    health {
        lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
        fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
}

Troubleshooting and Debugging

Common DNS Issues

Frequent DNS-related problems include:

  1. Resolution Failures
    • Misconfigured CoreDNS
    • Network policy restrictions
    • Invalid DNS policies
  2. Latency Problems
    • Insufficient CoreDNS resources
    • Cache misses
    • Network congestion
  3. Configuration Mistakes
    • Incorrect search paths
    • Invalid service entries
    • Plugin misconfiguration

Debugging Tools

Essential tools for DNS troubleshooting using the dnsutils pod:

# Basic DNS lookup
$ kubectl exec -it dnsutils -- nslookup kubernetes.default
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

# Using dig with specific DNS server
$ kubectl exec -it dnsutils -- dig @10.96.0.10 kubernetes.default.svc.cluster.local +short
10.96.0.1

# Checking reverse DNS lookup
$ kubectl exec -it dnsutils -- dig -x 10.96.0.1 +short
kubernetes.default.svc.cluster.local.

# Checking DNS search path configuration
$ kubectl exec -it dnsutils -- cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

# Testing external domain resolution
$ kubectl exec -it dnsutils -- dig kubernetes.io +short
147.75.40.148
147.75.40.147

Additionally, you can check CoreDNS logs:

# Checking CoreDNS logs
kubectl logs -n kube-system -l k8s-app=kube-dns

Advanced DNS Features

DNS Autoscaling

CoreDNS can be configured to automatically scale based on:

  • Cluster size
  • Query load
  • Resource utilization

Configuration example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  linear: |
    {
      "coresPerReplica": 256,
      "nodesPerReplica": 16,
      "min": 2,
      "max": 20
    }

Custom DNS Implementations

Support for external DNS providers:

  • Cloud provider DNS integration
  • External DNS controllers
  • Custom DNS zones

We can verify external DNS resolution using our dnsutils pod:

$ kubectl exec -it dnsutils -- dig example.com

; <<>> DiG 9.10.3-P4-Debian <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32478
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;example.com.                   IN      A

;; ANSWER SECTION:
example.com.            86400   IN      A       93.184.216.34

;; Query time: 30 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Thu May 15 12:35:24 UTC 2025
;; MSG SIZE  rcvd: 56

Conclusion

Kubernetes DNS serves as a critical component for service discovery and communication within clusters. Understanding its architecture, configuration, and troubleshooting approaches is essential for maintaining reliable cluster operations.

Key takeaways:

  • CoreDNS provides flexible, plugin-based DNS resolution
  • Proper DNS configuration is crucial for cross-namespace communication
  • Regular monitoring and optimization ensure reliable DNS operations

Common DNS Patterns

Here’s how to verify each DNS pattern using dnsutils:

# Service DNS pattern
$ kubectl exec -it dnsutils -- nslookup kubernetes.default.svc.cluster.local
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default.svc.cluster.local
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

# Pod DNS pattern (replace with your pod IP)
$ POD_IP=$(kubectl get pod dnsutils -o jsonpath='{.status.podIP}' | sed 's/\./-/g')
$ kubectl exec -it dnsutils -- nslookup $POD_IP.default.pod.cluster.local

# Headless service example (create a headless service first)
$ kubectl exec -it dnsutils -- nslookup some-pod.headless-service.default.svc.cluster.local

Configuration Templates

Basic CoreDNS configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        health
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }

Troubleshooting Cheatsheet

Common commands for DNS debugging with dnsutils:

# Deploy dnsutils if not already present
kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml

# Check CoreDNS pods
kubectl get pods -n kube-system -l k8s-app=kube-dns

# Verify CoreDNS service
kubectl get svc -n kube-system kube-dns

# Test DNS resolution
kubectl exec -it dnsutils -- nslookup kubernetes.default

# Test DNS resolution with verbose output
kubectl exec -it dnsutils -- dig kubernetes.default +noall +answer +comments

# Check SRV records for a service
kubectl exec -it dnsutils -- dig SRV _https._tcp.kubernetes.default.svc.cluster.local

# Check CoreDNS configuration
kubectl get configmap -n kube-system coredns -o yaml

# Monitor DNS metrics
kubectl port-forward -n kube-system deployment/coredns 9153:9153

You May Also Like