Understanding the kubeconfig file



kubectl

is a command-line tool used for interacting with Kubernetes clusters. kubectl allows users to manage Kubernetes resources and applications by providing commands to create, inspect, update, and delete various resources within a Kubernetes cluster.

In the previous post, we saw how to install kubectl and we mentioned, very briefly, how we'd be using the kubeconfig file to connect kubectl to our K8s cluster. Go to that post if you still haven't installed the kubectl command line tool:

How to Install kubectl to manage your K8s clusters

In this post, we'll dive deeper into the kubeconfig file. We'll see how it works, how it is structured and how to manage it from the kubectl

How to interact with a K8s cluster

In a K8s architecture the kube api server is the primary management component of the K8s cluster. It’s the one that coordinates and orchestrates all operations within the cluster. And it is also the one that exposes the K8s API which is used by

external users to perform management tasks in the cluster (e.g. - kubectl)
the diff K8s controllers to monitor the state of the cluster
the workers nodes to communicate with the server

So, we can manage our K8s cluster sending GET/POST request to the kube api server of our K8s cluster, which tipically is located at https://my-k8s-cluster:6443/api and specifying the certificate details for authentication. For example:

curl https://my-k8s-cluster:6443/api/v1/pods \
--key admin.key
--cert admin.crt
--cacert ca.crt

kubectl, behind the scenes, calls the K8s API. It is basically translating your commands into API requests to the kube API server. To interact with your K8s cluster, kubectl needs to know where your cluster is located and how to authenticate. We could do that like this:

kubectl get pods //
    --server my-k8s-cluster:6443 //
    --client-key admin.key //
    --client-certificate admin.crt //
    --certificate-authority ca.crt

Obviously providing the location and cert details on each query or command is a tedious task. To simplify this we move the auth details to a configuration file called kubeconfig and then specify this file as the kubeconfig option in our command:

kubectl get pods --kubeconfig [CONFIG_FILE]

We can make it even easier without specifying the kubeconfig file. By default, kubectl will look for the kubeconfig file under the path $HOME/.kube/config.

Lastly, if we want to use more than one kubeconfig file, we can use the env variable KUBECONFIG or by setting the flag --kubeconfig

kubectl will look first for this env variable and use its value if that exists. If there's no value or the env variable does not exist it will use the default path $HOME/.kube/config.

Structure of the kubeconfig file

The kubeconfig file has three sections:

Clusters

The various K8s clusters that you need access to. This is where we specify the URL to access each cluster.

Users

The user accounts with which you have access to your clusters. These users may have different privileges on different clusters. This is where we specify how each user authenticates.

Contexts

Marry clusters and users. A context define which user account will be used to access which cluster.

Here's an example

apiVersion: v1
kind: Config
clusters:
  - name: my-k8s-cluster
        cluster:
    certificate-authority: ca.crt
          server: https://my-k8s-cluster:6443
contexts:
  - name: my-k8s-admin@my-k8s-cluster
        context:
          cluster: my-k8s-cluster
          user: my-k8s-admin
users:
    - name: my-k8s-admin
    user:
    client-certificate: admin.crt
    client-key: admin.key

How to use the kubeconfig file

Ok, so to use kubectl we need to provide a kubeconfig file and specify in that file the details of the cluster and user I want to use. The question now is, what happens when I've got more than one cluster/user/context in the kubeconfig file? how does kubectl knows which context to use?

There are two ways to set this up.

First, we can add a field current-context in the kubeconfig file that specifies what context is the selected one.

Example:

apiVersion: v1
kind: Config    
current-context: admin-k8s@pqa-k8s-cluster

clusters: // values hidden
  - name: dev-k8s-cluster 
  - name: qa-k8s-cluster  
  - name: prod-k8s-cluster  
    

contexts: // values hidden
  - name: prod-k8s@prod-k8s-cluster
  - name: dev-k8s@dev-k8s-cluster
  - name: admin-k8s@pqa-k8s-cluster
    

users: // values hidden
  - name: admin-k8s
  - name: dev-k8s
  - name: prod-user-k8s

The second option is to switch contexts from the kubectl itself. kubectl provides a number of commands to manage the kubeconfig file. We can set the context we want to use like this:

kubectl config use-context admin-user@my-cluster

To get a full list of all commands in kubectl to manage the kubeconfig file we can run:

kubectl config -h