🥇Writing an operator for Kubernetes in Golang

Note. transl.: Operators are utility software for Kubernetes designed to automate the execution of routine actions on cluster objects upon certain events. We have already written about operators in this articlewhere they talked about the fundamental ideas and principles of their work. But if that material was rather a view from the side of the operation of ready-made components for Kubernetes, then the translation of the new article now proposed is already the vision of a developer / DevOps engineer, puzzled by the implementation of a new operator.

I decided to write this post with a real life example after my attempts to find documentation on creating an operator for Kubernetes, went through the study of the code.

The example that will be described is this: in our Kubernetes cluster, each Namespace represents a team's sandbox environment, and we wanted to restrict access to them so that teams can only play in their own sandboxes.

You can achieve what you want by assigning a user to a group that has RoleBinding to specific Namespace и ClusterRole with editing rights. The YAML representation will look like this:

---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: kubernetes-team-1
  namespace: team-1
subjects:
- kind: Group
  name: kubernetes-team-1
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: edit
apiGroup: rbac.authorization.k8s.io

(rolebinding.yamlIn raw)

Create such RoleBinding you can do it manually, but once you get over a hundred namespaces, it becomes a tedious task. This is where Kubernetes operators come in handy - they allow you to automate the creation of Kubernetes resources based on changes in resources. In our case, we want to create RoleBinding while creating Namespace.

First of all, let's define the function main, which does the required setup to run the statement and then invokes the statement action:

(Note. transl.: hereinafter, the comments in the code are translated into Russian. In addition, the indentation is corrected for spaces instead of [recommended in Go] tabs solely for the purpose of better readability within the framework of Habr's layout. After each listing, there are links to the original on GitHub, where English-language comments and tabs are saved.)

func main() {
  // Устанавливаем вывод логов в консольный STDOUT
  log.SetOutput(os.Stdout)

  sigs := make(chan os.Signal, 1) // Создаем канал для получения сигналов ОС
  stop := make(chan struct{})     // Создаем канал для получения стоп-сигнала

  // Регистрируем получение SIGTERM в канале sigs
  signal.Notify(sigs, os.Interrupt, syscall.SIGTERM, syscall.SIGINT) 

  // Goroutines могут сами добавлять себя в WaitGroup,
 // чтобы завершения их выполнения дожидались
  wg := &sync.WaitGroup{} 

  runOutsideCluster := flag.Bool("run-outside-cluster", false, "Set this flag when running outside of the cluster.")
  flag.Parse()
  // Создаем clientset для взаимодействия с кластером Kubernetes
  clientset, err := newClientSet(*runOutsideCluster)

  if err != nil {
    panic(err.Error())
  }

  controller.NewNamespaceController(clientset).Run(stop, wg)

  <-sigs // Ждем сигналов (до получения сигнала более ничего не происходит)
  log.Printf("Shutting down...")

  close(stop) // Говорим goroutines остановиться
  wg.Wait()   // Ожидаем, что все остановлено
}

(main.goIn raw)

We do the following:

We set up a handler for specific operating system signals to cause the operator to gracefully exit.
Use WaitGroupto gracefully stop all goroutines before the application terminates.
We provide access to the cluster by creating clientset.
Run NamespaceController, in which all our logic will be located.

Now we need a basis for logic, and in our case it is the mentioned NamespaceController:

// NamespaceController следит через Kubernetes API за изменениями
// в пространствах имен и создает RoleBinding для конкретного namespace.
type NamespaceController struct {
  namespaceInformer cache.SharedIndexInformer
  kclient           *kubernetes.Clientset
}

// NewNamespaceController создает новый NewNamespaceController
func NewNamespaceController(kclient *kubernetes.Clientset) *NamespaceController {
  namespaceWatcher := &NamespaceController{}

  // Создаем информер для слежения за Namespaces
  namespaceInformer := cache.NewSharedIndexInformer(
    &cache.ListWatch{
      ListFunc: func(options metav1.ListOptions) (runtime.Object, error) {
        return kclient.Core().Namespaces().List(options)
      },
      WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) {
        return kclient.Core().Namespaces().Watch(options)
      },
    },
    &v1.Namespace{},
    3*time.Minute,
    cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc},
  )

  namespaceInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
    AddFunc: namespaceWatcher.createRoleBinding,
  })

  namespaceWatcher.kclient = kclient
  namespaceWatcher.namespaceInformer = namespaceInformer

  return namespaceWatcher
}

(controller.goIn raw)

Here we are setting SharedIndexInformer, which will efficiently (using the cache) wait for namespace changes (read more about informers in the article "How does the Kubernetes scheduler actually work?"- approx. transl.). After that we connect EventHandler to the informer, thanks to which, when adding a namespace (Namespace) the function is called createRoleBinding.

The next step is to define this function createRoleBinding:

func (c *NamespaceController) createRoleBinding(obj interface{}) {
  namespaceObj := obj.(*v1.Namespace)
  namespaceName := namespaceObj.Name

  roleBinding := &v1beta1.RoleBinding{
    TypeMeta: metav1.TypeMeta{
      Kind:       "RoleBinding",
      APIVersion: "rbac.authorization.k8s.io/v1beta1",
    },
    ObjectMeta: metav1.ObjectMeta{
      Name:      fmt.Sprintf("ad-kubernetes-%s", namespaceName),
      Namespace: namespaceName,
    },
    Subjects: []v1beta1.Subject{
      v1beta1.Subject{
        Kind: "Group",
        Name: fmt.Sprintf("ad-kubernetes-%s", namespaceName),
      },
    },
    RoleRef: v1beta1.RoleRef{
      APIGroup: "rbac.authorization.k8s.io",
        Kind:     "ClusterRole",
        Name:     "edit",
    },
  }

  _, err := c.kclient.Rbac().RoleBindings(namespaceName).Create(roleBinding)

  if err != nil {
    log.Println(fmt.Sprintf("Failed to create Role Binding: %s", err.Error()))
  } else {
    log.Println(fmt.Sprintf("Created AD RoleBinding for Namespace: %s", roleBinding.Name))
  }
}

(controller.goIn raw)

We get the namespace as obj and convert it to an object Namespace. Then we define RoleBinding, based on the YAML file mentioned at the beginning, using the provided object Namespace and creating RoleBinding. Finally, we log whether the creation was successful.

The last function to be defined is − Run:

// Run запускает процесс ожидания изменений в пространствах имён
// и действия в соответствии с этими изменениями.
func (c *NamespaceController) Run(stopCh <-chan struct{}, wg *sync.WaitGroup) {
  // Когда эта функция завершена, пометим как выполненную
  defer wg.Done()

  // Инкрементируем wait group, т.к. собираемся вызвать goroutine
  wg.Add(1)

  // Вызываем goroutine
  go c.namespaceInformer.Run(stopCh)

  // Ожидаем получения стоп-сигнала
  <-stopCh
}

(controller.goIn raw)

Here we are talking WaitGroupthat we start the goroutine and then call namespaceInformerwhich has been previously defined. When a stop signal comes in, it will end the function, inform WaitGroup, which is no longer executing, and this function will exit.

Information about building and running this statement on a Kubernetes cluster can be found at repositories on GitHub.

On this statement, which creates RoleBinding upon appearance Namespace in a Kubernetes cluster, ready.

Source: habr.com

We write an operator for Kubernetes in Golang

Add a comment Отменить ответ