Components of an Orchestrator

From Build an Orchestrator in Go by Tim Boring

  • The evolution of application deployments
  • Classifying the components of an orchestration system


Kubernetes. Kubernetes. Kubernetes. If you’ve worked in or near the tech industry in the last five years, you’ve at least heard the name. Perhaps you’ve used it in your day job. In this article, we’re going to see what it takes to build our own Kubernetes; write the code to gain a better understanding about what Kubernetes is. And what Kubernetes is, in the general sense, is an orchestrator.

The (Not So) Good ‘Ole Days

Let’s take a journey back to 2002 and meet Michelle. Michelle is a system administrator for her company, and she is responsible for keeping her company’s applications up and running around the clock. How does she accomplish this?

Figure 1. This diagram represents a slice of Michelle’s world in 2002. The top image is a view of the rack in which her servers live. The bottom image is a view of those machines from the perspective of the operating system. Both views show the diversity that administrators dealt with across physical hardware and operating systems.

What is an orchestrator?

Before diving in head first and writing code, let’s define the term orchestrator. An orchestrator is a system that provides automation for deploying, scaling, and otherwise managing containers. In such systems, applications no longer run “naked” on bare metal or virtualized machines. Instead, they run inside containers. Table 1 summarizes the different features of each of the deployment and management models we’ve touched on.

The components of an orchestration system

An orchestrator automates deploying, scaling, managing containers. Next, let’s identify the components and their requirements that make those features possible. These components can be seen in Figure 2. They are:

  • The task
  • The job
  • The scheduler
  • The manager
  • The worker
  • The cluster
  • The CLI

The Task

The task is the smallest unit of work in an orchestration system and typically runs in a container. You can think of it like a process that runs on a single machine. A single task could run an instance of a reverse proxy like Nginx; or it could run an instance of an application like a RESTful API server; it could be a simple program that runs in an endless loop and does something silly, like ping a website and write the result to a database.

  1. The amount of memory, CPU, and disk it needs to run effectively
  2. What the orchestrator should do in case of failures, often called restart_policy
  3. The name of the container image used to run the task

The Job

The job is an aggregation of tasks. It has one or more tasks that typically form a larger logical grouping of tasks to perform a set of functions. For example, a job could be comprised of a RESTful API server and a reverse proxy.

  1. Each task that makes up the job
  2. Which data centers the job should run in
  3. The type of the job

The Scheduler

The scheduler decides what machine can best host the tasks defined in the job. It can be run as an independent process within the manager or as a completely separate service. The decision-making process can be as simple as selecting a node from a set of machine in a round-robin fashion, or as complex as calculating a score based upon a number of variables and selecting the node with the “best” score.

  1. Determine a set of candidate machines on which a task could run.
  2. Score the candidate machines from best to worst.
  3. Pick the machine with the best score.

The Manager

The manager is the brain of an orchestrator and the main entry point for users. In order to run jobs in the orchestration system, users submit their jobs to the manager. The manager, using the scheduler, then finds a machine where the job’s tasks can run. The manager also periodically collects metrics from each of its workers, which are used in the scheduling process.

  1. Accept requests from users to start and stop tasks.
  2. Schedule tasks onto worker machines.
  3. Keep track of tasks, their states, and the machine on which they run.

The Worker

The worker provides the brawns of an orchestrator. It’s responsible for running the tasks assigned to it by the manager. If a task fails for any reason, it must attempt to restart the task. The worker also makes metrics about its tasks and its overall machine health available for the master to poll.

  1. Running tasks as Docker containers.
  2. Accepting tasks to run from a manager.
  3. Providing relevant statistics to the manager for the purpose of scheduling tasks.
  4. Keeping track of its tasks and their state.

The Cluster

The cluster is the logical grouping of all the above components. An orchestration cluster could be run from a single physical or virtual machine. More commonly a cluster is built from multiple machines, from as few five to as many as thousands or more.


Finally, our CLI, the main user interface, should allow a user to:

  1. Start and stop tasks
  2. Get the status of tasks
  3. See the state of machines (i.e. the workers)
  4. Start the manager
  5. Start the worker
Figure 2. The components of an orchestration system. Regardless of what terms different orchestrators use, each has a scheduler, a manager, a worker, and they all operate on tasks.
Figure 3. Google’s Borg. At the bottom are a number of Borglets, or workers, which run individual tasks in containers. In the middle is the BorgMaster, or the manager, which uses the scheduler to place tasks on workers.
Figure 4. The Kubernetes architecture. The control plane, seen on the left, is equivalent to the manager function, or to Borg’s BorgMaster.
Figure 5. Nomad’s architecture, as it appears sparser, still functions similar to the other orchestrators.

Why implement an orchestrator from scratch?

If orchestrators such as Kubernetes and Nomad already exist, why write one from scratch? Couldn’t we look at the source code for them and get the same benefit?



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Manning Publications

Follow Manning Publications on Medium for free content and exclusive discounts.