ARTICLE

Introducing Knative Serving

From Knative in Action by Jacques Chester

  • The major components of Serving and what they do

A walkthrough

Your first deployment

$ kn service create hello-example \                        ❶
--image gcr.io/knative-samples/helloworld-go \ ❷
--env TARGET="First" ❸
Creating service 'hello-example' in namespace 'default': ❹
0.084s The Route is still working to reflect the latest desired specification. 0.260s Configuration "hello-example" is waiting for a Revision to become ready.
4.356s ...
4.762s Ingress has not yet been reconciled. 6.104s Ready to serve.
Service 'hello-example' created with latest revision 'hello-example-pjyvr-1' and URL: http://hello-example.default.35.194.0.173.nip.io
$ curl http://hello-example.default.35.194.0.173.nip.io
Hello First!

Your second deployment

$ kn service update hello-example \
--env TARGET=Second
Updating Service 'hello-example' in namespace 'default':
3.418s Traffic is not yet migrated to the latest revision.
3.466s Ingress has not yet been reconciled.
4.823s Ready to serve.
Service 'hello-example' updated with latest revision 'hello-example-bqbbr-2' and URL:
http://hello-example.default.35.194.0.173.nip.io
$ curl http://hello-example.default.35.194.0.173.nip.io
Hello Second!
func handler(w http.ResponseWriter, r *http.Request) {
target := os.Getenv("TARGET")
fmt.Fprintf(w, "Hello %s!\n", target)
}
$ kn revision list
NAME SERVICE GENERATION AGE CONDITIONS READY
REASON
hello-example-bqbbr-2 hello-example 2 2m3s 4 OK / 4 True hello-example-pjyvr-1 hello-example 1 3m15s 3 OK / 4 True

Conditions

  1. OK gives the quick summary about whether the news is good or bad. The ++ signals that everything is fine. The I signals an informational condition — not bad, but not as unambiguous as ++. If things were going badly, you’d see !!. If Knative doesn’t know what’s happening, you’ll see ??.
  2. TYPE is the unique condition being described. In this table we can see four being reported. The Ready condition, for example, surfaces the result of an underlying Kubernetes readiness probe. Of greater interest to us is the Active condition, which tells us whether there is an instance of the Revision running.
  3. AGE reports on when this Condition was last observed to have changed. In the example these are all three hours. But they don’t have to be.
  4. REASON allows a Condition to provide a clue as to deeper causes. For example, our Active condition shows NoTraffic as its reason.
$ kn revision describe hello-example-bqbbr-2
Name: hello-example-bqbbr-2
Namespace: default
Age: 7d
Image: gcr.io/knative-samples/helloworld-go (pinned to 5ea96b)
Env: TARGET=Second
Service: hello-example

Conditions:
OK TYPE AGE REASON
++ Ready 4h
++ ContainerHealthy 4h
++ ResourcesAvailable 4h
I Active 4h NoTraffic

$ curl http://hello-example.default.35.194.0.173.nip.io
# ... a pause while the container launches
Hello Second!

$ kn revision describe hello-example-bqbbr-2
Name: hello-example-bqbbr-2
Namespace: default
Age: 7d
Image: gcr.io/knative-samples/helloworld-go (pinned to 5ea96b)
Env: TARGET=Second
Service: hello-example

Conditions:
OK TYPE AGE REASON
++ Ready 4h
++ ContainerHealthy 4h
++ ResourcesAvailable 4h
++ Active 2s

Changing the image

$ kn service update hello-example \
--image gcr.io/knative-samples/helloworld-rust Updating Service
'hello-example' in namespace 'default':
49.523s Traffic is not yet migrated to the latest revision.
49.648s Ingress has not yet been reconciled. 49.725s Ready to serve.
Service 'hello-example' updated with latest revision 'hello-example-nfwgx-3' and URL: http://hello-example.default.35.194.0.173.nip.io
curl http://hello-example.default.35.194.0.173.nip.io
Hello world: Second

Splitting traffic

$ kn service update hello-example \
--traffic hello-example-bqbbr-2=50 \
--traffic hello-example-nfwgx-3=50 Updating Service 'hello-example' in namespace 'default':
0.057s The Route is still working to reflect the latest desired specification.
0.072s Ingress has not yet been reconciled.
1.476s Ready to serve.
Service 'hello-example' updated with latest revision 'hello-example-nfwgx-3'
(unchanged) and URL: http://hello-example.default.35.194.0.173.nip.io
$ curl http://hello-example.default.35.194.0.173.nip.io
Hello Second!
$ curl http://hello-example.default.35.194.0.173.nip.io
Hello world: Second
$ kn service update french-flashbacks-example \
--traffic un=25 \
--traffic deux=25 \
--traffic trois=25 \
--traffic quatre=25
$ kn service update french-flashbacks-example \
--traffic un=0 \
--traffic deux=0 \
--traffic trois=98 \
--traffic quatre=2
$ kn service update french-flashbacks-example \
--traffic trois=98 \
--traffic quatre=2
$ kn service update french-flashbacks-example \
--traffic @latest=100

Serving Components

  1. Reconcilers, responsible for acting on both user-facing concepts like Services, Revisions, Configurations and Routes as well as lower-level housekeeping;
  2. The “Webhook”, responsible for validating and enriching the Services, Configurations and Routes that users provide;
  3. Networking controllers that configure TLS certificates and HTTP ingress routing; and
  4. The Autoscaler/Activator/Queue-Proxy triad, which manage the business of comprehending and reacting to changes on traffic.

The Controller and Reconcilers

Figure 1. The serving controller and its Reconcilers

The Webhook

  • Setting default configurations. This includes values for timeouts, concurrency limits, container resources limits and garbage collection timing. This means that you only need to set values you want to override. I’ll touch on these as needed.
  • Injecting routing and networking information into Kubernetes. I’ll discuss this when I get to routing.
  • Validating that users didn’t ask for impossible configurations. For example, the webhook rejects negative concurrency limits. I’ll refer to these when needed.
  • Resolving partial docker image references to include the digest. For example, example/example:latest would be resolved to include the digest, and it looks like example/example@sha256:1a4bccf2…. This is one of the best things Knative can do for you, and the webhook deserves the credit for it.

Networking controllers

Certificates

Ingress

Autoscaler, Activator and Queue-Proxy

Figure 2. The triad of autoscaler, activator and queue-proxy
Figure 3. The Knative Pod Autoscaler is a control loop
Figure 4. The Activator’s role in managing cold starts
  1. The Ingress receives a new request. The Ingress sends the request to its configured target, which is the Activator.
  2. The Activator places the new request into a buffer.
  3. The Activator “pokes” the Autoscaler. The poke does two things: firstly, it carries information about requests waiting in the buffer. Secondly, the arrival of a poke signal prompts the Autoscaler to make an immediate scaling decision, instead of waiting until the next scheduled decision time.
  4. After considering the fact that there’s a request waiting to be served, but there are zero instances available to serve it, the Autoscaler decides that there ought to be one instance running. It sets a new scale target for Serving.
  5. As you wait for the Autoscaler and Serving to do their work, the Activator polls Serving to see if any instances are live.
  6. Serving’s hierarchy of controllers ultimately cause Kubernetes to launch an instance of your software.
  7. The Activator learns from its polling that an instance is now available and moves the request from its buffer to a proxy service.
  8. The proxy component sends the request to the instance, which responds normally.
  9. The proxy component sends the response back to the Ingress, which then sends it back to the requester.

Summary

  • kn is a CLI tool for interacting with Knative, including Serving.
  • kn service lets you view, create, update and configure Knative Services, including splitting traffic between Revisions.
  • Knative Serving has a controller process, which is a collection of components called “Reconcilers”. Reconcilers act as feedback controllers.
  • Reconcilers are for Serving’s core record types (Service, Route, Configuration and Revision), as well as housekeeping Reconcilers.
  • Knative Serving has a webhook process, which intercepts new and updated records you submit. It can then validate your submissions and inject additional information.
  • The Knative Pod Autoscaler is a feedback control loop. It compares the ratio of traffic to instances, and raises or lowers the desired number of instances that the serving controller controls.
  • The Activator is assigned Ingress routes when no instances are available. This assignment is made by the Autoscaler.
  • The Activator is responsible for “poking” the Autoscaler when new requests arrive, to trigger a scale-up.
  • Although instances are becoming available, the Activator remains on the data path as a throttling, buffering proxy for traffic.
  • When the Autoscaler believes there’s enough capacity to serve demand, it removes the Activator from the data path by updating Ingress routes.
  • Knative Serving’s Networking is highly pluggable. Core implementations are provided for two functions: Certificates and Ingress.
  • Certificate controllers accept a definition of desired Certificates and must provision new certificates or map existing certificates into your software.
  • Ingress controllers accept Routes and convert these into lower-level routing or traffic management configurations.
  • Ingress controller implementations include Istio-Gateway, Gloo, Ambassador and Kourier.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Manning Publications

Follow Manning Publications on Medium for free content and exclusive discounts.