End to End Encryption using EnRoute and Linkerd

Enable Zero Trust and Encrypt traffic from client to service

Introduction

This document describes how easily EnRoute OneStep integrates with Linkerd service-mesh. LinkerD is a CNCF graduated service mesh project which uses light-weight proxies to setup the mesh inside Kubernetes.

EnRoute integration with Linkerd can be achieved in One Step. Integrating EnRoute with Linkerd involves setting one flag

At a high level End-To-End encryption includes -

Encryption from Client to EnRoute and
Encryption from EnRoute to mTLS inside the Mesh

We mesh the workload and EnRoute by injecting linkerd proxy into their pods and then setup the client certificate.

We start by setting up the cluster, installing linkerd, adding an example workload emojivoto and meshing this workload by injecting linkerd
Next we install EnRoute, expose the emojivoto application without TLS, inject linkerd proxy in the EnRoute pod to add it to the mesh
As the last step, we use JetStack cert manager to generate, sign and install a let’s Encrypt certificate for the emojivoto app

We trace through each of the above steps while monitoring the cluster for linkerd proxy injection.

We also verify some of the above steps on the Bouyant cloud (with it’s free tier), that shows the above steps in different UI screens

Pre-Requisites

This setup needs an Kubernetes cluster. We have a cluster setup with two nodes -

‍

Check Kubernetes Cluster



kubectl get nodes

‍



Linkerd core checks
===================

kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version

...

linkerd-control-plane-proxy
---------------------------
√ control plane proxies are healthy
√ control plane proxies are up-to-date
√ control plane proxies and cli versions match

Status check results are √

Install EnRoute And Emojivoto Workload

Install Example Workload Emojivoto



curl -fsL https://run.linkerd.io/emojivoto.yml | kubectl apply -f -



Linkerd core checks
namespace/emojivoto created
serviceaccount/emoji created
serviceaccount/voting created
serviceaccount/web created
service/emoji-svc created
service/voting-svc created
service/web-svc created
deployment.apps/emoji created
deployment.apps/vote-bot created
deployment.apps/voting created
deployment.apps/web created

Install EnRoute OneStep

Add Helm Repos



helm repo add saaras https://charts.getenroute.io
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm search repo

Use Helm To Install EnRoute



helm install enroute-linkerd-meshed saaras/enroute \
	--set enrouteService.rbac.create=true \
	--create-namespace \
	--namespace demo



NAME: enroute-linkerd-meshed
LAST DEPLOYED: Mon Mar 21 03:19:46 2022
NAMESPACE: demo
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
𝙴𝚗𝚁𝚘𝚞𝚝𝚎 Ingress API Gateway Community Edition Installed!
-------------------------------------------------------
Request a free evaluation license for enterprise version
by sending an email to contact@saaras.io

Slack Channel         - https://slack.saaras.io
Getting Started Guide - https://getenroute.io/docs/getting-started-enroute-ingress-controller/
EnRoute Features      - https://getenroute.io/features/

Configure EnRoute For Linkerd

Meshing EnRoute with linkerd involves only setting one flag linkerd_enabled on the GlobalConfig



cat < < EOF | kubectl apply -f -
apiVersion: enroute.saaras.io/v1
kind: GlobalConfig
metadata:
  labels:
    app: web
  name: enable-linkerd
  namespace: default
spec:
  name: linkerd-global-config
  type: globalconfig_globals
  config: |
        {
          "linkerd_enabled": true
        }
EOF


globalconfig.enroute.saaras.io/enable-linkerd configured

This gets EnRoute ready to run along with Linkerd in meshed mode. This is the only configuration required to run EnRoute with linkerd with end-to-end encryption globally for all services.

By default EnRoute integration adds headers to help linkerd with routing choices in addition to letting Linkerd select endpoints for service. To change this default behavior and further customize it, the flags linkerd_header_disabled and linkerd_servicemode_disabled can be set to either disable the header or have EnRoute perform endpoint selection.

‍


---
apiVersion: enroute.saaras.io/v1
kind: GlobalConfig
metadata:
  labels:
    app: web
  name: enable-linkerd
  namespace: default
spec:
  name: linkerd-global-config
  type: globalconfig_globals
  config: |
        {
          "linkerd_enabled": true,
          "linkerd_header_disabled": false,
          "linkerd_servicemode_disabled": false
        }

Mesh EnRoute And Emojivoto Workload - Encryption Inside The Cluster

Inject Linkerd In Emojivoto Workload

The example workload wasn’t meshed into the linkerd mesh. To mesh it, we need to provide an annotation to the deployment to inject linkerd



kubectl get -n emojivoto deploy -o yaml | linkerd inject - | kubectl apply -f -



deployment "emoji" injected
deployment "vote-bot" injected
deployment "voting" injected
deployment "web" injected

deployment.apps/emoji configured
deployment.apps/vote-bot configured
deployment.apps/voting configured
deployment.apps/web configured

This installs linkerd proxy along with the example application pods

Inject Linkerd In The EnRoute Pod

Once EnRoute is ready to work with linkerd, we inject the linkerd proxy in the EnRoute pod



kubectl get -n demo deploy -o yaml | linkerd inject - | kubectl apply -f -



deployment "enroute-linkerd-meshed" injected
deployment.apps/enroute-linkerd-meshed configured

We use the Buoyant cloud to trace the status of linkerd injection and mesh formation. The picture below shows how the EnRoute deployment was mutated to add the linkerd proxy, it is captured a couple of screens on the bouyant cloud

The event screen also captures the linkerd proxy injection -

We can also use kubectl to check pods to see that linkerd got inserted in the pod



kubectl get pods -n demo -o jsonpath='{range .items[*]}{"\n"}{.metadata.name}{":\t"}{range .spec.containers[*]}{.image}{", "}{end}{end}' |sort



enroute-linkerd-meshed-5d9575d575-4bskt:        cr.l5d.io/linkerd/proxy:stable-2.11.1, saarasio/enroute-gwi:latest, redis:latest, envoyproxy/envoy:v1.21.1,

Encryption From Client To EnRoute

Install Service-Policy Chart To Expose The Service Externally With L7 Policies



helm install emojivoto-service-policy saaras/service-policy   \
	--set service.name=web-svc   \
	--set service.prefix=/ \
	--set service.port=80 \
	--namespace emojivoto \
	--set service.fqdn=enroute-linkerd-mesh.enroutedemo.com \
	--set service.enableTLS=false



NAME: emojivoto-service-policy
LAST DEPLOYED: Mon Mar 21 05:04:39 2022
NAMESPACE: emojivoto
STATUS: deployed
REVISION: 1
NOTES:
------------------ Configuration Graph -----------------------------------
Filters enabled for service [web-svc]
         |
    web-svc-80-luatestfilter
    kubectl edit -n emojivoto httpfilters.enroute.saaras.io web-svc-80-luatestfilter
         |
    web-svc-80-rl2
    kubectl edit -n emojivoto routefilters.enroute.saaras.io web-svc-80-rl2 (per-route ratelimit)
    kubectl edit -n emojivoto globalconfigs.enroute.saaras.io web-svc-rl-global-config (ratelimit engine global config)

𝙴𝚗𝚁𝚘𝚞𝚝𝚎 Community Edition Service Policy
----------------------------------------
Request an evaluation license for enterprise version - contact@saaras.io

Slack Channel         - https://slack.saaras.io
Getting Started Guide - https://getenroute.io/docs/getting-started-enroute-ingress-controller/
EnRoute Features      - https://getenroute.io/features/

Send some traffic



curl -I enroute-linkerd-mesh.enroutedemo.com



HTTP/1.1 200 OK
content-type: text/html
date: Mon, 21 Mar 2022 05:09:51 GMT
content-length: 560
x-envoy-upstream-service-time: 4
vary: Accept-Encoding
lua-filter-says: Hello
server: envoy



curl -I enroute-linkerd-mesh.enroutedemo.com

Enable End-To-End TLS By Enabling TLS For Downstream Client Connections

Install a certificate for example workload emojivoto and setup GatewayHost. An example for this can be found in the docs

The browser to EnRoute connection is encrypted using the certificate we installed on emojivoto app -

We can also see the workload and EnRoute are meshed in the “Meshed” column below

‍

We can also see that the incoming traffic shows up as Application TLS

The topology screen also displays the mTLS betwen services and direction of flow of traffic

Conclusion

EnRoute provides an extremely easy integration with linkerd.

EnRoute integrates easily with both linkerd and Istio service mesh. The integration with istio service mesh is covered in another article.

With an increased attack surface due to microservices, the need to secure and segment workloads using zero trust principles is paramount. EnRoute integrates with Service Meshes to enable zero-trust both at the cluster boundary and inside the cluster. Next steps to know more about Zero Trust, SPIFFE, Cryptographically Enforcable identities, identity based segmentation and integration with other service mesh can be found in articles below

‍