Linkerd vs Cilium

Five Key Differences in 2024

The service mesh landscape circa 2024 is a complex one with many different choices. In this comparison, we’ll walk you through the similarities and differences between Cilium Mesh and Linkerd, two service meshes available to Kubernetes users.

Just the facts

Created
CNCF status
Core component
Deployment
Mantainers
Known for
Linkerd
2016
Graduated CNCF project (as of 2021)
Ultralight Rust "microproxy"
Sidecar proxies
Buoyant
Simplicity, low cost of ownership
Cilium
2022 (service mesh), 2016 (CNI layer)
Graduated CNCF project (as of 2023)
Envoy (C++) general-purpose proxies
Per-node proxies
Isovalent (acquired by Cisco in 2023)
Use of eBPF
Created
CNCF status
Core component
Deployment
Mantainers
Known for
Linkerd
2016
Graduated CNCF project (as of 2021)
Ultralight Rust "microproxy"
Sidecar proxies
Buoyant
Simplicity, low cost of ownership
Cilium
2022 (service mesh), 2016 (CNI layer)
Graduated CNCF project (as of 2023)
Envoy (C++) general-purpose proxies
Per-node proxies
Isovalent (acquired by Cisco in 2023)
Use of eBPF

Cilium Overview

From the service mesh perspective, Cilium is essentially two projects: Cilium CNI, the Container Network Interface (CNI), and Cilium Mesh, the service mesh. Cilium started its life as a CNI implementation and added mesh functionality years later. The Cilium CNI is well-established, mature, and can be paired with any service mesh (including Linkerd). In this comparison, we'll primarily focus on Cilium Mesh.

Cilium is known for its popularization of eBPF, a feature of Linux that allows developers to run a restricted type of bytecode inside the kernel for maximum performance. For CNI, eBPF is useful for L3/L4 processing, and this powers much of Cilium CNI's feature set. However, eBPF is inherently limited in its ability to handle more complex L7 tasks such as parsing HTTP and gRPC traffic or handling mutual TLS.

Because of these limitations, Cilium Mesh relies heavily on Envoy proxy for service mesh features. In contrast to most service meshes, it deploys Envoy on a per-host (per-node) basis.

Linkerd Overview

Linkerd is an ultralight, security-first service mesh for Kubernetes. Linkerd adds critical security, observability, and reliability features to your Kubernetes stack with no code change required.

Linkerd was first created by Buoyant in 2016, coining the term "service mesh" and defining the category. Linkerd was donated to the CNCF in 2017, and became the first service mesh to achieve graduated status in 2023.

Linkerd is known for its simplicity and low maintenance cost. In contrast to most service meshes which use the C++ Envoy proxy, Linkerd uses a Rust-based "microproxy" which allows it to provide the operational and security benefits of sidecars without the complexity and resource consumption of Envoy-based meshes.

How do these projects stack up?

Linkerd and Cilium Mesh are very different beasts under the hood. Like most service meshes they both provide a set of features centered around reliability, security, and observability at the platform level. However, the design principles, architecture, implementation, and overall level of maturity are very different between these two projects.

In part this is because Cilium Mesh is a relatively new project, with even basic service mesh features such as gRPC load balancing—which Linkerd has featured since 2018—still in beta.

Below are the major differences between Cilium and Linkerd.

Difference I: Sidecar proxies vs Per-node proxies

While Cilium is known for its use of eBPF, the inherent limitations of this technology means that eBPF buys Cilium Mesh very little when it comes to service mesh functionality. As a result, the majority of service mesh features in Cilium are offloaded to Envoy.

Linkerd uses sidecar proxies. Cilium Mesh, by contrast, deploys one Envoy proxy per node. This per-node approach introduces significant operational and security concerns, including:

  • Large and indeterminate blast radius. The failure of a Cilium Envoy proxy on one node affects an arbitrary set of pods across services—whatever is scheduled on the node.
  • Unnecessary contention. High-traffic services in Cilium can easily choke low-traffic ones, and Envoy has no provisions for handling these "contended multi-tenancy" scenarios.
  • Poor security boundaries. All TLS key material for the pods on a node are held in memory by a single C++ process on the node, leaving the system vulnerable to Confused Deputy problems.

The per-node design was largely abandoned by the service mesh community in 2018, and it is unclear why Cilium chose to revive it almost half a decade later. You can read more about these considerations in our blog post, eBPF, Sidecars, and the Future of the Service Mesh.

Difference II: Attitude towards Security

While the project has only been around for a few years, Cilium Mesh already has a history of unfortunate security mis-steps that indicate an surprisingly immature approach to security.

In 2022, the authors of Cilium Mesh proclaimed support for mutual TLS but provided an implementation that was not, in fact, actually mutual TLS. When this was pointed out by security professionals and others in the community, Cilium Mesh rebranded its security approach as "mutual authentication". This "mutual authentication" approach had several deep-rooted flaws, and in 2023, was shown by outside parties to be deeply flawed at an architectural level. In 2024, Cilium Mesh addressed these flaws, but only partially, which is how things stand today.

This pattern of issuing grand security proclamations while delivering flawed solutions, which must be uncovered by third parties, is unfortunately not unique to Cilium. Cilium creator Isovalent has faced similar criticism about Tetragon, another project where they displayed similar behavior. In the case of Cilium, the insistence on developing custom security protocols is particularly puzzling given that Envoy has supported Mutual TLS—an accepted industry standard—for almost a decade.

By contrast, Linkerd's security approach has been grounded in accepted standards, well-tested libraries, and public reviews by independent third parties. Linkerd first shipped mutual TLS in 2019, building from well-tested security libraries such as Rustls. Linkerd has gradually improved this feature over the years, paying particular attention to minimizing configuration (zero config, on-by-default when possible) and other potential sources of human error. The Linkerd project has a clearly documented security policy that includes code review, code fuzzing, formal disclosure policies, and more. Linkerd periodically undergoes third-party security audits and publishes the unredacted results publicly.

In short: In contrast to Cilium Mesh, Linkerd has a long history of providing best-in-class security improvements to its adopters, and follows industry standards and best practices to do so.

Difference III: Performance

While eBPF has been described as a performance boost, Linkerd remains significantly faster than Cilium. In their 2024 third-party benchmark, Service Meshes Decoded: a performance comparison of Cilium vs. Linkerd vs. Cilium, UK-based cloud consulting firm LiveWyer concluded that "Linkerd is the fastest and most efficient mesh among all those tested."

Linkerd’s state-of-the-art, ultralight Rust “micro-proxy” is designed for the service mesh use case and is highly optimized to handle this traffic.

chromiumProject

Linkerd vs Istio vs Cilium 2024 benchmarks originally published by LiveWyer

bulb
Further reading: Linkerd vs Istio benchmarks
bulb
Further reading: Under the hood of Linkerd's state-of-the-art Rust proxy

Difference IV: Future-proofing

Cilium Mesh relies heavily on Envoy for L7 routing tasks. Envoy is a general-purpose proxy written in C++,  a legacy language known for its memory-related security vulnerabilities. By contrast, Linkerd’s data plane is built in Rust, a language that avoids entire classes of memory-related CVEs through advanced memory management.

This isn't just theoretical. Google’s Chromium project found that 70% of serious security bugs are due to memory safety problems. Microsoft also arrived at the same figure, saying: "One astonishing fact sticks out: the majority of vulnerabilities fixed and with a CVE assigned are caused by developers inadvertently inserting memory corruption bugs into their C and C++ code." (From A proactive approach to more secure code.)

Envoy-based service meshes have a continuous stream of Envoy CVEs due to the nature of C++, and Cilium Mesh adopters can expect these vulnerabilities to continue. With Linkerd, these CVEs are a non-issue.

bulb
Further reading: Why Linkerd doesn't use Envoy
chromiumProject

Google’s Chromium project found that 70% of serious security bugs are due to memory safety problems. Linkerd avoids these problems by using Rust.

Difference V: Level of Complexity

Linkerd’s core design philosophy is about minimalism. Linkerd is especially focused on reducing operational complexity, which is the human toil involved in maintaining, operating, and being on-call for a production service mesh. To accomplish this, Linkerd takes a multi-pronged approach, including:

  • Using a purpose-built Linkerd "microproxy" which is designed with operational simplicity as a goal from the start;.
  • A sidecar model that has clear operational and security boundaries (e.g. each microproxy is responsible for a single application replica, and their lifecycles are tied together);
  • Minimizing configuration and

This isn’t the case for Cilium, which builds its functionality out of Envoy, WireGuard, IPSec, and eBPF—all complex projects maintained by separate teams. Cilium's per-node implementation introduces operational complexity, as described above. For example, each Cilium Envoy proxy deployed on a node will handle traffic from the random set of pods deployed to that node, so any Envoy crash, error, or downtime (e.g. during upgrade) will affect an unpredictable part of the application. Cilium Mesh's configuration surface is much larger than Linkerd’s – in fact, users of Cilium Mesh are allowed to directly alter the configuration of  Envoy itself, which opens the door to whole new classes of misconfiguration and security errors.

At an empirical level, this complexity is also reflected in the size and nature of the code. The McCabe cyclomatic complexity score for Cilium Mesh alone (without Envoy) comes in at about 56K; for Envoy, it’s about 44K. By comparison, the same complexity for Linkerd is 11K for the control plane and the proxy combined. In other words, by this measure, the code complexity of Linkerd is approximately one tenth of that of Cilium.

bulb
Further reading: Linkerd's design principles
bulb
Further reading: Announcing Linkerd's CNCF Graduation

The bottom line

While Cilium CNI is popular and has often been deployed alongside Linkerd, Cilium Mesh is a different story. Given Cilium Mesh's history of poor design decisions and laissez-faire attitude towards security, it is hard to recommend Cilium Mesh as a viable service mesh choice for organizations that value security.

Want to give Linkerd a try? You can download and run the production-ready Buoyant Enterprise for Linkerd in minutes. Get started today!