Microservices with Cilium Service Mesh on Amazon EKS

Kubernetes has become the go-to platform for deploying microservices, and with it comes the challenge of service communication, observability, and security. While traditional service meshes like Istio and Linkerd use sidecars to address these, Cilium Service Mesh brings a sidecar-less approach using eBPF, setting a new standard for performance and simplicity. This is a first-hand account of our journey implementing Cilium on AWS EKS, the challenges, the wins, and most importantly — why this could be a game-changer for your Kubernetes workloads.

What is Cilium & Why It’s Different?

Cilium is a modern service mesh and CNI (Container Network Interface) built on eBPF, a Linux kernel technology enabling highly performant and secure networking.

Unlike Istio or Linkerd, Cilium doesn’t rely on sidecar proxies — meaning:

• No resource-hungry sidecars
• Kernel-level observability & security
• Higher scalability and throughput

Our Experiences on deploying Cilium Mesh and achieved Goals

• Deploy Cilium Service Mesh on Amazon EKS
• Validate sidecar-less architecture
• Ensure real-time observability using Hubble
• Measure performance against Istio
• Integrate Prometheus & Jaeger
• Enable multi-cluster support

Challenges We Faced with Istio (Before Cilium)

Why We Chose Cilium Over Istio/Linkerd

Architecture Comparison

• Hubble: Provides real-time observability into service communications using eBPF.
• Prometheus: Collects metrics and telemetry data for performance monitoring.
• Jaeger: Enables distributed tracing to analyze request flows across services.
• Cilium Agent (Node level): Enforces eBPF-based security and networking policies directly in the kernel for node traffic.
• Cilium Agent (Pod level): Manages pod-level traffic control without the need for sidecar proxies.
• eBPF-based policy enforcement: Provides fine-grained control over network and application-level traffic from within the kernel.
  • Load balancing
  • DNS-aware policies
  • Security rules
  • Traffic routing

Our Experience: What Made It Worthwhile:

When I first proposed replacing a traditional mesh like Istio with Cilium, I received a fair bit of skepticism, “Why fix something that isn’t broken?”

But here’s what we discovered:

• CPU usage dropped by 30% under the same load
• No need to manage or debug sidecars
• Hubble gave us complete traffic visibility without sidecar noise
• Faster pod start times (no injection delay)
• Simpler operational model — less YAML, fewer Helm values

And the best part, We didn’t lose any functionality. In fact, we gained clarity, performance, and maintainability.

Add-On Simplification: What Cilium Replaces in EKS

Cilium With vs. Without VPC CNI

If you are building microservices on Kubernetes and tired of sidecar complexity give Cilium a shot.

You’ll be amazed at the simplicity, power, and performance gains you get. This isn’t just a proof-of-concept — it’s the future of cloud-native networking and service mesh.

Our Experience deploying the Logistics Customer

What Worked Well

• Setup via Terraform made the entire provisioning and deployment reproducible and version-controlled.
• No sidecar injection issues, unlike previous challenges faced with Istio.
• Performance improved — CPU usage dropped ~20% on test workloads.
• Observability was exceptional using Hubble UI, without requiring external agents.
• kube-proxy replacement (optional but tested) made traffic routing more efficient.

Things to Consider

• Learning Curve: Understanding eBPF concepts and kernel-level observability takes time.
• AWS ALB/NLB support requires manual tweaks when not using VPC CNI.
• VPC CNI Chained Mode introduces partial visibility and limits eBPF optimizations.

Conclusion:

Cilium Service Mesh on Amazon EKS isn’t just another tool in the Kubernetes ecosystem. it represents a fundamental shift in how we think about networking, observability, and security in microservices.By eliminating the sidecar model and harnessing the power of eBPF, Cilium delivers superior performance, reduced complexity, and deeper visibility  all while integrating natively with Kubernetes.

In our PoC, the results were clear:

• Faster performance
• Rich, real-time observability
• Strong network security policies
• Simplified architecture
• True multi-cluster readiness

For teams aiming to scale efficiently, minimize resource overhead, and streamline operations, Cilium is not just an alternative, it’s an upgrade.  Contact us today

Date: 09/6/2025  :   Written by –