skip to Main Content

Andrew Seigner

Software Engineer at Buoyant

Using Linkerd to add visibility and control to microservices on Amazon ECS

Linkerd, our open source service mesh for cloud native applications, adds reliability and visibility to microservices by managing all of the internal communication between services. Deployed as a set of transparent layer 5/7 proxies, the Linkerd service mesh provides a consistent, global layer for monitoring and controlling all internal, service-to-service traffic within an application. (For more on the service mesh model, read William’s article, What’s a service mesh? And why do I need one?)

One of Linkerd’s strengths is its ability to integrate with many different environments (and to allow you to bridge environments!). In previous posts, we’ve covered how to use Linkerd with Kubernetes and DC/OS. In this post, we describe how to use Linkerd with Amazon ECS.

All commands and config files referenced in this post may be found in the linkerd-examples repo.

Overview

This post will show you how to set up Linkerd as a service mesh on ECS, using Consul for service discovery, linkerd-viz for monitoring, and a hello-world sample app, as seen in the diagram below:

Linkerd: A Service Mesh for ECS

Initial Setup

This post assumes you have already configured AWS with the proper IAM, key pairs, and VPCs for an ECS cluster. For more information on these topics, have a look at Amazon’s Setting Up with Amazon ECS guide.

Set a key pair you will use to access your instances, or omit the parameter to forego ssh access:

Next, create a Security Group:

Set up Consul

For demonstration purposes, we run a single Consul server outside of the ECS cluster:

Set up ECS

Create a new cluster

Create a Role Policy

Register Task Definitions

Create Launch Configuration

This step defines a Launch Configuration. We configure our ECS cluster to boot linkerd and consul on each ECS node.

Note ecs-user-data.txt dynamically generates config files for each of linkerd, consul-agent, and consul-registrator, using data specific to the ECS Instance it is running on.

Create an Auto Scaling Group

This step actually creates the EC2 instances, based on the Launch Configuration defined above. Upon completion, we should have two ECS nodes, each running linkerd, consul-agent, and consul-registrator.

We name our instances l5d-demo-ecs so we can programmatically find them later on.

Deploy the hello-world sample application

Now that all our foundational services are deployed, we can deploy a sample app. The hello-world task is composed of a hello service, a world service, and a world-v2 service. To demonstrate inter-service communication, we configure the hello service to call the world service via linkerd.

Note that we have deployed two instances of hello-world, which results in two hello containers, two world containers, and two world-v2 containers.

Did it work?

If everything deployed correctly, we should see 8 tasks running in our ECS dashboard:

ECS Tasks

We select an arbitrary ECS node, via the l5d-demo-ecs name, then curl the hello service via linkerd:

Now test routing:

If everything worked correctly, we should get a reply from the hello service, with data from the world service.

View Linkerd and Consul UIs:

Test dynamic request routing

One of Linkerd’s most powerful features is dynamic request routing. Here we’ll demonstrate routing a single request to the world-v2 service, rather than the default world service:

The request flow we just tested:

By setting the l5d-dtab header, we instructed linkerd to dynamically route all requests destined for world to world-v2, even though the request initially transited through the hello service.

Per-request routing with linkerd

For more information, have a look at Dynamic Request Routing.

Monitoring the services

Linkerd instruments all traffic and exposes these metrics, including top-line service metrics like success rates and latencies. By using the Linkerd service mesh, we can automatically collect these valuable metrics without having to modify our application!

Since Linkerd itself is purely distributed, however, we need to aggregate these results. For convenience, we provide a simple open source package, linkerd-viz, which can collect and displays metrics for all linkerd’s running in a cluster.

Prior to deploying linkerd-viz, let’s put some load through our system:

Now deploy a single linkerd-viz instance:

Now bring up the linkerd-viz dashboard:

If everything worked correctly, we should see a dashboard like this:

ECS linkerd-viz

Conclusion

In the above post, we’ve show how to deploy Linkerd on ECS to provide a service mesh: a dedicated layer for managing and monitoring all service-to-service communication. This is only the tip of the iceberg: Linkerd can also be used to merge ECS, Kubernetes, DC/OS, and other environments into a single logical service namespace; to implement complex traffic patterns like hybrid cloud and multi-cloud topologies; and much more.

Credits

The examples and configurations in this post drew heavily from some excellent blog posts. Have a look at them for other approaches to running ECS:

Further reading

There’s a lot more that you can do with Linkerd. For more details about this setup, see Getting Started: Running in ECS. For all commands and config files referenced in this post, see the linkerd-examples repo.
For more information about configuring Linkerd, see the linkerd Configuration page. Finally, for more information about linkerd-viz, see the linkerd-viz Github repo.

We hope this post was useful. We’d love to get your thoughts. Please join us in the Linkerd Discourse forums and the Linkerd Slack channel!