skip to Main Content

Andrew Seigner

Software Engineer at Buoyant

Our friends at Mesosphere recently announced Kubernetes support for DC/OS. As big fans of Kubernetes, Mesos, and Istio, the natural question in our minds was: does this mean we can run Istio on Mesos already? As in, today?

This isn’t just a thought exercise. A big part of Buoyant’s mission is to bring service mesh infrastructure to companies around the world, not just those running Kubernetes. We recently announced Linkerd Istio support, and are heavily invested in bringing the service mesh to everyone.

The tl;dr is: yes, you can run Istio on Mesos this way. Though it would be pretty silly to call this a full-fledged “Istio for Mesos” solution, in an upcoming blog post we’ll show how you can use Linkerd to bridge DC/OS and Kubernetes and make this a bit more realistic of a scenario.

Let’s get going!

As an introductory step, this post will guide you through the process of spinning up a fresh DC/OS cluster in AWS. We will then deploy Kubernetes on top and use it to set up the Istio service mesh. The following steps assume you have the AWS CLI installed. For more information consult the AWS CLI documentation.

Deploy DC/OS

First we need to set up DC/OS on AWS. To do this, deploy the default configuration with a single master. For full details on deploying DC/OS, have a look at the DC/OS installation documentation.

Download the official DC/OS CloudFormation template:

Set your AWS Keypair, and deploy the CloudFormation template. We set SlaveInstanceCount to 7 because Kubernetes needs more resources than the standard 5 instance configuration. Alternatively you could deploy DC/OS on instances larger than the default m3.xlarge.

Once fully deployed, retrieve the hostname for the DC/OS master, and use that to configure your DC/OS CLI.

Deploy Kubernetes

Next, setup Kubernetes on your new DC/OS cluster. For more information on running Kubernetes on DC/OS have a look at the Quickstart guide for Kubernetes on DC/OS.

Start by installing the beta-kubernetes package from the DC/OS catalog.

You can then see all the Kubernetes deploy tasks running in DC/OS.

Once the Kubernetes deployment is complete, you’ll need to configure kubectl to connect to it. Since the Kubernetes package installs onto private DC/OS nodes, we need to tunnel through our DC/OS master to reach it.

First, get the Public IP of your DC/OS master.

Now, start a tunnel on localhost:9000 through the DC/OS master to the Kubernetes API server, and then configure kubectl.

Confirm the cluster is operational.

The beta-kubernetes package in DC/OS does not include kube-dns or the kubernetes-dashboard.  Add those now.

You now have a fully functional Kubernetes deployment on your DC/OS cluster.

Deploy Istio

With a fully functional Kubernetes cluster, you can now setup Istio. You will need to make some modifications to Istio that are specific to DC/OS. There’s currently a known issue with the beta-kubernentes DC/OS package not supporting service accounts. By default, the Istio components use service accounts to find and connect to the Kubernetes API Server. As a workaround, you need to provide modified versions of Istio and Kubernetes (sample) app configs that include a kubeconfig file containing instructions for how our components connect to Kubernetes.

kubeconfig file, defined in our istio ConfigMap object

Modified Kubernetes config to load a kubeconfig file

These instructions are based on the Istio installation instructions.

First, download the Istio install files.

Now deploy Istio, metrics collection, and a sample app onto your Kubernetes cluster.

You should now be able to see the sample app.

To confirm metrics collection is working, add some load to the sample app and verify traffic via the Grafana dashboard.

Congratulations! You now have an application running on Kubernetes with DC/OS in AWS backed by the Istio service mesh.

Conclusion

This was a fun exercise in stacking technologies together, and an interesting demonstration of how various infrastructure layers can compose. In the future, we’ll show how Linkerd running on both Kubernetes and DC/OS can merge service namespaces and allow operators to migrate services between them, which is a more realistic of a use case for a cross-orchestrator service mesh. In the meantime, stay tuned and please feel free to stop by the Linkerd discussion forums, the Linkerd community Slack, or just contact us directly!