Automating K8s Mongo cluster on AWS IaaS including Prometheus exporters for monitoring

Creating a configurable Kubernetes cluster on AWS EC2 instance with MongoDB and Prometheus.

There is EKS in order to jump start Kubernetes(K8s) in AWS, though it is still evolving in few aspects. If the intent is to leverage many of bleeding edge flexibilities and features of K8s then the choice is to have it on AWS EC2.

This article provides high level steps to deploy the following along with link to Github repo with all scripts and configurations and readme with detailed instructions:

a) K8s on AWS EC2 with your configuration choices of instance types and VPC subnets leveraging ‘kops’

b) mongo cluster with sharding, including config and router with configurable choice of number of replicasets and shards

c) prometheus with the necessary exporters of node and mongo

The scripts and configurations are available through ‘1CloudHub GitHub’ public repository here.

Who can use this?

This will be helpful for developer, Ops or DevOps person to create kubernetes cluster in AWS IaaS and install mongoDB with minimal manual work.

Key benefits:

  • Scale the cluster nodes after launch.
  • Scale the mongoDB shards and replicaset based on the requirement.
  • Exporters can be customized and implemented as required.

Walk through of high level steps

PrerequisitesThe git repository instructions covers the Pre-requisites including IAM access role for the client instance from where the script to be executed, s3 buckets and installation of AWS cli, kops, kubectl and anscible.

K8s Cluster & Mongo Cluster: First it helps to deploy K8s on AWS EC2 leveraging kops. Once the cluster is up and running mongo db cluster will be deployed on top of the kops.

Kindly note, mongodb clusters are launched using Statefulsets and hence will have its own EBS volumes created and which provides additional reliability for the mongodb cluster on top of K8s.

Prometheus & exportersFinally this deploys prometheus along leveraging prometheus operator along with grafana. The required exporters including the mongodb exporter and node exporter are added along and starts to monitor mongodb cluster, k8s and nodes on which they run on.

Happy deployment folks.

Leave your feedback and feel free to extend and make the scripts suit your needs

Repository details:

Creating a configurable kubernetes cluster on AWS EC2 instance with
MongoDB and Prometheus. The following content will walk you…
github.com

 

Written by,

Geetha Pandiyan &

Umashankar N

Sharing is caring!