On-Premise Parse Server Deployment.

Task Overview :

Here I’m trying to provide an solution for on-premise application deployments to enterprise clients and here i’m doing it for Atlan, So we are going to deploy Parse Server on a cluster of machines.

This is the list of some foreseeable problems related to an on-premise deployment, and i’m trying to solve it in this task!

• Ease of clustered enterprise level deployments.
• Incremental remotely triggered application updates.
• Easy remote debugging.
• Health Alerts and Monitoring.
• Application Security (with source code protection).
• Disaster management.

Installation / Usage Instructions :

So basically for reaching the solution requirement i’m using this below given Platform and software's:

• Docker: Docker a tool designed to make it easier to create, deploy, and run applications by using containers.
• Google Kubernetes Engine: GKE clusters are fully managed by Google Site Reliability Engineers, ensuring your cluster is available and up-to-date. GKE runs on Container-Optimized OS.
• Google Cloud Platform: Google Cloud Platform is a suite of public cloud computing services offered by Google. The platform includes a range of hosted services for compute, storage and application development that run on Google hardware.
• Rancher: Rancher is an open source project that provides a container management platform built for organizations that deploy containers in production.
• Helm & Helm Charts: Helm helps you manage Kubernetes applications and Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.
• Prometheus and Grafana: Prometheus is an open-source systems monitoring and alerting toolkit and Grafana is an open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources.
• Weave Cloud: Weave Cloud is an automation and management platform for development and DevOps teams.
• Velero: Velero is an open source tool to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.

I am using Google Cloud Platform Compute-Engine VM instances for this task, So lets gets started for the deployment.

Step-by-Step Detailed Overview:

Step 1:

First I’m creating a Centos VM using Google Cloud Platform and then installing Docker on it. Once done with installing the Docker setup then i’m creating a Rancher container.

docker run -dti --restart unless-stopped -p 80:80 -p 443:443 rancher/rancher

using the above given link we will get running Rancher as container on port 80. We can access the web UI of Rancher using VM external IP.

Then i’m setting up a admin password for further use.

Step 2:

After successfully logged in to Rancher UI. I am creating a Kubernetes cluster for deploying Parse Server. Here I am using Google GKE (Google Kubernetes Engine) for creating a cluster with 3 nodes.

Then in Rancher Global tab select Clusters then Add Cluster and select Google GKE.

After selecting the Google GKE we will need GCP service account JSON Private key. Then go to GCP console and follow these steps for creating Key.

Then Upload key to ‘Read from a File’ section and fill up the required fields. after that you will see active running cluster i.e provided by Google GKE. In my case I named it as ‘Atlantask’.

Step 3:

In this step I am going to deploy the Parse Server application in ‘Atlantask’. Parse Server is a application that needs Database pod also, So here I am using MongoDB server in one pod and then installing Parse Server application in other pods.

So for this here I am using Helm Charts. Below are the steps to deploy Parse using Helm Chart.

Go to Rancher Global then Tools then Catalogs and add a Helm catalog by entering this url:

https://kubernetes-charts.storage.googleapis.com/

Then wait for it to turn into active state.

Then Go to Rancher Global then in Apps and click Launch button.

After that in the search bar, type “Parse” and you will see Parse Helm Chart available there.

After that click on the Parse Helm Chart. Then type the name you want I am typing parseserver and also select Target Project Default.

Then click on Launch. You will see it Active in some sec.

Then open Parse Server you can see 2 active pods one for MangoDB and one for Parse Server, And finally Parse Server is deployed Successfully on Kubernetes Cluster.

Step 4:

For remotely triggered application updates I am using Weave Cloud for setting up a pipeline for this project. For making changes directly from the Source Repository via CI/CD pipleline.

So, For this I am going to cloud.weave.works to manage CI/CD. In my case I already connected my cluster to Weave Cloud and named it is Atlan Task.

After successfully connected to the Weave Cloud. You will see all the live metrics, resources and graphs.

Now for the CI/CD click on the settings icon and then configure. Paste the Git source repository where configuration YAML is present.

Step 5:

In this step I am setting up Prometheus Monitoring and enabling Grafana Dashboard for live metrics monitoring.

For this go to Rancher Global then Clusters then Atlantask and then Dashboard

After that click on Enable Monitoring to see live metrics it will also enable Grafana Dashboard for you. In my case I already done that part, You will see enable option there in highlighted yellow field.

After clicking on any of Grafana icons. It will launch Grafana Dashboard. You will see beautiful graphical UI of Dashboard.

Step 6:

And in this last step I am setting up the Disaster management. After searching for backup tools I was able to find ‘Velero’ by Heptio to help me with backing up and restoring my Kubernetes cluster resources as well as persistent volumes.

Velero helps you with:

• Backup and restore of your Kubernetes cluster.
• Copy cluster resources from one cluster to another
• Replicate your production environment for development and testing environments.

For installing Install velero-cli. In my case I am installing it in GKE.

brew install velero

Create a Google Cloud Storage bucket.

gsutil mb gs://<gke-cluster-migrate-velero-placeholder-name>

Create service account / permissions / policies and then add credentials to your GKE Cluster.

To backup my entire cluster, I have used:

velero backup create atlantaskbk

To see the status of your backup just run:

velero get backups

This tools have saved me tons of hours as well as greatly simplified the overall process of mapping, backing up and restoring Kubernetes resources .

Use Cases, Edge Conditions and Assumptions

1. It is assumed that Source Code Protection is all managed by Professional coding team.
2. Configuration.yaml file in GitHub is only a sample code file. It is assumed that it can be used for automatic triggered updates.
3. It is assumed that clusters are managed by load balancers and can be scaled as per loads.
4. It is assumed that for saving up backup data we have good storage capacity and also it is stored in different Zone.

Workflow

All the actions of this is task are sequentially taken from the assignment problem document.

Ease of clustered enterprise level deployments:

For getting parse server up on clusters I used Rancher. Rancher runs inside a Docker container and gives UI for all the On-Prem cluster deployment requirements.

Incremental remotely triggered application updates:

For remotely triggered incremental updates, Here I used Weave Cloud because it also provides CI/CD.

Weave Cloud is an automation and management platform for development and DevOps teams.

Easy remote debugging:

For remote debugging, Here I used Prometheus because the metrics collected by Prometheus, Grafana are used for remote debugging only by cluster administrators or DevOps engineers.

Health Alerts and Monitoring:

Here also I used Prometheus and Grafana i.e provided by Rancher we didn't required to install any monitoring tool by ourselves, just we have to enable the monitoring api from the rancher.

And for Alerts here I used Weave Cloud because Alert system and Health Check is much easier here. The operation team members receive email alerts automatically when something is down in cluster. they don’t have to configure the notifiers here.

for testing you can send yourself an test email of verify it’s working or not.

here is the email that I received from Weave Cloud.

Application Security (with source code protection):

As we all know that in today’s age of malware and increasingly persistent hackers, software developers are often faced with more challenges like this. For prevent our code there are some precautions, let’s review just a few basic ones that you can consider.

• Copyright and patents: As we mentioned, lapsed copyrights are among the biggest reasons why many companies aren’t protecting their source code like they should. Don’t make the same mistake. Be sure that all your software and coding is protected by copyright law and necessary patents.
• Encryption and monitoring: Your source code is one of your most prized commodities on the programming side of things. So ensure that it includes the ability to encrypt the relevant data both in transit and at rest, as this will go a long way toward keeping it away from prying eyes.
• Redundancy: Although many effective ways to protect your source code are on the table, this is one case in which the more methods you have in place, the better off you’ll be in the long run. In many cases, businesses opt to limit redundancy, since it amounts to an excess expense of resources.

Disaster management:

For Disaster Management I am using Velero by Heptio. Velero helps you with:

• Backup and restore of your Kubernetes cluster.
• Copy cluster resources from one cluster to another
• Replicate your production environment for development and testing environments.

Data formats and Reporting

All the data that Prometheus and Grafana is providing from Rancher server, this data can be used by DevOps engineer or a system Admin for analyzing various statistical data from the running cluster. These statistics can then be used for processing information about the health of the application clusters, take productive measures checks and finally debugging the issues if any bugs are founded.

Metrices received from Prometheus monitoring

Performance and Scaling

Generally Scaling is used when traffic of work loads are increased and here we have Rancher for making it really easy to perform scaling by just few clicks.

For this go to Rancher then Atlantask then System then Apps, and click on parseserver application.

then scale it to 1 from 2

Now you can see your workload is increased with 2 running pods.

Unresolved issues

Incremental remotely triggered application updates is not currently working because I had put a sample configuration file on GitHub, It’s actually because I am not confirmed in providing the updates to Kubernetes Cluster or to the application deployed in the Parse Server.

Here I am using GitHub as a version control system.

Link : https://github.com/Prathmesh98/atlantask