Background:

First, With the rush of releasing Binge, endless zoom meetings and editing all those k8s yaml files, I have been itching to write a for loop (ie..do some coding) .

Second, I am tired of parsing/grepping/awking those kubectl output but I still had to go to my newrelic UI to see which pod is running on my EKS spot vs ondemand nodegroup and I am a commandline kinda guy and its too difficult to do in commandline.

So I decided to write a small tool just for myself, but didn’t feel like doing it in groovy which is my default scripting language (hey I come from java!). A few month ago, I picked up a GO book but realized it will take me more than a weekend to be productive in it and frankly I didn’t like the syntax. I like my language to be more abstract (again, I come from java!), so I picked up a python cheatsheet and started hacking.

I must admit its a great stress reliever when you are trying to deploy ~1000 pods running ~100 microservices in a fresh new EKS cluster for the first time in your organization and the whole internet is there to curse you if the platform is not delivering 🙂 (thats a different story for another day)

Result:

I hacked together a few lines of python code (again, my python skills are those coming from a cheatsheet and random googling over a weekend so no judging). But the output kk (kubectl++) came out to be quite handy and I find myself using it very often so I though I’d put it in my github.

https://github.com/sajid-moinuddin/kk

The idea behind is simple, you get all node info, all pod info and programatically join them and create a virtual PodNode Resource (just like any k8s resource). Now you can do things like:

#get me pod,namespace,nodename,spot/ondemand,pod-resources,restartcount from namespace x/y/z and by the way, exclude the daemonsets and format nicely as I say 

kk get podnode  
-f 'metadata.namespace=streamtech/content/commerce' 
-e 'metadata.owner_references[0].kind=DaemonSet' 
-o 'pod_name:62,pod.status.phase,node.metadata.labels.lifecycle,namespace,node.metadata.name:15,pod.spec.containers[0].resources.requests:35,pod.spec.containers[0].resources.limits:35,pod.status.container_statuses[0].restart_count'

offers-api            Running             spot                commerce            ip-10-100-58-11    {'cpu': '2', 'memory': '4Gi'}          {'cpu': '6', 'memory': '4Gi'}          0                   
offers-api            Running             spot                commerce            ip-10-100-62-11    {'cpu': '2', 'memory': '4Gi'}          {'cpu': '6', 'memory': '4Gi'}          0                   
offers-api            Running             spot                commerce            ip-10-100-63-3.    {'cpu': '2', 'memory': '4Gi'}          {'cpu': '6', 'memory': '4Gi'}          0                   
offers-api            Running             ondemand            commerce            ip-10-100-60-74    {'cpu': '2', 'memory': '4Gi'}          {'cpu': '6', 'memory': '4Gi'}          0                   

(venv) ➜  kk git:(master) kk -hk(kubectl)++ ... only better
Usage:
kk get (podnode|pn) [options]

Options:
    -h --help    show this
    -l STRING    label selector
    -f STRING    field selector
    -e STRING    exclude
    -o output    pydash format plus padding info, ie. node.metadata.name:62
    --offline    do not fetch new data, work on the last fetched data
    --json       print the -o elements in json format
    -w           watch mode

Example:
#kk get podnode -f 'metadata.namespace=streamtech/content/commerce' -e 'metadata.owner_references[0].kind=DaemonSet' -o 'pod_name:62,pod.status.phase,node.metadata.labels.lifecycle,namespace,node.metadata.name:15,pod.spec.containers[0].resources.requests:35,pod.spec.containers[0].resources.limits:35,pod.status.container_statuses[0].restart_count' | sort -k3

#kk get podnode -f 'metadata.namespace=streamtech/content/commerce' -e 'metadata.owner_references[0].kind=DaemonSet' -o 'pod_name:62,pod.status.phase,node.metadata.labels.lifecycle,namespace,node.metadata.name:15,pod.spec.containers[0].resources.requests:30,pod.spec.containers[0].resources.limits:30'

#kk get podnode -f 'metadata.namespace=streamtech/content/commerce'  -e 'metadata.owner_references[0].kind=DaemonSet'   -o 'pod_name:62,node.metadata.labels.lifecycle,namespace,node.metadata.name:62' -w

#kk get podnode -f 'metadata.namespace=streamtech/content/commerce'  -e 'metadata.owner_references[0].kind=DaemonSet'   -o 'pod_name,node.metadata.labels.lifecycle,namespace,node.metadata' --json

I have used the python-builder library which looked like maven. I am yet to spend any time to understand how to publish this as a pip module (I havent spend much time understanding python modules and packages yet…)

NOTE:
>> this can be a very expensive operation on your kubectl api so use this with care in prod environment.
>> It also has a –offline mode (this works on the last fetched data) which can be used to first get a dump of the data and look at them at different angel
>> You can also use a –json format that will dump the raw jason