My story with Jenkins-X is a Love/Hate one. First time I saw a demo of preview environment on kubernetes with jenkins-x last year, I was hooked :
I have never seen anything like it and I still dont know if there is anything in the market that is close to it even now. At the same time I was reading the book Accelerate and the book world found its match in real world. It was easy to quickly get it up and running and do a few demo and have some stakeholder buy-in into GitOps. We even went live in production with jenkins-x with some of the batch jobs in datalake (lower-risk). However when the honeymoon period is over and we wanted to roll out across 10+ teams, the reality hits and we found productionizing jenkins-x on a non GCP platform is not only HARD but almost impossible. I have had the good fortune of talking to some of the jenkins-x core product people and until recently they have been focused purely on GCP.
The CI part of jenkins-x is too powerful to pass on so we marched on! Many weekend and debugging GO sourcecode and countless hours in jenkins-x slack channel, we got jenkins-x working on EKS cluster ( this has changed since last year when all this happened and the installation option is so much better with jenkins-x boot). We were probably 1 yr too early into the product. But boy am I happy for that investment! Once you get one up and running, there is nothing like it. However, the jenkins-x installation I had was very much a PET. I got it installed and I had a lot of NOTES for it but its not GITOPS itself. So I didn’t want to have it installed 20x time in all the EKS clusters we got per business units plus dev/staging etc. We’d need a few people to maintain those 20 jenkins-x installation. Anything that doesn’t support native gitops style deployment is a massive maintainance overhead.
So we installed jenkins-x in 1 cluster only (We call it GitOps cluster) and do all our CI pipeline there and it produces Helm + docker images following gitops best practices etc. Jenkins-x has a helm chart deployer called “Environment Controller“, but at the time we tested it about 5 month ago, it wasn’t working with bitbucket in EKS. So the search continued. There are quite a few tools in the market to do gitops (weave flux is a good one) , but we stumbled on ARGO by chance (I was playing with KUBEFLOW and kustomize+argo is their main CD tool). As nice surprise, Argo could install the helm environment repo jenkins-x creates out of the box with 0 modification (you need to install argo 1.3.x with helm hook support)
If you want to see Argo in action, checkout their own CD projects:
https://cd.apps.argoproj.io/ (use your github to login)
https://github.com/argoproj/argoproj-deployments/tree/master/argocd
A really really awesome feature is auto-generated helm graph , its really awesome if you have devs who are not that familiar with kubectl but they can still get started by visualizing the deployed helm charts. Here is how our Nginx stack looks like : (You can read more about this here: ( https://medium.com/@sajid2045/aws-eks-ingress-option-alb-nginx-fc64a4a5ea9f )
And here is how the Git Repo vs Cluster State looks like:
Finally! This is how it looks like E2E (https://sajidmoinuddin.files.wordpress.com/2019/12/gitopse2e-1.png):
Lazy Programmer's Shortcut 
