Config Connector is a great tool for managing Google Cloud resources using Kubernetes manifests. We show you how to achieve enhanced control by replacing the add-on installation with a manual one.
For enhanced control of the Config Connector version, replace the add-on installation with a manual one
With Config Connector, you can manage GCP resources using Kubernetes manifests. It is a great tool created by Google and available as an add-on installation on GKE clusters ( that fit the requirements). These are just some of its advantages and use cases:
- Creating underlying infrastructure for a service deployed to GKE using the same pipelines and paradigms (it's just extra YAML manifests)
- Using a central "control" cluster to spin up other clusters, effectively achieving Infrastructure as Code using Kubernetes
- Allowing for for spinning up infrastructure (think dev environments or even a new production cluster) with a single kubectl apply
Config Connector works by deploying an Operator StatefulSet (configconnector-operator-system/configconnector-operator), which in turn creates all CustomResourceDefinition resources (CRDs) and all supporting workloads in the cnrm-system namespace.
As Config Connector is constantly evolving, new features are often added as new versions are released (a new resource type to be managed by Config Connector or even a field added to a CRD). Unfortunately, the version of Config Connector installed using the GKE add-on is tied to the GKE master version. This means that, in order to update the Config Connector version installed by the add-on, you'll need to update your GKE version (probably to the next minor version, at least), which is not always desirable. Furthermore, the bundled version is usually a few versions behind the latest available Config Connector version, which means that even then you might not get a new feature or a fix added in the latest version.
From testing, I can tell that the following versions of Config Connector are installed with the GKE add-on:
- v1.69.0 for GKE 1.21.x
- v1.71.0 for GKE 1.22.x
The latest Config Connector version at the time of writing this is v1.83.0.
Manual installation
To improve control of the Config Connector version on a GKE cluster, you can replace the add-on installation with a manual installation [1]. Note that doing so transfers responsibility for updating Config Connector to you, as it will not be updated automatically with your GKE cluster anymore.
Replacing the add-on installation with a manual one
1. disable the add-on installation on the cluster:
gcloud container clusters update <CLUSTER_NAME> --update-addons ConfigConnector=DISABLED
2. delete the Config Connector operator StatefulSet:
kubectl -nconfigconnector-operator-system delete statefulset configconnector-operator
- 3. install the latest (or any desired version) of the Config Connector operator as per the documentation [2]
With this process, all the CRDs should remain intact, as well as the ConfigConnector object [3], so all resources created with Config Connector should remain unaffected as well.
Make sure that the new operator pod is running using the desired version:
$ kubectl -nconfigconnector-operator-system get statefulsets configconnector-operator -oyaml |grep operator-version | head -n1
cnrm.cloud.google.com/operator-version: 1.83.0
and that it recreated all the workloads running in the cnrm-system namespace (this might take a couple of minutes to reconcile):
$ kubectl -ncnrm-system get po -oyaml |grep /version
cnrm.cloud.google.com/version: 1.83.0
cnrm.cloud.google.com/version: 1.83.0
cnrm.cloud.google.com/version: 1.83.0
cnrm.cloud.google.com/version: 1.83.0
From now on, if you want to upgrade Config Connector, you have to do this manually as described in the documentation [4].
Reverting to the add-on installation
If you upgrade your GKE cluster to a version that contains the desired Config Connector version in the add-on, you can revert to the add-on installation and offload management back to GKE.
The process basically reverses the previous instructions:
- enable the Config Connector add-on on your cluster:
gcloud container clusters update <CLUSTER_NAME> --update-addons ConfigConnector=ENABLED
- delete the Config Connector operator StatefulSet:
kubectl -nconfigconnector-operator-system delete statefulset configconnector-operator
If you watch pods in the configconnector-operator-system namespace, you'll eventually see that the operator pod starts up again (since the StatefulSet is recreated by the add-on) and is running the version managed by the add-on. The workloads in the cnrm-system namespace will also be replaced during the operator's first reconciliation loop.
[1] https://cloud.google.com/config-connector/docs/how-to/advanced-install#manual
[2] https://cloud.google.com/config-connector/docs/how-to/advanced-install#installing_the_operator
[3] https://cloud.google.com/config-connector/docs/how-to/install-upgrade-uninstall#addon-configuring
[4] https://cloud.google.com/config-connector/docs/how-to/advanced-install#upgrading