Add design for cluster profile
@@ -92,24 +92,80 @@ the required manifests while keeping a single master source.
The current installation profile is called `self-managed-high-availability`. All current
manifests must specify it. Future profiles may choose.
-#### Environment variable
+#### Design
-A cluster profile is specified to the CVO as an identifier in an environment
-variable. For a given cluster, only one CVO profile may be in effect.
-
-NOTE: The mechanism by which the environment variable is set on the CVO deployment is
-out of the scope of this design.
+A cluster profile is specified to the CVO as an identifier an environment variable.
```
CLUSTER_PROFILE=[identifier]
```
-This environment variable would have to be specified in the CVO deployment. When
-no `CLUSTER_PROFILE=[identifier]` variable is specified, the `default` cluster profile
-is in effect.
+
+For a given cluster, only one cluster profile may be in effect.
+
+The profile is a set-once property and cannot be changed once the cluster has started.
+
+The cluster-version-operator picks the value for the cluster profile according to this order:
+* if it is defined and not empty, the environment variable,
+* otherwise, the default profile `self-managed-high-availability`.
+
+Clusters in a version unaware of the cluster profile must upgrade to the `self-managed-high-availability` profile.
+
+When upgrading, outgoing CVO will forward the cluster profile information to the incoming CVO with the environment variable.
`include.release.openshift.io/[identifier]=true` would make the CVO render this manifest only when `CLUSTER_PROFILE=[identifier]`
has been specified.
+##### Usage
+
+###### Without the installer
+
+IBM Cloud and others platforms that are managing their own deployment of the CVO should pass the env. variable.
+For instance, IBM Cloud already uses `EXCLUDE_MANIFESTS` env. variable. Cluster profile will be set like this env. variable.
+
+Upgrade will have to preserve the initial cluster profile.
+
+###### With the installer
+
+This method will be used by CodeReady Containers and single-node production edge clusters.
+
+Users must set `OPENSHIFT_INSTALL_EXPERIMENTAL_CLUSTER_PROFILE` env. variable in their shell before running the installer if they want to use non-default profile.
+
+Example:
+```
+$ OPENSHIFT_INSTALL_EXPERIMENTAL_CLUSTER_PROFILE=single-node-developer openshift-install create manifest
+$ OPENSHIFT_INSTALL_EXPERIMENTAL_CLUSTER_PROFILE=single-node-developer openshift-install create cluster
+```
+
+##### Release phases
+
+The `CLUSTER_PROFILE` env. variable will break the upgrade as this new template variable in CVO manifests is not known by outgoing CVO.
+This requires to deploy cluster profile in 2 phases.
+
+```
+apiVersion: v1
+kind: Pod
+metadata:
+ name: bootstrap-cluster-version-operator
+ namespace: openshift-cluster-version
+...
+ env:
+ - name: CLUSTER_PROFILE
+ value: "{{.ClusterProfile}}"
+```
+
+###### Phase 1
+
+* Add the cluster profile `ClusterProfile` in the [`manifestRenderConfig`](https://github.com/openshift/cluster-version-operator/blob/b59561c40240d2a52048923b1b94ed7385cab957/pkg/payload/render.go#L104) object used to render all manifests (esp. CVO manifests).
+* Read the env. variable and select manifests with the right include property `include.release.openshift.io/[identifier]=true`.
+ It will default to `self-managed-high-availability` in this phase.
+
+This will probably need to be backported.
+
+###### Phase 2
+
+* Add the cluster profile env. variable in CVO manifests,
+* Operators can add manifests for the non-default profile.
+
## Implementation History
* Teach cluster-version operator about profiles, [cvo#404](https://github.com/openshift/cluster-version-operator/pull/404), in flight.
