PGO YAML

pgo.yaml Configuration

The pgo.yaml file contains many different configuration settings as described in this section of the documentation.

The pgo.yaml file is broken into major sections as described below:

Cluster

Setting Definition
BasicAuth If set to "true" will enable Basic Authentication. If set to "false", will allow a valid Operator user to successfully authenticate regardless of the value of the password provided for Basic Authentication. Defaults to "true".
CCPImagePrefix newly created containers will be based on this image prefix (e.g. crunchydata), update this if you require a custom image prefix
CCPImageTag newly created containers will be based on this image version (e.g. centos8-13.2-4.6.2), unless you override it using the –ccp-image-tag command line flag
Port the PostgreSQL port to use for new containers (e.g. 5432)
PGBadgerPort the port used to connect to pgbadger (e.g. 10000)
ExporterPort the port used to connect to postgres exporter (e.g. 9187)
User the PostgreSQL normal user name
Database the PostgreSQL normal user database
Replicas the number of cluster replicas to create for newly created clusters, typically users will scale up replicas on the pgo CLI command line but this global value can be set as well
Metrics boolean, if set to true will cause each new cluster to include crunchy-postgres-exporter as a sidecar container for metrics collection, if set to false (default), users can still add metrics on a cluster-by-cluster basis using the pgo command flag –metrics
Badger boolean, if set to true will cause each new cluster to include crunchy-pgbadger as a sidecar container for static log analysis, if set to false (default), users can still add pgbadger on a cluster-by-cluster basis using the pgo create cluster command flag –pgbadger
Policies optional, list of policies to apply to a newly created cluster, comma separated, must be valid policies in the catalog
PasswordAgeDays optional, if set, will set the VALID UNTIL date on passwords to this many days in the future when creating users or setting passwords, defaults to 60 days
PasswordLength optional, if set, will determine the password length used when creating passwords, defaults to 8
ServiceType optional, if set, will determine the service type used when creating primary or replica services, defaults to ClusterIP if not set, can be overridden by the user on the command line as well
Backrest optional, if set, will cause clusters to have the pgbackrest volume PVC provisioned during cluster creation
BackrestPort currently required to be port 2022
DisableAutofail optional, if set, will disable autofail capabilities by default in any newly created cluster
DisableReplicaStartFailReinit if set to true will disable the detection of a “start failed” states in PG replicas, which results in the re-initialization of the replica in an attempt to bring it back online
PodAntiAffinity either preferred, required or disabled to either specify the type of affinity that should be utilized for the default pod anti-affinity applied to PG clusters, or to disable default pod anti-affinity all together (default preferred)
SyncReplication boolean, if set to true will automatically enable synchronous replication in new PostgreSQL clusters (default false)
DefaultInstanceMemory string, matches a Kubernetes resource value. If set, it is used as the default value of the memory request for each instance in a PostgreSQL cluster. The example configuration uses 128Mi which is very low for a PostgreSQL cluster, as the default amount of shared memory PostgreSQL requests is 128Mi. However, for test clusters, this value is acceptable as the shared memory buffers won’t be stressed, but you should absolutely consider raising this in production. If the value is unset, it defaults to 512Mi, which is a much more appropriate minimum.
DefaultBackrestMemory string, matches a Kubernetes resource value. If set, it is used as the default value of the memory request for the pgBackRest repository (default 48Mi)
DefaultPgBouncerMemory string, matches a Kubernetes resource value. If set, it is used as the default value of the memory request for pgBouncer instances (default 24Mi)
DisableFSGroup If set to true, this will disable the use of the fsGroup for the containers related to PostgreSQL, which is normally set to 26. This is geared towards deployments that use Security Context Constraints in the mode of restricted (default false)

Storage

Setting Definition
PrimaryStorage required, the value of the storage configuration to use for the primary PostgreSQL deployment
BackupStorage required, the value of the storage configuration to use for backups, including the storage for pgbackrest repo volumes
ReplicaStorage required, the value of the storage configuration to use for the replica PostgreSQL deployments
BackrestStorage required, the value of the storage configuration to use for the pgbackrest shared repository deployment created when a user specifies pgbackrest to be enabled on a cluster
WALStorage optional, the value of the storage configuration to use for PostgreSQL Write Ahead Log
StorageClass optional, for a dynamic storage type, you can specify the storage class used for storage provisioning (e.g. standard, gold, fast)
AccessMode the access mode for new PVCs (e.g. ReadWriteMany, ReadWriteOnce, ReadOnlyMany). See below for descriptions of these.
Size the size to use when creating new PVCs (e.g. 100M, 1Gi)
Storage.storage1.StorageType supported values are either dynamic, create, if not supplied, create is used
SupplementalGroups optional, if set, will cause a SecurityContext to be added to generated Pod and Deployment definitions
MatchLabels optional, if set, will cause the PVC to add a matchlabels selector in order to match a PV, only useful when the StorageType is create, when specified a label of key=value is added to the PVC as a match criteria

Storage Configuration Examples

In pgo.yaml, you will need to configure your storage configurations depending on which storage you are wanting to use for Operator provisioning of Persistent Volume Claims. The examples below are provided as a sample. In all the examples you are free to change the Size to meet your requirements of Persistent Volume Claim size.

HostPath Example

HostPath is provided for simple testing and use cases where you only intend to run on a single Linux host for your Kubernetes cluster.

  hostpathstorage:
    AccessMode:  ReadWriteMany
    Size:  1G
    StorageType:  create

NFS Example

In the following NFS example, notice that the SupplementalGroups setting is set, this can be whatever GID you have your NFS mount set to, typically we set this nfsnobody as below. NFS file systems offer a ReadWriteMany access mode.

  nfsstorage:
    AccessMode:  ReadWriteMany
    Size:  1G
    StorageType:  create
    SupplementalGroups:  65534

Storage Class Example

Most Storage Class providers offer ReadWriteOnce access modes, but refer to your provider documentation for other access modes it might support.

  storageos:
    AccessMode:  ReadWriteOnce
    Size:  1G
    StorageType:  dynamic
    StorageClass:  fast

Miscellaneous (Pgo)

Setting Definition
Audit boolean, if set to true will cause each apiserver call to be logged with an audit marking
ConfigMapWorkerCount The number of workers created for the worker queue within the ConfigMap controller (defaults to 2)
ControllerGroupRefreshInterval The refresh interval for any per-namespace controller with a refresh interval (defaults to 60 seconds)
DisableReconcileRBAC Whether or not to disable RBAC reconciliation in targeted namespaces (defaults to false)
NamespaceRefreshInterval The refresh interval for the namespace controller (defaults to 60 seconds)
NamespaceWorkerCount The number of workers created for the worker queue within the Namespace controller (defaults to 2)
PgclusterWorkerCount The number of workers created for the worker queue within the PGCluster controller (defaults to 1)
PGOImagePrefix image tag prefix to use for the Operator containers
PGOImageTag image tag to use for the Operator containers
PGReplicaWorkerCount The number of workers created for the worker queue within the PGReplica controller (defaults to 1)
PGTaskWorkerCount The number of workers created for the worker queue within the PGTask controller (defaults to 1)

Storage Configuration Details

You can define n-number of Storage configurations within the pgo.yaml file. Those Storage configurations follow these conventions -

  • they must have lowercase name (e.g. storage1)
  • they must be unique names (e.g. mydrstorage, faststorage, slowstorage)

These Storage configurations are referenced in the BackupStorage, ReplicaStorage, and PrimaryStorage configuration values. However, there are command line options in the pgo client that will let a user override these default global values to offer you the user a way to specify very targeted storage configurations when needed (e.g. disaster recovery storage for certain backups).

You can set the storage AccessMode values to the following:

  • ReadWriteMany - mounts the volume as read-write by many nodes
  • ReadWriteOnce - mounts the PVC as read-write by a single node
  • ReadOnlyMany - mounts the PVC as read-only by many nodes

These Storage configurations are validated when the pgo-apiserver starts, if a non-valid configuration is found, the apiserver will abort. These Storage values are only read at apiserver start time.

The following StorageType values are possible -

  • dynamic - this will allow for dynamic provisioning of storage using a StorageClass.
  • create - This setting allows for the creation of a new PVC for each PostgreSQL cluster using a naming convention of clustername. When set, the Size, AccessMode settings are used in constructing the new PVC.

The operator will create new PVCs using this naming convention: dbname where dbname is the database name you have specified. For example, if you run:

pgo create cluster example1 -n pgouser1

It will result in a PVC being created named example1 and in the case of a backup job, the pvc is named example1-backup

Note, when Storage Type is create, you can specify a storage configuration setting of MatchLabels, when set, this will cause a selector of key=value to be added into the PVC, this will let you target specific PV(s) to be matched for this cluster. Note, if a PV does not match the claim request, then the cluster will not start. Users that want to use this feature have to place labels on their PV resources as part of PG cluster creation before creating the PG cluster. For example, users would add a label like this to their PV before they create the PG cluster:

kubectl label pv somepv myzone=somezone -n pgouser1

If you do not specify MatchLabels in the storage configuration, then no match filter is added and any available PV will be used to satisfy the PVC request. This option does not apply to dynamic storage types.

Example PV creation scripts are provided that add labels to a set of PVs and can be used for testing: $COROOT/pv/create-pv-nfs-labels.sh in that example, a label of crunchyzone=red is set on a set of PVs to test with.

The pgo.yaml includes a storage config named nfsstoragered that when used will demonstrate the label matching. This feature allows you to support n-number of NFS storage configurations and supports spreading a PG cluster across different NFS storage configurations.

Overriding Storage Configuration Defaults

pgo create cluster testcluster --storage-config=bigdisk -n pgouser1

That example will create a cluster and specify a storage configuration of bigdisk to be used for the primary database storage. The replica storage will default to the value of ReplicaStorage as specified in pgo.yaml.

pgo create cluster testcluster2 --storage-config=fastdisk --replica-storage-config=slowdisk -n pgouser1

That example will create a cluster and specify a storage configuration of fastdisk to be used for the primary database storage, while the replica storage will use the storage configuration slowdisk.

pgo backup testcluster --storage-config=offsitestorage -n pgouser1

That example will create a backup and use the offsitestorage storage configuration for persisting the backup.

Using Storage Configurations for Disaster Recovery

A simple mechanism for partial disaster recovery can be obtained by leveraging network storage, Kubernetes storage classes, and the storage configuration options within the Operator.

For example, if you define a Kubernetes storage class that refers to a storage backend that is running within your disaster recovery site, and then use that storage class as a storage configuration for your backups, you essentially have moved your backup files automatically to your disaster recovery site thanks to network storage.