Navigation :

Crunchy PostgreSQL Operator > Using Custom Resources

Using Custom Resources

Operator Architecture with CRDs

As discussed in the architecture overview, the heart of the PostgreSQL Operator, and any Kubernetes Operator, is one or more Custom Resources Definitions, also known as “CRDs”. CRDs provide extensions to the Kubernetes API, and, in the case of the PostgreSQL Operator, allow you to perform actions such as:

Creating a PostgreSQL Cluster
Updating PostgreSQL Cluster resource allocations
Add additional utilities to a PostgreSQL cluster, e.g. pgBouncer for connection pooling and more.

The PostgreSQL Operator provides the pgo client as a convenience for interfacing with the CRDs, as manipulating the CRDs directly can be a tedious process. For example, there are several Kubernetes objects that need to be set up prior to creating a pgcluster custom resource in order to successfully deploy a new PostgreSQL cluster.

The Kubernetes community trend has been to move towards supporting a “custom resource only” workflow for using Operators, and this is something that the PostgreSQL Operator aims to do as well. Certain workflows are fully driven by Custom Resources (e.g. creating a PostgreSQL cluster), while others still need to interface through the pgo client (e.g. adding a PostgreSQL user).

The following sections will describe the functionality that is available today when manipulating the PostgreSQL Operator Custom Resources directly.

Custom Resource Workflows

Create a PostgreSQL Cluster

The fundamental workflow for interfacing with a PostgreSQL Operator Custom Resource Definition is for creating a PostgreSQL cluster. There are several that a PostgreSQL cluster requires to be deployed, including:

Secrets
- Information for setting up a pgBackRest repository
- PostgreSQL superuser bootstrap credentials
- PostgreSQL replication user bootstrap credentials
- PostgresQL standard user bootstrap credentials

Additionally, if you want to add some of the other sidecars, you may need to create additional secrets.

The good news is that if you do not provide these objects, the PostgreSQL Operator will create them for you to get your Postgres cluster up and running!

The following goes through how to create a PostgreSQL cluster called hippo by creating a new custom resource.

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

cat <<-EOF > "${pgo_cluster_name}-pgcluster.yaml"
apiVersion: crunchydata.com/v1
kind: Pgcluster
metadata:
  annotations:
    current-primary: ${pgo_cluster_name}
  labels:
    crunchy-pgha-scope: ${pgo_cluster_name}
    deployment-name: ${pgo_cluster_name}
    name: ${pgo_cluster_name}
    pg-cluster: ${pgo_cluster_name}
    pgo-version: 4.6.0
    pgouser: admin
  name: ${pgo_cluster_name}
  namespace: ${cluster_namespace}
spec:
  BackrestStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ""
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  PrimaryStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ${pgo_cluster_name}
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  ReplicaStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ""
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  annotations: {}
  ccpimage: crunchy-postgres-ha
  ccpimageprefix: registry.developers.crunchydata.com/crunchydata
  ccpimagetag: centos8-13.1-4.6.0
  clustername: ${pgo_cluster_name}
  database: ${pgo_cluster_name}
  exporterport: "9187"
  limits: {}
  name: ${pgo_cluster_name}
  namespace: ${cluster_namespace}
  pgDataSource:
    restoreFrom: ""
    restoreOpts: ""
  pgbadgerport: "10000"
  pgoimageprefix: registry.developers.crunchydata.com/crunchydata
  podAntiAffinity:
    default: preferred
    pgBackRest: preferred
    pgBouncer: preferred
  port: "5432"
  tolerations: []
  user: hippo
  userlabels:
    pgo-version: 4.6.0
EOF

kubectl apply -f "${pgo_cluster_name}-pgcluster.yaml"

And that’s all! The PostgreSQL Operator will go ahead and create the cluster.

As part of this process, the PostgreSQL Operator creates several Secrets that contain the credentials for three user accounts that must be present in order to bootstrap a PostgreSQL cluster. These are:

A PostgreSQL superuser
A replication user
A standard PostgreSQL user

The Secrets represent the following PostgreSQL users and can be identified using the below patterns:

PostgreSQL User	Type	Secret Pattern	Notes
`postgres`	Superuser	`<clusterName>-postgres-secret`	This is the PostgreSQL superuser account. Using the above example, the name of the secret would be `hippo-postgres-secret`.
`primaryuser`	Replication	`<clusterName>-primaryuser-secret`	This is for the managed replication user account for maintaining high availability. This account does not need to be accessed. Using the above example, the name of the secret would be `hippo-primaryuser-secret`.
User	User	`<clusterName>-<User>-secret`	This is an unprivileged user that should be used for most operations. This secret is set by the `user` attribute in the custom resources. In the above example, the name of this user is `hippo`, which would make the Secret `hippo-hippo-secret`

To extract the user credentials so you can log into the database, you can use the following JSONPath expression:

# namespace that the cluster is running in
export cluster_namespace=pgo
# name of the cluster
export pgo_cluster_name=hippo
# name of the user whose password we want to get
export pgo_cluster_username=hippo

kubectl -n "${cluster_namespace}" get secrets \
  "${pgo_cluster_name}-${pgo_cluster_username}-secret" -o "jsonpath={.data['password']}" | base64 -d

Customizing User Credentials

If you wish to set the credentials for these users on your own, you have to create these Secrets before creating a custom resource. The below example shows how to create the three required user accounts prior to creating a custom resource. Note that if you omit any of these Secrets, the Postgres Operator will create it on its own.

# this variable is the name of the cluster being created
pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
cluster_namespace=pgo

# this is the superuser secret
kubectl create secret generic -n "${cluster_namespace}" "${pgo_cluster_name}-postgres-secret" \
  --from-literal=username=postgres \
  --from-literal=password=Supersecurepassword*

# this is the replication user secret
kubectl create secret generic -n "${cluster_namespace}" "${pgo_cluster_name}-primaryuser-secret" \
  --from-literal=username=primaryuser \
  --from-literal=password=Anothersecurepassword*

# this is the standard user secret
kubectl create secret generic -n "${cluster_namespace}" "${pgo_cluster_name}-hippo-secret" \
  --from-literal=username=hippo \
  --from-literal=password=Moresecurepassword*


kubectl label secrets -n "${cluster_namespace}" "${pgo_cluster_name}-postgres-secret" "pg-cluster=${pgo_cluster_name}"
kubectl label secrets -n "${cluster_namespace}" "${pgo_cluster_name}-primaryuser-secret" "pg-cluster=${pgo_cluster_name}"
kubectl label secrets -n "${cluster_namespace}" "${pgo_cluster_name}-hippo-secret" "pg-cluster=${pgo_cluster_name}"

Create a PostgreSQL Cluster With Backups in S3

A frequent use case is to create a PostgreSQL cluster with S3 or a S3-like storage system for storing backups. This requires adding a Secret that contains the S3 key and key secret for your account, and adding some additional information into the custom resource.

Step 1: Create the pgBackRest S3 Secrets

As mentioned above, it is necessary to create a Secret containing the S3 key and key secret that will allow a user to create backups in S3.

The below code will help you set up this Secret.

# this variable is the name of the cluster being created
pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
cluster_namespace=pgo
# the following variables are your S3 key and key secret
backrest_s3_key=yours3key
backrest_s3_key_secret=yours3keysecret

kubectl -n "${cluster_namespace}" create secret generic "${pgo_cluster_name}-backrest-repo-config" \
  --from-literal="aws-s3-key=${backrest_s3_key}" \
  --from-literal="aws-s3-key-secret=${backrest_s3_key_secret}"

unset backrest_s3_key
unset backrest_s3_key_secret

Step 2: Create the PostgreSQL Cluster

With the Secrets in place. It is now time to create the PostgreSQL cluster.

The below manifest references the Secrets created in the previous step to add a custom resource to the pgclusters.crunchydata.com custom resource definition. There are some additions in this example specifically for storing backups in S3.

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo
# the following variables store the information for your S3 cluster. You may
# need to adjust them for your actual settings
export backrest_s3_bucket=your-bucket
export backrest_s3_endpoint=s3.region-name.amazonaws.com
export backrest_s3_region=region-name

cat <<-EOF > "${pgo_cluster_name}-pgcluster.yaml"
apiVersion: crunchydata.com/v1
kind: Pgcluster
metadata:
  annotations:
    current-primary: ${pgo_cluster_name}
  labels:
    crunchy-pgha-scope: ${pgo_cluster_name}
    deployment-name: ${pgo_cluster_name}
    name: ${pgo_cluster_name}
    pg-cluster: ${pgo_cluster_name}
    pgo-version: 4.6.0
    pgouser: admin
  name: ${pgo_cluster_name}
  namespace: ${cluster_namespace}
spec:
  BackrestStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ""
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  PrimaryStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ${pgo_cluster_name}
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  ReplicaStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ""
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  annotations: {}
  backrestStorageTypes:
  - s3
  backrestS3Bucket: ${backrest_s3_bucket}
  backrestS3Endpoint: ${backrest_s3_endpoint}
  backrestS3Region: ${backrest_s3_region}
  backrestS3URIStyle: ""
  backrestS3VerifyTLS: ""
  ccpimage: crunchy-postgres-ha
  ccpimageprefix: registry.developers.crunchydata.com/crunchydata
  ccpimagetag: centos8-13.1-4.6.0
  clustername: ${pgo_cluster_name}
  database: ${pgo_cluster_name}
  exporterport: "9187"
  limits: {}
  name: ${pgo_cluster_name}
  namespace: ${cluster_namespace}
  pgDataSource:
    restoreFrom: ""
    restoreOpts: ""
  pgbadgerport: "10000"
  pgoimageprefix: registry.developers.crunchydata.com/crunchydata
  podAntiAffinity:
    default: preferred
    pgBackRest: preferred
    pgBouncer: preferred
  port: "5432"
  tolerations: []
  user: hippo
  userlabels:
    pgo-version: 4.6.0
EOF

kubectl apply -f "${pgo_cluster_name}-pgcluster.yaml"

Create a PostgreSQL Cluster with TLS

There are three items that are required to enable TLS in your PostgreSQL clusters:

A CA certificate
A TLS private key
A TLS certificate

It is possible create a PostgreSQL cluster with TLS using a custom resource workflow with the prerequisite of ensuring the above three items are created.

For a detailed explanation for how TLS works with the PostgreSQL Operator, please see the TLS tutorial.

Step 1: Create TLS Secrets

There are two Secrets that need to be created:

A Secret containing the certificate authority (CA). You may only need to create this Secret once, as a CA certificate can be shared amongst your clusters.
A Secret that contains the TLS private key & certificate.

This assumes that you have already generated your TLS certificates where the CA is named ca.crt and the server key and certificate are named server.key and server.crt respectively.

Substitute the correct values for your environment into the environmental variables in the example below:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo
# this is the local path to where you stored the CA and server key and certificate
export cluster_tls_asset_path=/path/to

# create the CA secret. if this already exists, it's OK if it fails
kubectl create secret generic postgresql-ca -n "${cluster_namespace}" \
  --from-file="ca.crt=${cluster_tls_asset_path}/ca.crt"

# create the server key/certificate secret
kubectl create secret tls "${pgo_cluster_name}-tls-keypair" -n "${cluster_namespace}" \
  --cert="${cluster_tls_asset_path}/server.crt" \
  --key="${cluster_tls_asset_path}/server.key"

Step 2: Create the PostgreSQL Cluster

The below example uses the Secrets created in the previous step and creates a TLS-enabled PostgreSQL cluster. Additionally, this example sets the tlsOnly attribute to true, which requires all TCP connections to occur over TLS:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

cat <<-EOF > "${pgo_cluster_name}-pgcluster.yaml"
apiVersion: crunchydata.com/v1
kind: Pgcluster
metadata:
  annotations:
    current-primary: ${pgo_cluster_name}
  labels:
    crunchy-pgha-scope: ${pgo_cluster_name}
    deployment-name: ${pgo_cluster_name}
    name: ${pgo_cluster_name}
    pg-cluster: ${pgo_cluster_name}
    pgo-version: 4.6.0
    pgouser: admin
  name: ${pgo_cluster_name}
  namespace: ${cluster_namespace}
spec:
  BackrestStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ""
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  PrimaryStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ${pgo_cluster_name}
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  ReplicaStorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ""
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  annotations: {}
  ccpimage: crunchy-postgres-ha
  ccpimageprefix: registry.developers.crunchydata.com/crunchydata
  ccpimagetag: centos8-13.1-4.6.0
  clustername: ${pgo_cluster_name}
  database: ${pgo_cluster_name}
  exporterport: "9187"
  limits: {}
  name: ${pgo_cluster_name}
  namespace: ${cluster_namespace}
  pgDataSource:
    restoreFrom: ""
    restoreOpts: ""
  pgbadgerport: "10000"
  pgoimageprefix: registry.developers.crunchydata.com/crunchydata
  podAntiAffinity:
    default: preferred
    pgBackRest: preferred
    pgBouncer: preferred
  port: "5432"
  tls:
    caSecret: postgresql-ca
    tlsSecret: ${pgo_cluster_name}-tls-keypair
  tlsOnly: true
  user: hippo
  userlabels:
    pgo-version: 4.6.0
EOF

kubectl apply -f "${pgo_cluster_name}-pgcluster.yaml"

Modify a Cluster

There following modification operations are supported on the pgclusters.crunchydata.com custom resource definition:

Modify Resource Requests & Limits

Modifying the resources, limits, backrestResources, backRestLimits, pgBouncer.resources, or pgbouncer.limits will cause the PostgreSQL Operator to apply the new values to the affected Deployments.

For example, if we wanted to make a memory request of 512Mi for the hippo cluster created in the previous example, we could do the following:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

kubectl edit pgclusters.crunchydata.com -n "${cluster_namespace}" "${pgo_cluster_name}"

This will open up your editor. Find the resources block, and have it read as the following:

resources:
  memory: 256Mi

The PostgreSQL Operator will respond and modify the PostgreSQL instances to request 256Mi of memory.

Be careful when editing these values for a variety of reasons, mainly that modifying these values will cause the Pods to restart, which in turn will create potential downtime events. It’s best to modify the values for a deployment group together and not mix and match, i.e.

PostgreSQL instances: resources, limits
pgBackRest: backrestResources, backrestLimits
pgBouncer: pgBouncer.resources, pgBouncer.limits

Scale

Once you have created a PostgreSQL cluster, you may want to add a replica to create a high-availability environment. Replicas are added and removed using the pgreplicas.crunchydata.com custom resource definition. Each replica must have a unique name, e.g. hippo-rpl1 could be one unique replica for a PostgreSQL cluster.

Using the above example cluster, hippo, let’s add a replica called hippo-rpl1 using the configuration below. Be sure to change the replicastorage block to match the storage configuration for your environment:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this helps to name the replica, in this case "rpl1"
export pgo_cluster_replica_suffix=rpl1
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

cat <<-EOF > "${pgo_cluster_name}-${pgo_cluster_replica_suffix}-pgreplica.yaml"
apiVersion: crunchydata.com/v1
kind: Pgreplica
metadata:
  labels:
    name: ${pgo_cluster_name}-${pgo_cluster_replica_suffix}
    pg-cluster: ${pgo_cluster_name}
    pgouser: admin
  name: ${pgo_cluster_name}-${pgo_cluster_replica_suffix}
  namespace: ${cluster_namespace}
spec:
  clustername: ${pgo_cluster_name}
  name: ${pgo_cluster_name}-${pgo_cluster_replica_suffix}
  namespace: ${cluster_namespace}
  replicastorage:
    accessmode: ReadWriteMany
    matchLabels: ""
    name: ${pgo_cluster_name}-${pgo_cluster_replica_suffix}
    size: 1G
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""
  tolerations: []
  userlabels:
    pgo-version: 4.6.0
EOF

kubectl apply -f "${pgo_cluster_name}-${pgo_cluster_replica_suffix}-pgreplica.yaml"

Add this time, removing a replica must be handled through the pgo client.

Monitoring

To enable the monitoring (aka metrics) sidecar using the crunchy-postgres-exporter container, you need to set the exporter attribute in pgclusters.crunchydata.com custom resource.

Add a Tablespace

Tablespaces can be added during the lifetime of a PostgreSQL cluster (tablespaces can be removed as well, but for a detailed explanation as to how, please see the Tablespaces section).

To add a tablespace, you need to add an entry to the tablespaceMounts section of a custom entry, where the key is the name of the tablespace (unique to the pgclusters.crunchydata.com custom resource entry) and the value is a storage configuration as defined in the pgclusters.crunchydata.com section above.

For example, to add a tablespace named lake to our hippo cluster, we can open up the editor with the following code:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

kubectl edit pgclusters.crunchydata.com -n "${cluster_namespace}" "${pgo_cluster_name}"

and add an entry to the tablespaceMounts block that looks similar to this, with the addition of the correct storage configuration for your environment:

tablespaceMounts:
  lake:
    accessmode: ReadWriteMany
    matchLabels: ""
    size: 5Gi
    storageclass: ""
    storagetype: dynamic
    supplementalgroups: ""

pgBouncer

pgBouncer is a PostgreSQL connection pooler and state manager that can be useful for high-availability setups as well as managing overall performance of a PostgreSQL cluster. A pgBouncer deployment for a PostgreSQL cluster can be fully managed from a pgclusters.crunchydata.com custom resource.

For example, to add a pgBouncer deployment to our hippo cluster with two instances and a memory limit of 36Mi, you can edit the custom resource:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

kubectl edit pgclusters.crunchydata.com -n "${cluster_namespace}" "${pgo_cluster_name}"

And modify the pgBouncer block to look like this:

pgBouncer:
  limits:
    memory: 36Mi
  replicas: 2

Likewise, to remove pgBouncer from a PostgreSQL cluster, you would set replicas to 0:

pgBouncer:
  replicas: 0

Start / Stop a Cluster

A PostgreSQL cluster can be start and stopped by toggling the shutdown parameter in a pgclusters.crunchydata.com custom resource. Setting shutdown to true will stop a PostgreSQL cluster, whereas a value of false will make a cluster available. This affects all of the associated instances of a PostgreSQL cluster.

Manage Annotations

Kubernetes Annotations can be managed for PostgreSQL, pgBackRest, and pgBouncer Deployments, as well as being able to apply Annotations across all three. This is done via the annotations block in the pgclusters.crunchydata.com custom resource definition. For example, to apply Annotations in the hippo cluster, some that are global, some that are specific to each Deployment type, you could do the following.

First, start editing the hippo custom resource:

# this variable is the name of the cluster being created
export pgo_cluster_name=hippo
# this variable is the namespace the cluster is being deployed into
export cluster_namespace=pgo

kubectl edit pgclusters.crunchydata.com -n "${cluster_namespace}" "${pgo_cluster_name}"

In the hippo specification, add the annotations block similar to this (note, this explicitly shows that this is the spec block. Do not modify the annotations block in the metadata section).

spec:
  annotations:
    global:
      favorite: hippo
    backrest:
      chair: comfy
    pgBouncer:
      pool: swimming
    postgres:
      elephant: cool

Save your edits, and in a short period of time, you should see these annotations applied to the managed Deployments.

Delete a PostgreSQL Cluster

A PostgreSQL cluster can be deleted by simply deleting the pgclusters.crunchydata.com resource.

It is possible to keep both the PostgreSQL data directory as well as the pgBackRest backup repository when using this method by setting the following annotations on the pgclusters.crunchydata.com custom resource:

keep-backups: indicates to keep the pgBackRest PVC when deleting the cluster.
keep-data: indicates to keep the PostgreSQL data PVC when deleting the cluster.

PostgreSQL Operator Custom Resource Definitions

There are several PostgreSQL Operator Custom Resource Definitions (CRDs) that are installed in order for the PostgreSQL Operator to successfully function:

pgclusters.crunchydata.com: Stores information required to manage a PostgreSQL cluster. This includes things like the cluster name, what storage and resource classes to use, which version of PostgreSQL to run, information about how to maintain a high-availability cluster, etc.
pgreplicas.crunchydata.com: Stores information required to manage the replicas within a PostgreSQL cluster. This includes things like the number of replicas, what storage and resource classes to use, special affinity rules, etc.
pgtasks.crunchydata.com: A general purpose CRD that accepts a type of task that is needed to run against a cluster (e.g. take a backup) and tracks the state of said task through its workflow.
pgpolicies.crunchydata.com: Stores a reference to a SQL file that can be executed against a PostgreSQL cluster. In the past, this was used to manage RLS policies on PostgreSQL clusters.

Below takes an in depth look for what each attribute does in a Custom Resource Definition, and how they can be used in the creation and update workflow.

Glossary

create: if an attribute is listed as create, it means it can affect what happens when a new Custom Resource is created.
update: if an attribute is listed as update, it means it can affect the Custom Resource, and by extension the objects it manages, when the attribute is updated.

`pgclusters.crunchydata.com`

The pgclusters.crunchydata.com Custom Resource Definition is the fundamental definition of a PostgreSQL cluster. Most attributes only affect the deployment of a PostgreSQL cluster at the time the PostgreSQL cluster is created. Some attributes can be modified during the lifetime of the PostgreSQL cluster and make changes, as described below.

Specification (`Spec`)

Attribute	Action	Description
annotations	`create`, `update`	Specify Kubernetes Annotations that can be applied to the different deployments managed by the PostgreSQL Operator (PostgreSQL, pgBackRest, pgBouncer). For more information, please see the “Annotations Specification” below.
backrestConfig	`create`	Optional references to pgBackRest configuration files
backrestLimits	`create`, `update`	Specify the container resource limits that the pgBackRest repository should use. Follows the Kubernetes definitions of resource limits.
backrestRepoPath	`create`	Optional reference to the location of the pgBackRest repository.
BackrestResources	`create`, `update`	Specify the container resource requests that the pgBackRest repository should use. Follows the Kubernetes definitions of resource requests.
backrestS3Bucket	`create`	An optional parameter that specifies a S3 bucket that pgBackRest should use.
backrestS3Endpoint	`create`	An optional parameter that specifies the S3 endpoint pgBackRest should use.
backrestS3Region	`create`	An optional parameter that specifies a cloud region that pgBackRest should use.
backrestS3URIStyle	`create`	An optional parameter that specifies if pgBackRest should use the `path` or `host` S3 URI style.
backrestS3VerifyTLS	`create`	An optional parameter that specifies if pgBackRest should verify the TLS endpoint.
BackrestStorage	`create`	A specification that gives information about the storage attributes for the pgBackRest repository, which stores backups and archives, of the PostgreSQL cluster. For details, please see the `Storage Specification` section below. This is required.
backrestStorageTypes	`create`	An optional parameter that takes an array of different repositories types that can be used to store pgBackRest backups. Choices are `posix` and `s3`. If nothing is specified, it defaults to `posix`. (`local`, equivalent to `posix`, is available for backwards compatibility).
ccpimage	`create`	The name of the PostgreSQL container image to use, e.g. `crunchy-postgres-ha` or `crunchy-postgres-ha-gis`.
ccpimageprefix	`create`	If provided, the image prefix (or registry) of the PostgreSQL container image, e.g. `registry.developers.crunchydata.com/crunchydata`. The default is to use the image prefix set in the PostgreSQL Operator configuration.
ccpimagetag	`create`	The tag of the PostgreSQL container image to use, e.g. `centos8-13.1-4.6.0`.
clustername	`create`	The name of the PostgreSQL cluster, e.g. `hippo`. This is used to group PostgreSQL instances (primary, replicas) together.
customconfig	`create`	If specified, references a custom ConfigMap to use when bootstrapping a PostgreSQL cluster. For the shape of this file, please see the section on Custom Configuration
database	`create`	The name of a database that the PostgreSQL user can log into after the PostgreSQL cluster is created.
disableAutofail	`create`, `update`	If set to true, disables the high availability capabilities of a PostgreSQL cluster. By default, every cluster can have high availability if there is at least one replica.
exporter	`create`,`update`	If `true`, deploys the `crunchy-postgres-exporter` sidecar for metrics collection
exporterLimits	`create`, `update`	Specify the container resource limits that the `crunchy-postgres-exporter` sidecar uses when it is deployed with a PostgreSQL instance. Follows the Kubernetes definitions of resource limits.
exporterport	`create`	If `Exporter` is `true`, then this specifies the port that the metrics sidecar runs on (e.g. `9187`)
exporterResources	`create`, `update`	Specify the container resource requests that the `crunchy-postgres-exporter` sidecar uses when it is deployed with a PostgreSQL instance. Follows the Kubernetes definitions of resource requests.
limits	`create`, `update`	Specify the container resource limits that the PostgreSQL cluster should use. Follows the Kubernetes definitions of resource limits.
name	`create`	The name of the PostgreSQL instance that is the primary. On creation, this should be set to be the same as `ClusterName`.
namespace	`create`	The Kubernetes Namespace that the PostgreSQL cluster is deployed in.
nodeAffinity	`create`	Sets the node affinity rules for the PostgreSQL cluster and associated PostgreSQL instances. Can be overridden on a per-instance (`pgreplicas.crunchydata.com`) basis. Please see the `Node Affinity Specification` section below.
pgBadger	`create`,`update`	If `true`, deploys the `crunchy-pgbadger` sidecar for query analysis.
pgbadgerport	`create`	If the `PGBadger` label is set, then this specifies the port that the pgBadger sidecar runs on (e.g. `10000`)
pgBouncer	`create`, `update`	If specified, defines the attributes to use for the pgBouncer connection pooling deployment that can be used in conjunction with this PostgreSQL cluster. Please see the specification defined below.
pgDataSource	`create`	Used to indicate if a PostgreSQL cluster should bootstrap its data from a pgBackRest repository. This uses the PostgreSQL Data Source Specification, described below.
pgoimageprefix	`create`	If provided, the image prefix (or registry) of any PostgreSQL Operator images that are used for jobs, e.g. `registry.developers.crunchydata.com/crunchydata`. The default is to use the image prefix set in the PostgreSQL Operator configuration.
podAntiAffinity	`create`	A required section. Sets the pod anti-affinity rules for the PostgreSQL cluster and associated deployments. Please see the `Pod Anti-Affinity Specification` section below.
policies	`create`	If provided, a comma-separated list referring to `pgpolicies.crunchydata.com.Spec.Name` that should be run once the PostgreSQL primary is first initialized.
port	`create`	The port that PostgreSQL will run on, e.g. `5432`.
ReplicaStorage	`create`	A specification that gives information about the storage attributes for any replicas in the PostgreSQL cluster. For details, please see the `Storage Specification` section below. This will likely be changed in the future based on the nature of the high-availability system, but presently it is still required that you set it. It is recommended you use similar settings to that of `PrimaryStorage`.
replicas	`create`	The number of replicas to create after a PostgreSQL primary is first initialized. This only works on create; to scale a cluster after it is initialized, please use the `pgo scale` command.
resources	`create`, `update`	Specify the container resource requests that the PostgreSQL cluster should use. Follows the Kubernetes definitions of resource requests.
serviceType	`create`, `update`	Sets the Kubernetes Service type to use for the cluster. If not set, defaults to `ClusterIP`.
shutdown	`create`, `update`	If set to true, indicates that a PostgreSQL cluster should shutdown. If set to false, indicates that a PostgreSQL cluster should be up and running.
standby	`create`, `update`	If set to true, indicates that the PostgreSQL cluster is a “standby” cluster, i.e. is in read-only mode entirely. Please see Kubernetes Multi-Cluster Deployments for more information.
syncReplication	`create`	If set to `true`, specifies the PostgreSQL cluster to use synchronous replication.
tablespaceMounts	`create`,`update`	Lists any tablespaces that are attached to the PostgreSQL cluster. Tablespaces can be added at a later time by updating the `TablespaceMounts` entry, but they cannot be removed. Stores a map of information, with the key being the name of the tablespace, and the value being a Storage Specification, defined below.
tls	`create`	Defines the attributes for enabling TLS for a PostgreSQL cluster. See TLS Specification below.
tlsOnly	`create`	If set to true, requires client connections to use only TLS to connect to the PostgreSQL database.
tolerations	`create`,`update`	Any array of Kubernetes Tolerations. Please refer to the Kubernetes documentation for how to set this field.
user	`create`	The name of the PostgreSQL user that is created when the PostgreSQL cluster is first created.
userlabels	`create`	A set of key-value string pairs that are used as a sort of “catch-all” as well as a way to add custom labels to clusters. This will disappear at some point.

Storage Specification

The storage specification is a spec that defines attributes about the storage to be used for a particular function of a PostgreSQL cluster (e.g. a primary instance or for the pgBackRest backup repository). The below describes each attribute and how it works.

Attribute	Action	Description
accessmode	`create`	The name of the Kubernetes Persistent Volume Access Mode to use.
matchLabels	`create`	Only used with `StorageType` of `create`, used to match a particular subset of provisioned Persistent Volumes.
name	`create`	Only needed for `PrimaryStorage` in `pgclusters.crunchydata.com`.Used to identify the name of the PostgreSQL cluster. Should match `ClusterName`.
size	`create`	The size of the Persistent Volume Claim (PVC). Must use a Kubernetes resource value, e.g. `20Gi`.
storageclass	`create`	The name of the Kubernetes StorageClass to use.
storagetype	`create`	Set to `create` if storage is provisioned (e.g. using `hostpath`). Set to `dynamic` if using a dynamic storage provisioner, e.g. via a `StorageClass`.
supplementalgroups	`create`	If provided, a comma-separated list of group IDs to use in case it is needed to interface with a particular storage system. Typically used with NFS or hostpath storage.

Node Affinity Specification

Sets the node affinity for the PostgreSQL cluster and associated deployments. Follows the Kubernetes standard format for setting node affinity, including preferred and required node affinity.

To set node affinity for a PostgreSQL cluster, you will need to modify the default attribute in the node affinity specification. As mentioned above, the values that default accepts match what Kubernetes uses for node affinity.

For a detailed explanation for node affinity works. Please see the high-availability documentation.

Attribute	Action	Description
default	`create`	The default pod anti-affinity to use for all PostgreSQL instances managed in a given PostgreSQL cluster. Can be overridden on a per-instance basis with the `pgreplicas.crunchydata.com` custom resource.

Pod Anti-Affinity Specification

Sets the pod anti-affinity for the PostgreSQL cluster and associated deployments. Each attribute can contain one of the following values:

required
preferred (which is also the recommended default)
disabled

For a detailed explanation for how this works. Please see the high-availability documentation.

Attribute	Action	Description
default	`create`	The default pod anti-affinity to use for all Pods managed in a given PostgreSQL cluster.
pgBackRest	`create`	If set to a value that differs from `Default`, specifies the pod anti-affinity to use for just the pgBackRest repository.
pgBouncer	`create`	If set to a value that differs from `Default`, specifies the pod anti-affinity to use for just the pgBouncer Pods.

PostgreSQL Data Source Specification

This specification is used when one wants to bootstrap the data in a PostgreSQL cluster from a pgBackRest repository. This can be a pgBackRest repository that is attached to an active PostgreSQL cluster or is kept around to be used for spawning new PostgreSQL clusters.

Attribute	Action	Description
restoreFrom	`create`	The name of a PostgreSQL cluster, active or former, that will be used for bootstrapping the data of a new PostgreSQL cluster.
restoreOpts	`create`	Additional pgBackRest restore options that can be used as part of the bootstrapping operation, for example, point-in-time-recovery options.

TLS Specification

The TLS specification makes a reference to the various secrets that are required to enable TLS in a PostgreSQL cluster. For more information on how these secrets should be structured, please see Enabling TLS in a PostgreSQL Cluster.

Attribute	Action	Description
caSecret	`create`	A reference to the name of a Kubernetes Secret that specifies a certificate authority for the PostgreSQL cluster to trust.
replicationTLSSecret	`create`	A reference to the name of a Kubernetes TLS Secret that contains a keypair for authenticating the replication user. Must be used with `CASecret` and `TLSSecret`.
tlsSecret	`create`	A reference to the name of a Kubernetes TLS Secret that contains a keypair that is used for the PostgreSQL instance to identify itself and perform TLS communications with PostgreSQL clients. Must be used with `CASecret`.

pgBouncer Specification

The pgBouncer specification defines how a pgBouncer deployment can be deployed alongside the PostgreSQL cluster. pgBouncer is a PostgreSQL connection pooler that can also help manage connection state, and is helpful to deploy alongside a PostgreSQL cluster to help with failover scenarios too.

Attribute	Action	Description
limits	`create`, `update`	Specify the container resource limits that the pgBouncer Pods should use. Follows the Kubernetes definitions of resource limits.
replicas	`create`, `update`	The number of pgBouncer instances to deploy. Must be set to at least `1` to deploy pgBouncer. Setting to `0` removes an existing pgBouncer deployment for the PostgreSQL cluster.
resources	`create`, `update`	Specify the container resource requests that the pgBouncer Pods should use. Follows the Kubernetes definitions of resource requests.
serviceType	`create`, `update`	Sets the Kubernetes Service type to use for the cluster. If not set, defaults to the `ServiceType` set for the PostgreSQL cluster.
tlsSecret	`create`	A reference to the name of a Kubernetes TLS Secret that contains a keypair that is used for the pgBouncer instance to identify itself and perform TLS communications with PostgreSQL clients. Must be used with the parent Spec `TLSSecret` and `CASecret`.

Annotations Specification

The pgcluster.crunchydata.com specification contains a block that allows for custom Annotations to be applied to the Deployments that are managed by the PostgreSQL Operator, including:

PostgreSQL
pgBackRest
pgBouncer

This also includes the option to apply Annotations globally across the three different deployment groups.

Attribute	Action	Description
backrest	`create`, `update`	Specify annotations that are only applied to the pgBackRest deployments
global	`create`, `update`	Specify annotations that are applied to the PostgreSQL, pgBackRest, and pgBouncer deployments
pgBouncer	`create`, `update`	Specify annotations that are only applied to the pgBouncer deployments
postgres	`create`, `update`	Specify annotations that are only applied to the PostgreSQL deployments

`pgreplicas.crunchydata.com`

The pgreplicas.crunchydata.com Custom Resource Definition contains information pertaning to the structure of PostgreSQL replicas associated within a PostgreSQL cluster. All of the attributes only affect the replica when it is created.

Specification (`Spec`)

Attribute	Action	Description
clustername	`create`	The name of the PostgreSQL cluster, e.g. `hippo`. This is used to group PostgreSQL instances (primary, replicas) together.
name	`create`	The name of this PostgreSQL replica. It should be unique within a `ClusterName`.
namespace	`create`	The Kubernetes Namespace that the PostgreSQL cluster is deployed in.
nodeAffinity	`create`	Sets the node affinity rules for this PostgreSQL instance. Follows the Kubernetes standard format for setting node affinity.
replicastorage	`create`	A specification that gives information about the storage attributes for any replicas in the PostgreSQL cluster. For details, please see the `Storage Specification` section in the `pgclusters.crunchydata.com` description. This will likely be changed in the future based on the nature of the high-availability system, but presently it is still required that you set it. It is recommended you use similar settings to that of `PrimaryStorage`.
serviceType	`create`, `update`	Sets the Kubernetes Service type to use for this particular instance. If not set, defaults to the value in the related `pgclusters.crunchydata.com` custom resource.
userlabels	`create`	A set of key-value string pairs that are used as a sort of “catch-all” as well as a way to add custom labels to clusters. This will disappear at some point.
tolerations	`create`,`update`	Any array of Kubernetes Tolerations. Please refer to the Kubernetes documentation for how to set this field.

Using Custom Resources

Custom Resource Workflows

Create a PostgreSQL Cluster

Customizing User Credentials

Create a PostgreSQL Cluster With Backups in S3

Step 1: Create the pgBackRest S3 Secrets

Step 2: Create the PostgreSQL Cluster

Create a PostgreSQL Cluster with TLS

Step 1: Create TLS Secrets

Step 2: Create the PostgreSQL Cluster

Modify a Cluster

Modify Resource Requests & Limits

Scale

Monitoring

Add a Tablespace

pgBouncer

Start / Stop a Cluster

Manage Annotations

Delete a PostgreSQL Cluster

PostgreSQL Operator Custom Resource Definitions

Glossary

pgclusters.crunchydata.com

Specification (Spec)

Storage Specification

Node Affinity Specification

Pod Anti-Affinity Specification

PostgreSQL Data Source Specification

TLS Specification

pgBouncer Specification

Annotations Specification

pgreplicas.crunchydata.com

Specification (Spec)

`pgclusters.crunchydata.com`

Specification (`Spec`)

`pgreplicas.crunchydata.com`

Specification (`Spec`)