WAL Management

In Crunchy Postgres for Kubernetes, archiving of the write-ahead log (WAL) is handled by pgBackRest, the same tool used to manage backups and restores. It's important to keep an archive of WAL for recovery purposes. A backup only ever captures the state of your database on disk. WAL captures the state of your database in memory. Together, a backup and WAL can restore your database to its production state just before an outage.

Keeping a separate WAL volume

It's best to keep WAL on a separate volume from your pgdata directory. Doing so is more performant and prevents disk exhaustion on the pgdata volume from affecting WAL storage. You can provision a dedicated WAL volume like this:

spec:
  instances:
    - name: instance1
      walVolumeClaimSpec:
        accessModes:
        - "ReadWriteOnce"
        resources:
          requests:
            storage: 1Gi
Copy

WAL archiving

When pgBackRest archives WAL, log files get copied out of the wal directory and compressed at their destination. Postgres can then recycle WAL files in the wal directory, reducing the amount of space required for normal operations. Crunchy Postgres for Kubernetes v5.7+ configures pgBackRest to use asynchronous archiving for robust and performant offloading of WAL.

WAL can be stored in either mounted storage or a cloud-based object store. A mounted volume can be allocated like this:

spec:
  backups:
    pgBackRest:
      repos:
      - name: repo1
        volume:
          volumeClaimSpec:
            accessModes:
            - "ReadWriteOnce"
            resources:
              requests:
                storage: 1Gi
Copy

An object store, like s3, can be allocated like this:

spec:
  backups:
    pgBackRest:
      repos:
      - name: repo1
        s3:
          bucket: "the-name-of-your-bucket"
          endpoint: "s3.us-east-1.amazonaws.com"
          region: "us-east-1"
Copy

For details on configuring different object stores and using multiple repos, see our tutorial on Backup Configuration.

For further information on the relationship between WAL retention and backup retention, see the --repo-retention-archive section of the pgBackRest configuration docs.

If for any reason you would like to opt out of asynchronous archiving, apply the following configuration:

spec:
  backups:
    pgbackrest:
      global:
        archive-async: n
Copy

WAL archive logging

Logs for WAL archiving can be found in pgdata/pgbackrest/log/. The log level can be adjusted through pgBackRest's global settings.

spec:
  backups:
    pgbackrest:
      global:
        log-level-console: warn
        log-level-file: warn
Copy

Log levels less than error are not recommended. See the pgBackRest Configuration docs for further details.