patroni.postgresql package

Submodules

Module contents

class patroni.postgresql. Postgresql ( config : Dict [ str , Any ] ) View on GitHub

Bases: object

POSTMASTER_START_TIME = 'pg_catalog.pg_postmaster_start_time()'

TL_LSN = "CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_{0}file_name(pg_catalog.pg_current_{0}_{1}()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_current_{0}{2}_{1}(), '0/0')::bigint END, pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_last_{0}_replay_{1}(), '0/0')::bigint, pg_catalog.pg_{0}_{1}_diff(COALESCE(pg_catalog.pg_last_{0}_receive_{1}(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_{0}_replay_paused()"

__init__ ( config : Dict [ str , Any ] ) → None View on GitHub

_before_stop ( ) → None View on GitHub: Synchronously run a script prior to stopping postgres.

_checkpoint_locations_from_controldata ( data : Dict [ str , str ] ) → Tuple [ int , int ] | None View on GitHub

Get shutdown checkpoint location.

Parameters :: data – dict object with values returned by pg_controldata tool.
Returns :: a tuple of checkpoint LSN for the cleanly shut down primary, and LSN of prev wal record (SWITCH) if we know that the checkpoint was written to the new WAL file due to the archive_mode=on.

_cluster_info_state_get ( name : str ) → Any | None View on GitHub

_do_before_stop ( cmd : str ) → None View on GitHub

_do_stop ( mode : str , block_callbacks : bool , checkpoint : bool , on_safepoint : Callable [ [ ... ] , Any ] | None , on_shutdown : Callable [ [ int , int ] , Any ] | None , before_shutdown : Callable [ [ ... ] , Any ] | None , stop_timeout : int | None ) → Tuple [ bool , bool ] View on GitHub

_get_gucs ( ) → CaseInsensitiveSet View on GitHub

Get all available GUCs based on postgres --describe-config output.

Returns :: all available GUCs in the local Postgres server.

_pre_promote ( ) → bool View on GitHub: Runs a fencing script after the leader lock is acquired but before the replica is promoted. If the script exits with a non-zero code, promotion does not happen and the leader key is removed from DCS.

_query ( sql : str , * params : Any ) → List [ Tuple [ Any , ... ] ] View on GitHub

Execute sql query with params and optionally return results.

Parameters :

sql – SQL statement to execute.
params – parameters to pass.

Returns :

a query response as a list of tuples if there is any.

Raises :

Error if had issues while executing sql .

PostgresConnectionException : if had issues while connecting to the database.

RetryFailedError : if it was detected that connection/query failed due to PostgreSQL restart.

_version_file_exists ( ) → bool View on GitHub

_wait_for_connection_close ( postmaster : PostmasterProcess ) → None View on GitHub

_wait_promote ( wait_seconds : int ) → bool | None View on GitHub

static _wal_position ( is_primary : bool , wal_position : int , received_location : int | None , replayed_location : int | None ) → int View on GitHub

property available_gucs : CaseInsensitiveSet View on GitHub: GUCs available in this Postgres server.

call_nowait ( cb_type : CallbackAction ) → None View on GitHub: pick a callback command and call it without waiting for it to finish

property callback : Dict [ str , str ] View on GitHub

property can_advance_slots : bool View on GitHub: True if :attr: major_version is greater than 110000.

can_create_replica_without_replication_connection ( replica_methods : List [ str ] | None ) → bool View on GitHub: go through the replication methods to see if there are ones that does not require a working replication connection.

property cb_called : bool View on GitHub

check_for_startup ( ) → bool View on GitHub: Checks PostgreSQL status and returns if PostgreSQL is in the middle of startup.

check_startup_state_changed ( ) → bool View on GitHub

Checks if PostgreSQL has completed starting up or failed or still starting.

Should only be called when state == ‘starting’

Returns :: True if state was changed from ‘starting’

checkpoint ( connect_kwargs : Dict [ str , Any ] | None = None , timeout : float | None = None ) → str | None View on GitHub

property cluster_info_query : str View on GitHub

Returns the monitoring query with a fixed number of fields.

The query text is constructed based on current state in DCS and PostgreSQL version:

function names depend on version. wal/lsn for v10+ and xlog/location for pre v10.
for primary we query timeline_id (extracted from pg_walfile_name()) and pg_current_wal_lsn()
for replicas we query pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn(), and pg_is_wal_replay_paused()
for v9.6+ we query primary_slot_name and primary_conninfo from pg_stat_get_wal_receiver()
for v11+ with permanent logical slots we query from pg_replication_slots and aggregate the result
for standby_leader node running v9.6+ we also query pg_control_checkpoint to fetch timeline_id
if sync replication is enabled we query pg_stat_replication and aggregate the result. In addition to that we get current values of synchronous_commit and synchronous_standby_names GUCs.

If some conditions are not satisfied we simply put static values instead. E.g., NULL, 0, ‘’, and so on.

configure_server_parameters ( ) → None View on GitHub

connection ( ) → connection3 | Connection3 [ Any ] View on GitHub

controldata ( ) → Dict [ str , str ] View on GitHub: return the contents of pg_controldata, or non-True value if pg_controldata call failed

property create_replica_methods : List [ str ] View on GitHub

property data_dir : str View on GitHub

data_directory_empty ( ) → bool View on GitHub

property database : str View on GitHub

property enforce_hot_standby_feedback : bool View on GitHub

ensure_major_version_is_known ( ) → bool View on GitHub

Calls configure_server_parameters() if _major_version is not known

Returns :: True if _major_version is set, otherwise False

follow ( member : Leader | Member | None , role : str = 'replica' , timeout : float | None = None , do_reload : bool = False ) → bool | None View on GitHub

Reconfigure postgres to follow a new member or use different recovery parameters.

Method may call on_role_change callback if role is changing.

Parameters :

member – The member to follow
role – The desired role, normally ‘replica’, but could also be a ‘standby_leader’
timeout – start timeout, how long should the start() method wait for postgres accepting connections
do_reload – indicates that after updating postgresql.conf we just need to do a reload instead of restart

Returns :

True - if restart/reload were successfully performed, False - if restart/reload failed None - if nothing was done or if Postgres is still in starting state after timeout seconds.

get_guc_value ( name : str ) → str | None View on GitHub

get_history ( timeline : int ) → List [ Tuple [ int , int , str ] | Tuple [ int , int , str , str , str ] ] View on GitHub

get_major_version ( ) → int View on GitHub

Reads major version from PG_VERSION file

Returns :: major PostgreSQL version in integer format or 0 in case of missing file or errors

get_postgres_role_from_data_directory ( ) → str View on GitHub

get_primary_timeline ( ) → int View on GitHub

Returns :: current timeline if postgres is running as a primary or 0.

get_replica_timeline ( ) → int | None View on GitHub

get_replication_connection_cursor ( host : str | None = None , port : int | str = 5432 , ** kwargs : Any ) → Iterator [ cursor | Cursor [ Any ] ] View on GitHub

property global_config : GlobalConfig | None View on GitHub

handle_parameter_change ( ) → None View on GitHub

initdb ( * args : str , ** kwargs : Any ) → bool View on GitHub

Builds and executes the initdb command.

Parameters :

args – List of arguments to be joined into the initdb command.
kwargs – Keyword arguments to pass to subprocess.call .

Returns :

True if the result of subprocess.call`, the exit code, is ``0 .

is_healthy ( ) → bool View on GitHub

is_primary ( ) → bool View on GitHub

is_running ( ) → PostmasterProcess | None View on GitHub: Returns PostmasterProcess if one is running on the data directory or None. If most recently seen process is running updates the cached process based on pid file.

is_starting ( ) → bool View on GitHub

last_operation ( ) → int View on GitHub

latest_checkpoint_location ( ) → int | None View on GitHub

Get shutdown checkpoint location.

Note

In case if checkpoint was written to the new WAL file due to the archive_mode=on we return LSN of the previous wal record (SWITCH).

Returns :: checkpoint LSN for the cleanly shut down primary.

property lsn_name : str View on GitHub

property major_version : int View on GitHub

move_data_directory ( ) → None View on GitHub

parse_wal_record ( timeline : str , lsn : str ) → Tuple [ str , str , str , str ] | Tuple [ None , None , None , None ] View on GitHub

property pending_restart : bool View on GitHub

pg_control_exists ( ) → bool View on GitHub

pg_control_timeline ( ) → int | None View on GitHub

pg_ctl ( cmd : str , * args : str , ** kwargs : Any ) → bool View on GitHub

Builds and executes pg_ctl command

Returns :: !True when return_code == 0, otherwise !False

pg_isready ( ) → str View on GitHub

Runs pg_isready to see if PostgreSQL is accepting connections.

Returns :: ‘ok’ if PostgreSQL is up, ‘reject’ if starting up, ‘no_resopnse’ if not up.

pg_stat_replication ( ) → List [ Dict [ str , Any ] ] View on GitHub

Returns :: a result set of ‘SELECT * FROM pg_stat_replication’.

pg_tblspc_realpaths ( ) → Dict [ str , str ] View on GitHub: Returns a dict containing the symlink (key) and target (values) for the tablespaces

pg_wal_realpath ( ) → Dict [ str , str ] View on GitHub: Returns a dict containing the symlink (key) and target (value) for the wal directory

pgcommand ( cmd : str ) → str View on GitHub

Return path to the specified PostgreSQL command.

Note

If postgresql.bin_name.*cmd* was configured by the user then that binary name is used, otherwise the default binary name cmd is used.

Parameters :: cmd – the Postgres binary name to get path to.
Returns :: path to Postgres binary named cmd .

postmaster_start_time ( ) → str | None View on GitHub

primary_conninfo ( ) → str | None View on GitHub

primary_slot_name ( ) → str | None View on GitHub

promote ( wait_seconds : int , task : CriticalTask , before_promote : Callable [ [ ... ] , Any ] | None = None ) → bool | None View on GitHub

query ( sql : str , * params : Any , retry : bool = True ) → List [ Tuple [ Any , ... ] ] View on GitHub

Execute sql query with params and optionally return results.

Parameters :

sql – SQL statement to execute.
params – parameters to pass.
retry – whether the query should be retried upon failure or given up immediately.

Returns :

a query response as a list of tuples if there is any.

Raises :

Error if had issues while executing sql .

PostgresConnectionException : if had issues while connecting to the database.

RetryFailedError : if it was detected that connection/query failed due to PostgreSQL restart or if retry deadline was exceeded.

received_location ( ) → int | None View on GitHub

received_timeline ( ) → int | None View on GitHub

reload ( block_callbacks : bool = False ) → bool View on GitHub

reload_config ( config : Dict [ str , Any ] , sighup : bool = False ) → None View on GitHub

remove_data_directory ( ) → None View on GitHub

replay_paused ( ) → bool View on GitHub

replayed_location ( ) → int | None View on GitHub

replica_cached_timeline ( primary_timeline : int | None ) → int | None View on GitHub

replica_method_can_work_without_replication_connection ( method : str ) → bool View on GitHub

replica_method_options ( method : str ) → Dict [ str , Any ] View on GitHub

replication_state ( ) → str | None View on GitHub

Checks replication state from pg_stat_get_wal_receiver() .

Note

Available only since 9.6

Returns :: streaming , in archive recovery , or None

replication_state_from_parameters ( is_primary : bool , receiver_state : str | None , restore_command : str | None ) → str | None View on GitHub

Figure out the replication state from input parameters.

Note

This method could be only called when Postgres is up, running and queries are successfuly executed.

Is_primary :

True is postgres is not running in recovery

Receiver_state :

value from pg_stat_get_wal_receiver.state or None if Postgres is older than 9.6

Restore_command :

value of restore_command GUC for PostgreSQL 12+ or postgresql.recovery_conf.restore_command if it is set in Patroni configuration

Returns :

None for the primary and for Postgres older than 9.6;
’streaming’ if replica is streaming according to the pg_stat_wal_receiver view;
’in archive recovery’ if replica isn’t streaming and there is a restore_command

reset_cluster_info_state ( cluster : Cluster | None , nofailover : bool = False , global_config : GlobalConfig | None = None ) → None View on GitHub

Reset monitoring query cache.

It happens in the beginning of heart-beat loop and on change of synchronous_standby_names .

Parameters :

cluster – currently known cluster state from DCS
nofailover – whether this node could become a new primary. Important when there are logical permanent replication slots because "nofailover" node could do cascading replication and should enable hot_standby_feedback
global_config – last known GlobalConfig object

Restarts PostgreSQL.

When timeout parameter is set the call will block either until PostgreSQL has started, failed to start or timeout arrives.

Returns :: True when restart was successful and timeout did not expire when waiting.

resume_wal_replay ( ) → None View on GitHub

property role : str View on GitHub

schedule_sanity_checks_after_pause ( ) → None View on GitHub: After coming out of pause we have to: 1. configure server parameters if necessary 2. sync replication slots, because it might happen that slots were removed 3. get new ‘Database system identifier’ to make sure that it wasn’t changed

property server_version : int View on GitHub

set_enforce_hot_standby_feedback ( value : bool ) → None View on GitHub

set_pending_restart ( value : bool ) → None View on GitHub

set_role ( value : str ) → None View on GitHub

set_state ( value : str ) → None View on GitHub

slots ( ) → Dict [ str , int ] View on GitHub

Start PostgreSQL

Waits for postmaster to open ports or terminate so pg_isready can be used to check startup completion or failure.

Returns :: True if start was initiated and postmaster ports are open, False if start failed, and None if postgres is still starting up

property state : str View on GitHub

stop ( mode : str = 'fast' , block_callbacks : bool = False , checkpoint : bool | None = None , on_safepoint : Callable [ [ ... ] , Any ] | None = None , on_shutdown : Callable [ [ int , int ] , Any ] | None = None , before_shutdown : Callable [ [ ... ] , Any ] | None = None , stop_timeout : int | None = None ) → bool View on GitHub

Stop PostgreSQL

Supports a callback when a safepoint is reached. A safepoint is when no user backend can return a successful commit to users. Currently this means we wait for user backends to close. But in the future alternate mechanisms could be added.

Parameters :

on_safepoint – This callback is called when no user backends are running.
on_shutdown – is called when pg_controldata starts reporting Database cluster state: shut down
before_shutdown – is called after running optional CHECKPOINT and before running pg_ctl stop

property supports_multiple_sync : bool View on GitHub

True if Postgres version supports more than one synchronous node.

Type :: returns

synchronous_commit ( ) → str View on GitHub

Returns :: "synchronous_commit" GUC value.

synchronous_standby_names ( ) → str View on GitHub

Returns :: "synchronous_standby_names" GUC value.

property sysid : str View on GitHub

terminate_postmaster ( postmaster : PostmasterProcess , mode : str , stop_timeout : int | None ) → bool | None View on GitHub

terminate_starting_postmaster ( postmaster : PostmasterProcess ) → None View on GitHub: Terminates a postmaster that has not yet opened ports or possibly even written a pid file. Blocks until the process goes away.

time_in_state ( ) → float View on GitHub

timeline_wal_position ( ) → Tuple [ int , int , int | None ] View on GitHub

wait_for_port_open ( postmaster : PostmasterProcess , timeout : float ) → bool View on GitHub: Waits until PostgreSQL opens ports.

wait_for_startup ( timeout : float = 0 ) → bool | None View on GitHub

Waits for PostgreSQL startup to complete or fail.

Returns :: True if start was successful, False otherwise

property wal_dir : str View on GitHub

property wal_flush : str View on GitHub: For PostgreSQL 9.6 onwards we want to use pg_current_wal_flush_lsn()/pg_current_xlog_flush_location().

property wal_name : str View on GitHub

waldump ( timeline : int | str , lsn : str , limit : int ) → Tuple [ bytes | None , bytes | None ] View on GitHub

patroni.postgresql. null_context ( ) View on GitHub