ref/transforms

ref/transforms

Transformation Functions

Some data types are implemented in a different enough way that a transformation function is necessary. This function must be written in Common lisp and is searched in the pgloader.transforms package.

Some default transformation function are provided with pgloader, and you can use the –load command line option to load and compile your own lisp file into pgloader at runtime. For your functions to be found, remember to begin your lisp file with the following form:

(in-package #:pgloader.transforms)

The provided transformation functions are:

  • zero-dates-to-null

    When the input date is all zeroes, return nil, which gets loaded as a PostgreSQL NULL value.

  • date-with-no-separator

    Applies zero-dates-to-null then transform the given date into a format that PostgreSQL will actually process:

    In:  "20041002152952"
    Out: "2004-10-02 15:29:52"
    
  • time-with-no-separator

    Transform the given time into a format that PostgreSQL will actually process:

    In:  "08231560"
    Out: "08:23:15.60"
    
  • tinyint-to-boolean

    As MySQL lacks a proper boolean type, tinyint is often used to implement that. This function transforms 0 to ‘false’ and anything else to ‘true’.

  • bits-to-boolean

    As MySQL lacks a proper boolean type, BIT is often used to implement that. This function transforms 1-bit bit vectors from 0 to f and any other value to t..

  • int-to-ip

    Convert an integer into a dotted representation of an ip4. :

    In:  18435761
    Out: "1.25.78.177"
    
  • ip-range

    Converts a couple of integers given as strings into a range of ip4. :

    In:  "16825344" "16825599"
    Out: "1.0.188.0-1.0.188.255"
    
  • convert-mysql-point

    Converts from the astext representation of points in MySQL to the PostgreSQL representation. :

    In:  "POINT(48.5513589 7.6926827)"
    Out: "(48.5513589,7.6926827)"
    
  • integer-to-string

    Converts a integer string or a Common Lisp integer into a string suitable for a PostgreSQL integer. Takes care of quoted integers. :

    In:  "\"0\""
    Out: "0"
    
  • float-to-string

    Converts a Common Lisp float into a string suitable for a PostgreSQL float:

    In:  100.0d0
    Out: "100.0"
    
  • hex-to-dec

    Converts a string containing an hexadecimal representation of a number into its decimal representation:

    In:  "deadbeef"
    Out: "3735928559"
    
  • set-to-enum-array

    Converts a string representing a MySQL SET into a PostgreSQL Array of Enum values from the set. :

    In: "foo,bar"
    Out: "{foo,bar}"
    
  • empty-string-to-null

    Convert an empty string to a null.

  • right-trim

    Remove whitespace at end of string.

  • remove-null-characters

    Remove NUL characters (0x0) from given strings.

  • byte-vector-to-bytea

    Transform a simple array of unsigned bytes to the PostgreSQL bytea Hex Format representation as documented at http://www.postgresql.org/docs/9.3/interactive/datatype-binary.html

  • sqlite-timestamp-to-timestamp

    SQLite type system is quite interesting, so cope with it here to produce timestamp literals as expected by PostgreSQL. That covers year only on 4 digits, 0 dates to null, and proper date strings.

  • sql-server-uniqueidentifier-to-uuid

    The SQL Server driver receives data fo type uniqueidentifier as byte vector that we then need to convert to an UUID string for PostgreSQL COPY input format to process.

  • unix-timestamp-to-timestamptz

    Converts a unix timestamp (number of seconds elapsed since beginning of 1970) into a proper PostgreSQL timestamp format.

  • varbinary-to-string

    Converts binary encoded string (such as a MySQL varbinary entry) to a decoded text, using the table’s encoding that may be overloaded with the DECODING TABLE NAMES MATCHING clause.