Writing a Foreign Data Wrapper

• Requirements
• Known Issues and Limitations
• Header Files
• Foreign Data Wrapper Functions
• Foreign Data Wrapper Callback Functions
• Foreign Data Wrapper Helper Functions
• Greenplum Database Considerations
• Building a Foreign Data Wrapper Extension with PGXS
• Deployment Considerations

• You must develop your code on a system with the same hardware and software architecture as that of your Greenplum Database hosts.
• Your code must be written in a compiled language such as C, using the version-1 interface. For details on C language calling conventions and dynamic loading, refer to C Language Functions in the PostgreSQL documentation.
• Symbol names in your object files must not conflict with each other nor with symbols defined in the Greenplum Database server. You must rename your functions or variables if you get error messages to this effect.
• Review the foreign table introduction described in Accessing External Data with Foreign Tables.

• The Greenplum Database 6 distribution does not install any foreign data wrappers.
• Greenplum Database uses the mpp_execute option value for foreign table scans only. Greenplum does not honor the mpp_execute setting when you write to, or update, a foreign table; all write operations are initiated through the master.

• foreign/fdwapi.h - FDW API structures and callback function signatures
• foreign/foreign.h - foreign-data wrapper helper structs and functions
• catalog/pg_foreign_table.h - foreign table definition
• catalog/pg_foreign_server.h - foreign server definition

CREATE FUNCTION NEW_fdw_handler()
  RETURNS fdw_handler
  AS ‘MODULE_PATHNAME‘
LANGUAGE C STRICT;

Most of the effort in writing a foreign-data wrapper is in implementing the callback functions. The FDW API callback functions, plain C functions that are not visible or callable at the SQL level, are described in Foreign Data Wrapper Callback Functions.

The validator function is responsible for validating options provided in CREATE and ALTER commands for its foreign-data wrapper, as well as foreign servers, user mappings, and foreign tables using the wrapper. The validator function must be registered as taking two arguments, a text array containing the options to be validated, and an OID representing the type of object with which the options are associated. For example:

CREATE FUNCTION NEW_fdw_validator( text[], oid )
  RETURNS void
  AS ‘MODULE_PATHNAME‘
LANGUAGE C STRICT;

/*
 * FdwRoutine is the struct returned by a foreign-data wrapper‘s handler
 * function.  It provides pointers to the callback functions needed by the
 * planner and executor.
 *
 * More function pointers are likely to be added in the future.  Therefore
 * it‘s recommended that the handler initialize the struct with
 * makeNode(FdwRoutine) so that all fields are set to NULL.  This will
 * ensure that no fields are accidentally left undefined.
 */
typedef struct FdwRoutine
{
	NodeTag		type;

	/* Functions for scanning foreign tables */
	GetForeignRelSize_function GetForeignRelSize;
	GetForeignPaths_function GetForeignPaths;
	GetForeignPlan_function GetForeignPlan;
	BeginForeignScan_function BeginForeignScan;
	IterateForeignScan_function IterateForeignScan;
	ReScanForeignScan_function ReScanForeignScan;
	EndForeignScan_function EndForeignScan;

	/*
	 * Remaining functions are optional.  Set the pointer to NULL for any that
	 * are not provided.
	 */

	/* Functions for updating foreign tables */
	AddForeignUpdateTargets_function AddForeignUpdateTargets;
	PlanForeignModify_function PlanForeignModify;
	BeginForeignModify_function BeginForeignModify;
	ExecForeignInsert_function ExecForeignInsert;
	ExecForeignUpdate_function ExecForeignUpdate;
	ExecForeignDelete_function ExecForeignDelete;
	EndForeignModify_function EndForeignModify;
	IsForeignRelUpdatable_function IsForeignRelUpdatable;

	/* Support functions for EXPLAIN */
	ExplainForeignScan_function ExplainForeignScan;
	ExplainForeignModify_function ExplainForeignModify;

	/* Support functions for ANALYZE */
	AnalyzeForeignTable_function AnalyzeForeignTable;
} FdwRoutine;

#include "foreign/foreign.h"

ForeignTable *table = GetForeignTable(foreigntableid);
if (table->exec_location == FTEXECLOCATION_ALL_SEGMENTS)
{
    ...
}
else if (table->exec_location == FTEXECLOCATION_ANY)
{
    ...
}
else if (table->exec_location == FTEXECLOCATION_MASTER)
{
    ...
} 

If the foreign table was not created with an mpp_execute option setting, the mpp_execute setting of the foreign server, and then the foreign data wrapper, is probed and used. If none of the foreign-data-related objects has an mpp_execute setting, the default setting is master.

If a foreign-data wrapper supports mpp_execute ‘all‘, it will implement a policy that matches Greenplum segments to data. So as not to duplicate data retrieved from the remote, the FDW on each segment must be able to establish which portion of the data is their responsibility. An FDW may use the segment identifier and the number of segments to help make this determination. The following code snippet demonstrates how a foreign-data wrapper may retrieve the segment number and total number of segments:

int segmentNumber = GpIdentity.segindex;
int totalNumberOfSegments = getgpsegmentCount();

MODULE_big = base_fdw
OBJS = base_fdw_1.o base_fdw_2.o

PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)

PG_CPPFLAGS = -I$(shell $(PG_CONFIG) --includedir)
SHLIB_LINK = -L$(shell $(PG_CONFIG) --libdir)
include $(PGXS)

A description of the directives used in this Makefile follows:

• MODULE_big - identifes the base name of the shared library generated by the Makefile
• PG_CPPFLAGS - adds the Greenplum Database installation include/ directory to the compiler header file search path
• SHLIB_LINK adds the Greenplum Database installation library directory ($GPHOME/lib/) to the linker search path
• The PG_CONFIG and PGXS variable settings and the include statement are required and typically reside in the last three lines of the Makefile.

To package the foreign-data wrapper as a Greenplum Database extension, you create script (newfdw--version.sql) and control (newfdw.control) files that register the FDW handler and validator functions, create the foreign data wrapper, and identify the characteristics of the FDW shared library file.

Example foreign-data wrapper extension script file named base_fdw--1.0.sql:

CREATE FUNCTION base_fdw_handler()
  RETURNS fdw_handler
  AS ‘MODULE_PATHNAME‘
LANGUAGE C STRICT;

CREATE FUNCTION base_fdw_validator(text[], oid)
  RETURNS void
  AS ‘MODULE_PATHNAME‘
LANGUAGE C STRICT;

CREATE FOREIGN DATA WRAPPER base_fdw
  HANDLER base_fdw_handler
  VALIDATOR base_fdw_validator;

Example FDW control file named base_fdw.control:

# base_fdw FDW extension
comment = ‘base foreign-data wrapper implementation; does not do much‘
default_version = ‘1.0‘
module_pathname = ‘$libdir/base_fdw‘
relocatable = true

When you add the following directives to the Makefile, you identify the FDW extension control file base name (EXTENSION) and SQL script (DATA):

EXTENSION = base_fdw
DATA = base_fdw--1.0.sql

• The FDW shared library must be installed to the same file system location on the master host and on every segment host in the Greenplum Database cluster. You specify this location in the .control file. This location is typically the $GPHOME/lib/postgresql/ directory.
• The FDW .sql and .control files must be installed to the $GPHOME/share/postgresql/extension/ directory on the master host and on every segment host in the Greenplum Database cluster.
• The gpadmin user must have permission to traverse the complete file system path to the FDW shared library file and extension files.