Accelerate the Value of Data

Create a Snowpipe in Snowflake (Azure)

Learn how to create a Snowpipe that copies data to the target table from the external stage in Snowflake.

Before you create a Snowpipe in Snowflake, you must have created a JSON file format, an event notification integration, a landing data table, and an external stage. For more information, see topicsCreate a JSON file format in Snowflake, Create event notification integrations in Snowflake (Azure) , Create a landing data table in Snowflake, and Create an external stage (Azure).

A Snowflake Snowpipe loads data from files as soon as they are available in a stage to the location specified in the Snowpipe. Create a Snowpipe to copy data from the external stage to your landing data tables in Snowflake.

To create a Snowpipe in your Snowflake environment:
  1. In the Snowflake application worksheet area, create a new SQL worksheet.
  2. In the SQL worksheet, run this command:
    create pipe "<database_name>"."<schema_name>"."<pipe_name>" auto_ingest=true
    integration=<notification_integration_name>
    as
      copy into "<database_name>"."<schema_name>"."<landing_table_name>"
    from (select $1:uri::string, $1:version::string, $1:timestamp::string, $1:deleted::boolean, $1:linked::boolean, $1:objectType::string, $1:type::string, $1, current_timestamp() from @"<database_name>"."<schema_name>"."<stage_name>")
      file_format = "<database_name>"."<schema_name>"."<file_format_name>";
    where:
    CREATE OR REPLACE PIPE "EDL_DEV_SILVER"."RELTIO_SNOWFLAKE_SCHEMA"."RELTIO_SNOWFLAKE_PIPE"
    auto_ingest=true
    integration='RELTIO_SNOWFLAKE_NOTIFY_INT'
    AS
    COPY INTO "EDL_DEV_SILVER"."RELTIO_SNOWFLAKE_SCHEMA"."RELTIO_SNOWFLAKE_LANDING_TBL"
    FROM (
        SELECT $1:uri::string, $1:version::string, $1:timestamp::string,
               $1:deleted::boolean, $1:linked::boolean, $1:objectType::string,
               $1:type::string, $1, current_timestamp()
        FROM @"EDL_DEV_SILVER"."RELTIO_SNOWFLAKE_SCHEMA"."RELTIO_SNOWFLAKE_EXT_STG"
    )
    file_format = EDL_DEV_SILVER.RELTIO_SNOWFLAKE_SCHEMA.reltio_snowflake_json_fmt;
    
  3. In the SQL worksheet, view the Successfully created notification message.
For general information, see Create pipe in the Snowflake SQL Command Reference.