Snowflake

Learn about streaming data into snowflake using Couchdrop.

Overview

Couchdrop can insert data into Snowflake through file actions, allowing you to ingest data from cloud storage, SFTP or FTP servers, APIs, and inbound webhooks into your Snowflake tables.

Rather than maintaining separate file-transfer and ingestion infrastructure, you can use Couchdrop to receive files and pass their contents into Snowflake as part of an Automation. This provides a way to connect existing file-based workflows with your Snowflake data warehouse.

How Snowflake works with Couchdrop

Snowflake is configured in Couchdrop as a platform integration. Unlike storage connections, a Snowflake integration does not appear as a virtual folder because Couchdrop needs information about the destination table and how the incoming data maps to it.

The setup has two parts:

1. Create the Snowflake connection as a system integration in Couchdrop.
2. Use that connection in a file action within an Automation.

When configuring the Snowflake file action, you provide information about the Snowflake table and how the incoming data should be inserted.

How to connect Snowflake to Couchdrop

Before you start

To connect Snowflake to Couchdrop, you will need:

• A Snowflake account with access to the table you want Couchdrop to use
• Your Snowflake Account ID
• A Snowflake programmatic access token
• The name of the destination Snowflake table
• A location or inbound method in Couchdrop that provides the data you want to ingest

Depending on your Snowflake configuration, you may also need the database, schema, warehouse, and role.

1

Create a Snowflake integration

Log in to Couchdrop and add a new system integration.

Select Snowflake from the list of available integrations.

2

Configure the Snowflake connection

Enter the required Snowflake connection details.

Setting	Description
Integration name	A name used to identify the Snowflake connection within Couchdrop.
Snowflake Account ID	The Account ID for the Snowflake account Couchdrop will connect to.
Snowflake Table Name	The Snowflake table Couchdrop will use for the integration.
Snowflake Access Token	The programmatic access token Couchdrop uses to authenticate to Snowflake.
Snowflake Database	Optional. Defaults to the DEFAULT_NAMESPACE user property in Snowflake if empty.
Snowflake Schema	Optional. Defaults to the DEFAULT_NAMESPACE user property in Snowflake if empty.
Snowflake Warehouse	Optional. Defaults to the DEFAULT_WAREHOUSE user property in Snowflake if empty.
Snowflake Role	Optional. Defaults to the DEFAULT_ROLE user property in Snowflake if empty.

Click Add System Integration and follow the wizard to save the connection.

3

Use Snowflake in an Automation

Create an Automation or file action and select Snowflake as the platform type.

Use the Snowflake connection you created, then configure the file action with the Snowflake table schema and how the incoming data maps to the destination table.

Next Steps

Once the Snowflake system integration is connected, you can select it as the platform for a file action within a Couchdrop Automation. The Automation can then use data arriving through the configured Couchdrop inbound method as input for the Snowflake workflow.

Creating a Snowflake programmatic access token

Couchdrop uses a Snowflake programmatic access token to authenticate the connection.

Snowflake can generate a token for a user with:

ALTER USER IF EXISTS <example_user>
ADD PROGRAMMATIC ACCESS TOKEN example_token;

Snowflake returns the token secret when the token is created. This is the only time the token secret is displayed, so copy and store it securely before closing the result.

For a comprehensive guide on generating an access token, see https://docs.snowflake.com/en/user-guide/programmatic-access-tokens

Common issues connecting to Snowflake

The network policy setup for your user in Snowflake can block us from authenticating and connecting to your Snowflake account. The authentication policy setup for your user in Snowflake can prevent the network policy from blocking us, if it is an issue, by running

CREATE|ALTER AUTHENTICATION POLICY <my_policy_name>
  PAT_POLICY=(
    NETWORK_POLICY_EVALUATION = NOT_REQUIRED
  );

The authentication policy can also block us from connecting if the authentication methods have been configured, and do not include PROGRAMMATIC_ACCESS_TOKEN in the available authentication methods. To see if this is the case you can run

DESCRIBE AUTHENTICATION POLICY <authentication_policy_name>;

To add the authentication method to your authentication policy, you can run

ALTER AUTHENTICATION POLICY <authentication_policy_name>
  SET AUTHENTICATION_METHODS = ('PROGRAMMATIC_ACCESS_TOKEN');

To generate the programmatic access token

ALTER USER IF EXISTS <example_user> ADD PROGRAMMATIC ACCESS TOKEN example_token;