Snowflake connection
To access your data in Snowflake, create a connection asset for it.
Snowflake is a cloud-based data storage and analytics service.
Create a connection to Snowflake
To create the connection asset, you need the following connection details:
- Account name: The full name of your account
- Database name
- Role: The default access control role to use in the Snowflake session
- Warehouse: The virtual warehouse
Credentials
Authentication method:
-
Username and password
-
Key-Pair: Enter the contents of the private key and the key passphrase (if configured). These properties must be set up by the Snowflake administrator. For information, see Key Pair Authentication & Key Pair Rotation in the Snowflake documentation.
-
Okta URL endpoint: If your company uses native Okta SSO authentication, enter the Okta URL endpoint for your Okta account. Example:
. Leave this field blank if you want to use the default authentication of Snowflake. For information about federated authentication provided by Okta, see Native SSO.https://<okta_account_name>.okta.com
Choose the method for creating a connection based on where you are in the platform
- In a project
- Click Assets > New asset > Connect to a data source. See Adding a connection to a project.
- In a catalog
- Click Add to catalog > Connection. See Adding a connection asset to a catalog.
- In a deployment space
- Click Import assets > Data access > Connection. See Adding data assets to a deployment space.
- In the Platform assets catalog
- Click New connection. See Adding platform connections.
Next step: Add data assets from the connection
Snowflake setup
Running SQL statements
To ensure that your SQL statements run correctly, refer to the Snowflake SQL Command Reference for the correct syntax.
Configuring lineage metadata import for Snowflake
When you create a metadata import for the Snowflake connection, you can set options specific to this data source, and define the scope of data for which lineage is generated. For details about metadata import, see Designing metadata imports.
To import lineage metadata for Snowflake, complete these steps:
- Create a data source definition. Select Snowflake as the data source type.
- Create a connection to the data source in a project.
- Create a metadata import. Learn more about options that are specific to Snowflake data source:
- When you define a scope, you can analyze the entire data source or use the include and exclude options to define the exact databases and schemas that you want to be analyzed. See Include and exclude lists.
- Optionally, you can provide external input in the form of a .zip file. You add this file in the Add inputs from file field. The file must have a supported structure. See External inputs.
- Optionally, specify advanced import options.
Include and exclude lists
You can include or exclude assets up to the schema level. Provide databases and schemas in the format database/schema. Each part is evaluated as a regular expression. Assets which are added later in the data source will also be included or excluded if they match the conditions specified in the lists. Example values:
: all schemas inmyDB/
database.myDB
: all schemas inmyDB2/.*
database.myDB2
:myDB3/mySchema1
schema frommySchema1
database.myDB3
: any schema in mymyDB4/mySchema[1-5]
database with a name that starts withmyDB4
and ends with a digit between 1 and 5.mySchema
External inputs
If you use external Snowflake SQL scripts, you can add them in a .zip file as an external input. You can organize the structure of a .zip file as subfolders that represent databases and schemas. After the scripts are scanned, they are added under respective databases and schemas in the selected catalog or project. The .zip file can have the following structure:
<database_name> <schema_name> <script_name.sql> <database_name> <script_name.sql> <script_name.sql> replace.csv
The
file contains placeholder replacements for the scripts that are added in the .zip file. For more information about the format, see Placeholder replacements.replace.csv
Advanced import options
- Table stages extraction
- You can add a regular expression to list table stages from which you want staged files to be extracted. Use a fully qualified name and enclose each segment with double quotation marks. Leave the field empty if you do not want to extract staged files from any table stages. Example value:
\\\"mydb\\\"\\.\\\"schema1\\\"\\.\\\".*\\\"|\\\"mydb\\\"\\.\\\"myschema\\\"\\.\\\"abc.*\\\
- Transformation logic extraction
- You can enable building transformation logic descriptions from SQL code in SQL scripts.
Learn more
Parent topic: Supported connections