CSV file format for importing metadata asset details
The CSV file for importing asset metadata details must conform to specific formatting rules.
Limitations
-
File encoding is UTF-8.
-
The maximum recommended size of a file is 10,000 lines.
-
You can specify columns definition only for data assets.
-
You can import and export data assets with custom attributes only if the attributes belong to a group. If you import or export such data assets anyway, the custom attributes fields are not included.
-
You must follow asset definition by its column definition.
-
Unexpected extra properties are included in the exported CSV file. Do not edit these properties.
Asset Property: data_asset>>dataset Asset Property: data_asset>>mime_type Asset Property: data_asset>>properties>>name Asset Property: data_asset>>properties>>name Asset Property: data_asset>>semantic_description>>confidence Asset Property: data_asset>>semantic_description>>generated_date Asset Property: data_asset>>semantic_description>>generated_description Asset Property: data_asset>>semantic_description>>status Asset Property: discovered_asset>>connection_id Asset Property: discovered_asset>>connection_path Asset Property: discovered_asset>>discovery_id Asset Property: discovered_asset>>extended_metadata>>name Asset Property: discovered_asset>>extended_metadata>>name Asset Property: discovered_asset>>extended_metadata>>name Asset Property: discovered_asset>>first_imported_timestamp Asset Property: discovered_asset>>last_discovered_timestamp Asset Property: discovered_asset>>last_imported_timestamp Asset Property: discovered_asset>>last_job_action Asset Property: discovered_asset>>last_job_run_id Asset Property: discovered_asset>>metadata_import_id Asset Property: discovered_asset>>outdated_reason Asset Property: discovered_asset>>outdated_timestamp Asset Property: key_analyses>>fk_assigned Asset Property: key_analyses>>fk_assigned_as_pk Asset Property: key_analyses>>fk_defined Asset Property: key_analyses>>fk_defined_as_pk Asset Property: key_analyses>>fk_suggested Asset Property: key_analyses>>fk_suggested_as_pk Asset Property: key_analyses>>key_analysis_area_id Asset Property: key_analyses>>overlap_assigned Asset Property: key_analyses>>overlap_suggested Asset Property: key_analyses>>pk_assigned Asset Property: key_analyses>>pk_defined Asset Property: key_analyses>>pk_suggested Asset Property: metadata_enrichment_info>>MDE_instrumented Asset Property: term_assignment_profile>>attachment_id Asset Property: term_assignment_profile>>completed_date Asset Property: term_assignment_profile>>messages Asset Property: term_assignment_profile>>messages Asset Property: term_assignment_profile>>messages Asset Property: term_assignment_profile>>semexp_completed_date Asset Property: term_assignment_profile>>semexp_messages Asset Property: term_assignment_profile>>semexp_start_date Asset Property: term_assignment_profile>>semexp_status Asset Property: term_assignment_profile>>start_date Asset Property: term_assignment_profile>>status
Header row
The headers in the first row of the CSV file represent which properties are imported for the assets.
- The header row must be the first row in the file and must not be repeated.
- The header must include the mandatory headers.
Mandatory headers
The header row of the CSV file must include the following mandatory headers:
- Name
- Example:
my_asset
. - Type
- All asset types are supported. Columns definition is supported only for the
data_asset
type.
Example:data_asset
Optional headers
To specify more detailed asset metadata, add appropriate optional headers to your CSV file.
- Asset Resource Key
- Uniquely identifies the asset in the catalog. Automatically included in every export CSV file.
Example:Test asset
.
Owner
: Emails of the asset owners. Owners must be catalog collaborators. Multiple values are allowed.
Example: [email protected]
- Description
- Asset or column descriptions.
Example:This is a test asset
. - Tag
- Tags to identify assets or columns. Multiple values are allowed.
Example:tag1
- Term
- Assigned business terms. The business term must exist. Use the path format. Multiple values are allowed.
Example:AlexTestCategory>>sub category>>category one>>new term
- Classification
- Assigned classifications. Use the path format. Multiple values are allowed.
Example:AlexTestCategory>>sub category>>category>>new classification
. - Data Class
- Data class that is assigned to a column in a data asset. Use the path format. The root category
[uncategorized]
must always be present. A blank entry does not replace it.
Example:[uncategorized]>>Computer Host Name
- Column Source Type
- Column data type.
Example:varchar
. - Column Size
- Column data size.
Example:1024
- Column Type Nullable
- If column type could be null, use a boolean.
Example:TRUE
- Column Native Type
- Column native data type.
Example,enum
. - Asset Property
- Asset custom property. Multi columns must have the attribute path in the header. The property is defined in the column, the value of the property in the row. For column names,
"Asset property:"
is used as the prefix.enum
types path must end withname
. Multiple values are allowed.
Example:
Column name:Asset Property: azub_group_test>>prop_ahhatx
Column value:value of prop1
- Column Property
- Column custom property. Multi columns must have the attribute path in the header. The property is defined in the column, the value of the property in the row. For column names,
"Asset property:"
is used as the prefix.enum
types path must end withname
. Multiple values are allowed.
Example:
Columns name:Column Property: col_group_zevkys>>prop_qblrui
Columns value:host1
Formatting category and attribute paths
-
You must specify the full category and attribute paths. If you do not specify the category, the default category is [uncategorized].
-
To delimit the category path and attribute path, use two greater than
>>
symbols between each level of the category or attribute hierarchy and between the category path or attribute path and the artifact name. -
List the category hierarchy or attribute hierarchy from the top-level category name through the final category or attribute name, and separate each with the
>>
symbols.For example, to create a third level category or attribute, your Category field might look like this:
myCategory1>>myCategory2>>myCategory3
The root category [uncategorized] must always be present. A blank entry does not replace it. You can't specify the [uncategorized] category as a secondary category and it can't have subcategories.
Example metadata asset CSV file
Asset Resource Key,Name,Type,Owner,Owner,Description,Tag,Tag,Term,Classification,Classification,Data Class,Asset Property: custom-attribute-721f6090.6acd460b.21klmcp7s.os41g6v.8a9jav.pkhvcsenvacjnqhcr9je1>>property_details,Asset Property: custom-attribute-721f6090.6acd460b.21klmcp7s.os41g6v.8a9jav.pkhvcsenvacjnqhcr9je1>>property_details,Column Source Type,Column Size,Column Type Nullable,Column Native Type,Column Property: custom-attribute-721f6090.6acd460b.21klmcp71.uk11ngl.jqagjc.9pbt16a766iga30rvf1gs#column>>property_details>>name,Column Property: custom-attribute-721f6090.6acd460b.7et5e21rq.hhip32d.ki6dir.k9q71bq85s80mhr8h19uu#column>>property_details
0000:0000:0000:0000:0000:FFFF:A92C:974B|50000|BANK:/BANK1/BANK_CLIENTS,BANK_CLIENTS,data_asset,admin,jjw-admin,Bank clients database,asset tag1,asset tag2,Locations>>testbusinessterm,[uncategorized]>>Confidential,[uncategorized]>>Personal Information,,Owner1,Owner2,,,,,,
0000:0000:0000:0000:0000:FFFF:A92C:974B|50000|BANK:/BANK1/BANK_CLIENTS,CLIENT_ID,column,,,,tag1,tag2,,,,[uncategorized]>>Identifier,,,char,50,FALSE,CHAR,XL,
0000:0000:0000:0000:0000:FFFF:A92C:974B|50000|BANK:/BANK1/BANK_CLIENTS,NAME,column,,,,,,,[uncategorized]>>Sensitive Personal Information,,[uncategorized]>>Person Name,,,char,128,TRUE,CHAR,,2024-08-27T04:00:00.000Z
Learn more
Parent topic: Adding and updating asset metadata