Metadata Propagator

The metadata propagator package allows to propagate a set of selected metadata through lineage. The package gives the flexibility to select which metadata to propagate and where to propagate them.

Warning

By default metadata are propagated only if the target asset has only one dependency (upstream if the propagation is downstream, downstream if the propagation is upstream). The only exception are tags that are propagated upstream even if the asset has more than one dependency. To relax this constraint the flag Check direct dependency can be used.

Configuration

Credentials

• Atlan API Token: this is required to generate the dynamic dropdowns (see the Filters and Impacted Asset Filters screens)

Configurations

• Attributes to propagate:
  • Description: userDescription has priority over description, i.e. if both are populated only the userDescription is propagated. The description (either userDescription or description) is always propagated to the userDescription field of the target asset.
  • Owner Users
  • Owner Groups
  • Announcement: type, title and message are propagated.
  • Assigned Terms
  • Certificate
  • Tags: tags are propagated only upstream. Dowstream tags propagation is already covered by the out of the box tags propagation feature.
  • Readme
• Custom metadata to propagate: list of custom metadata to be propagated. Format: custom metadata 1.attribute 1@@@custom metadata 2.attribute 2.

• Propagation direction: whether to propagate metadata across the lineage or to related assets.

  LineageHierarchy

  • Lineage direction: whether the propagate the metadata downstream or upstream through lineage.
  • Lineage Depth: propagation depth. If empty metadata are propagated through the full lineage.

  Warning

  Hierarchy is currently supported for:

  • Tableau workbook
  • Tableau dashboard

  • Connections

    Info

    Connection assets are supported for connections that come under warehouse category. The list of connections that come under the warehouse category can be viewed from atlan-python by finding connections with AtlanConnectionCategory.WAREHOUSE as category defination.

    • The assets picked for propagating selected attribute to, from the connection asset are flitered/picked using Impacted Asset Filters.

    Provide connection asset in Source Filters for this feature to work. To do so, use the Asset Type and Qualified Name filter to give connection as an asset type with qualified name of the connection asset. Do not select any connection from the Connection filter, if the goal is to propagate from the connection asset to assets in impacted asset filter.

  Please reach out to your Atlan counterpart to request support for additional assets.

• Propagation mode:

  • Append: whether to append the propagated metadata to the ones already defined in the target asset. This option is valid only for Owner Users, Owner Groups, Assigned Terms and Tags.
  • Replace: whether to fully replace the metadata of the target asset with the one propagated.
  • Propagate only if empty: whether to propagate the metadata only if the corresponding metadata on the target asset is not pupulated.

• Propagate empty announcement: whether the propagate the empty/no announcement. Please note that if selected, it will remove all announcements from the propagated assets.

  Warning

  Only announcements propagated by this package will be removed.

• Operation type:

  • Preview: the package generates a csv file with the list of metadata that will be propagated.
  • Propagate: the package propagates the metadata according to the configuration defined.

• Check direct dependency: whether to propagate the metadata only if only if the target asset has only one dependency (upstream if the propagation is downstream, downstream if the propagation is upstream) or always propagate metadata. If No is selected there could be conflicts during propagation.

  Warning

  It's recommended to start with the Preview operation type to double check that the workflow is propagating metadata according to your expectations. Reverting back propagation requires ad-hoc scripts.

Filters

This screen allows to apply filters on the asset population. Only the metadata of the assets selected by this set of filters will be propagated.

Each filterable attribute has an operator and a value. The operator is always a dropdown menu while the value can be either a dropdown or a free text field. The package considers the filter only if both the operator and the value are filled. The only exceptions are the operators “Is Empty” and “Is Not Empty” where the value is not required (and ignored if filled). The available filters are:

• Connector
• Connection
• Asset Type
• Name
• Certificate
• Qualified Name
• Tag
• Description
• Announcement
• Owner User
• Owner Group
• Has Terms

Impacted Asset Filters

This screen allows to apply filters on the upstream/downstream assets. Metadata are propagated to assets that are upstream/dowstream to the one selcted in the above Filter section and that match the filters defined in this section. Leave it empty to propagate the metadata to all upstream/dowstream assets.

Each filterable attribute has an operator and a value. The operator is always a dropdown menu while the value can be either a dropdown or a free text field. The package considers the filter only if both the operator and the value are filled. The only exceptions are the operators “Is Empty” and “Is Not Empty” where the value is not required (and ignored if filled). The available filters are:

• Connector
• Connection
• Asset Type
• Name
• Certificate
• Qualified Name
• Tag
• Description
• Announcement
• Owner User
• Owner Group

Warning

Process, BIProcess, ColumnProcess, DbtColumnProcess, DbtProcess, Query assets are filtered out by default.

What it does

The package performs the following steps:

• Translate the UI input into an ElasticSearch query.
• Retrieve assets from Atlan.
• Retrieve the dowstream/upstream/related assets of the one selected.
• Filter them according to the rules defined in the Impacted Asset Filters section.
• Propagate the metadata.

---