Skip to content

Metadata propagator

The metadata propagator package allows to propagate a set of selected metadata through lineage. The package gives the flexibility to select which metadata to propagate and where to propagate them.

Warning

By default metadata are propagated only if the target asset has only one dependency (upstream if the propagation is downstream, downstream if the propagation is upstream). The only exception are tags that are propagated upstream even if the asset has more than one dependency. To relax this constraint the flag Check direct dependency can be used.

Configuration

Credentials

  • Atlan API Token: this is required to generate the dynamic dropdowns (see the Filters and Impacted Asset Filters screens)

Configurations

  • Attributes to propagate:

    • Description

      Description: priority order for different attributes

      The userDescription takes priority over description when both are present. For more information, see How does version control work for description changes in source tools vs. Atlan?

    • Owner Users

    • Owner Groups
    • Announcement: type, title and message are propagated.
    • Assigned Terms
    • Certificate
    • Tags: tags are propagated only upstream. Dowstream tags propagation is already covered by the out of the box tags propagation feature.
    • Readme
    • Domain

  • Custom metadata to propagate: list of custom metadata to be propagated. Format: <custom_metadata_1_name>.<attribute_1_name>@@@<custom_metadata_2_name>.<attribute_2_name>.

    Custom metadata format: example

    Separate multiple custom metadata with @@@ and each custom metadata is composed by the name of the custom metadata and the name of the attribute to propagate, separated by a dot. See the example below. 

        |-- Scores
    |   |-- Missing Metadata
    |   |-- Score Contributors
    |-- SLA
        |-- Update Frequency
    For the given custom metadata, the value to enter is: Scores.Missing Metadata@@@SLA.Update Frequency

  • Propagation direction: whether to propagate metadata across the lineage or to related assets.

    • Lineage direction: whether the propagate the metadata downstream or upstream through lineage.
    • Lineage Depth: propagation depth. If empty metadata are propagated through the full lineage.
    • Check direct dependency: Propagate metadata only if the target asset has a single dependency—upstream for downstream propagation, downstream for upstream propagation. If you select No, propagation may result in conflicts during propagation.

    Warning

    Hierarchy is currently supported for:

    • Tableau workbook
    • Tableau dashboard

    • Connections

      Info

      Connection assets are supported for connections that come under warehouse category. The list of connections that come under the warehouse category can be viewed from atlan-python by finding connections with AtlanConnectionCategory.WAREHOUSE as category defination.

      • The assets picked for propagating selected attribute to, from the connection asset are flitered/picked using Impacted Asset Filters.

      Provide connection asset in Source Filters for this feature to work. To do so, use the Asset Type and Qualified Name filter to give connection as an asset type with qualified name of the connection asset. Do not select any connection from the Connection filter, if the goal is to propagate from the connection asset to assets in impacted asset filter.

    Please reach out to your Atlan counterpart to request support for additional assets.

    • Relationship Names: comma delimited relationship names to find assets related to the source asset to propagate metadata to. For more information, see relationship names.

    Example

    To propagate metadata from a Table to glossary terms attached to the table and columns under the table, provide this string: meanings,columns

  • Propagation mode:

    • Append: whether to append the propagated metadata to the ones already defined in the target asset. This option is valid only for Owner Users, Owner Groups, Assigned Terms and Tags.
    • Replace: whether to fully replace the metadata of the target asset with the one propagated.
    • Propagate only if empty: controls whether metadata is propagated only when the corresponding field on the target asset is empty. If the target already has a value, propagation is skipped for that attribute.

  • Propagate empty announcement: whether the propagate the empty/no announcement. Please note that if selected, it will remove all announcements from the propagated assets.

    Warning

    Only announcements propagated by this package will be removed.

  • Operation type:

    • Preview: the package generates a csv file with the list of metadata that will be propagated.
    • Propagate: the package propagates the metadata according to the configuration defined.

    Warning

    It's recommended to start with the Preview operation type to double check that the workflow is propagating metadata according to your expectations. Reverting back propagation requires ad-hoc scripts.

  • Mode: whether the propagation is executed for all assets (full) or just for the ones updated since the last time the workflow was executed (incremental). The incremental option is valid only if opearation type is set to 'Propagate'.

  • Asset name match: whether to propagate the metadata only to assets that match the name of the source asset.

  • Case sensitive match: whether to propagate the metadata only to assets that match the name of the source asset in a case sensitive way. This is only valid if the Asset name match is set to Yes.

Filters

This screen allows to apply filters on the asset population. Only the metadata of the assets selected by this set of filters will be propagated.

Each filterable attribute has an operator and a value. The operator is always a dropdown menu while the value can be either a dropdown or a free text field. The package considers the filter only if both the operator and the value are filled. The only exceptions are the operators Is Empty and Is Not Empty where the value isn't required (and ignored if filled). The available filters are:

  • Connector
  • Connection
  • Asset Type
  • Name
  • Certificate
  • Qualified Name
  • Tag
  • Description
  • Announcement
  • Owner User
  • Owner Group
  • Has Terms

Impacted asset filters

This screen allows to apply filters on the upstream/downstream assets. Metadata are propagated to assets that are upstream/dowstream to the one selcted in the above Filter section and that match the filters defined in this section. Leave it empty to propagate the metadata to all upstream/dowstream assets.

Each filterable attribute has an operator and a value. The operator is always a dropdown menu while the value can be either a dropdown or a free text field. The package considers the filter only if both the operator and the value are filled. The only exceptions are the operators Is Empty and Is Not Empty where the value isn't required (and ignored if filled). The available filters are:

  • Connector
  • Connection
  • Asset Type
  • Name
  • Certificate
  • Qualified Name
  • Tag
  • Description
  • Announcement
  • Owner User
  • Owner Group

Warning

Process, BIProcess, ColumnProcess, DbtColumnProcess, DbtProcess, Query assets are filtered out by default.

What it does

The package performs the following steps:

  • Translate the UI input into an ElasticSearch query.
  • Retrieve assets from Atlan.
  • Retrieve the dowstream/upstream/related assets of the one selected.
  • Filter them according to the rules defined in the Impacted Asset Filters section.
  • Propagate the metadata.