Skip to content

Metadata Propagator

The metadata propagator package allows to propagate a set of selected metadata through lineage. The package gives the flexibility to select which metadata to propagate and where to propagate them.

Warning

By default metadata are propagated only if the target asset has only one dependency (upstream if the propagation is downstream, downstream if the propagation is upstream). The only exception are tags that are propagated upstream even if the asset has more than one dependency. To relax this constraint the flag Check direct dependency can be used.

Configuration

Credentials

  • Atlan API Token: this is required to generate the dynamic dropdowns (see the Filters and Impacted Asset Filters screens)

Configurations

  • Attributes to propagate:
    • Description: userDescription has priority over description, i.e. if both are populated only the userDescription is propagated. The description (either userDescription or description) is always propagated to the userDescription field of the target asset.
    • Owner Users
    • Owner Groups
    • Announcement: type, title and message are propagated.
    • Assigned Terms
    • Certificate
    • Tags: tags are propagated only upstream. Dowstream tags propagation is already covered by the out of the box tags propagation feature.
    • Readme
  • Custom metadata to propagate: list of custom metadata to be propagated. Format: custom_metadata_1.attribute_1@@@custom_metadata_2.attribute_2.
  • Lineage direction: whether the propagate the metadata downstream or upstream through lineage.
  • Lineage Depth: propagation depth. If empty metadata are propagated through the full lineage.
  • Propagation mode:
    • Append: whether to append the propagated metadata to the ones already defined in the target asset. This option is valid only for Owner Users, Owner Groups, Assigned Terms and Tags.
    • Replace: whether to fully replace the metadata of the target asset with the one propagated.
    • Propagate only if empty: whether to propagate the metadata only if the corresponding metadata on the target asset is not pupulated.
  • Operation type:
    • Preview: the package generates a csv file with the list of metadata that will be propagated.
    • Propagate: the package propagates the metadata according to the configuration defined.

  • Check direct dependency: whether to propagate the metadata only if only if the target asset has only one dependency (upstream if the propagation is downstream, downstream if the propagation is upstream) or always propagate metadata. If No is selected there could be conflicts during propagation.

    Warning

    It's recommended to start with the Preview operation type to double check that the workflow is propagating metadata according to your expectations. Reverting back propagation requires ad-hoc scripts.

Filters

This screen allows to apply filters on the asset population. Only the metadata of the assets selected by this set of filters will be propagated.

Each filterable attribute has an operator and a value. The operator is always a dropdown menu while the value can be either a dropdown or a free text field. The package considers the filter only if both the operator and the value are filled. The only exceptions are the operators “Is Empty” and “Is Not Empty” where the value is not required (and ignored if filled). The available filters are:

  • Connector
  • Connection
  • Asset Type
  • Name
  • Certificate
  • Qualified Name
  • Tag
  • Description
  • Announcement
  • Owner User
  • Owner Group

Impacted Asset Filters

This screen allows to apply filters on the upstream/downstream assets. Metadata are propagated to assets that are upstream/dowstream to the one selcted in the above Filter section and that match the filters defined in this section. Leave it empty to propagate the metadata to all upstream/dowstream assets.

Each filterable attribute has an operator and a value. The operator is always a dropdown menu while the value can be either a dropdown or a free text field. The package considers the filter only if both the operator and the value are filled. The only exceptions are the operators “Is Empty” and “Is Not Empty” where the value is not required (and ignored if filled). The available filters are:

  • Connector
  • Connection
  • Asset Type
  • Name
  • Certificate
  • Qualified Name
  • Tag
  • Description
  • Announcement
  • Owner User
  • Owner Group

Warning

Process, BIProcess, ColumnProcess, DbtColumnProcess, DbtProcess assets are filtered out by default.

What it does

The package performs the following steps:

  • Translate the UI input into an ElasticSearch query.
  • Retrieve assets from Atlan.
  • Retrieve the dowstream/upstream assets of the one selected.
  • Filter them according to the rules defined in the Impacted Asset Filters section.
  • Propagate the metadata.