Informatica Power center Lineage builder(no transformations).

The Informatica power center lineage builder package allows to build lineage between oracle assets already present in Atlan. The package uses the REP_TBL_MAPPING file generated by Informatica power center to look upon mappings. The package currently supports only table level lineage.

Warning

• Assets not existing in Atlan are to be ignored by default, so if source or target does not exist in Atlan, the mapping will be ignored
• The package today looks for a standard naming convention for oracle tables. To determine which schema the table belongs to we use a configuration and also a (optional) suffix may need to be provided if schema names in Atlan have additional suffixes than in Informatica Power center
• Only table level lineage is handled

Configuration

Oracle Connection

• (required) Connection to be selected from the list of connections crawled in Atlan. Lineage will be established only across assets under this connection.

Oracle Database

• (required) Schemas under this database will be considered for lineage.

Import Mapping file from

• Import mapping file from: select how you want to provide the input file(s) to be processed

  Direct file upload(s)Object storage

  Directly upload CSV file(s) containing the metadata.

  Limited file sizes

  This option is generally limited to ~10MB for each file. For anything larger, use object storage.

  Retrieve the metadata files from cloud object storage.

  • Cloud object store the object store from which to fetch the metadata file(s).

    S3GCSADLS

    • AWS access key: your AWS access key.
    • AWS secret key: your AWS secret key.
    • Region: your AWS region.
    • Bucket: your AWS bucket.

    Reusing Atlan's backing S3 store

    When your Atlan tenant is deployed in AWS, you can leave all of these blank to reuse the backing store of Atlan itself. You can also set up a cross-account bucket policy for Atlan to have access to your S3 bucket and the objects within it, and leave these blank.

    • Project ID: the ID of your GCP project.
    • Servive account JSON: your service account credentials, as JSON.
    • Bucket your GCS bucket.

    Reusing Atlan's backing GCS store

    When your Atlan tenant is deployed in GCP, you can leave all of these blank to reuse the backing store of Atlan itself.

    • Azure client ID: the unique application (client) ID assigned to your app by Azure AD when the app was registered.
    • Azure client secret: your Azure client secret (it's actual value, not its identifier).
    • Azure tenant ID: the unique identifier of the Azure Active Directory instance.
    • Storage account name: name of your storage account.
    • Container: your ADLS container.

    Reusing Atlan's backing ADLS store

    When your Atlan tenant is deployed in Azure, you can leave all of these blank to reuse the backing store of Atlan itself.

Cloud file path

(conditionally required) Provide the cloud file path(along with the file name). This is required only when the file is uploaded to a cloud provider.

Schema name

(required) Case sensitive comma separated schema names which are prefixed in the power center mapping file to the respective table.

Schema suffix

(optional) Case sensitive schema suffix (if any) should be provided here. This may be needed if the schema name prefixed in the table is not the entire name as scanned by Atlan. Schema name + Schema suffix will be used to scan Atlan for crawled schemas.

Table Name separator

(required) In the power center mapping file the table name and schema name will be separated by this character, please specify the same

Output option

=== "Preview Lineage"

A csv file with the list of matching assets is generated with the following columns:
- source_qualified_name
- target_qualified_name
- source_schema
- target_schema
!!! recommendation 
   "How to download the file"
     The file with the lineage preview can be downloaded from the Argo workflow log screen.     
!!! warning
    The file with the lineage preview is always generated even if **Preview Lineage** is not selected as ouput type.

=== "Generate Lineage"

  Processes assets (lineage) are created between assets matched in (5).
  !!! warning
      If *Process connection* is blank the process assets will be assigned to the source assets connection.

=== "Delete Lineage"

    Processes assets (lineage) between assets matched in (5) are deleted.
    !!! warning
         Only processes (lineage) created using this package are deleted.

What it does

The package performs the following steps:

• Reads the input file
• Checks for the schemas crawled by Atlan under the DB and connection provided as input, any uncrawled schemas in schema name config is ignored
• Extract all the source and tables from the mapping file (Columns SOURCE_NAME and TARGET_NAME from REP_TBL_MAPPING file)
• Assign source and target schemas to the tables. This is done in the following order:
• a) Split the table name by table name separator and extract the first element(this is the schema name)
• b) Concat the output of the previous step with the schema suffix config (if any) and assign the schema to it
• Retrieve the tables from Atlan and filter out the original mapping to only what exists.
• Create fully qualified name of the table (source and target).
• Create lineage preview
• If output option is to Generate Lineage then Lineage is generated(If lineage exists then lineage creation for those assets is skipped )
• If output option is to Delete Lineage then Lineage is Deleted(If lineage does not exist then lineage deletion for those asset is skipped )
• If output option is to Preview Lineage then the Lineage candidates csv file is only generated and no actions are performed. Irrespective of the output option the preview file is generated.

---