PBI OLS Dataset Enricher

The Power BI Admin APIs that our native workflows use do not return PBI table metadata due to OLS (Object Level Security). This results in missing PBI table assets which breaks upstream lineage to the Data Warehouse. The PBI OLS Dataset Enricher package creates the missing Power BI Tables along with their columns, column data types, and processes linking PBI tables to downstream Datasets and linking upstream to DW tables (both Table Level Lineage & Column Level Lineage).

Assumptions

The customer has the following available for use: - Tabular Editor - Service Principal (Used to authenticate Tabular Editor script against PBI workspace and Semantic Model) - Premium Capacity workspace or Fabric Workspace - S3 Bucket (Extract will be uploaded here)

Pre-requisites

• AWS S3 bucket with access provisioned via AWS Access Key & Secret which allows access by a non-AWS application Resources:
  • Creating a bucket
  • Creating an IAM policy to access Amazon S3 resources
  • Managing access keys for IAM users

Credentials

Authentication Mechanism:

User-based authentication

To configure user-based authentication: - Create an AWS IAM user by following the steps in the AWS Identity and Access Management User Guide. - On the Set permissions page, attach your IAM policy to this user. - Once the user is created, view or download the user's access key ID and secret access key.

Role-based authentication

To configure role-based authentication, attach your IAM policy to the EC2 role that Atlan uses for its EC2 instances in the EKS cluster. Please raise a support ticket to use this option.

Role delegation-based authentication

To configure role delegation-based authentication: - Raise a support ticket to get the ARN of the Node Instance Role for your Atlan EKS cluster. - Create a new role in your AWS account by following the steps in the AWS Identity and Access Management User Guide. - When prompted for policies, attach your IAM policy to this role. - When prompted, create a trust relationship for the role using the following trust policy. (Replace <atlan_nodeinstance_role_arn> with the ARN received from Atlan support.) { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "<atlan_nodeinstance_role_arn>" }, "Action": "sts:AssumeRole", "Condition": {} } ] } - Now, reach out to Atlan support with: - The name of the role you created above. - The ID of the AWS account where the role was created.

Configuration

• Connection: Select the PBI connection from dropdown.

What it does

1. A Tabular Editor script extracts a BIM file from the PowerBI Semantic Model (aka PBI Dataset) and places it on S3 bucket.
2. The workflow extracts this BIM file from S3 having a dataset's details which is then used for processing.
3. Extracts the dataset, tables, columns and power-query-expressions details from the BIM file.
4. Publishes the assets extracted above to Atlan using atlan-crawler/generic-publish template.
5. Parses the power-query-expressions to create lineage processes between the PBI Tables and their columns to the upstream DW tables and their columns. This creates upstream lineage.
6. Creates lineage process between the PBI table and the PBI dataset for downstream lineage.

---