Skip to content

Generic Miner

The generic miner package allows you to mine query history and create lineage. The supported connections are:

  • Athena
  • MySQL
  • Oracle
  • Postgres
  • Snowflake

Use cases

  • Mine and parse SQL queries to generate lineage at table and column level.

Configuration

Follow the steps described in this article to generate the file with the list of queries to parse.

  • Connection Qualified Name: qualified name of the connection to mine.
  • Parsing language: SQL language to use to parse the queries.
  • Bucket Name: enter the bucket.
  • Bucket Prefix: enter the S3 prefix (directory) within the bucket where the files are located.
  • Bucket Region: if you are using your own S3 bucket, you must enter the name of the S3 region in which the bucket exists. (If you are reusing Atlan's bucket, you can leave this blank.).
  • SQL Json key: enter the JSON key containing the SQL query value.
  • Default Database Json Key: enter the JSON key containing the name of the default database.
  • Default Schema Json Key: enter the JSON key containing the name of the default schema.
  • Session ID Json Key: enter the JSON key containing the session ID under which the query ran.