Native Data Quality
by Data Meaning Services Group Inc
Run your data quality rules inside Snowflake or Databricks, with nothing for your team to schedule.
Your data quality rules, run natively in Snowflake or Databricks.
Your business approved its data quality rules. They often still run in separate ETL jobs, or a separate tool, that read the data, test it and keep the results somewhere else. Snowflake can now run data quality checks itself, and Databricks can run them inside your pipelines or the moment a table updates. Native Data Quality puts every approved rule into those checks and keeps them there: nothing for your team to build or schedule, no data quality tool to buy, and the steward who owns a rule gets an email when it fails.
What it does
- Every rule running natively. Each approved rule in your GitHub repository becomes a native check on its own table: a data metric function with an expectation on Snowflake; on Databricks, a check the product runs in a Databricks job whenever the table changes, or an expectation inside your Lakeflow pipeline.
- The same rules in every environment. One versioned copy of each rule deploys to development, test and production. You promote a change by moving a git tag.
- Checks that survive your loads. On Snowflake, a table that is dropped and rebuilt loses its checks without warning. The product puts them back on its next run, by default within the hour.
- Failures reach the right person. When a check fails, the steward named in the rule gets one email listing every failed check on the tables they look after.
- It watches itself. If a run cannot keep the checks up to date, your data quality owner gets an email, a reminder each day while it lasts, and one when it works again.
What gets deployed in your Azure subscription
- Azure Functions that keep each environment's checks matching your rules and email the stewards, hourly by default
- A Key Vault for the read-only GitHub token, and storage closed to the internet on a private network
- A NAT gateway with one fixed public IP address, so your platform admin can allow-list the product
- Azure Communication Services for the emails, and a managed identity the product signs in with
The product signs in to Snowflake and Databricks with its own Azure identity: no Snowflake or Databricks password is stored. It reads only what your rules need, and the checks run inside your platform. Data Meaning does not host the deployment and does not receive your data.
Before you deploy
- A GitHub repository for your rules (we provide a template) and a read-only token for it.
- Snowflake Enterprise Edition or higher, or a Databricks workspace with Unity Catalog and a serverless or pro SQL warehouse.
- After the install, your platform admin runs a short setup that the deployment's outputs fill in.
How the engagement works
The solution template is free to deploy. Data Meaning moves your approved rules into native checks in about six weeks, for one fixed fee: rules as code, native checks in development, test and production, load changes, plus the hourly restore of any check a rebuild removes, the steward alert and a Power BI data quality report, then a handover where your team adds a rule on its own. You pay Microsoft for Azure consumption, your platform for the compute the checks use, and Data Meaning for the engagement.