Skip to main content

Posts

Showing posts with the label unity catalogue

Data Masking When Ingesting Into Databricks

  Photo by Alba Leader Data masking is a data security technique that involves hiding data by changing its original numbers and letters. It's a way to create a fake version of data that's similar enough to the actual data, while still protecting it. This fake data can then be used as a functional alternative when the real data isn't needed.  Unity Catalog  is not a feature within Databricks. Instead, Databricks provides the  Delta Lake  feature, which includes data governance capabilities such as row filters and column masking. Unity Catalog in Databricks allows you to apply data governance policies such as row filters and column masks to sensitive data. Let’s break it down: Row Filters : Row filters enable you to apply a filter to a table so that subsequent queries only return rows for which the filter predicate evaluates to true. To create a row filter, follow these steps: Write a SQL user-defined function (UDF) to define the filter policy. CREATE FUNCTIO...