Databricks on Google Cloud, Part 6: Governance and security

Part 5 put Acme’s tables in a bucket it owns, gs://acme-lake-supply, and connected that bucket to Unity Catalog through a storage credential and an external location. The data now has a home. This part answers the questions a security team asks next: who may read which rows, whose key encrypts the files, where the passwords live, and how you prove afterwards who did what.

Two words first. Governance is the set of rules that decide who may use which data and how, plus the records that show the rules were followed. Security is the set of controls that enforce those rules and protect the data itself: encryption, credentials and monitoring. On Databricks, most of the governance lives in Unity Catalog; most of the security splits between Databricks and Google Cloud services such as Cloud Key Management Service (Cloud KMS) and Cloud Audit Logs.

I use the same running example as the rest of the series: Acme Consumer Brands, workspace acme-prod-euw2-supply in europe-west2, catalog acme_supply. Identity (who a person or a program is) is Part 2’s topic; this part starts once Databricks knows who is asking.

If you prefer to watch first, here is the heart of this part as a short animated explainer.

Video: who may see which rows, whose key encrypts the data and who did what, in three minutes. The narration is a synthetic voice; the script and animation are mine.

What are the key ideas?

Unity Catalog: Databricks’ governance layer. It holds the names of all governed objects and decides, on every query, whether the caller may use them.
Metastore: the top-level container in Unity Catalog. There is one per region, and every workspace in that region attaches to it.
Securable object: anything you can grant a permission on, such as a catalog, a table or a volume. A privilege is one such permission, for example SELECT (read).
Customer-managed encryption key (CMEK): an encryption key that you create and control in Cloud KMS, which a service then uses to encrypt your data. If you disable the key, the data becomes unreadable. That control is the point. Prerequisite 3 explains envelope encryption and CMEK in general.
Secret: a credential such as a password, a token or an API key, stored somewhere safe so that code can use it without the value appearing in a notebook. Prerequisite 3 covers impersonation and secret rotation.
Audit log: a record of who did what, when and from where. Databricks keeps one for actions in Databricks; Google Cloud keeps one for actions in your projects.

Here is how those ideas map onto six questions about one workspace:

the six questions governance and security answer for Acme's workspace acme-prod-euw2-supply: who may see what (Unity Catalog privileges, owners, bindings, filters, masks and ABAC policies), where a number came from (lineage and tags), whose key encrypts the data (Cloud KMS keys for managed services, workspace storage and the lake bucket), where credentials live (secret scopes, Unity Catalog secrets, Secret Manager), who did what (system.access.audit and Cloud Audit Logs), and which rules the workspace must follow (the compliance security profile)

Throughout the post I protect two tables. The first is the gold table the whole series uses. The second, a silver table of individual deliveries, holds a column of personal data, which gives the masking examples something real to hide:

the two tables this post protects: acme_supply.supply_gold.supply_service, one row per brand, region and week, owned by supply (four rows shown), and acme_supply.supply_silver.deliveries, one row per delivery, with a driver_phone column that counts as personal data (three rows shown); all values are fictional

Prerequisite 1 explains why the medallion layers (bronze, silver, gold) exist, and the business and data prerequisite explains every column of supply_service.

What is Unity Catalog, and how is it organised?

The bottom line: Unity Catalog is one register of every table, file and model in a region, and every name in it has exactly three parts. Every query asks that register two things: does this name exist, and may this caller use it?

A public library works the same way. Picture one central library per city. Inside it, books are grouped into sections (Science, History), each section has shelves, and every book has a call number such as Science / Chemistry / Organic Basics. Two sections can both hold a book called Introduction, because the full call numbers differ. Some things in the building are not books at all, such as the keys to the archive room, and those are kept at the front desk rather than on a shelf. Map it across: the city library is the metastore (one per region), a section is a catalog, a shelf is a schema, a book is a table (or view, function, model or volume), and the front desk holds the objects that are not data, such as storage credentials.

The Unity Catalog overview describes it in one line, and the rest of this section unpacks that line:

Unity Catalog: “the unified governance layer for data and AI built into Databricks”. It names every table as catalog.schema.table, and the same three-level namespace holds its other securable objects (objects you can grant permissions on): views (saved queries), functions, models and volumes. A volume is governed storage for files that are not tables, such as Acme’s raw delivery JSON files under /Volumes/acme_supply/supply_bronze/landing/; Part 5 explains managed and external volumes. Unity Catalog also controls access, records lineage (which tables were built from which) and logs access for audit. It is enabled automatically for workspaces created after 6 March 2024.

How does this apply to Acme? The city is europe-west2, the section is acme_supply, the shelf is supply_gold and the book is supply_service. This is how Unity Catalog’s namespace looks for Acme’s supply data:

the Unity Catalog three-level namespace at Acme: one metastore, a catalog called acme_supply, and schemas that hold tables, views, functions, models and volumes, each with a three-level name; a volume is governed storage for files that are not tables, reached by a /Volumes path

Why is it called a three-level namespace? A namespace is a scheme for naming things so that two objects never share a full name. Unity Catalog uses three levels: a catalog (Acme uses one per data domain, acme_supply), a schema inside it (one per medallion layer, supply_gold), and the object itself (supply_service). Two teams can both have a table called orders because their full names differ in the first or second level. The metastore sits above the catalogs but is not part of the name: a workspace only ever sees one metastore, the one for its region.

A few securable objects sit directly under the metastore instead of inside a schema, because they are not data: storage credentials and external locations (which Part 5 built for gs://acme-lake-supply), service credentials (which let code call other Google Cloud services, covered below), connections (for Lakehouse Federation to systems such as BigQuery, covered in Part 8; this kind of federation is about querying data, not signing in, as Part 2 explains) and shares for OpenSharing, the protocol formerly called Delta Sharing.

Who runs it? The admin privileges page names three roles. An account admin creates metastores and workspaces and links them. A workspace admin manages one workspace and can create catalogs. A metastore admin governs the top-level objects in one metastore; the role is optional, and an account admin who creates a metastore becomes its first metastore admin. Part 2 shows where each kind of role lives, in Google Cloud IAM, the Databricks account, the workspace or Unity Catalog. My view: give the metastore admin role to a small group, such as data-platform-admins, never to one person.

How do privileges and inheritance work?

Start from the rule the permissions model page states plainly: initially, users have no access to data, and a user or group must be explicitly granted a privilege to perform an action. Nothing is open by default. The bottom line: to read a table you need a way through every container above it as well as the right on the table itself, and a right granted on a container flows down to everything inside it, now and later.

Where people get tripped up is thinking that SELECT on a table is enough. It is not. An office building shows why. To open filing cabinet 7 on the second floor, you need three things: a badge that opens the front door, a badge that opens the second-floor door, and the key to the cabinet. A cabinet key without the front-door badge gets you nowhere. Now suppose facilities gives you one key that opens every cabinet on the second floor: when a new cabinet is installed next year, the same key opens it. The front door is USE CATALOG, the floor door is USE SCHEMA, the cabinet key is SELECT, and the floor-wide key is a grant on a schema that every table inside inherits.

Privileges come in a few families:

Usage privileges, USE CATALOG and USE SCHEMA, open a container. They grant no data on their own; they are the doors you must pass through to reach anything inside.
Data privileges act on the object: SELECT reads a table or view, MODIFY writes to a table, READ VOLUME reads files in a volume, and EXECUTE calls a function.
Creation privileges such as CREATE SCHEMA (on a catalog) and CREATE TABLE (on a schema).
BROWSE shows that objects exist, with their names, descriptions and tags, without showing any data. Databricks recommends granting it on catalogs to the account users group so that people can discover data and request access.
MANAGE lets someone grant privileges on, transfer ownership of, and drop an object without owning it.

Inheritance is what makes this workable. When you grant a privilege on a catalog or a schema, it applies to all current and future objects inside it.

How does this apply to Acme? The building is acme_supply, the second floor is supply_gold, and the floor-wide key is one grant. Acme grants SELECT once, on the schema supply_gold, and every gold table, including ones created next year, is readable by the group. When Sam from the supply team runs a query, Unity Catalog checks three things in order:

how Unity Catalog decides whether Sam, a member of supply-analysts, may read supply_service: it checks USE CATALOG on acme_supply, then USE SCHEMA on supply_gold, then SELECT, which Sam holds on the table through a grant on the schema that every current and future table inherits; only when all three pass does the query run, with any row filter and column mask applied

The SQL is ordinary ANSI SQL, the standard dialect most databases share. You run it in a notebook or the SQL editor, or manage the same grants with the Databricks CLI or Terraform:

-- Who can do what on the gold schema?
SHOW GRANTS ON SCHEMA acme_supply.supply_gold;

-- principal          actionType   objectType  objectKey
-- supply-analysts    SELECT       SCHEMA      acme_supply.supply_gold
-- supply-analysts    USE SCHEMA   SCHEMA      acme_supply.supply_gold

-- Take it away again
REVOKE SELECT ON SCHEMA acme_supply.supply_gold FROM `supply-analysts`;

The output shape is the one the manage privileges page shows; I have not run this against a live metastore. Notice that the grantee is a group, supply-analysts, which Acme syncs from Microsoft Entra ID as Part 2 describes. Granting to groups rather than people means joining or leaving the team is a directory change, not a SQL change.

The full rule set for common operations fits in one diagram:

which Unity Catalog privileges each common operation needs: reading a table needs USE CATALOG, USE SCHEMA and SELECT; writing needs MODIFY; creating a table needs CREATE TABLE on the schema; reading volume files needs READ VOLUME; calling a function needs EXECUTE; BROWSE shows metadata without data; MANAGE lets someone grant, transfer ownership and drop without owning the object

Two details trip people up. First, ALL PRIVILEGES is a shorthand for every applicable privilege on an object, but it deliberately leaves out MANAGE, READ METADATA, EXTERNAL USE SCHEMA and EXTERNAL USE LOCATION, so granting it does not hand over administration. Second, MODIFY is a composite privilege whose children are INSERT, UPDATE and DELETE (all three in Beta as of November 2026, per the privileges reference), so you can now let a pipeline append without letting it delete.

Who owns an object, and what does ownership give?

The bottom line: every object has exactly one owner, ownership is a status rather than a grant, and in production that owner should be a group, never a person.

Think of a house. The name on the deed is the owner: they can do anything with the house, hand out keys, sell it or knock it down, and the deed does not appear on any list of key holders, because it is not a key. A letting agent can hand out keys and arrange repairs without owning the house; that is MANAGE. If the house is in the name of someone who has emigrated and left no forwarding address, nobody can sell it or change the locks. That last case is the one to design out.

Every securable object has exactly one owner: a user, a service principal (an identity for automation) or a group. Whoever creates an object becomes its first owner. The owner can do everything with the object, grant and revoke privileges on it, transfer ownership and drop it. The permissions page adds two subtleties:

Ownership does not show up as a grant. An owner never appears with ALL PRIVILEGES in SHOW GRANTS, because the rights come from ownership itself.
Ownership does not inherit downwards, but control does. The owner of the catalog acme_supply does not own the schema supply_gold, yet can manage it and everything in it.

How does this apply to Acme? The practical consequence: if Lena, an engineer, creates supply_service and later leaves Acme, the table is owned by a person who no longer exists in the directory. Transfer production objects to a group as part of the deployment:

ALTER TABLE acme_supply.supply_gold.supply_service OWNER TO `supply-engineers`;
ALTER SCHEMA acme_supply.supply_gold OWNER TO `supply-engineers`;

MANAGE is the lighter alternative when someone needs to administer an object without owning it. One deliberate exception is worth knowing: schema owners do not get EXTERNAL USE SCHEMA by default, the privilege that lets engines outside Databricks read a schema’s tables. That stops an owner from accidentally exposing data to other engines; Part 8 shows when Acme grants it for BigQuery.

How do you keep production data out of development workspaces?

Here is the surprise for many teams: by default, every catalog is reachable from every workspace attached to the same metastore. Acme’s development workspace acme-dev-euw2-shared attaches to the same europe-west2 metastore as production, so without further action anyone with a grant on acme_supply could query production data from a development notebook. The bottom line: grants decide who may read a catalog; a workspace binding decides from where. The common misconception is that separate workspaces mean separate data. On a shared metastore they do not.

A simple example: a company with a head office and a branch office shares one filing system. By default, anyone with clearance can pull the payroll file from either office. The company then adds a rule: payroll files can only be opened in the head office, whatever your clearance. A manager with full clearance who asks for payroll at the branch is refused, and at the head office she gets it. The offices are workspaces, the clearance is a grant, and the rule tied to a place is a binding.

How does this apply to Acme? The head office is acme-prod-euw2-supply, the branch is acme-dev-euw2-shared, and the payroll file is the acme_supply catalog.

Workspace-catalog binding fixes that. You bind a catalog to a list of workspaces, and any other workspace is refused, even for users who hold an explicit grant. You can also make a binding read-only, which blocks writes from that workspace. Information schema queries, Catalog Explorer and lineage all respect the binding, so an unbound catalog simply appears greyed out.

workspace-catalog binding at Acme: both workspaces attach to the same europe-west2 metastore, but the catalog acme_supply is bound only to acme-prod-euw2-supply and acme_supply_dev only to acme-dev-euw2-shared; a query from the dev workspace to acme_supply fails even for a user who holds SELECT, and the storage credential acme-lake-sc is bound to the production workspace in the same way

The same binding exists for storage credentials, external locations and service credentials, and Part 2 recommended binding Acme’s credentials for exactly this reason. One exception to the open default: the workspace catalog that Databricks creates automatically for a new workspace is bound only to that workspace from the start. With the Databricks CLI, binding is two steps, taken from the binding page (workspace IDs are placeholders; I have not run these):

# 1. Stop the catalog being open to every workspace on the metastore
databricks catalogs update acme_supply --isolation-mode ISOLATED

# 2. Bind it to the production workspace only
databricks workspace-bindings update-bindings catalog acme_supply --json '{
  "add": [{"workspace_id": <prod-workspace-id>, "binding_type": "BINDING_TYPE_READ_WRITE"}]
}'

How do row filters and column masks hide data inside a table?

The bottom line: a row filter hides whole rows, a column mask rewrites one column’s values, and both run at query time on data that never changes.

A simple example first. A hospital keeps one patient list. When a nurse on Ward 3 opens it, she sees only Ward 3’s patients; the other wards’ rows are simply not there for her. When the reception desk opens the same list, every patient appears, but the phone number column shows *** 123 instead of the full number. Nobody made a second list. The rule “show only my ward” is a row filter, the rule “show only the last three digits” is a column mask, and both are applied the moment someone opens the list.

How does this apply to Acme? The wards are regions and the phone numbers are drivers’ phones. Grants work on whole objects: you can read supply_service or you cannot. Often that is too coarse. Acme’s planners for Western Europe should see only their region’s rows, and only the privacy team should see drivers’ phone numbers in full. Unity Catalog has two tools for this, described on the row filters and column masks page:

A row filter is a SQL user-defined function (UDF), a function you write once and store in Unity Catalog, that returns TRUE or FALSE for each row. Rows where it returns FALSE vanish from the result. This is row-level security.
A column mask is a UDF that receives a column’s value and returns either the value or a replacement, such as the last three digits. Each column can have one mask.

Both run at query time, for every query, from every tool: a notebook, a SQL warehouse, a dashboard or a JDBC client. The stored data never changes. Acme keeps its governance functions in a schema of their own, acme_supply.governance, so that a function’s own permissions are easy to audit:

CREATE FUNCTION acme_supply.governance.region_filter(region STRING)
RETURN IF(is_account_group_member('supply-admins'), TRUE, region = 'Western Europe');

CREATE FUNCTION acme_supply.governance.phone_mask(p STRING)
RETURN CASE WHEN is_account_group_member('supply-privacy') THEN p
            ELSE concat('*** ', right(p, 3)) END;

is_account_group_member() checks the caller’s membership of an account-level group, the groups Part 2 syncs from Entra ID. The filter above is deliberately simple; with more regions, the docs suggest a small mapping table of group to region that the function looks up. Then attach both functions:

the same queries return different results for different people: a row filter on supply_service lets planners-weu see only Western Europe rows while supply-admins see every region, and a column mask on deliveries.driver_phone shows the full number to supply-privacy and only the last three digits to everyone else; the table data never changes, Unity Catalog applies both functions at query time

Two limits matter in practice. On compute, you need standard access mode on Databricks Runtime 12.2 LTS or above, or dedicated access mode on 15.4 LTS or above; older runtimes fail securely, meaning they return no data rather than unfiltered data. On dedicated compute, Databricks hands the filtering to serverless compute, so serverless must be enabled in the workspace and you may be charged for serverless compute. And order matters when you clean up: run ALTER TABLE ... DROP ROW FILTER before DROP FUNCTION, or the table becomes inaccessible until you remove the orphaned reference.

What are tags, and how does ABAC apply one rule to many tables?

A filter or mask attached table by table has an obvious weakness: the day someone creates a new table with phone numbers and forgets the mask, the numbers are exposed. Attribute-based access control (ABAC) solves this by attaching rules to descriptions of data instead of to tables. (Prerequisite 3 compares role-based and attribute-based access control in general.)

The bottom line: table-level rules protect the tables you remember; ABAC protects every table whose data carries a label, including tables nobody has created yet.

A supermarket makes the idea concrete. It could keep a list of every product that needs an ID check at the till, and update the list whenever a new beer arrives. Or it can print a sticker, “age restricted”, and set one rule at every till: anything with this sticker needs ID. A new beer gets the sticker on delivery and the rule applies with no change at the tills. Only the store manager is allowed to apply or remove the stickers, so nobody can peel one off to skip the check. The sticker is a tag, the controlled sticker roll is a governed tag, and the till rule is an ABAC policy.

How does this apply to Acme? The sticker is pii = phone, the till rule is one policy on the catalog acme_supply, and the store manager is supply-privacy.

The description is a tag, a key and value attached to a catalog, schema, table or column, such as pii = phone. A governed tag is a tag defined at the account level with a tag policy: a fixed list of allowed values and a list of who may assign it. Only people with the ASSIGN permission on the tag can apply it, so nobody can quietly untag a column to escape a policy. An account can define up to 1,000 governed tags.

An ABAC policy then says: for every table under this catalog (or schema) whose columns carry this tag, apply this mask or filter, for these people, except those. Acme replaces its per-table phone mask with one policy:

ALTER TABLE acme_supply.supply_silver.deliveries
  ALTER COLUMN driver_phone SET TAGS ('pii' = 'phone');

CREATE POLICY mask_pii_phone
ON CATALOG acme_supply
COMMENT 'Mask phone numbers everywhere in acme_supply'
COLUMN MASK acme_supply.governance.phone_mask
TO `account users` EXCEPT `supply-privacy`
FOR TABLES
MATCH COLUMNS has_tag_value('pii', 'phone') AS c
ON COLUMN c;

The syntax follows the policy reference; I have not run it. My advice: if a table already has a table-level mask on the same column, remove it first, so that only one rule decides what people see.

attribute-based access control at Acme: an account-level governed tag pii with allowed values phone, email and name; supply-privacy may assign it; columns tagged pii = phone in any table of the catalog acme_supply, including a table created later, are masked by one policy attached to the catalog, for all account users except supply-privacy

When the returns team creates supply_silver.returns next month and a privacy steward tags customer_phone, the policy covers it with no further work. For ABAC evaluation, tags on a catalog or schema flow down to the tables beneath them, but never to columns. Tagging can be automated too: Data Classification scans tables with an AI model and suggests or applies tags for sensitive data; its results land in a system table that is in Public Preview as of October 2026.

The requirements page lists what you need: serverless compute, or standard or dedicated compute on Databricks Runtime 16.4 or above (dedicated with fine-grained access control enabled). Quotas include 100 policies per catalog and 20 principals per policy. Row filter and column mask policies carry no preview label in the docs as of October 2026, while policies attached at the metastore level, DENY policies and policies on views are Beta. Which mechanism should you choose?

three ways to restrict rows and columns in Unity Catalog: table-level row filters and column masks for one table at a time, ABAC policies that apply to every table whose governed tags match, and dynamic views that expose a filtered or reshaped version of base tables; Databricks recommends ABAC for rules that span many tables

My view: use ABAC for anything that recurs, such as personal data, regions or confidentiality levels, and keep table-level filters for the odd table with unique logic. Dynamic views still have a place when you want to publish a curated or joined extract to people who should not see the base tables at all. As a litmus test, simplest option first:

1. Should some people see none of the table?        -> no rows or masks needed: just don't grant SELECT
2. One table, logic nobody else will reuse?         -> table-level row filter or column mask
3. The same rule on many tables (PII, region)?      -> governed tag + ABAC policy
4. People must never see the base table at all?     -> dynamic view, granted instead of the table

How does lineage show where a number came from?

Data lineage is the record of how data moved: which files fed which table, which job built which table from which, and which dashboards read the result. According to the lineage page, Unity Catalog captures it automatically for queries that run on Databricks, down to the column, and combines it across all workspaces attached to the metastore. Nobody has to document it by hand. The bottom line: lineage is a recorded family tree for every table and column, captured automatically, so “where did this number come from?” becomes a query rather than an investigation.

A food recall shows why that matters. A supermarket learns that one farm’s lettuce is contaminated. Because every crate is logged from farm to depot to store to salad, it can answer three questions in minutes: which salads contain that farm’s lettuce (impact), which farm supplied the salad a customer complained about (root cause), and proof that the lettuce never reached the stores it should not have (compliance). Without the log, it would pull every salad off every shelf.

How does this apply to Acme? The farm is a delivery file in the landing volume, the depot is the silver table, the salad is supply_service, and the store shelf is the dashboard:

lineage for Acme's supply data: delivery files in the volume landing feed deliveries_raw, a pipeline builds deliveries in silver, a nightly job builds supply_service in gold and an AI/BI dashboard reads it; column lineage shows otif_pct derived from units_ordered and units_delivered; below, three rows of system.access.table_lineage that record the same flow

Lineage answers three questions an architect meets every week. Impact analysis: before the team renames units_delivered, which tables, jobs and dashboards break? Root cause: the weekly dashboard shows an odd otif_pct for Northern Europe; which upstream table changed? Compliance: where does the personal data in deliveries.driver_phone flow? If the answer is “nowhere beyond silver”, that is evidence an auditor can see.

You read lineage in two places. The Lineage tab in Catalog Explorer draws the graph and keeps lineage indefinitely for data captured after 1 September 2024. The system tables system.access.table_lineage and system.access.column_lineage let you query it with SQL, over a rolling 395-day window by default, which is how you would build an automated check such as “alert if any table downstream of a column tagged pii is outside the silver schema”. To see lineage you need at least BROWSE on the table’s catalog, and the catalog must be bound to your workspace.

One honest gap: lineage only covers what runs through Databricks, through Spark DataFrames or Databricks SQL. A BigQuery job that reads the same files directly is invisible to it unless you register it as an external lineage object.

Which data does Databricks encrypt, and with whose key?

The bottom line: everything is already encrypted; a customer-managed key changes who controls the key, and each key covers only the storage it is attached to.

A simple example. A bank locks your documents in a safe-deposit box with the bank’s own key. That is encrypted, but the bank decides. Now the bank offers a box that opens only with a key you keep in your own drawer: you see each time it is used, you can change it, and if you throw it away nobody can open the box. The catch people miss: that key opens the safe-deposit box only. The filing cabinet at your home is a different lock. The bank’s key is the default Google-managed or Databricks-managed key, your own key is a customer-managed encryption key (CMEK), and the home filing cabinet is any storage, such as a lake bucket, that the key was never attached to.

Everything Databricks stores is encrypted at rest by default: in the control plane with keys Databricks manages, and in your project with keys Google manages. Traffic between cluster workers is encrypted too; the data security page notes that Google encrypts all VM-to-VM traffic within a VPC network. So why bring your own key?

Because “encrypted” and “under our control” are different things. With a customer-managed encryption key, the key lives in Acme’s Cloud KMS project. Acme decides who may use it, sees every use in its own logs, rotates it on its own schedule, and can disable it, which makes the data unreadable everywhere at once. Regulated industries often require exactly that last ability.

A few Cloud KMS terms first:

A key ring is a named group of keys in one location, such as europe-west2. Its location is fixed at creation.
A key is the named object you grant permissions on. A symmetric key uses the same secret to encrypt and decrypt, which is what storage encryption needs.
A key version holds the actual key material. Rotation creates a new primary version for new encryption; old versions stay available to decrypt old data.

The customer-managed keys page defines two uses on Google Cloud:

Managed services: data Databricks keeps in its control plane, namely notebook source and interactive results, secrets in Databricks secret scopes, Databricks SQL queries and query history, AI/BI dashboards (created after 1 November 2024), Git credentials, AI Search indexes and Model Serving images. For serverless workspaces, this key also covers the workspace storage.
Workspace storage: the DBFS root and workspace system data buckets that Databricks creates in your project (Part 5 shows what they hold), and the persistent disks of classic cluster VMs. Serverless disks are not covered, because they are short-lived and destroyed with the workload.

The setup rules come from Configure customer-managed keys, which carries a Public Preview label as of October 2026:

It needs the Enterprise tier. (The overview table on the data security page says Premium; the feature page says Enterprise. I would plan for Enterprise and confirm with Databricks.)
The key is a symmetric Cloud KMS key in the same region as the workspace. It may live in a different project, so Acme keeps its keys in a dedicated project, acme-kms, run by the security team.
An account admin registers the key as an encryption key configuration in the account console. One configuration can serve both uses and several workspaces.
You can attach it only when you create the workspace. Adding it later means a new workspace.
Whoever creates the workspace needs cloudkms.cryptoKeys.getIamPolicy and cloudkms.cryptoKeys.setIamPolicy on the key, so that Databricks can grant itself the role it needs.

Who actually uses the key? The least-privilege workspace guide, where you grant everything yourself, spells it out: for workspace storage, the Compute Engine and Cloud Storage service agents (Google-managed identities that act inside your project, as Part 2 explained) get Cloud KMS CryptoKey Encrypter/Decrypter; for managed services, the workspace service account gets it. In the standard flow, the creator’s getIamPolicy and setIamPolicy permissions let Databricks make the grants it needs; after creation, gcloud kms keys get-iam-policy on the key shows exactly who holds the role. This is how Acme would build it with Terraform:

resource "google_kms_key_ring" "euw2" {
  project  = "acme-kms"
  name     = "acme-euw2"
  location = "europe-west2"             # must match the workspace region
}

resource "google_kms_crypto_key" "dbx" {
  name            = "dbx-workspace"
  key_ring        = google_kms_key_ring.euw2.id
  rotation_period = "7776000s"          # 90 days
}

resource "databricks_mws_customer_managed_keys" "dbx" {
  provider   = databricks.accounts
  account_id = var.databricks_account_id
  gcp_key_info {
    kms_key_id = google_kms_crypto_key.dbx.id
  }
  use_cases = ["MANAGED_SERVICES", "STORAGE"]
}

resource "databricks_mws_workspaces" "supply" {
  # ... name, location, project and network as in Parts 1 and 3
  managed_services_customer_managed_key_id = databricks_mws_customer_managed_keys.dbx.customer_managed_key_id
  storage_customer_managed_key_id          = databricks_mws_customer_managed_keys.dbx.customer_managed_key_id
}

The resource and attribute names come from the Databricks Terraform provider; I have not applied this. One provider detail: on Google Cloud, the account-level provider must authenticate with a Google-issued OIDC token (an OpenID Connect identity token), not a Databricks OAuth token, because creating the key configuration validates the key in your project. Part 2 explains when each of those token types applies.

How does this apply to Acme? The safe-deposit box is the Databricks control plane and workspace storage; the home filing cabinet is gs://acme-lake-supply. Now the most common misunderstanding. “We turned on CMEK in Databricks, so our data is under our key.” Not quite. Unity Catalog tables do not live in the workspace storage buckets; they live in Acme’s own bucket, which the Databricks key configuration never touches:

which key protects what at Acme: the Databricks managed services key covers notebooks, secrets, queries and dashboards in the control plane and the storage of serverless workspaces; the workspace storage key covers the two workspace storage buckets and classic cluster disks in acme-dbx-prod; Unity Catalog tables in gs://acme-lake-supply need the bucket's own default key, and BigQuery tables need a BigQuery key; a disabled key makes that data unreadable

One side effect to plan for: with a managed services key, fields such as statement_text in the query history system table appear empty until an account admin also configures encryption for the system catalog.

How does a Cloud KMS key protect Acme’s own lake bucket?

You protect the lake bucket in Cloud Storage itself, by giving it a default key. Every new object written to the bucket, which for a Delta table means every Parquet data file, every _delta_log JSON commit and every checkpoint, is then encrypted with that key. The Cloud Storage CMEK page sets the rules:

The key ring must be in the same location as the bucket. Acme’s bucket is in europe-west2, so the key ring is too. Dual-region buckets need a key ring that matches the dual-region’s location code.
The project’s Cloud Storage service agent performs the encryption and decryption, so the agent, not the reader, needs the Cloud KMS role.
CMEK encrypts the object’s data and its CRC32C and MD5 checksums. Object names and most metadata use Google’s own keys, so you can still list and delete objects with a disabled key, and you should never put sensitive data in object names.
A default key applies to new objects. Existing objects keep their old encryption until they are rewritten.

# Let the acme-lake project's Cloud Storage service agent use the key
gcloud storage service-agent --project=acme-lake \
  --authorize-cmek=projects/acme-kms/locations/europe-west2/keyRings/acme-euw2/cryptoKeys/lake-supply

# Make it the bucket's default key
gcloud storage buckets update gs://acme-lake-supply \
  --default-encryption-key=projects/acme-kms/locations/europe-west2/keyRings/acme-euw2/cryptoKeys/lake-supply

Both commands are from Use customer-managed encryption keys; I have not run them against Acme’s projects. Here is what happens on every read:

how a CMEK-protected read works on gs://acme-lake-supply: compute reads an object as the storage credential's service account, which needs no Cloud KMS role; the Cloud Storage service agent of the acme-lake project, which holds Cloud KMS CryptoKey Encrypter/Decrypter on the key lake-supply, asks Cloud KMS to unwrap the object's key and returns plaintext; if the key is disabled or the agent loses the role, the read fails

This is the pleasant part of Google’s model. Unity Catalog’s storage credential service account, acme-lake-sc, needs only storage roles on the bucket; it never needs a Cloud KMS role. Permission to read data and permission to use the key stay separate.

The unpleasant part is the blast radius. If someone disables the key version or removes the service agent’s role, Cloud Storage refuses to decrypt, and because every Delta read starts by reading the _delta_log files, whole tables stop working, not just some files. The change is not instant either: the Compute Engine CMEK page mentions a delay of up to an hour before a disabled key stops working, and that cluster VMs cannot boot while a disk’s key is revoked. Destruction is slower and safer than it sounds: a destroyed key version first waits in a scheduled for destruction state for 30 days by default, during which you can restore it, as the destroy and restore page explains. After that, objects encrypted with it are gone for good, and you keep paying to store them until you delete them.

The same pattern covers the rest of Acme’s estate: BigQuery datasets take a default key through the BigQuery encryption service agent (Part 8 covers the BigQuery side), and a Pub/Sub topic used for ingestion takes its own key. When a compliance officer asks “is everything under our key?”, the honest answer is a list: the Databricks managed services key, the workspace storage key, the lake bucket’s default key, and a key on each BigQuery dataset and Pub/Sub topic in the data path.

Where should credentials live: secret scopes, Unity Catalog secrets or Secret Manager?

The bottom line: avoid the credential if you can; if you cannot, store it once, in the place whose access rules match who needs it.

An office makes the order clear. The best door is one that recognises your badge, so there is no code to share. If a code is unavoidable, such as the code for a supplier’s portal, you can tape it inside one team’s drawer (only that room can use it), put it in the company safe with a sign-out log (any office can request it), or leave it in the building landlord’s safe because the landlord already manages it for other tenants. The badge is a keyless credential, the drawer is a workspace secret scope, the company safe is a Unity Catalog secret, and the landlord’s safe is Secret Manager.

How does this apply to Acme? The best secret is one you never create. Unity Catalog storage credentials give Databricks keyless access to Cloud Storage, and service credentials do the same for other Google Cloud services: Databricks generates a Google service account, you grant it roles, and code borrows it by name. Some credentials cannot be avoided, though, such as a partner’s API token or the BigQuery federation key that Part 2 stored as secret('bq', 'acme_federation_key'). Those that remain should be short-lived or rotated (Prerequisite 3 explains secret rotation). There are three places to keep them:

A Databricks secret scope (the docs now call it a workspace secret scope) is a named collection of secrets in one workspace, stored in an encrypted database that Databricks runs. Access is set with secret ACLs (access control lists) at three levels: READ, WRITE and MANAGE. With a managed services key, the scope is encrypted with Acme’s key. On Google Cloud the docs describe only these Databricks-backed scopes; Azure Databricks also offers scopes backed by Azure Key Vault, and there is no Secret Manager-backed equivalent.
A Unity Catalog secret is a securable object with a three-level name, such as acme_supply.governance.partner_api_token. It is available in every workspace attached to the metastore and governed by READ SECRET, WRITE SECRET and CREATE SECRET, plus REFERENCE SECRET (see a secret and its metadata without its value), with the same inheritance and auditing as tables. Reading one with dbutils needs Databricks Runtime 17.3 LTS or above, or serverless environment version 4. Since 2 November 2026 a schema can hold up to 1,000 Unity Catalog secrets and a metastore up to 10,000 (up from 100 and 1,000), and BROWSE on a catalog also lists its secrets and their metadata, never their values. For example, granting BROWSE on acme_supply to account users lets an engineer find acme_supply.governance.partner_api_token and request access, without being able to read it.
Secret Manager is Google Cloud’s own service. A secret there has versions, can be regional for data residency, and is governed by Google IAM, typically roles/secretmanager.secretAccessor. Databricks reads it through a service credential, which keeps one source of truth for teams that also run Cloud Run or GKE.

four places a credential can live for Databricks on Google Cloud: nowhere, by using a keyless storage or service credential; a Unity Catalog secret governed by READ SECRET; a workspace-level Databricks secret scope governed by ACLs; or Google Secret Manager, read through a Unity Catalog service credential whose service account holds Secret Accessor

Reading from Secret Manager looks like this in a notebook. The service credential acme-secret-reader maps to a Google service account that holds Secret Accessor on the one secret it needs:

from google.cloud import secretmanager

creds = dbutils.credentials.getServiceCredentialsProvider("acme-secret-reader")
client = secretmanager.SecretManagerServiceClient(credentials=creds)
name = "projects/acme-dbx-prod/secrets/partner-api-token/versions/latest"
token = client.access_secret_version(name=name).payload.data.decode("utf-8")

The getServiceCredentialsProvider call is the pattern the service credentials usage page shows for Pub/Sub; I have applied it to the Secret Manager client library and have not run it. One difference matters: values read with dbutils.secrets.get are redacted, shown as [REDACTED] if printed, while a value you fetch yourself is an ordinary string. Redaction is a safety net, not a wall: the secrets page warns that workspace admins and anyone with read access can still reach a secret’s value (the workspace secret scopes page says “it is not possible to fully prevent these users from viewing secret contents”), so keep the ACLs tight.

My view: keyless credentials first, Unity Catalog secrets for anything shared across workspaces, Secret Manager when another platform owns the secret, and secret scopes only for workspace-local cases. As a litmus test, in that order:

1. Can a storage or service credential do it keylessly?   -> no secret at all
2. Is the secret shared by Databricks workspaces only?    -> Unity Catalog secret
3. Does Cloud Run, GKE or another team also use it?       -> Secret Manager, read via a service credential
4. Only then, for one workspace's local need               -> workspace secret scope

How do you see who did what?

There are two trails, and you need both. The bottom line: Databricks records which person read which table; Google Cloud records which service account touched which bucket object; only together do they tell the whole story.

A bank branch shows the split. The branch’s own ledger says that a customer, Ms Shah, withdrew documents from her box at 10:02. The vault door’s log says that staff key 14 opened the vault at 10:02. The ledger alone does not prove the vault was opened; the door log alone does not say for whom. The ledger is the Databricks audit log, the door log is Cloud Audit Logs, and staff key 14 is the storage credential’s service account. Acme’s version follows, and the section closes by putting the two side by side.

Acme's two audit trails: Databricks records account and workspace actions such as sign-ins, Unity Catalog reads and grant changes in the system table system.access.audit, and can also deliver JSON files to a bucket; Google Cloud records actions in Acme's projects in Cloud Audit Logs, where Admin Activity logs are always on and Data Access logs for Cloud Storage must be turned on; the security team joins the two by time and identity

The Databricks trail. The audit log system table system.access.audit records account and workspace events: sign-ins, Unity Catalog reads, permission changes, secret reads, cluster and job actions. A system table is a read-only table that Databricks hosts in a special catalog called system and shares with your account through OpenSharing; its data stays in the metastore’s region. According to the system tables reference, the audit table is in Public Preview as of November 2026, keeps 395 days by default, and holds regional data for workspace events and global data for account events. Each row looks like this:

four rows from system.access.audit as they might look for Acme: Sam's getTable and generateTemporaryTableCredential when he reads supply_service, Lena's updatePermissions on supply_silver, and a service principal's getSecret on the BigQuery federation key; each row also carries workspace_id, source_ip_address, request_id and the response status

generateTemporaryTableCredential is the event the audit log reference recommends for “who queried what and when”, because it is logged whenever Unity Catalog hands compute a short-lived credential to read a table. A first query for Acme’s security team:

-- Who read supply_service in the last 7 days, and from which address?
SELECT event_time, user_identity.email, source_ip_address
FROM system.access.audit
WHERE event_date >= current_date() - INTERVAL 7 DAYS
  AND service_name = 'unityCatalog'
  AND action_name = 'generateTemporaryTableCredential'
  AND request_params.table_full_name = 'acme_supply.supply_gold.supply_service'
ORDER BY event_time DESC;

I have not run this query; the column and parameter names come from the audit system table page and the reference. Access to system tables is itself governed: account admins who are also metastore admins can read them, and anyone else needs USE CATALOG on system, plus USE SCHEMA and SELECT on the schema. Some parameters that contain SQL are hidden unless you are an account admin or in the databricks_pii_access group. Verbose audit logs, a workspace setting, add the text of each notebook command and SQL statement, which helps investigations and is itself sensitive.

If a SIEM (security information and event management system, the tool a security operations team uses to collect and correlate logs) needs raw files, you can also configure audit log delivery on the Premium plan. Databricks writes JSON files to a bucket you own at gs://<bucket>/<prefix>/workspaceId=<id>/date=<yyyy-mm-dd>/auditlogs_<id>.json, with account events under workspaceId=0. Give the account log-delivery@databricks-prod-master.iam.gserviceaccount.com Storage Admin on a dedicated bucket, expect files within about an hour, and note the limits: two account-level configurations, and configurations can be disabled but never edited or deleted. Databricks itself recommends the system table over exporting, because audit data is sensitive.

The Google Cloud trail. Cloud Audit Logs record what happened inside Acme’s projects. Admin Activity logs, which cover changes such as the workspace service account creating cluster VMs or someone changing a bucket’s IAM policy, are always on and cannot be disabled. Data Access logs, which record reads of user data such as acme-lake-sc reading an object, are off by default for every service except BigQuery. Turn on DATA_READ for Cloud Storage in the data project, accepting the extra log volume:

resource "google_project_iam_audit_config" "lake_reads" {
  project = "acme-lake"
  service = "storage.googleapis.com"
  audit_log_config {
    log_type = "DATA_READ"
  }
}

Now the two trails line up. The Databricks trail says Sam read supply_service at 08:02; the Google trail says acme-lake-sc read these objects from gs://acme-lake-supply at the same moment. Together they tell a complete story.

Other system tables matter here too: system.access.table_lineage and column_lineage from the previous section, system.query.history for queries on SQL warehouses and serverless compute, and system.access.outbound_network, which Part 4 uses to show what the serverless egress policy blocked. The billing tables belong to Part 9.

Which compliance features exist on Google Cloud?

Some workloads need more than good configuration. Databricks sells an Enhanced Security and Compliance add-on, described on the compliance page, with three parts:

The compliance security profile adds a hardened compute image (a CIS Level 1 hardened operating system with FIPS 140 validated encryption modules), TLS 1.2 or higher for traffic within the cluster and for egress, enhanced security monitoring and automatic cluster update. On Google Cloud you can select the compliance standards C5, HIPAA, HITRUST, PCI-DSS and TISAX, or enable the profile without a standard.
Enhanced security monitoring puts agents on cluster VMs: file integrity monitoring, antivirus and malware detection, and vulnerability scanning. Their findings land in the audit logs, and reviewing them is your responsibility.
Automatic cluster update restarts clusters during a maintenance window so they run the latest images.

Three cautions from the configuration page. It needs the Enterprise tier plus the add-on. Enabling the profile is meant to be permanent: once a workspace has processed regulated data, the only way back is a new workspace. And free-text fields such as workspace names, cluster names, tags and job names may be processed outside the compliance boundary, so never put sensitive data in them. The regional support table includes europe-west2, but check each standard and feature for your region, and check the Databricks Trust Center for the current list of certifications on Google Cloud before you commit to a customer.

What can go wrong?

troubleshooting governance and security on Databricks on Google Cloud: missing usage privileges, workspace-catalog bindings, unsupported runtimes for filtered tables, functions dropped before their row filter, disabled Cloud KMS keys or lost service agent roles, CMEK workspace creation without key IAM permissions, empty query text under managed services CMEK, and audit log delivery blocked by bucket permissions or VPC Service Controls

Three of these deserve a sentence each. Keys cause the worst outages because they fail everywhere at once, so treat a change to a key’s IAM policy or state as a production change with an approver. Ownership by individuals causes the slowest pain: nobody notices until the person leaves and a grant needs changing. Data Access logs are the gap auditors find: Admin Activity logs exist by default, so teams assume reads are logged too, and they are not until you turn them on.

What would I do now?

Decide on customer-managed keys before creating the workspace, because the key configuration can only be attached at creation. If the answer is yes, create acme-kms, the key ring and the keys first.
Give the lake bucket its own default key and grant the Cloud Storage service agent, then check that every new Delta file shows the key in its kmsKeyName metadata.
Build the grants as code: groups only, USE CATALOG and USE SCHEMA plus SELECT at schema level, ownership transferred to groups, and every production catalog and credential bound to the production workspace.
Create one governed tag for personal data and one ABAC masking policy per catalog, and turn on Data Classification so new columns get tagged.
On day one, grant the security team read access to system.access.audit and turn on Data Access logs for Cloud Storage in the data project, so both trails exist before anyone needs them.

What did we learn?

Unity Catalog names every object as catalog.schema.object under one regional metastore, and nothing is readable until someone grants USE CATALOG, USE SCHEMA and a data privilege such as SELECT, which then inherits to every current and future object below.
Owners have full control over what they own, so production objects should belong to groups, and workspace-catalog binding keeps production catalogs out of development workspaces even for users who hold grants.
Row filters and column masks change what each person sees at query time, and ABAC policies apply the same rule to every column that carries a governed tag, including tables created later.
A Databricks key configuration covers the control plane and the workspace storage buckets and disks, but Unity Catalog tables live in your bucket, which needs its own Cloud KMS default key; in both cases the service agent uses the key, not the reader.
Credentials are safest when they do not exist; otherwise keep them in Unity Catalog secrets, secret scopes or Secret Manager read through a service credential.
system.access.audit records what happened in Databricks and Cloud Audit Logs record what happened in your projects, and you need Data Access logs turned on for the two trails to meet.

What comes next?

Databricks on Google Cloud, Part 7: The platform on Google Cloud, runtime, Photon, SQL, Lakeflow and AI moves from protecting the data to using it: Databricks Runtime and Photon, SQL warehouses, Lakeflow pipelines and jobs, and the AI features. Every one of them runs under the grants, filters, keys and audit trails set up here.

Sources

Opinions are my own. Every fact in this post comes from the public documentation linked above. Acme Consumer Brands and its data are fictional.