Unity Catalog Setup

Prerequisites

• Account-Level Admin Access — Required to enable Unity Catalog and create metastores. Regular workspace admins cannot enable this feature.
• Cloud Storage for Metastore — A cloud storage path (S3 bucket, ADLS container, GCS bucket) to store Unity Catalog metadata. This location must be empty and dedicated.
• Cloud IAM Credentials — Appropriate IAM roles or service accounts in your cloud provider to allow Databricks to access the metastore storage location.
• Workspace Attached to AWS/Azure/GCP Account — Your workspace must be associated with your cloud account (not a community edition workspace).

Step 1: Create a Metastore

The metastore is the cloud-backed central repository for all Unity Catalog metadata.

-- Via SQL (in any workspace in the account)
CREATE METASTORE my_metastore
  PROVIDER_NAME aws  -- or 'azure', 'gcp'
  LOCATION 's3://my-bucket/metastore-root'  -- cloud storage path

Alternatively, use the Databricks account console (admin.databricks.com) under Admin → Metastores to create visually.

Step 2: Assign Metastore to Workspace

A workspace can use only one metastore. All workspaces assigned to the same metastore share catalogs and permissions.

-- Via account console or API
-- Associate the workspace with the metastore

Once assigned, run a test query to verify:

SELECT current_metastore();

Step 3: Create Catalogs and Schemas

Catalogs are top-level containers. Schemas organize tables within catalogs.

-- Create a catalog (requires account-level admin or owner role)
CREATE CATALOG finance_prod;

-- Create a schema within the catalog
CREATE SCHEMA finance_prod.transactions;

-- Create a table in the schema (3-level path)
CREATE TABLE finance_prod.transactions.orders (
  order_id INT,
  customer_id INT,
  order_date TIMESTAMP,
  amount DECIMAL(10, 2)
);

Step 4: Set Up External Locations and Storage Credentials

Required for external tables and volumes.

-- Create a storage credential (AWS example)
CREATE STORAGE CREDENTIAL s3_credentials
  PROVIDER amazon
  CREDENTIAL_PROVIDER_TYPE assume_role
  WITH (ROLE_ARN = 'arn:aws:iam::123456789012:role/UnityCatalogRole');

-- Create an external location tied to a storage credential
CREATE EXTERNAL LOCATION s3_data_lake
  URL 's3://my-data-bucket/external-data'
  WITH (STORAGE_CREDENTIAL = s3_credentials);

Step 5: Grant Permissions

Control who can access catalogs, schemas, and tables using the principle of least privilege.

-- Grant USE CATALOG to a group (enables browsing the catalog)
GRANT USE CATALOG ON CATALOG finance_prod TO `data-team@company.com`;

-- Grant USE SCHEMA to access a specific schema
GRANT USE SCHEMA ON SCHEMA finance_prod.transactions TO `data-team@company.com`;

-- Grant SELECT to read table data
GRANT SELECT ON TABLE finance_prod.transactions.orders TO `data-team@company.com`;

-- Grant MODIFY for write access
GRANT MODIFY ON TABLE finance_prod.transactions.orders TO `data-team@company.com`;

-- Grant ALL PRIVILEGES (full control, use sparingly)
GRANT ALL PRIVILEGES ON CATALOG finance_prod TO `admin-group@company.com`;

Step 6: View and Audit Permissions

-- Show all grants on a catalog
SHOW GRANTS ON CATALOG finance_prod;

-- Show all grants on a specific schema
SHOW GRANTS ON SCHEMA finance_prod.transactions;

-- Show grants for a specific principal
SHOW GRANTS ON SCHEMA finance_prod.transactions TO `data-team@company.com`;

-- View who has access to a table
SHOW GRANTS ON TABLE finance_prod.transactions.orders;

Best Practices

• Use Groups, Not Individuals — Grant permissions to groups (e.g., data-engineers, analysts) for easier management and scalability.
• Hierarchy Matters — Permissions granted at the catalog level inherit to schemas and tables. Design your hierarchy strategically.
• Separate Environments — Create separate catalogs for dev, staging, and production to enforce isolation and prevent accidental modifications.
• Audit Regularly — Use SHOW GRANTS and audit logs to review who has access to sensitive data.
• Plan Storage Credentials — One credential can back multiple external locations. Consolidate for easier management.