Databricks Genie and AI/BI Dashboards

Genie is Databricks' text-to-SQL chat experience: a business user asks a question in natural language, Genie converts it to SQL against tables in Unity Catalog, executes the query on a SQL Warehouse, and returns the answer with the generated SQL visible for inspection. AI/BI Dashboards (the GA evolution of Lakeview) are the dashboarding side of the same product family — AI-native dashboards built on the same UC-aware semantic layer, with embedded Genie chat as a first-class citizen.

Together they form Databricks' answer to self-service BI: a curated set of governed tables, a natural-language interface for exploration, and dashboards for the canonical questions.

1. What Genie Is

Genie sits over a curated Genie Space — a scoped collection of Unity Catalog tables and views, plus the metadata that grounds the model: column descriptions, business glossary terms, sample questions, and certified SQL examples. When a user types a question, Genie:

1. Selects relevant tables from the Space using the metadata.
2. Generates SQL using a foundation model conditioned on the schema and the curated examples.
3. Executes the SQL on the Space's SQL Warehouse using the questioner's identity (so UC row/column policies apply).
4. Returns the result as a table or chart, alongside the generated SQL for the user to inspect, edit, and re-run.

The "show the SQL" property is what makes Genie credible for analysts: the model is not a black box, it's an SQL author whose work you can audit and correct on the spot. Corrections that you mark as certified get fed back into the grounding context for future questions.

2. Creating a Genie Space

The setup happens in the workspace UI under Genie → New Space. The conceptual fields:

• Name and description — what business domain does this Space cover ("Sales Analytics", "Customer Support Operations", "Fleet Telemetry").
• SQL Warehouse — the compute that will execute generated queries. A serverless small warehouse is the typical starting point.
• Tables — usually 5 to 30 Gold-layer tables and views from a single subject area. Resist the urge to dump the entire catalog into one Space; a focused Space dramatically improves SQL quality.
• Instructions — free-form natural-language guidance to the model: business definitions, default filters, naming conventions ("'active customers' means status='ACTIVE' and last_login_at >= current_date - 90").
• Sample questions — questions the user is most likely to ask, shown as suggestions on the Space landing screen.
• Certified queries — vetted (question, SQL) pairs that act as both UI shortcuts and few-shot examples for the model.

Permissions are standard Unity Catalog: a user can use the Space only if they have SELECT on the underlying tables. Row filters and column masks are honored at execution time, so two users asking the same question may legitimately get different results.

3. Grounding: Instructions, Examples, Certified Queries

The single biggest lever on Genie quality is the metadata you give it. Three things matter:

Rich Column Descriptions in Unity Catalog

Genie reads UC column comments. Tables with sparse or generic comments (id, amount, status) produce ambiguous SQL. Invest in describing every column the model will see:

ALTER TABLE main.sales.fct_orders
  ALTER COLUMN order_id     COMMENT 'Surrogate primary key generated at ingest',
  ALTER COLUMN customer_id  COMMENT 'Foreign key to dim_customer.customer_id; natural business key',
  ALTER COLUMN order_ts     COMMENT 'Order placement time in UTC; use this column for any time-based filter',
  ALTER COLUMN net_revenue  COMMENT 'Order total after discounts and refunds, in USD; the canonical revenue column',
  ALTER COLUMN status       COMMENT 'One of: PLACED, SHIPPED, DELIVERED, CANCELLED, RETURNED';

ALTER TABLE main.sales.fct_orders
  SET TBLPROPERTIES (
    'comment' = 'One row per customer order. Use this table for revenue, AOV, and order-volume questions.'
  );

Space-Level Instructions

Use the Instructions field to encode business semantics that aren't visible in the schema:

- The canonical revenue column is `net_revenue` in `main.sales.fct_orders`. Do not use `gross_revenue`.
- "Active customer" means a customer with an order in the last 90 days. Always join to fct_orders and filter on order_ts.
- Treat status='CANCELLED' and status='RETURNED' as excluded from any "completed sale" question unless the user explicitly asks for them.
- All time columns are UTC. When the user says "today" or "this week", use UTC, not local time.
- Currency is always USD. Do not attempt currency conversion.

Certified Queries

A certified query is a verified (question, SQL, optional chart) triple. They are the highest-leverage grounding signal — they teach the model the joins, filters, and idioms that match how this business actually thinks.

-- Certified query: "What is daily revenue for the last 30 days?"
SELECT
  DATE_TRUNC('day', order_ts) AS order_date,
  SUM(net_revenue)            AS daily_revenue
FROM main.sales.fct_orders
WHERE order_ts >= current_timestamp() - INTERVAL 30 DAYS
  AND status NOT IN ('CANCELLED', 'RETURNED')
GROUP BY 1
ORDER BY 1;

-- Certified query: "Top 10 customers by revenue this quarter"
SELECT
  c.customer_id,
  c.customer_name,
  SUM(o.net_revenue) AS revenue
FROM main.sales.fct_orders o
JOIN main.sales.dim_customer c USING (customer_id)
WHERE o.order_ts >= DATE_TRUNC('quarter', current_date())
  AND o.status NOT IN ('CANCELLED', 'RETURNED')
GROUP BY 1, 2
ORDER BY revenue DESC
LIMIT 10;

Five to ten well-chosen certified queries per Space cover a surprising fraction of real-world questions. The pattern: ship the Space, watch what users ask in the first two weeks, and certify the answers to the most common questions.

4. AI/BI Dashboards

AI/BI Dashboards are the dashboard product in the same family as Genie — the rebrand and GA of Lakeview Dashboards. Each dashboard is composed of tiles backed by SQL queries against Unity Catalog tables, using the same SQL Warehouses as Genie. Two design ideas distinguish them from older Databricks SQL dashboards:

• AI-assisted authoring. A "describe a chart" prompt lets the author type a question and get a starting tile (query + visualization) generated automatically, which they can then refine.
• Embedded Genie chat. Each dashboard can have an attached Genie Space so viewers can ask follow-up questions directly from the dashboard surface, scoped to the same tables and grounding.

A dashboard is itself a UC object — serializable to YAML, manageable through Asset Bundles, and grantable through standard GRANT statements. The same dashboard definition can be promoted dev → staging → prod via a bundle target override.

# Sketch of a dashboard resource in a Databricks Asset Bundle
resources:
  dashboards:
    sales_overview:
      display_name: Sales Overview ${bundle.target}
      file_path: ./dashboards/sales_overview.lvdash.json
      warehouse_id: ${var.warehouse_id}
      parent_path: /Shared/dashboards

5. Practical Tips for Reliable Genie

Genie quality is a function of how disciplined you are about scope and metadata. The patterns that consistently work:

• Narrow the Space. One subject area, ideally Gold-layer tables only. Each extra table multiplies the surface for ambiguous joins.
• Hide raw Bronze and Silver tables. Don't expose tables with cryptic source-system names. Build clean Gold views with business-friendly column names and add only those to the Space.
• Comment every column. If the model has to guess what amt_2 is, it will guess.
• Define the canonical metric column. If two columns could plausibly answer "revenue," explicitly tell Genie which one to use.
• Certify the top questions. One well-written certified query is worth fifty paragraphs of instructions.
• Review feedback regularly. Genie collects thumbs-up/thumbs-down on responses; treat the down-votes as a backlog for new certified queries or instruction edits.
• Use a serverless SQL Warehouse. Cold-start a classic warehouse and your first question takes 4 minutes; serverless responds in seconds.

6. Genie vs Code Interpreter vs Cortex Analyst

Three products in adjacent territory, optimized for different deployment contexts:

Genie and Cortex Analyst are the closest pair: both are text-to-SQL bound to a governed warehouse. The main difference is grounding ergonomics — Genie leans on UC metadata plus a UI for instructions and certified queries, while Cortex Analyst leans on a YAML semantic model. Pick the one that matches the warehouse you're already on; the underlying capability gap is small.

Code Interpreter is solving a different problem: arbitrary-files exploration without governance. It is complementary to Genie, not competitive — you'd reach for Code Interpreter when there is no warehouse table to query in the first place.

↑ Back to Top