Posts from the data workshop. Pipelines that actually run, models that don't break, dashboards somebody reads.
Unity Catalog accepts PRIMARY KEY, UNIQUE and FOREIGN KEY on Delta tables. None of them is enforced on write, and adding RELY tells the optimizer to believe the declaration anyway — so a violation returns wrong results faster. That is one topic out of 130 in a Databricks warehouse handbook now open on Crosshire Learn: 23 chapters, 72 pages, one worked warehouse on every page, and 26 platform facts each carrying the date it was last checked.
The Query History page shows you the last statements and how long each took. It won't tell you which cluster is pegged right this minute, or that a query is running slower than its own last-7-day baseline. Two SELECTs over the same system tables close both gaps.
If you run Claude Code, you already produce a goldmine of structured data on how you build. AURA is a local-first, open-source pipeline that turns that transcript exhaust into an itemized record — every dollar, every prompt, every tool call — reconciled to the same total on every page, and never leaving your machine.
ACCOUNT_USAGE — the Snowflake view that lists everyone who can read a table — is restricted to ACCOUNTADMIN for security, so most engineers can't use it. And SHOW GRANTS ON the table shows only the first hop, not the people who reach it through a chain of roles. Here is how to answer who-can-read anyway: a low-privilege stored procedure that walks the grant graph live with the one command every role already has.
You've inherited a Databricks account, or the bill jumped and nobody can say why. The full library is one hundred SELECTs — but one hundred with no order of operations is just a bigger homework assignment. This is the order: ten queries, three moves — trust the numbers, chase the money, check the risk — and a findings sheet you can defend.
The console shows a bill and a usage chart. Neither tells you what Predictive Optimization is maintaining, what VACUUM actually reclaimed, or which tables nobody has touched in months — that lives in two system tables and four plain SELECTs.
When a Databricks bill jumps, every tool hands you a bigger number and asks to be trusted. These are the one hundred plain SELECTs we run instead — against your own system tables, open on GitHub. A number is a claim; the query behind it is the receipt.
Reliability is a cost line on Databricks. The Jobs console shows you run history — not the cost-ranked failure backlog, the SKU arbitrage, or the runs that quietly failed a task and reported SUCCEEDED anyway.
A serving endpoint doesn't bill for the tokens it generates — it bills for staying ready to generate them. Three questions, four plain SELECTs over Databricks system tables, and the console runs none of them for you.
Your risk register is scattered across audit logs and information_schema, and nothing in the console ranks it — the createGrant that slipped a service principal into admins sits in the same flat feed as every routine one. Four questions, four plain SELECTs over Unity Catalog system.* tables, no dashboard required.
The naive-Opus agent ran 67 tool calls and produced no report. A re-run lost its own best finding. Three contaminations quietly invalidated the control group. Every dispute was settled the same way: grep the trace.
63.55 GB of disk spill — one query spilling 26.75 GB — and I had written 'clean' in my manual analysis notes. A naive agent with one sentence of context falsified two of my clean-bill claims. Then skilled-Opus made a units error the naive agent didn't. No analyst dominated. The union of all three, after four adjudication queries, beat every individual.
Four agents, one directed question, same account. Skilled-Sonnet answered correctly for $0.19 in 248 seconds. Naive-Sonnet answered confidently, fluently, and wrong — and its trace contains zero queries against the attribution column.
The engineered audit prompt made skilled-Opus complete a full run in 14 minutes for $1.77 — and the cache-units bug that derailed an earlier run did not recur. That is the payoff. This note shows how to build the prompt, what the three operating tiers cost, and why the idle-burn run-rate works out to roughly $980 a month.
An AI agent found a notebook I forgot to close — 98.9% of one warehouse's credits, burned doing nothing — for $1.77, and named the exact function responsible. That finding is the least interesting thing here. The lab around it cost under $15, and what it measured contradicts both loud opinions about agentic analysis.
One pass over a Snowflake account export surfaced $2,090 a year of idle compute, 61 findings, and zero users with MFA — none of it visible on the invoice the day before. Every figure traces back to the row that produced it.
One query against a month of Claude Code transcripts surfaced $2,666 in spend, 32,600 tool calls, and a single agent eating two-thirds of the bill — none of it visible the day before. Aura is the local-first analytics platform that reads it back.
We attached a NULL_COUNT DMF to customers.email and watched the dashboard for a fortnight. It read 11 % null. Then we added WITHIN GROUP (signup_region). The next run wrote four rows instead of one. DE was 40 %. The same column, the same scan, two readings — and the second one was the page-worthy unit the first had been hiding.
We opened a Snowflake worksheet, ran one query, and twelve of fourteen users came back entitled to spend Cortex credits — none of them ever asked. The lock Snowflake shipped this month is real. The door has been open since the account was created.
We deleted three rows on Snowflake. Queried them on Databricks. Got three rows back. Iceberg v3 went GA on Snowflake on 2026-05-07 and Public Preview on Databricks since April — both vendor blogs say interop is solved, neither ran the other engine. The matrix is what happens when you do, feature by feature, with the verbatim error string for the cell where the docs say no.
We migrated one one-line dbt helper from macro to 1.11 UDF. The unit test failed with Unknown function CENTS_TO_DOLLARS — and the failure was the post. What the compiled SQL diff actually says, what state:modified does to a one-character body change, and why dbt test alone stops being enough.
When Databricks anomaly detection fires, the email names the monitor and stops. The gap — `daily_row_count.last_value = 412` against `min_predicted_value = 9400` — sits two struct levels deep in `system.data_quality_monitoring.table_results`. Most practitioner posts never show that row.
Unity Catalog ABAC has no UPDATE POLICY verb. An edit is a re-create against the same name — so the audit trail your CISO reads under-reports drift unless you look for duplicate createPolicy events on the same policy_name.
Same Unity Catalog. Same source tables. Four dashboards returned four numbers for one Slack question — a 5,231-customer spread. None of the SQL was wrong. Each dashboard had its own definition of active. Unity Catalog metric views are the one-place fix.
A managed Snowpark Container Services notebook with AUTO_SUSPEND_SECS = 0 held a warehouse warm for 26 hours and burned €70 — 99.8% of it on nothing a human asked for. The documented fix is one parameter on the notebook (IDLE_AUTO_SHUTDOWN_TIME_SECONDS). An optional backstop is your own SPCS compute pool. The bleed pattern, the proof, and the queries that find it on your own account.
Most of the logins in your Snowflake account are not logins — they are driver OAuth refreshes and service-account connects that nobody is reading. One real account: 691 events, six interactive, and a method that surfaces what a single-account analyst would miss.
Snowflake shipped Adaptive Compute. The pitch is that you stop managing warehouses. We ran the numbers (50 concurrent queries across nine warehouse configurations) to see if the scaling claim holds and whether the operational reduction is worth the move.