The Medallion Architecture is not a Standard

Walk into almost any data team right now and ask how their pipeline is structured. There’s a good chance they’ll tell you: Bronze, Silver, Gold. The medallion architecture. Three layers, clearly named, widely understood. It has the feel of a standard.

It isn’t one. It’s a naming convention Databricks popularised in their documentation and marketing material, and at some point the industry collectively decided it was the canonical way to organise a data platform. Teams adopted it not because they evaluated it and found it fit their needs, but because it was what everyone else was doing. That distinction matters, and it causes real problems.

Where it came from and what it actually says

The underlying idea is simple and genuinely sensible: keep a raw, immutable copy of your data; do some cleaning and conforming in a middle layer; serve business-ready data to consumers at the top. Ingest, clean, model. That three-layer pattern is not new or controversial. You can find it in data warehouse literature going back decades under various names.

What Databricks did was give the layers a memorable brand: Bronze for raw, Silver for cleaned, Gold for aggregated or business-ready. The Delta Lake ecosystem made the pattern easy to implement, the documentation made it look like architecture, and the community ran with it. By the time analysts and engineers across the industry were referencing it, the marketing origin had been forgotten. It had become industry vocabulary.

That would be harmless if the labels came with clear definitions of what each layer should contain and why. They don’t. Ask five teams what belongs in Silver versus Gold and you’ll get five different answers. Some teams put lightly cleaned data in Silver. Others put fully modelled, Kimball-style fact and dimension tables there. Some reserve Gold for aggregations only. Some have four layers because they couldn’t agree where something fit. The naming has the appearance of structure without imposing any.

What goes wrong when teams adopt it blindly

The most immediate problem is storage. Raw data in a lakehouse is already cheap, but many teams end up storing three full copies of most of their data: one raw, one partially transformed, one final. For high-volume pipelines, that cost compounds fast, and the justification is usually not “we need this for a specific reason” but “that’s how the medallion architecture works.”

The Gold layer is where the real chaos tends to accumulate. Because the name implies readiness-for-consumption, teams treat it as the place to put anything that gets queried by a BI tool or an analyst. Over time it becomes a dumping ground: some tables are granular transaction data, some are pre-aggregated weekly metrics, some are one-off tables built for a specific dashboard that nobody uses anymore. The layer exists, but it has no coherent logic. You have three layers and you still need a data dictionary to explain what anything in Gold actually means.

The pattern also maps poorly onto streaming pipelines. Batch-oriented Bronze/Silver/Gold logic assumes discrete runs and clean separation between ingestion and transformation. Real-time pipelines don’t work like that. The latency requirements and stateful processing involved in streaming mean you’re often better off thinking in terms of raw topics, enrichment stages, and serving tables; architecture shaped by the data flow, not a label someone applied to batch jobs.

The pattern isn’t the problem

To be clear: raw, cleaned, modelled is a sound way to think about data pipeline layers. Keeping immutable raw data is good practice. Separating transformation concerns makes things easier to debug and rebuild. None of that is wrong.

What’s wrong is treating the Bronze/Silver/Gold labels as a substitute for actually thinking about your pipeline. The labels describe stages of transformation, but they say nothing about what your data model should look like, how your consumers will query it, or what your data products actually need to be. Those are the questions that determine whether a data platform is useful, and they have nothing to do with what you call your layers.

What to do instead

Name your layers based on what they mean to your organisation. Raw and staging and marts (the naming dbt made popular) is explicit about purpose: raw is ingested source data, staging is cleaned and typed, marts are modelled for consumers. Landing, intermediate, presentation works just as well. What matters is that the names reflect a clear contract: here’s what this layer contains, here’s who it’s for, here’s what can be trusted in it. Any naming convention that your team understands and enforces consistently is better than borrowed terminology that carries no shared meaning.

More importantly: model for your consumers. Before you decide how many layers you need or what they contain, understand who queries your data and what they need. An analyst building a recurring revenue report needs something different from a data scientist building a churn model. A real-time product dashboard has different requirements from a weekly finance extract. Kimball’s dimensional modelling is still one of the most practical frameworks for thinking through this, because it forces you to start with the business question and work backwards to the physical model. The fact and dimension structure it produces is intuitive to query and maps well to how business users think about events and attributes.

The result of that thinking might well look like three pipeline layers. It might look like two, or four, or a more complex directed graph with multiple output paths. The structure should follow the problem, not the other way around.

When your Gold layer is a mess and nobody agrees on what belongs there, the answer is not to rename it. It is to go back to the questions you should have started with: who needs this data, what will they do with it, and what does it need to look like for that to work? Those questions don’t have a branded answer. They have an engineering answer, and it’s worth working it out.