DATA ENGINEERING
Synapse to Databricks migration, without stopping the business
Migrating from Azure Synapse to Databricks means moving storage to Delta Lake, rebuilding pipelines around Spark and Autoloader, and re-pointing every downstream report — usually while the business keeps running on the old platform. M3Stack has delivered this for a global enterprise, covering more than ten business areas fed by eight source systems including Salesforce, Oracle ERP Cloud, Marketo and SharePoint, with no disruption to finance and sales reporting during the move. We structure the target estate in bronze, silver and gold layers so raw landing data, cleansed conformed data and business-ready tables stay separated and governed. Orchestration runs through Azure Data Factory across both batch and streaming paths. The result is one continuously refreshed data estate that leadership can trust and every team can build on, rather than a set of pipelines only one person understands.
01 / SCOPE
What the migration actually covers
Estate assessment
What exists, what is actually used, and what should not survive the move. Most estates carry pipelines nobody has read in years — migrating them wholesale just moves the problem.
Delta Lake foundation
Storage rebuilt on Delta Lake so tables are transactional, versioned and time-travellable, instead of files that are correct only if every job ran in the right order.
Medallion architecture
Bronze for raw landing data, silver for cleansed and conformed, gold for business-ready tables — general ledger, receivables, payables, orders, quotes.
Ingestion rebuild
Autoloader streaming ingestion for continuously arriving sources, batch where batch is genuinely right, and the connectors to bring each source system in cleanly.
Orchestration
Azure Data Factory coordinating batch and streaming paths together, so dependencies are explicit and a failure is visible rather than silently stale.
Governance and lineage
Access control, data quality checks and end-to-end lineage, so any number on a dashboard can be traced back to the source system it came from.
02 / PROOF
Enterprise lakehouse migration
Ten business areas, eight source systems, no disruption
A global enterprise's finance and sales reporting was running on an ageing data platform that was getting slower and costlier to grow. We moved the entire estate to Databricks — more than ten business areas, fed by eight different source systems — without disrupting the business running on top.
Sources included Salesforce, Oracle ERP Cloud, Marketo and SharePoint alongside internal finance and quoting systems. The result: one governed, continuously refreshed home for the company's data that leadership can trust and every team can build on.
03 / HOW IT RUNS
Phased cutover, reconciled at every step
Discover
Inventory the estate, find what is genuinely used, and agree what gets migrated, rebuilt or retired.
Architect
Design the target lakehouse — layers, ingestion patterns, orchestration and governance — before moving anything.
Build
Migrate business area by business area. Old and new run in parallel until the numbers reconcile.
Launch & support
Cut over once reports match, decommission the old estate, and stay on while the platform beds in.
04 / QUESTIONS
Common questions
What does migrating from Azure Synapse to Databricks involve?
It means moving storage onto Delta Lake, rebuilding pipelines around Spark and Autoloader, re-pointing every downstream report, and re-establishing governance on the new platform — usually while the business keeps running on the old one. The technical move is rarely the hard part. Sequencing it so finance and sales reporting never goes dark is.
Can a Databricks migration happen without downtime?
Yes, when it is sequenced as a phased cutover rather than a single switch. We delivered a Synapse to Databricks migration for a global enterprise covering more than ten business areas fed by eight source systems, with no disruption to the finance and sales reporting running on top of it. Old and new run in parallel until outputs reconcile, and each business area moves only once its numbers match.
What is a medallion architecture?
A medallion architecture separates a data estate into three layers. Bronze holds raw landing data exactly as it arrived. Silver holds cleansed and conformed data with types and keys resolved. Gold holds business-ready tables such as general ledger, accounts receivable, accounts payable, orders and quotes. Keeping the layers distinct means a bad upstream change can be traced and reprocessed without rebuilding everything downstream of it.
Which source systems can be integrated into a Databricks lakehouse?
We have integrated Salesforce, Oracle ERP Cloud, Marketo, SharePoint and several internal finance and quoting systems into a single governed lakehouse, orchestrated through Azure Data Factory across both batch and streaming paths. Most enterprise sources can be brought in; the work is in reconciling how each system defines the same entity, not in the connectors.
How do you prove the migrated data is correct?
By running both platforms in parallel and reconciling outputs before anything is switched off. Each business area is signed off only when its reports produce the same numbers on Databricks as they did on the previous platform. That reconciliation step is what makes a migration safe to complete rather than a source of quiet, long-lived reporting errors.
05 / ALSO FROM M3STACK
Related work
06 / OPEN CHANNEL
Stuck on a data platform
that's outgrown you?
Tell us what you're running today and what's breaking. We'll reply within 24 hours.
team@m3stack.com