Exporting tables from a SQL Server backup (.bak) to Parquet without restoring it

rferraton

Author : Romain Ferraton To get one table out of a SQL Server full backup, you normally restore the whole database first: disk space for all of it, an instance, then a SELECT INTO. FastBCP (our bulk export tool) now treats the .bak as a plain binary file. It reads it once to index it, locates the table’s pages, decodes them into Arrow columns and writes Parquet. No SQL Server instance, no credentials, nothing touches production. Threads split the table’s pages and share one open backup. On a 320 GB striped, compressed production backup (383 tables), the cold open took 2m29s (~2.2 GB/s from disk). After that, a 175M-row table exported in 29.6 s, and later opens took 0.3 s. Supported: SQL Server 2017–2025, full and copy-only backups, stripes, MS_XPRESS/ZSTD compression, rowstore and columnstore. Not supported yet: differential/log backups, TDE or encrypted backups. The only output is Parquet. Happy to answer questions about the backup format internals.

hbarka

I love me my Microsoft SQL Server. Where was this thirty years ago. Finding a table from a bak file in a matter of minutes without restore is miraculous.

mrtimo

Parquet files are a game changer. I don't know how people are living without them, especially pairing them with DuckDB for basic analysis that was previously happening with pandas etc.