AI glossary · Advanced · Data engineering
Columnar storage
Columnar storage is a way of storing table data grouped by column rather than by row. It makes analytical work that reads only a few columns to compute sums or averages much faster and cheaper.
About this term
- Level
- Advanced · Specialist terms that need the inner workings
- Category
- Data, ops & platforms
- Fields
- Data engineering
- Added
- 2026-10-09
How it works
A typical database stores one customer's name, age, address and sign-up date together in one row. Columnar storage keeps names with names and ages with ages, each stored separately. So when computing the 'average age', it reads only the age column and leaves the rest untouched. Because similar values sit together in the same column, they also compress well, further reducing the amount read from disk.
Row-based storage vs. columnar storage
Row-based storageOne customer = one row
- Fast to add or update one record
- Reads every column even for a single average
- Databases for transaction processing
Columnar storageOne column = one group
- Reads only the columns needed
- Similar values together compress well
- Poor for writing one record at a time
Why it matters
Analytical queries usually read only a few columns but a very large number of rows. Columnar storage is built for exactly this pattern, so it became the default storage method for data warehouses and OLAP systems. Columnar file formats such as Parquet are used almost as a standard in data lakes, and are also often used to store and filter AI training data. Reading less data also lowers cloud costs that are charged by the amount scanned.
Things to know
It is at a disadvantage for inserting and updating records one at a time, because adding one row means touching every column group. Systems work around this weakness by briefly collecting new data and writing it all at once. It is common to split usage, keeping a service's real-time transaction records in a row-based database and analytics in a columnar store.
Example
When a data team counts daily visits from hundreds of millions of web log entries using only the 'date' and 'page' columns, it only needs to read those two columns if the logs are stored in a columnar file format like Parquet. This is also why LLM training datasets downloaded from public repositories are often provided as Parquet files.