Instantly view Parquet files in JupyterLab with Arbalister

2026-01-29T16:18:00+00:00

A colleague sends you a SQLite file, a Parquet dataset, or an Avro snapshot.
You need to explore the content, but you’re not sure what’s inside or even how to open it. You shouldn’t have to write code, craft SQL queries, or recall the syntax for read_parquetin a Python library. Instead, you should be able to click the file once and instantly see the tables and metadata.

Despite being the de-facto standard for data science and exploration, JupyterLab lacks this basic capability out of the box. Of course, there are extremely well designed libraries to read, process, and transform tabular data — but opening and viewing a file with a table should be simpler. A workflow with libraries like Pandas or Polars require you to install relevant packages, create a notebook, and iterate on queries to get a full picture of the data.

import pandas as pd

pd.set_option("display.max_rows", 50)
pd.set_option("display.max_columns", 20)
pd.set_option("display.width", None)

df = pd.read_parquet("data/2025/PERFORMANCE_SNAPSHOT_2025_01_05.parquet")
df

Whether for a newcomer to data science, or someone that needs to investigate many different files with hard to remember schemas, this is a bit cumbersome. Comma separated value files (CSV) have better support: double click on the file to see it open (see figure below). That is a great way to understand the business logic and answer questions such as What type of information is in this file? What do values typically look like? Does it seem to contain all the data I need? What is the difference between customer_id and user_id?

The JupyterLab CSV viewer

Arbalister, our new JupyterLab extension, changes this. With Arbalister, you can double-click to instantly view a wide range of tabular data files: Parquet, CSV, Avro, ORC, SQLite, and more…

Arbalister viewer can open Parquet files and more

The extension will load data as it is displayed, making it possible to view it almost instantaneously even if the file does not fit in memory. A toolbar lets the user select some reading options, such as the delimiter for a CSV file, or selecting the table to display in the SQLite file!

Arrow over the wire

Fueling it all is the Apache Arrow ecosystem, an in-memory format for tabular data used in most data science libraries. Plenty of files types can already be read into an Arrow data structure so we exploit it heavily in Arbalister (with Apache Datafusion in our case).

The table displayed in the user’s browser reuses JupyterLab Lumino DataGrid with a custom data model. In the latter, we divide the whole table in chunks across both the rows and columns axes (e.g. 512 rows and 24 columns). When we need to display data from a chunk that is not already available, we make an HTTP request to a server-side extension that open the file, read the relevant portion into an Arrow table and return it to the client. The table is returned as Arrow IPC (Inter Process Communication, a binary format close to the original Arrow memory) in the response body. In the client, a lightweight Arrow implementation can read it back efficiently.

Arbalister client-server architecture

When nothing else is happening, the implementation will also pre-fetch some of the next chunks in the background so that scrolling past a chunk boundary is imperceptible.

Because it all works in Arrow, which is standard for tabular data, adding support for new types of files that can be read into the Arrow format is extremely easy. Vortex, Lance, or whichever other format becomes relevant. But that is not all…

The existing SQLite support is made possible through ADBC, an SQL connector that reads databases into Arrow format. SQLite databases fit into single files, which fit the abstraction used in Arbalister, but ADBC could also be used to add general database exploration (Postgres etc.) into Arbalister. With Datafusion, we could also add visualization for data lakehouses: partitioned datasets (e.g. DeltaLake or Apache Iceberg) over remote object storage (S3). With Intelligent predicate and projections pushdowns, we could visualize remote tera-byte table instantly by only downloading the relevant parts.

Looking further

There are multiple things that would make great improvements to Arbalister. We have already mentioned:

Optional support for newer file types: Vortex, Lance;
Object storage and data lakehouses support;
SQL databases support.

Another direction is support for JupyterLite, a WebAssembly distribution of JupyterLab running entirely in the browser. This setting is great for education because it does not require more than a static file server to host.
Being able to work directly with Parquet files in the browser is great for making dashboard and interactive demonstrations. Arrow and Datafusion already have some experimental support for WebAssembly so this is not so far-fetched.

Finally Arbalister could use a few features to investigate deeper questions about the data. For instance, beyond the What does the data look like? question, some column filters could help answer questions such as What does the data look like in November 2024 for John Doe? Perhaps even an SQL console for more advanced queries.

An extension point

The server extension being only a way to serve Arrow data, it can be reused by other libraries to use it in other ways. For instance, if we knew that our table contains time-series data, a second viewer could display it as a graph.
Similarly for geospatial data.

If you are interested to use or contribute to Arbalister, head to our GitHub!

About the Author

Antoine Prouvost is a senior scientific software engineer at QuantStack. His work on Arbalister was funded by Bloomberg.

Jupyter Blog - Antoine Prouvost

Instantly view Parquet files in JupyterLab with Arbalister

Arrow over the wire

Looking further

An extension point

About the Author