Jupyter Notebook format workshop outcomes
The Jupyter Community Workshop on the notebook file format took place at the Safran Campus near Paris from February 28th to March 2nd. It was a great opportunity to gather various Jupyter stakeholders from private and public affiliations to bootstrap new features for the Notebook file format.
Jupyter Community Workshops are a series of events designed to bring together small groups of Jupyter community members and core contributors for high-impact strategic work and community engagement on focused topics.
Community discussions
The community has lots of ideas to improve the Notebook format. So we split in three smaller groups with the aim of drafting Jupyter Enhancement Proposal (JEP): the markdown group, the text-format group and the cell types group.
The markdown group focused on backward compatible enhancement for the Markdown cells. The discussions focused on two subjects:
- The specification of the Markdown flavor (pre-proposal): the goals are to specify which Markdown flavor (e.g. GitHub, CommonMark, MyST,…) is used for the cell source, how to store rendered output for easier cross-compatibility and what is the default Markdown flavor.
- The persistence of user expression (pre-proposal): in order to display inline expressions within Markdown cells, the results obtained from the kernel should be stored in the notebook. This proposal aims to define the schema modification to store such information.
The text-format group lays out a specification for an official Jupyter notebook textual format. The discussion went on after the meeting to prepare that proposal.
Finally the cell-types group took the hypothesis of starting from the blank page to create the best Jupyter notebook format building on top of 10-years of experience. The discussion will take time to settle down on a new specification. So if you are interested, join the weekly discussion (see the meeting notes for all the details). In addition to the fully new specification, two backward compatible enhancements have been proposed:
- Adding $schema to the notebook format and deprecate the nbformat version keys (see proposal).
- Adding extraSchema to the notebook format to optionally extend the schema to specify in particular metadata (see pre-proposal).
Follow-up
Six JEP’s are foreseen from the workshop discussions. But as mentioned earlier, the community has lots of great other ideas (like SQL cells, low-/no-code cells for inputs or visualization). So we would like to encourage anyone interested by any Jupyter enhancement to open issue on the Jupyter Enhancement Proposals repository (see the guidelines for more information).
With the new Jupyter governance in place, the new Software Steering Council is responsible for ensuring those proposals get reviewed and go through the approval process.
Acknowledgements
I deeply want to thank all participants to the workshop that took the time (some of them despite time zone difference) to bring very constructive and thoughtful discussion.
We are really grateful to Bloomberg and Amazon Web Services for their donations to the Jupyter Community Workshops program. This event would not have been possible without their generous support.
We are also grateful to Safran Group for hosting this workshop and the NumFOCUS foundation for helping and mentoring this workshop organization.
