Jupyter Blog - publishing

What’s New in MyST Markdown 1.8.2

2026-03-26T15:10:00+00:00

The Jupyter Book team just released mystmd 1.8.2 and a new version of the MyST react theme! Below are a few things that stand out.

Interactive widgets with the `{anywidget}` directive

You can now embed JavaScript widgets in your MyST pages using the new {anywidget} directive. Widgets follow the anywidget specification, which gives plugin authors an extension point for embedding JS capabilities in their documents.

A widget is a JavaScript module that exports a render function. You pass it initial state as JSON in the directive body, and it renders an interactive element on the page. You could use this for interactive figures, explorable explanations, or custom visualizations without leaving MyST.

A little confetti demo that we use to show off embedding widgets in a MyST document with anywidget!

Note*: Widget support is experimental. The interfaces may change as we learn more about usage patterns. Only load widgets from sources you trust, since they execute JavaScript in the reader’s browser!*

Table of contents improvements

The {toc} directive now supports a children context option, which lets you display only the child pages of the current page rather than the full project table of contents. Use it on landing pages and index pages that introduce a section and link to its sub-pages. You can see an example in the Jupyter Book getting started guide.

Better link handling across multi-site projects

If your project spans multiple domains (e.g., a docs site and a blog), you can now configure a domain to be treated as “internal” for link styling. Links to that domain will look like in-site navigation instead of external links. Check out the Getting Started guide for an example.

Upgrade

To get the latest version of mystmd:

npm install -g mystmd

or for pip users:

pip install -U mystmd

The theme updates automatically. Delete your _build folder and mystmd will download the latest theme on your next build.

Changelogs

Find an ongoing list of releases in the Jupyter Book ecosystem here:

jupyterbook.org/releases

You can also read the original mystmd release notes and myst-theme release notes.

The templating system of nbconvert 6

2020-09-26T07:31:00+00:00

One of the main changes in nbconvert 6 is the refactor of the template system, which should be easier to extend and build upon.

In this article, we dive into the template system, and provide a tutorial on how to build a custom template for nbconvert or Voilà.

From Classic to Lab

My notebooks look different!

If you are accustomed to convert notebook files to HTML by typing

jupyter nbconvert notebook.ipynb --to html

you may have noticed differences in the generated HTML when switching to the latest release of nbconvert. In fact, nbconvert now produces the same DOM structure as JupyterLab’s notebook implementation, which is styled with JupyterLab’s CSS.

One can even apply the dark theme:

jupyter nbconvert xleaflet.ipynb --to html --HTMLExporter.theme=dark

The HTML output of nbconvert with the lab template and the dark theme

While JupyterLab uses CodeMirror to render code cells, nbconvert makes use of the Pygments library to produce syntax-colored static HTML. To mimick the JupyterLab CodeMirror styling, we created a Pygments theme called jupyterlab-pygments. JupyterLab Pygments uses JupyterLab’s CSS variables for coloring and will therefore reflect the theme that is applied to the notebook.

Note: at the moment, only the default light and dark themes are supported, but we plan on adding support for third-party JupyterLab themes after the release of JupyterLab 3, which introduces a new packaging system for extensions.

But I wanted my notebooks to look the same!

Well, if you want to retain the classic notebook styling that was used in earlier versions of CSS, it is still possible using the classic template.

jupyter nbconvert xleaflet.ipynb --to html --template classic

The HTML output of nbconvert with the classic template

With this template, you retrieve the original style of nbconvert outputs and of the classic notebook.

Another perk of the new nbconvert release is the WebPDF exporter. The WebPDF exporter supports the same templates and themes as the HTML exporter, and produces a PDF output that renders the same rich content as the HTML exporter, such as rich HTML tables, widgets etc.

jupyter nbconvert xleaflet.ipynb --to webpdf --HTMLExporter.theme=dark

The WebPDF output of nbconvert with the lab template and the dark theme

Executing the notebook before rendering

Nbconvert’s two main categories of transformations are preprocessors and exporters. Preprocessors take a notebook as an input, and return a transformed notebook, while exporters return other types of content, such as HTML or PDF. An important preprocessor is the ExecutePreprocessor, which spawns a kernel for the notebook, execute all cells, and populate outputs.

It can be invoked before the export by passing --execute. For example, the xleaflet.ipynb notebook uses the xeus-cling C++ kernel and makes use of the xleaflet interactive widget, which can be displayed when converting to HTML or with the WebPDF exporter.

jupyter nbconvert xleaflet.ipynb --to html --execute

The HTML output of nbconvert with the execute preprocessor

Similarly, the WebPDF exporter will also display interactive widgets!

The PDF output of nbconvert with the execute preprocessor

Note: Future plans for the WebPDF exporter include offering more options to users with respect to page breaks, and providing bookmarks for the main sections of the document.

Nbconvert templates

Unlike with earlier versions of nbconvert, templates are now directories, which may contain a jinja template but also other assets, such as macros, CSS files etc. The nbconvert template system also provides an inherittance mechanism which makes it simple to tweak existing templates in a derived one, by overriding bits of it.

Selecting a template

Most exporters in nbconvert are subclasses of TemplateExporter, and make use of jinja to render notebooks into the destination format. Nbconvert templates can be selected by name with the --template command line option.

For example, the reveal template, shipped with nbconvert, turns Jupyter notebooks into HTML slideshows using the RevealJS library. Which cells should be skipped, or where breaks betweens slides should be, are specified in the notebook cell metadata. The classic notebook and JupyterLab both provide means to set the appropriate values.

To select the reveal template, simply type:

jupyter nbconvert <path-to-notebook> --to html --template reveal

In the case of the xleaflet.ipynb notebook showed earlier, we get:

Creating a reveal slideshow with the dark theme

This demonstrates that the nbconvert templating system can be used to completely override how we look at notebook documents. Using metadata, we can even create arbitrary layouts and rich views of the same content.

Where are nbconvert templates installed?

Nbconvert 6 templates are directories containing resources such as jinja templates and other assets. They are installed in the data directory of nbconvert, namely <installation prefix>/share/jupyter/nbconvert.

Running jupyter --paths shows all Jupyter directories and search paths. For example, on Linux, jupyter --paths returns:

$ jupyter --paths
config:
    /home/<username>/.jupyter
    /<sys-prefix>/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/<username>/.local/share/jupyter
    /<sys-prefix>/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/<username>/.local/share/jupyter/runtime

In our case, only the data section is relevant. Listing the content of <sys-prefix>/share/jupyter/nbconvert/templates in a raw installation of nbconvert will show

$ ls <sys-prefix>/share/jupyter/nbconvert/templates
asciidoc base classic compatibility html lab latex markdown python reveal rst script

The base template should not be used directly, but is typically inherited from. Thecompatibility directory provides some content for backward compatibility with earlier verions of nbconvert. Three templates are available for the HTML exporter: lab, classic, and reveal.

The content of nbconvert templates

The conf.json file

Nbconvert templates all include a conf.json file used to indicate the base template that it is inheriting from, the mimetype corresponding to that template (which determines which exporters are compatible with it, and which file is the entry point), and preprocessors to run when using that template before running the exporter. For example, inspecting the configuration of the reveal template we see that

it inherits from the lab template,
exports text/html, and therefore will only work with the HTML and WebPDF exporters.
and runs two preprocessors called 100-pygments and 500-reveal:

{
  "base_template": "lab",
  "mimetypes": {
    "text/html": true
  },
  "preprocessors": {
    "100-pygments": {
      "type":         "type": "nbconvert.preprocessors.CSSHTMLHeaderPreprocessor",
        "enabled": true
    },
    "500-reveal": {
      "type": "nbconvert.exporters.slides._RevealMetadataPreprocessor",
      "enabled": true
    }
  }
}

The CSSHTMLHeaderPreprocessor inlines the CSS required for the syntax highlighting of input cells.
The RevealMetadataPreprocessor massages the notebook metadata and consumes the information required to set up the layout of the slideshow.

Nbconvert walks up the inheritance structure determined by conf.json and produces an agregated configuration, merging the dictionaries of registered preprocessors. The ordering of the preprocessor names determines the order in which they will be run.

Jinja templates

Besides the conf.json file, nbconvert templates most typically include jinja templates files. They may also override files from the base templates, or provide extra content.

For example, inspecting the content of the classic template located in share/jupyter/nbconvert/templates/classic, we find the following content:

share/jupyter/nbconvert/templates/classic
├── static
│   └── styles.css
├── conf.json
├── index.html.j2
└── base.html.j2

we see that it includes the index.html.j2 jinja template (which is the main entry point for HTML exporters) as well as CSS and a base template file in base.html.j2. The only preprocessor listed in conf.json is the pygments syntax highlighting…

Inheritance in Jinja

In nbconvert, jinja templates can inherit from any other jinja template available in its current directory or base template directory by name. Jinja templates of other directories can be addressed by their path from the Jupyter data directory. Using the path is also useful when using a jinja template that may be overriden locally.

For example, in the reveal template, index.html.j2 extends base.html.j2 which is in the same directory, and base.html.j2 extends lab/base.html.j2. This approach allows using content that is available in other templates or may be overriden in the current template.

Building a custom template

Now, let’s create a custom template! If you work at ACME Corporation, you may want to create a template that follows the graphical charter of ACME Corp, and includes the logo in a banner.

Besides the logo, titles should also use the “ACME Regular” font, which has a cartoon-style look. The other parts of the template are inherited from the regular lab template.

Setting up a logo banner, and the font change, the acme nbconvert template produces the following result with a very simple notebook:

jupyter nbconvert acme.ipynb --to html --template acme

The HTML output of nbconvert with the acme template

Now you can create sophisticated templates making use of sophisticated front-end framework and processing the notebook metadata in creative ways!

The source for the ACME nbconvert template is available here.
Beyond the template files in share/jupyter/nbconvert/acme, the repo provides the logic for packaging this template into a PyPI wheel with data files. You will also find content related to the use of that template with Voilà, which is the subject of the next section.

From nbconvert templates to Voilà templates

Voilà turns Jupyter notebooks into standalone web applications and dashboards.

It differs from nbconvert in that the output web application is connected to a Jupyter kernel, allowing it to respond to user input through widget controls and other UI components, while with nbconvert, any action that requires a roundtrip to the kernel will not work.

However, Voilà leverages the nbconvert template system to benefit from their flexibility in overriding the front-end looks and behavior. From a user standpoint, the system is made so that the same templates will be usable for both systems.

However, template authors interested in advanced features of Voilà may be interested in the following information.

Where are Voilà templates installed?

Voilà templates are installed in the <installation prefix>/share/jupyter/voila directory (while nbconvert templates are in <installation prefix>/share/jupyter/nbconvert).
Just like nbconvert templates, Voilà templates are directories, they use the same conf.json configuration mechanism.
Voilà can use nbconvert HTML templates without modification.
When there exists an nbconvert and a Voilà template of the same name, the conf.json files are recursively merged, as well as the content of the directory, with a higher precedence for the Voilà template.

Overriding nbconvert templates with Voilà

When specifying the acme template that we developed earlier, with command

voila xleaflet.ipynb --template acme

Voilà will look pick up the acme nbconvert template. Since this template inherits from lab and Voilà has an overridden lab template (with e.g. the logic for rendering widgets), it will pick up the Voilà flavor of the lab template. Most typically, the Voilà flavor of a template does not add much on top of nbconvert besides boilerplate such as

a call to the macro that includes the JavaScript assets for the Voilà front-end logic.
calls to macro related to error logging when using the Voilà preview.
calls to macros related to the progressive rendering of notebooks as it is being executed, and the display of an “in-progress” spinner.

These macros are provided in the base Voilà template, and can also be overridden in derived templates.

Future developments

In the coming weeks and months, we plan on polishing the experience of nbconvert users.

JupyterLab 3, which should be released shortly, includes a more dynamic extension system which does not require the main JupyterLab application to be rebuilt. We plan on adding support for a category of lab extensions called “mime renderers“ which are used for rich rendering of data in cell outputs. This should enable the use of complex mime types such as GeoJSON or Vega visualizations in nbconvert and Voilà.
We are working on improving the reveal template, to include a custom reveal theme making use of JupyterLab CSS variables, so that it can be easily combined with JupyterLab themes.
The JupyterLab 3 extension system may also enable us to enable third-party JupyterLab themes in nbconvert.
The WebPDF exporter should expose options on output format and where page breaks should be. At the moment, we prevent page breaks until the maximum dimensions of PDF documents are reached.

Acknowledgements

Many people were involved in the nbconvert 6 release! The full list of contributors is available here.

Among them, we are especially indebted to Maarten Breddels, who was the main architect of the new template system.
We owe the split of the execute preprocessor and the new nbclient package to Matthew Seal and David Brochart. Matthew took on a large amount of maintenance work on the project over the past year.

The work on Voilà and nbconvert by the QuantStack team was funded by Bloomberg.

About the Author

Sylvain Corlay is the CEO of QuantStack, an open-source software development team specialized scientific computing comprising maintainers of major projects of the ecosystem.

As an open-source developer, Sylvain is mostly active in the Jupyter ecosystem, and the general PyData stack. He is currently a steering committee member for Project Jupyter, and a member of the board of directors of NumFOCUS.

Announcing the new Jupyter Book

2020-08-12T19:55:00+00:00

Note: this announcement is cross-posted between the Jupyter Blog and the Executable Book Project updates blog

Jupyter Book is an open source project for building beautiful, publication-quality books, websites, and documents from source material that contains computational content. With this post, we’re happy to announce that Jupyter Book has been re-written from the ground up, making it easier to install, faster to use, and able to create more complex publishing content in your books. It is now supported by the Executable Book Project, an open community that builds open source tools for interactive and executable documents in the Jupyter ecosystem and beyond.

The new Jupyter Book interface, highlighting how you can insert code outputs into your content

This post is a short overview of the new features in Jupyter Book, and gives some background on the future of the project.

You can also browse the new Jupyter Book documentation, or check out the Jupyter Book GitHub repository.

❗❗ Jupyter Book is still in beta, so things are always changing. We’d love your feedback, ideas, and PRs for how to make the project better!❗❗

What does the new Jupyter Book do?

The new version of Jupyter Book will feel very similar. However, it has a lot of new features due to the new Jupyter Book stack underneath (more on that later).

The new Jupyter Book has the following main features (with links to the relevant documentation for each):

✅ Write publication-quality content in markdown
You can write in either Jupyter markdown, or an extended flavor of markdown with publishing features. This includes support for rich syntax such as citations and cross-references, math and equations, and figures.

✅ Write content in Jupyter Notebooks
This allows you to include your code and outputs in your book. You can also write notebooks entirely in markdown to execute when you build your book.

✅ Execute and cache your book’s content
For .ipynb and markdown notebooks, execute code and insert the latest outputs into your book. In addition, cache and re-use outputs to be used later.

✅ Insert notebook outputs into your content
Generate outputs as you build your documentation, and insert them in-line with your content across pages.

✅ Add interactivity to your book
You can toggle cell visibility, include interactive outputs from Jupyter, and connect with online services like Binder.

✅ Generate a variety of outputs
This includes single- and multi-page websites, as well as PDF outputs.

✅ Build books with a simple command-line interface
You can quickly generate your books with one command, like so: jupyter-book build mybook/

These are just a few of the major changes that we’ve made. For a more complete idea of what you can do, check out the Jupyter Book documentation

Major changes

In the next few sections we’ll talk about some major changes that you may notice in Jupyter Book.

An enhanced flavor of markdown

The biggest enhancement to Jupyter Book is support for the MyST Markdown language. MyST stands for “Markedly Structured Text”, and is a flavor of markdown that implements all of the features of the Sphinx documentation engine, allowing you to write scientific publications in markdown. It draws inspiration from RMarkdown and the reStructuredText ecosystem of tools. Anything you can do in Sphinx, you can do with MyST as well.

MyST Markdown is a superset of Jupyter Markdown (AKA, CommonMark), meaning that any default markdown in a Jupyter Notebook is valid in Jupyter Book. If you’d like extra features in markdown such as citations, figures, references, etc, then you may include extra MyST Markdown syntax in your content.

For example, here’s how you can include a citation in the new Jupyter Book:

A sample citation. Here we see how you can include citation syntax in-line with your markdown, and then insert a bibliography later on in your page.

And here’s how you can include a figure:

A sample figure. Here we see how you can insert a figure with a caption into your book, and control some aspects of how the figure is displayed with options.

A smarter build system

While the old version of Jupyter Book used a combination of Python and Jekyll to build your book’s HTML, the new Jupyter Book uses Python all the way through. This means that building the HTML for your book is as simple as:

jupyter-book build mybookname/

In addition, the new build system leverages Jupyter Cache to execute notebook content only if the code is updated, and to insert the outputs from the cache at build time. This saves you time by avoiding the need to re-execute code that hasn’t been changed.

An example build process. Here the jupyter-book command-line interface is used to convert a collection of content into an HTML book.

More book output types

By leveraging Sphinx, Jupyter Book will be able to support more complex outputs than just an HTML website. For example, we are currently prototyping PDF Outputs, both via HTML as well as via LaTeX. This gives Jupyter Book more flexibility to generate the right book for your use case.

You can also run Jupyter Book on individual pages. This means that you can write single-page content (like a scientific article) entirely in Markdown.

A new stack

The biggest change under-the-hood is that Jupyter Book now uses the Sphinx documentation engine instead of Jekyll for building books. By leveraging the Sphinx ecosystem, Jupyter Book can more effectively build on top of community tools, and can contribute components back to the broader community.

Instead of being a single repository, the old Jupyter Book repository has now been separated into several modular tools. Each of these tools can be used on their own in your Sphinx documentation, and they can be coordinated together via Jupyter Book:

The MyST markdown parser for Sphinx allows you to write fully-featured Sphinx documentation in Markdown.
MyST-NB is an .ipynb parser for Sphinx that allows you to use MyST Markdown in your notebooks. It also provides tools for execution, cacheing, and variable insertion of Jupyter Notebooks in Sphinx.
The Sphinx Book Theme is a beautiful book-like theme for Sphinx, build on top of the PyData Sphinx Theme.
Jupyter Cache allows you to execute a collection of notebooks and store their outputs in a hashed database. This lets you cache your notebook’s output without including it in the .ipynb file itself.
Sphinx-Thebe converts your “static” HTML page into an interactive page with code cells that are run remotely by a Binder kernel.
Finally, Jupyter Book also supports a growing collection of Sphinx extensions, such as sphinx-copybutton, sphinx-togglebutton, sphinx-comments, and sphinx-panels.

We’ll write a more developer-focused post in the future to describe each of these components in more detail.

What next?

Jupyter Book and its related projects will continue to be developed as a part of the Executable Book Project, a community that builds open source tools for high-quality scientific publications from computational content in the Jupyter ecosystem and beyond.

If you’d like to learn more about Jupyter Book or get started building your own books, then check out the new Jupyter Book documentation. Jupyter Book is still in beta and is constantly being improved, so your feedback and contributions are always welcome.

If there are particular features you’d like to see, open an issue or give a 👍 to a pre-existing issue, and it will be bumped up on the feature request leaderboard.

Get involved

If you’d like to contribute to any of the projects listed in this post, you are highly encouraged to do so! The Executable Book Project (and thus Jupyter Book) is run as an open project that welcomes contributions from others.

We are excited about all the new features and improvements in the new Jupyter Book, and look forward to seeing the new books that the community creates with this stack!

Rendering Notebooks on GitHub

2015-05-07T19:47:00+00:00

We are pleased to announce that, starting today, and as announced on the GitHub blog, Jupyter/IPython notebook (.ipynb) files will render directly on GitHub. This feature works for notebooks in any of the supported Jupyter programming languages on both public and private repos. This capability will complement nbviewer and make it easier for GitHub users to create, view and share notebooks on GitHub.

Here are some great examples of notebook based content that can now be viewed directly on GitHub:

Cameron Davidson-Pilon’s Probabilistic Programming and Bayesian Methods for Hackers.
Lorena Barba’s Aerodynamics-Hydrodynamics with Python.
Benjamin Laken’s Monsoon Analysis.

We want to thank the GitHub team for bringing this new feature to GitHub. In particular @tclem, @arfon, @sshirokov, and @skalnik have all done an incredible job building this at GitHub. The first discussion of how to bring notebook rendering to GitHub happened almost 18 months ago, and a lot of work made this feature possible today. With over 200,000 notebooks available on GitHub, you already have a lot to read!

We are excited to see how this feature will impact science education and research on GitHub, as well as encourage the adoption of the Jupyter notebook as a standard file format across a wide range of fields. Jupyter notebook files are a JSON-based open document format that supports code and results, narrative text, images, and equations in one file. These documents simplify the creation and communication of computational narratives and make the sharing and replication of scientific results and data analyses simpler.

nbviewer

One key piece of infrastructure in sharing notebooks has been the nbviewer service, which started as a side project of mine in the late summer of 2012. Nbviewer has grown through deep integration with GitHub, with the ability to browse all public users, organizations, repositories, branches, tags, and even Gists. This would not have been possible without @rgbkrk and @bollwyvl, who handle the development and deployment of nbviewer. It would also not have been possible without Rackspace and Fastly, who provide free hosting and fast static asset distribution. Nbviewer has steadily grown to render hundreds of thousands of notebooks every week, and usage is still increasing. Based on this, we believe that rendering on GitHub will be a highly used and appreciated feature.

Although you can now view notebooks directly on GitHub, we are not planning to stop the development of nbviewer. While rendering directly in GitHub repositories is convenient and allows viewing notebooks in private repositories, there are number of reasons that nbviewer will continue to remain an important part of project Jupyter.

First, there are many notebooks not hosted on GitHub. As before, these notebooks can still be viewed on nbviewer. Furthermore, the Jupyter/IPython team are actively working on improving integration with other content providers, building on our experience with GitHub. For example, a recent pull request of @bollwyvl streamlines the process of adding new providers to nbviewer, with proposals for DropBox, GitLab, Google Drive, Stash, and other hosting and collaboration platforms.

Second, because of security concerns, some features available on nbviewer will not be available on GitHub. For example, GitHub will not render any dynamic output display that uses JavaScript, custom CSS, and most custom HTML embedded in Markdown or in outputs. When this is the case, and GitHub cannot fully render your notebooks, you will see an icon that allows you to view the full notebook on nbviewer.

Because nbviewer does not handle authentication, it will continue to support custom CSS, HTML and JavaScript in notebook. Thus, for highly customized or dynamic notebooks, nbviewer will still be the ideal place to share them.

We would love your feedback! If you have comments or find bugs with notebook rendering on GitHub or nbviewer, please open an issue on our nbviewer repo. Also, please send a few tweets to thank all the people without whom this would have not happened.

The Jupyter/IPython Team