Jupyter Kernel Gateway
Last Friday, the Jupyter Steering Council voted to make the incubating Kernel Gateway project a top-level Jupyter project. I want to thank everyone for their contributions so far — code from my teammates at IBM and beyond; advice from the Jupyter development team; and questions, issues, and requirements from end users.
What is it?
The kernel gateway is a web server that clients can use to spawn and communicate with Jupyter kernels. Currently, it supports two such mechanisms:
- A 100% Jupyter Notebook server compatible API for:
- Creating, listing, and deleting Jupyter kernels via HTTP
- Communicating with those kernels using the Jupyter kernel protocol via Websockets
- A notebook-defined API that maps HTTP verbs (e.g., GET, POST) and resources (e.g.,
/foo/:bar) to code that executes on a kernel
What problem does it solve?
People are creating novel web applications that use Jupyter kernels as interactive code execution engines. For example:
- pyxie uses kernels to evaluate code from a text area on a web page
- Thebe uses kernels to evaluate code from multiple input areas on a web page
- gist exec uses kernels to evaluate code from GitHub gists rendered as web pages
- Atom Notebook uses kernels to enable a notebook experience in the Atom text editor
- Eclairjs uses the Apache Toree kernel as a means of making Apache Spark API calls from JavaScript
- Jupyter dashboard server uses kernels to evaluate code from notebooks rendered as interactive dashboards
These applications have two important properties in common:
- They spawn kernels using provisioning APIs that run separate, and often remote, from the user-facing applications themselves.
- They communicate with kernels using Websockets rather than directly with ZeroMQ.
Developers can deploy instances of the full Jupyter Notebook server to satisfy these requirements. However, this approach is less than ideal:
- The notebook UI is exposed, but only the programmatic API is used.
- The notebook authentication mechanism is form- and cookie-based and meant for humans, not programmatic clients.
- The notebook transport mechanisms and APIs serve the notebook user experience, and are not meant to be replaced or extended to support other clients.
The kernel gateway addresses these issues, and others, in a Python package that can be used across new Jupyter-enabled applications.
What’s next?
In the short term, we plan to bring the documentation, test suite, and code into line with other Jupyter projects. Over time, we hope to separate the different APIs layers into more maintainable packages, and gather requirements from users developing new kernel-powered applications.
For More Information
- Visit the jupyter/kernel_gateway GitHub project for the source and issue tracker.
- Check out the posts about the kernel gateway on the IBM Emerging Technology blog. Then visit the jupyter/kernel_gateway_demos project to try some of the examples yourself.
- Reach-out on the Jupyter Google Group or in the jupyter/jupyter gitter channel.
- Chat with me at the Berkeley Data Science Meetup on March 29th, 2016.
