Dashboarding with JupyterLab 3

2021-01-08T10:25:00+00:00

Project Jupyter offers a complete suite of open-source tools for the scientific computing community, reaching from the exploratory phase of a project to the presentation of the results. In this last but not least phase is where Voilà takes place, turning a technical document such as Notebook into an interactive presentation or a stand-alone web app ready to be shared with non-technical readers. You can learn more about Voilà in this blog post.

Voilà is an easy-to-use command-line tool that shines in its simplicity and flexibility. All its versatility comes from a custom template system built upon Jinja templates that allows everyone to create a custom UI to share Jupyter Notebooks. Examples of the versatility provided for the template system are voila-reveal and voila-gridstack.

Voila-gridstack is a Voilà template started by Bartosz Telenczuk to turn notebooks into dashboards following the specification introduced by the legacy jupyter-dashboards project. The idea behind is to be able to change the layout of the cells to re-configure your dashboards using drag-and-drop. Once you have your desired layout, its configuration stays in the metadata of the notebook. This makes it simple to carry around or share the notebook and its layout configuration.

With the release of JupyterLab 3.0, we wanted to update the voila-gridstack template by adding an editor for JupyterLab.

Configure your layout

The first way to open the voila-gridstack editor is by right-clicking on the notebook and selecting open with “Voilà Gridstack”. The editor is itself a JupyterLab Document Widget, which can be seen as another view of the same notebook document. The editor can also be launched from the classic notebook with a toolbar button.

Opening JupyterLab-Gridstack

Once the editor is open, every cell is decorated with a toolbar as a drag-and-drop handle and a button to remove the cell from the dashboard. When removing a cell, it is just marked as hidden in the metadata but it is not deleted from the notebook file. Besides, handles are provided for resizing cells in the layout.

Configuring your layout

When opening the editor for the first time in a new notebook, the metadata of the notebook is modified to follow jupyter-dashboards’ specification. By default, you will see an empty editor. To add new cells just drag them from the notebook view. It is not possible to add cells from another notebook.

Adding new cells

Finally, you can save the layout into the metadata of the notebook by clicking the “save” button or with the ctrl+s shortcut and then it’s time to hit the render button and Voilà! You will see your layout as a stand-alone web app.

Voilà! Your stand-alone web app

Try it online!

You can try the jupyterlab-gridstack extension online without installing anything by just clicking on the following image:

Installation

Voila-gridstack is available as a conda package, and you can easily install it by typing the following command in a conda environment:

mamba install -c conda-forge jupyterlab voila-gridstack

Acknowledgements

The development of Voilà and related packages at QuantStack is sponsored by Bloomberg.

The example notebook used as an example in this post was one of the example notebooks of the legacy Jupyter dashboards project.

About the authors

Carlos Herrero is a Computer Engineer passionate about AI and its applications to robotics. Currently working at QuantStack helping to develop Open Source projects.

Carlos Herrero

Jeremy Tuloup is a Scientific Software Developer at QuantStack, working on JupyterLab, Voilà, and various other projects within the Jupyter ecosystem.

Jeremy Tuloup

JupyterLab, the Cloud Robotics Command Station

2020-08-11T13:19:00+00:00

JupyterLab-ROS with Voilà to quickly make a standalone web app

The ROS open-source project (Robot Operating System) has become a de-facto industry standard in the robotics community. Very much like with Project Jupyter, a large collection of software has been built upon well-specified communication protocols. ROS has become a rich ecosystem of standardized tools for building and distributing ROS-based software, exploratory robotics development, and much more.

With Industry 4.0, the number of robots and smart devices has been increasing exponentially. We are talking about fully automated factories with hundreds of robots and thousands of connected sensors, generating large amounts of raw data.

There is an opportunity in bridging the ROS and the open-source data science ecosystems and tools.

For this reason, we set ourselves to bring together these ecosystems,

by producing a collection of JupyterLab extensions for ROS, turning JupyterLab into a cloud robotics command station,
by integrating ROS packages with conda / mamba package management system, to enable their installation alongside other open-source data-science packages.

JupyterLab, the cloud robotics command station

While the traditional developer tools from the ROS stack such as RViz are desktop-based, we think that a robotics development environment running in a web browser would provide a lot of flexibility. Also, the ideal robotics developer platform should be plugin-based, and easily extensible to account for the diversity of use cases to enable, from simple interactive scripting to visualization or large datasets. Finally, we should build this environment upon broadly adopted foundations, with open governance, and not let a single entity hold the keys to the ROS ecosystem.

Jupyter is a natural candidate to serve as the foundation for a web-based robotics development environment.

JupyterLab is a web-based interactive development environment that has many of the much-needed features

In JupyterLab, everything is a plugin, including the core components to the application. Anyone can produce a “remix” of core and third-party JupyterLab extensions to their needs.
Very much in the spirit of ROS, one of the keys to Jupyter’s success is that the project was built upon well-documented and specified protocols and file formats that anyone could implement.
Jupyter is a multi-stakeholder project, not backed by a single corporation, but by a community of developers at a variety of companies, universities, as well as individual contributors.

Today, we are announcing the first beta of JupyterLab-ROS, a collection of JupyterLab plugins to integrate ROS with the JupyterLab platform. We combine the best of the data-science and robotics worlds to help developers build custom solutions for the industry. ROS can run in a high-performance server while JupyterLab is accessed in a web-browser (on Windows, OS X or Linux).

JupyterLab-ROS

JupyterLab-ROS automatically starts a rosmaster and rosbridge_server making all the painful setup of WebSockets connections completely transparent to users. A status bar widget allows you to start and stop the rosmaster server with a simple click and displays its state in real-time. Internally this widget runs a launch file which can be changed from the JupyterLab settings UI to launch additional nodes.

Master status widget

Another important component of the ROS stack is RViz, a desktop-based 3-D visualization tool. It’s web-based counterpart in JupyterLab is Zethus, started by Rapyuta Robotics. This plugin supports most of the display types supported by RViz. It also provides an info panel that displays the raw messages in realtime, and a web-based version of rqt_graph for visualizing the ROS node graph.

Zethus

A robotics station would be incomplete without a tool for debugging. For that purpose, ROS provides rosconsole, a package which allows developers to send messages to rosout and make them available on every node. The console gives access to these debugging messages in JupyterLab and provides additional features such as filtering by level and by node.

Log Console

Robotics Notebooks

A key component to the Jupyter stack is the Notebook, an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. It’s frequently used to do data cleaning and transformation, numerical simulation, statistical modelling, data visualization, machine learning etc.

The notebook could be the perfect tool for robotics researchers and educators to make (and share) experiments in control algorithms, working in a dynamic and interactive development environment that allows quick prototyping and exploratory analysis while having access to the sophisticated mathematical libraries already well integrated with Jupyter. It can be tedious to have a different platform for interacting with robots and processing the data obtained from these, JupyterLab-ROS combines both toolsets and enables an integrated workflow.

At the same time, many projects are becoming more and more interdisciplinary. Currently, it is common to see projects mixing topics like machine learning and robotics. JupyterLab-ROS opens a new world of possibilities for machine learning researchers that are interested in applying their models to robots: now they have the possibility of connecting a robot to a development environment that they are very familiar with.

JupyROS is another powerful library that leverages ipywidgets and bqplot to integrate ROS in Jupyter Notebooks, being able to show real time plots from ROS messages and create intuitive forms to control robots in minutes.

Interactive Robotics in the Jupyter Notebook, with live plotting

ROS Package management

Modern scientific computing applications typically depend on a large number of libraries. Conda-forge is a community initiative to package scientific libraries for the conda package manager, and has become a de-facto standard source of packages for the scientific computing community, with tens of thousands of available packages, and thousands of contributors.

During the last year, we have worked hard to package more and more of ROS as conda packages. Conda-packages for ROS Melodic, including RViz are now available on all three operating systems! You can find the packages on the RoboStack channel (a more detailed blog post about this is going to follow soon).

Conda can be used to create virtual environments for ROS projects, which makes it possible to install Melodic, Kinect and Dashing side-by-side on the same Linux system, and have exact control over package versions.

We are also working on Mamba, a conda-compatible package manager implemented in C++. Together with Boa this effort will help to continuously build and release ROS packages on conda.

Try it online!

You can try the JupyterLab-ROS extension online without installing anything by just clicking on the following binder link:

JupyterLab-ROS is under active development. We are working on new features such as a rosbag widget to play and record bag files and improving existing ones like the roslaunch widget. If you are interested in contributing to JupyterLab-ROS you can find the code at this repository in the RoboStack organization. Contributions are always welcome!

Installation

You can install JupyterLab-ROS using Mamba, the first step to create a new environment with JupyterLab and the ROS packages necessary, then you can install the extension.

mamba create -n test -c conda-forge -c robostack python=3.6 nodejs=12 jupyterlab ros-melodic-ros-core ros-melodic-rosauth ros-melodic-rospy ros-melodic-rosbridge-suite ros-melodic-rosbag ros-melodic-tf2-web-republisher

conda activate test

pip install jupyter-ros-server

If you have any questions don’t hesitate to contact us on the RoboStack or QuantStack chats.

This work has many collaborators around the world, and we are especially grateful to Deep Chaitanya & Ashish Chaudhary from Rapyuta Robotics, Sean Yen from Microsoft for his amazing work on vinca that automagically generates the conda-recipes, Tobias Fischer for many patches and pushing us to bring OS X support to the conda packages as well. Many thanks to Jeremy Tuloup for his help with the JupyterLab extension system.

About the authors

Wolf Vollprecht works as a scientific and robotics software developer for QuantStack in Paris and Berlin.

Carlos Herrero is a Computer Engineer passionate about AI and his applications on robotics. Currently working at QuantStack helping to develop Open Source projects.

Jupyter Blog - Carlos Herrero

Dashboarding with JupyterLab 3

Configure your layout

Try it online!

Installation

Acknowledgements

About the authors

JupyterLab, the Cloud Robotics Command Station

JupyterLab, the cloud robotics command station

Robotics Notebooks

ROS Package management

Try it online!

Installation

About the authors