ROS @ Jupyter

2019-04-07T21:15:00+00:00

This may be the right time for Jupyter-based developer tools, as cloud robotics is taking off. A PR2 robot in the browser, and making him dance through the traditional Qt interface

Historically, the ROS (Robot Operating System) community has relied on Qt for building complex user interfaces. Nowadays, the Jupyter notebook and the ipywidgets framework offer a compelling alternative for several reasons:

Code and interface are not separated — both are in the same notebook.
Complex widgets using browser technology are possible: from JavaScript sliders to 3D with WebGL, real time video streaming with WebRTC…
Works with any web browser — not bound to Linux, and no Qt applications need to be compiled.
Doesn’t need to run locally! Applications can run on a server far away, without any manual setup or installation procedure.

However, ROS does not play nicely with Jupyter. For example, rospy is inherently multi-threaded (every new ROS topic subscriber spawns a new thread). Debug printouts in callbacks result in Jupyter printing that content into the output area of the active notebook cell, quickly flooding the user interface — and there is no easy way to stop logging threads once started.

ipywidgets to the rescue: jupyter-ros

That is why jupyter-ros was created. It is a suite of plugins to the Jupyter ecosystem to make working with ROS inside Jupyter a breeze.

Subscribing with jupyter-ros produces an interactive widget

First, when you subscribe using jupyter-ros, the function returns an ipywidget with a start/stop button and a dedicated output area for debug prints. Internally this re-routes all print outs from your subscriber thread to this Jupyter cell, and gives full control over the thread (by being able to stop and restart it at any time).

The jupyros publishing sugar

If you call the publish function in jupyter-ros, a widget is automatically generated from the message specification. For example, a ROS message String field automatically becomes a text input widget, a Float32 becomes a FloatSlider…

Live plotting IMU data from a BBC micro:bit

Another interesting functionality is the live plotting, similar to rqt_plot. For this we use bqplot, a “jupyter-native” solution to plotting. With jupyter-ros, you can select a couple of fields from your message, and they will be automatically plotted to a bqplot.

Going 3-D

Most robots are three dimensional, and so should be the visualizations! The go-to tool in the ROS ecosystem at the moment is RViz, a very powerful 3-D visualization tool.

Some of RViz’s functionality has already been ported over to the web browser as part of the RobotWebTools effort. We’ve been able to piggy-back on their impressive work, and we’re releasing the first iteration of ROS3D Jupyter widgets. Currently, they allow you to programatically plug together RViz like visualizations for different data types, such as laser scans, robot trajectories, and 3D (URDF) models of the robot!

Thanks to the ipywidgetification, you can now bring complex visualizations to the web, without writing any JavaScript, and arrange those visualizations freely in JupyterLab.

Interactive widgets showing ROS data inside JupyterLab

Jupyter, Widgets, and Cloud Robotics

We hope that Jupyter, and the jupyter-ros widgets will play a big role in cloud robotics. In cloud robotics, some of the software powering one or multiple robots runs on powerful computers in data centers. For monitoring purposes, or development, Jupyter and JupyterLab are perfect candidates. Robot customers will be able to login to a single user-friendly interface, without having to install any custom software on their machine, or run a specialized operating system (ROS usually runs on Ubuntu). Exciting possibilities arise: it is already possible to run a Docker container running JupyterLab and jupyter-ros on the leading cloud robotics platforms, Rapyuta Robotics and their freshly launched platform rapyuta.io.

Installation & source

The jupyter-ros widgets can be installed from PyPI using

$ pip install jupyros
$ jupyter nbextension enable --py --sys-prefix jupyros

The examples and the source code can be found on github: https://github.com/robostack/jupyter-ros

This work is still early stages and you might find some rough edges. We look forward to collaborating with the community to polish these widgets to the highest standards!

About QuantStack

QuantStack is located in the center of Europe (Paris). We are core Jupyter developers, and love ROS, and the cloud. If you are interested in working with us on professional user interfaces for developers or clients in the cloud, do not hesitate to send us an email: wolf.vollprecht@quantstack.net

A Diagram Editor for JupyterLab

2018-02-26T11:33:00+00:00

JupyterLab viewing LaTeX source code and a PDF document

The new JupyterLab interface is much more than a replacement for the classic notebook. It aims to bring together all the pieces required for a complete scientific workflow. The extension-based architecture of JupyterLab comes with a number of components already enabled:

a Jupyter notebook,
a text editor,
a file browser in the sidebar,
a number of editors and viewers for various file formats,

and much more. However, some pieces are still missing to complete the picture for a scientific authoring environment. One would be a featureful LaTeX editor. The first LaTeX editor for JupyterLab is a step in the right direction and offers an easy way to live-compile tex documents. Another piece is — of course — a means to produce diagrams, flow charts and draw figures!

Drawing charts and diagrams

On the occasion of the Paris Jupyter Widgets workshop, I started working on a feature to fill that gap and built a JupyterLab extension for the Draw.io diagram editor.

Draw.io is a diagram editor that runs in the web browser and is Apache 2.0 licensed. It’s got a really mature code base, which has been around for many years. However, unlike the other components used by JupyterLab, Draw.io has not yet embraced the new JavaScript packaging tooling such as NPM, which complicated the integration with JupyterLab a little bit, but it all paid off eventually!

Now, I am really pleased to announce the first release of the draw.io extension, a fully fledged integration for JupyterLab of the fully-fledged diagram editor!

The Draw.io JupyterLab extension takes advantages of the JupyterLab architecture: i.e. registering a new mime type (.dio) with the file explorer to open files, and adding a launcher button and menu items. Besides that, multiple synchronized views of the same diagrams can be displayed at the same time, allowing a user to visualize the same content with different zoom levels, or with a bare text editor.

Installation

You can install the jupyterlab-drawio extension with the following command:

jupyter labextension install jupyterlab-drawio

This should set up the extension inside your JupyterLab environment. I hope this will be a useful extension for the larger community. All the code is available on GitHub: https://github.com/QuantStack/jupyterlab-drawio. Don’t hesitate to open issues and come contribute to jupyterlab-drawio.

The future

There are other projects just waiting to be packaged for use inside of JupyterLab: one great webapp for JupyterLab would probably be the ShareLaTeX application, which is Open Source as well and provides a very nicely integrated editing experience for LaTeX documents, with autocomplete of LaTeX commands and reference search. Eventually, we might be able to integrate with the official ShareLaTeX server for a collaborative, hosted, editing experience for LaTeX documents from inside JupyterLab.

Maybe we as a community can come together and start building integrations for these amazing free tools into JupyterLab!

To conclude, thanks to all who’ve organized and participated in the workshop (especially Sylvain for the organization). I’ve used the opportunity to chat with the core developers and get their helpful input: Steven, Afshin, and Jason, thanks for helping me out in getting this off the ground and making JupyterLab! And honestly, the biggest shoutout has to go to the people who’ve worked on improving draw.io and thankfully open sourced this amazing code base: the entire draw.io team.

About the Author

Wolf Vollprecht is a scientific software developer at QuantStack, passionate about High-Performance Computing and Robotics. He is one of the core developers of xtensor.

Jupyter Blog - Wolf Vollprecht