Jupyter Blog - Matthias Bussonnier

Release of IPython 5.4, 6.1, and rlipython.

2017-05-31T22:47:00+00:00

Slightly over a month ago we released IPython 6.0 : the first version of IPython to be compatible only with Python 3. The reception of IPython 6.0 was great, and the codebase way more maintainable. So while 6.0 was mostly focused on internal refactoring and working upstream to make the release of a Python 3 only version possible, the next releases can focus on bug fixes and new features. Today we not only have the new release of IPython 6.1, but IPython 5.4 as well (the LTS branch targeting Python 2.7). Read on for more information! As usual, you can upgrade using pip and the 🔥 Conda Forge 🔥 builder is warming up: IPython 5.4, and 6.1 should be available via these channels soon.

IPython 5.4 (and 6.1) major new features

IPython 5.x is the LTS branch which is still compatible with Python 2. The IPython team ♥️ Python 2 users so we continue to make fixes to our 5.x branch. You can install it today by making sure you have pip 9 or above:

$ pip --version
pip 9.0.1

Then upgrade IPython:

$ pip install ipython --upgrade

If you do not have pip 9 and are using Python 2, pip may download IPython 6, which will not work on your system. If you are on Python 3, see the next section to upgrade to IPython 6.1 but read on in this section for the new features.

IPython 5.4 got a number of new features that went into 6.0, plus a number of API improvements from 6.1 allowing the underlying library to target both 5.4+ (Python 2+3) and 6.1+ (Python 3) without having conditional branches. This should make the transition easier for users and developers

You can see the what’s new for:

`TerminalIPythonApp.interactive_shell_class`

In 6.0 and 5.4 IPython gained a new c.TerminalIPythonApp.interactive_shell_class option, to customize the class used to start the terminal frontend. This should enable you to use custom interfaces, such as reviving the former readline interface which is now a separate package. See the next section for more.

Define `_repr_mimebundle_`

In IPython 6.1+ and 5.4+ object can now define _repr_mimebundle_ in place of multiple _repr_*_ methods and return a dictionary of MIME types and MIME data for rich display. IPython provides _ipython_display_ but this mechanism was complex and difficult to use for frontends with different architectures, such as nteract and hydrogen. This new _repr_mimebundlde_ methods are thus simpler for both backend and frontends developers to implement, and are already handled in projects like nteract. Our own JupyterLab, the next-generation evolution of the Jupyter Notebook, takes full advantage of these capabilities to expose rich and flexible data display capabilities that work not only in notebook documents, but across all of JupyterLab.

This new model will simplify things for client implementations and will facilitate the publication of custom mimetypes (GeoJson, Plotly, DataFrames, etc.). See the Custom Display Logic example notebook for examples and technical details.

Execution Heuristics

In IPython 5.4+ and 6.1 +, the heuristic for execution in the command line interface is now more biased toward executing for single statement. Use Ctrl-O to force the insertion of a new line. The execution semantics of Enter were still debated, so if you like to configure that you can define your own function which decides what to do.

Display IDs

IPython 5.4 and 6.1+ implement display ids, which can be used to update a displayed object. This should provide simple patterns to implement common cases of dynamic display updates (see Updating Displays.ipynb notebook). It will also simplify the implementation of some types of interactive user interface elements, such as progress-bars. Display objects with IDs can be updated from other cells or frontends.

The display function is now always available

The display() function is now available by default in the IPython session, without users having to import anything. display() is the "moral equivalent” of print(), but takes advantage of the rich display capabilities of Jupyter (and defaults to printing a string for objects lacking rich representations). By making display() appear, in practice, as if it was built-in, we hope to encourage library authors to provide their objects with rich and informative representations that users can now more conveniently access, whether at the IPython REPL or in any of the richer clients in the Jupyter ecosystem (JupyterLab, classic Notebook, nteract, etc.). Note that this automatic import of display() is only done in the live IPython kernel; scripts and libraries that rely on display and may be run outside of IPython still need to import it using from IPython.display import display.

Rlipython

Many users were pleased with the transition from readline to prompt_toolkit for IPython 5.0, this gave IPython multiline editing, nice completion and highlighting as you type. Still some users are missing the old readline interface (for its reactiveness an custom keybindings) though are still using IPython 4.x series.

While prompt toolkit is still the default interface, you can now use the new configuration option c.TerminalIPythonApp.interactive_shell_class .With this you can now replace the Prompt_Toolkit interface. For one of your liking.

Rlipython is one of these alternative interface that resurect readline and is lighter, and more responsive on some system than default IPython. Using it still give you most of the new features of IPython 5.x or 6.x ! This should also please many of our users who use vi keybinding in terminals as this should respect .inputrc.

To try rlipython, install it with pip,

$ pip install rlipython

And follow the instructions.

We look forward to your contribution and please give lots of 💖 by to @ivanov (tweet) for making this possible.

IPython 6.1 and Python 3 only features

IPython 6.1 is the first minor release of our Python 3 only branch, it is the branch that get the most work, and start to use great new features such as type annotations, async/await [more to be added]. You can install it today please upgrade pip to pip 9 if you can, then upgrade IPython:

$ pip install ipython --upgrade

You’ll get all the new features in 5.4, some of which are already in IPython 6.0, plus a number of additional enhancements:

You can read more on the what’s new for IPython 6 what’s new.

In this release, we had a number of bug fixes and improvements from developers at TwoSigma, who participated in an internal open-source hackathon for Jupyter/IPython in May. Hackathon attendees tackled some long standing, hard to debug issues, and feature requests. Among these:

Quotes in a filename are always escaped during tab-completion on non-Windows, which should make many of our windows users happy. As none of the core devs use windows this one needed someone to simply use some elbow grease!
Variables now shadow magics in autocompletion which make autocompletion of things like matplotlib simpler and less surprising. - Magic aliases can now have parameters.
?? /pinfo2 machinery will show docstrings if source can’t be retrieved.

The completer also got a number of small improvements, and is smarter in many cases. For example, it understands the arguments to the %config and %colors magics. Long methods and snake_case function names can now be completed from abbreviations. For example, if foo_bar_kitten is an existing object, f_b<tab> will find it as a possible completion.

Look out for a blog post from TwoSigma.

Only a small number of improvements are describe here, and we encourage you to read the full change logs if you want to know more.

We hope that all these releases will make you more productive. We welcome your contributions, and questions. Feel free to open issues on GitHub, we’ll do our best to respond.

Release of IPython 6.0

2017-04-19T22:37:00+00:00

It is with great pleasure that today we released IPython 6.0 — almost a year after the 5.0 version.
Users on Python 3.3 and above can get this latest version with all its new features by asking your package manager to upgrade IPython. If you are using pip:

Ensure you have pip 9+:

pip --version

if not upgrade pip:

pip install pip --upgrade

Then upgrade IPython.

pip install ipython --upgrade

or if you have conda:

conda update ipython

It might take a few hours/days for the various conda channels to be updated.

The What’s New page has an extensive description of the changes, which includes new, modified and deprecated features.

Highlighted feature: Jedi completion

One of the biggest changes is integration with Jedi, which provides completions using static analysis, and can display completion types. This will provide a foundation for a richer tab completion experience (e.g., it allows extracting and displaying function signatures in the completer). As of now, these features are available only for command line users, but work is underway in IPykernel and in various frontends (like nteract) to make use of these features.

Sunsetting Python 2 support

As we announced last year when we released IPython 5.0, IPython 6 and above require Python 3. The 5.x branch is still maintained and will be regularly released, so Python 2 users will still see updates, but new features in IPython 6.x will not be backported.

We have worked to make installing IPython on Python 2 and 3 as seamless as possible, but you may still encounter errors. If you do, please check the [readme](https://github.com/ipython/ipython/blob/master/README.rst and if you still have a problem, open an issue on Github.

If you want to hear more about the process of creating a seamless Python 2 sunset and are planning on attending PyCon 2017, we encourage you to attend our talk “Ending Py2/Py3 compatibility in a user friendly manner” on Saturday, May 20, 2017 at 15:15–16:00 PDT.

Make sure you have Pip 9 or above

For most end users the only thing you should pay attention is to have pip version 9, or above. To check your pip version, run:

pip --version

If you have a version of pip older than 9.0.1, you should upgrade pip to its latest version before upgrading ipython ****:

pip install pip --upgrade

Pip 9+ will install the latest compatible version of IPython – that is to say IPython 5.x if you are on Python 2, and IPython 6.x if you are on Python 3. Make sure this is the case for all your Python environments, and build scripts. Users of Ubuntu Trusty 14.04 LTS should be aware that the default pip is too old and will not install the correct version of IPython. Don’t forget to upgrade pip in your various install script, continuous integration settings and docker images.

The Python 3 statement

We care deeply about the seamless coexistence of Python 2 and Python 3 software. This is one of the reasons we started this transition with a new release that is only available for Python 3, while still maintaining the 5.x release series, which is compatible with Python 2. If you want to read more about these reasons you can find more information at the python 3 statement.

As expected, we discovered many challenges and pitfalls as we moved from a single source to a Python3-only source. In the course of overcoming these obstacles, we learned a great deal. We have documented the the risks and errors you can encounter (and their solutions) at the practicalities section of the python 3 statement.

If you are a library maintainer planning to stop support for an older version of Python (even a minor version), and you want to limit the frustration faced by users who use your library on the older version of Python, we could use your help! If you want to add to the resources available for trouble-shooting this transition, please join the conversation on our GitHub repository.

Migrating to Python3-only source

The migration of the IPython source code to be Python3-only was not an easy task, and was – as far as we can tell – unexplored territory. But we’ve bitten the bullet and hopefully anticipated most of the problems that can arise. With this, we hope to make the transition easier for other projects who want to follow our lead.

First, let’s speak about our personal experience writing Python3-only source code.

The size of the IPython codebase has decreased by about 1500 lines of Python code relative to the last release. Of course, that’s not solely due to the removal of Python 2 support, but a non-negligible amount is. And this reduction is even more remarkable in light of completely new features that required adding hundreds of lines of code. A large number of conditionals are gone, making the code more straightforward, easier to read, and simpler to maintain. Across the codebase we saw improvements in code compactness just from removing utility functions that existed only to provide identical behavior across Python 2 and Python 3. Even then, parts of the codebase remain affected by “Python 2 code”, so the potential gains are even greater. We will continue our quest to remove things as we go, and, as usual, contributions are welcome.

This change eases the burden on contributors to IPython. Contributors can can spend less time thinking “what about Python 2”, or rewriting a pull request because the Python 2 test suite fails. At the same time, our tests now complete more quickly on continuous integration services because they need to run on fewer versions of Python.

A couple of new APIs are using type annotations, Python3-only syntax which, properly used, make the code clearer and easier to refactor, and allow the documentation to focus on usage / reason than to describe the types of the function. We can also make use of nifty Python 3 features like keyword-only arguments, which is definitely appreciated when designing APIs.

From a developer point of view we are extremely pleased with having the possibility to write Python3-only code, and are looking forward to even more improvements like pathlib.

Keeping a Python 2 stable branch

Keeping a Python 2 stable branch, and making the install experience as seamless as possible for Python 2 users was a key point in our decision to migrate to requiring Python 3. One of our first questions was how could we make sure that Python 2 users using pip install would get the latest 5.x version of IPython still compatible with their system (and not the latest >6.x version that would not be compatible)? This lead us to a number of upstream patches (we’re giving a talk at PyCon, come see us) and a number of guides as to how to make the transitions as a seamless as possible. We patched pip, pypi-legacy, warehouse (we had to learn postgres for that one), amended peps and made use of recent improvements in setuptools to allow Python users to peacefully coexist. The only thing the user should need to do is to make sure to use pip 9+ , and to use pip install . and pip install -e . instead of invoking setup.py directly.

Migrating more code to Python3-only

IPython is the first of the Python packages we maintain which has been migrated from Python 2+3 compatibility to requiring Python 3. Some of our more recent projects (like JupyterHub) have required Python 3 from their beginnings. We expect more Jupyter & IPython projects will move to requiring Python 3 in the future.

As the Jupyter protocol is language agnostic, it is perfectly possible to run Python 2 code in a Python 2 kernel using a Notebook Server running on Python 3. Especially since JupyterHub already requires Python 3, we envisage that the single-user notebook server will follow suit soon.

Conclusion

We hope you will enjoy this release. It will be the base for some awesome features, like async/await REPL. We hope our work in the packaging ecosystem will minimise the inevitable teething difficulties from our first release to require Python 3. We look forward for your feedback.

HN and reddit