Jupyter Blog - Outreachy

From intern to mentor

2023-06-19T08:02:00+00:00

Four years ago, Project Jupyter participated for the first time in the Outreachy program. And at the end of last year, during the December round, three more awesome interns joined the project. Because I was lucky enough to be part of this program both times, first time as an intern and second time as a mentor, in this post I will share the parts that were the most impactful and contributed the most to my growth as both mentor and engineer. I hope this will inspire other interns, to start their journey, but also other community members into becoming mentors.

When I first interacted with the Jupyter community, I remember I was impressed by how welcoming everyone was, how well organized they seemed, how promptly questions got answered and how fast code got reviewed. This was especially impressive during the contribution phase when prospect interns were everywhere at the same time, we were generating crazy amount of work for mentors and the community when asking for guidance and tackling tasks.

When the internship ended I realized how much I appreciated this process compared to other internships. The contribution phase made me feel very motivated because of the opportunity to work on real tasks, with impact for the entire community.

All the work traffic, immense quantity of GitHub notifications, emails, questions, etc., were totally opaque to me as an intern, they seemed to just be magically handled by mentors. I now know this seamless interaction is made of some very special ingredients like team work, time commitment and hard work, nothing magic about it.

Scoping the project, the most challenging part

I believe scoping the project was the part that I found the most challenging as a first time mentor.

The plan I had in mind for how to create the project roadmap, was to define a few tasks that were not very tightly scoped, choose one to start with, allocate time for the intern to research about it, have synchronous meetings to collaboratively refine it, then implement it, review, and then repeat for next task. And although this was a good first plan, I failed to recognize when it stopped working and didn’t realize it’s important to pause and adapt the plan.

It took me some time and guidance from others in the community, to realize that sometimes, constraints are beneficial and allowing complete implementation freedom over a task, can be overwhelming. For finding the appropriate [freedom — constraint] balance I needed to experimented with different task sizes, until tasks had a less overwhelming learning curve. This fosters and grows confidence and improves time management skills.

I found this to be particularly tricky because this balance can be hard to achieve when creating a project roadmap. The hidden complexities of a project, existing skills, or any prior experience managing a project from scratch, are hard to know and plan around in the very first iteration.

Another challenge of being a mentor that is usually hidden is the context switch penalty between mentor time and daily job responsibilities. A mentor needs to fine tune the amount of intern interaction and involvement in the project implementation so that context switches from daily responsibilities are as soft as possible. It’s also critical that a mentor has full support and transparent communication with the daily job management so that work responsibilities don’t spill over into mentoring time, otherwise this can lead to stress and a sub-optimal experience for the intern.

Asking questions publicly, the biggest achievement

The open source comes with a challenge attached to it by default, which is being comfortable communicating in the open and asking questions publicly. The time and work that needs to go into fostering this skill needs to be taken into account from the very beginning and it’s extremely important to consider it from the project proposal and roadmap definition phase. It will help set the correct project expectations, and expose an otherwise hidden complexity of open source project maintainability.

This was a challenge that was more difficult to overcome than expected, because in our case it involved, a first time mentor, a first time intern, and a project that required diving into the codebase of lots of other different projects in the ecosystem, so guidance from the entire community and asking questions was crucial.

My intern, Sheila was very determined and courageous and she received great care and support from the entire community in overcoming this obstacle. She is now a key maintainer of the internship project she worked on, asking and answering questions without issues. Checkout out her many achievements captured in this list (not complete) and her presentation about the Reusable JupyterHub Pytest Plugin at JupyterCon. A recording of her talk should soon be available on JupyterCon’s YouTube channel.

Mentor training, mentor shadowing, co-mentoring, the best allies

I am extremely grateful to have had the support of the other mentors and the community during theses three mentoring months.

In particular, I am very grateful for Sarah Gibson, the Jupyter strategic lead and our Community Coordinator with Outreachy, for organizing for us to get mentor training beforehand through the OLS community. I found that to be very helping and ground setting and it made me feel more empowered.

I also appreciated and found really useful the mentor shadowing sessions that I did with Sarah and the demo session that Min RK hold with all the interns and mentors. They were both great opportunities for me to observe and learn from the other mentors and get feedback from them.

Co-mentoring was something that was considered during this round, but because we wanted to select all these great interns we ended up not mentoring in teams. But this appears as a great approach of introducing first-time mentors.

It was all worth it, the takeway

The JupyterHub project got to improve its documentation, accessibility and testing thanks to this program and to the three amazing interns that worked with us during this round ❤️. In addition, this internship round helped us create a place where people can learn about JupyterHub’s experience with Outreachy and a template repository to be used for quickly generating other JupyterHub Python sub-repositories that follow the standards of the organization.

On the personal side now that the internship is over, I can happily say that it was very enlightening and humbling to be on the mentoring side as well. Even if it’s challenging, you get an amazing feeling when you give back and channel all the effort and kindness that shaped you and that you received as a mentee from other people.

Introducing JupyterHub’s Outreachy interns! — December 2022 Cohort

2022-12-07T07:45:00+00:00

As part of the community strategic support project funded by CZI’s EOSS grant series, the JupyterHub sub-project has funding to support Outreachy interns through four cohorts over the next two years. We would like to introduce you to the first cohort and the projects they will be working on!

Ogoh Blessing Onyowoicho — Improve Accessibility in JupyterHub

Accessibility is the ability of tools (in our case web tools) to be used by a variety of communities with different disabilities. There are a variety of standards and tools for evaluating and ensuring that a web page can be used effectively by as many people as possible. Work by the Accessibility team is ongoing to define a set of standard tools to improve accessibility across the Jupyter ecosystem.

The JupyterHub project is working to improve the accessibility of its pages to ensure we are providing tools that are as useful as they can be to as many people as we can. During the internship, we will evaluate JupyterHub’s accessibility, find ways to improve it, and integrate accessibility testing into the development process, in collaboration with the Accessibility team, to ensure we do a better job going forward.

Ogoh Blessing says:

My name is Ogoh Blessing Onyowoicho. I am a self-taught Frontend developer based in Lagos, Nigeria and I am an Outreachy intern working on Improving the accessibility of JupyterHub.
I am excited to work on JupyterHub because it is giving me the opportunity to use skills that I have accrued over the years to proffer solutions to problems that affect people’s lives directly. On hearing accessibility, the first thought one has is that it involves building web products that people with special needs can use seamlessly. Though this is part of it, accessibility goes way beyond this. It involves building products that different users (e.g users at different locations, users with different devices etc) can use easily. The thought of contributing to improving the experience of so many people alone excites me.
In the coming months, I hope to learn and continue to hone my skills as I am guided by my mentors and members of the community I get to interact with.

Allan Wasega — Restructure and improve JupyterHub documentation by implementing the Diátaxis framework

JupyterHub has a range of documentation that covers both developer and user audiences in order to help them deploy, maintain, and use their own instance of a JupyterHub. The success of an open source software project to (i) be adopted by users, and (ii) receive meaningful contributions relies heavily on the quality, navigability and accessibility of documentation so that users and developers have all the information they need to achieve what they want to do.

A framework for organising technical documentation has arisen called diátaxis. It takes a systematic approach to understanding user requirements of documentation throughout the lifecycle of interaction with a product and posits that different user needs require different approaches in creation of the documentation, as well as a layout to navigate these different “modes” of documentation.

This project will focus on a refactoring of the documentation for the JupyterHub package. We will begin by performing a review of the present documentation, categorise these into the diataxis framework, and then restructure the documentation files in the repository. Once we have transformed the documentation into this framework, it will be much easier to identify missing and unclear documentation (those that were difficult to categorise). We can then begin to curate resources that can fill the gaps and improve documentation that is not specific enough.

Allan says:

I am Allan Wasega, from Nairobi, Kenya. Broadly, I like to describe myself as a researcher and a writer. Researcher because looking into things to find patterns or hidden information has always been of interest to me. Writer because I figured early on that words allow me to express myself and to communicate to a larger audience than that inside my head :). As a Computer Science student, I looked for ways to bring these two skills together within the realm of computing and that is how I found myself in the technical writing space.
As an undergraduate student, I used Jupyter Notebooks extensively for most of my programming assignments and projects. As a result, when making my Outreachy application, the JupyterHub project immediately piqued my interest because I did not know that there was a way to serve Jupyter Notebooks to multiple users, such as within a classroom setting. Presently, I serve as a mentor at KamiLimu, which is a mentorship program for students pursuing technology-aligned courses in tertiary institutions in Kenya. One of the objectives of KamiLimu is to introduce students to tech specializations such as Data Science and Machine Learning, which, from experience, uses Jupyter Notebooks! Therefore, I am excited to work on JupyterHub because I will be learning about and helping to improve a product I hold dear and which I can use to advance the skills of the next generation of computer technologists in Kenya.
First, I hope to gain a deep understanding of how JupyterHub works so as to spread the word about it and its functionalities. Second, I hope to apply and advance my technical writing skills. I encountered the Diataxis Framework earlier this year while working as a technical writer at Tingle Software, a Nairobi-based software company. I have also been following Daniele Procida (the author of the framework) on Twitter for a while now and his work has been quite inspiring. Through this project, I will apply the Diataxis Framework to restructure JupyterHub’s documentation and, in doing so, further my understanding of the framework. Finally, I am passionate about mentorship, especially in the tech field. Therefore, besides expanding my professional and personal networks, this project will enable me to learn how to conduct mentorship within a global setting, and I can, in turn, apply this knowledge within my local community.

Sheila Kahwai — Create a reusable JupyterHub pytest plugin

JupyterHub is a modular and extensible project, with parts, like the proxy, authenticator and spawner, that can be easily changed and extended. Testing the functionality of these components against JupyterHub is important and it requires various hub setups that can sometimes become complicated.

Currently, each of these hub components and the hub itself define their own testing infrastructure, building everything from the ground up using the pytest framework. But some of this complex work is either repetitive across JupyterHub sub-projects, or under-specified for some of them.

This project will abstract out these common parts into a separate testing framework. This framework should be a pip-installable pytest plugin that would provide various hub functionalities through pytest fixtures. These fixtures can then be re-used by JupyterHub and its components to bootstrap their own testing suite.

Integrating this plugin will drive some important refactoring work of the current testing architecture of JupyterHub and have a great impact in improving the overall test coverage, maintainability and continuity of the JupyterHub project.

Sheila says:

My name is Sheila Kahwai. I am a self-taught python developer from Nairobi, Kenya, working to specialize in back-end engineering.
I am excited to work on JupyterHub because it has made many products I have used throughout my learning journey. It is a privilege to work with a diverse community that has created products that are very useful to equally diverse users like myself.
While creating a reusable JupyterHub pytest plugin, I hope to gain more insight into creating plugins for massive codebases to improve maintainability and scalability with clean and reusable code. Through this project, I look forward to improving the overall testing infrastructure of the various JupyterHub components.

Welcome to the interns! We’re so excited to start working with you!

The Jupyter Contributor In Residence, update 1

2020-05-07T15:22:00+00:00

This year Jupyter was awarded a one-year “Essential Open Source Software for Science” grant from the Chan Zuckerberg Initiative, which we are using to bring on our first “Contributor in Residence” (many thanks, CZI ❤). This is a short report back from Jupyter’s first “Contributor in Residence”, Georgiana Dolocan. We are now about one third of the way through her tenure as CIR. Editor’s note: Georgiana has also made all of her own GIFs, because she’s just awesome like that ;-)

The JupyterHub and Binder Contributor in Residence. The first keepalive message.

At the end of last year, I found out that the CZI grant (so nicely put together by Chris Holdgraf), was accepted. I was very happy to have gotten the chance to continue working alongside an amazing community that I grew very fond of from day one that I started contributing.

Plan 1.0

Being the first project of its kind, we had millions of ideas about what the Contributor in Residence (CIR) project should be about. We discussed and converged towards a common plan, then we decided to continuously iterate once we got some real life feedback.

See this GitHub issue with the end result of the brainstorming, and with various milestones set in place. In short, the purpose of the CIR is to work on the little day-to-day tasks that occur across a project (respond to questions, review PRs, fix bugs, write documentation, maintain infrastructure), nothing fancy, no big projects.

The biggest little achievements

Among the little day-to-day tasks, there are a few achievements that are more noticeable and worthwhile to mention, that will potentially have a bigger impact than others.

🚩 The first blog post about The-Littlest-JupyterHub is out

The superheroes and the magic wand is the first blog post about TLJH on the Jupyter Blog. It first started as a technical blog post but while writing and constantly relating to TLJH as having superpowers to explain how it works, it turned rapidly into a superhero story.

TLJH is an important project in the Jupyter ecosystem and one that I care about a lot, so stay tuned as there are more blog posts to come about it, and although technical this time, I won’t restrain myself from talking about its superpowers.

🚩 The TLJH CI tests were refactored and an upgrade test was added

Are you familiar with that feeling when a pull request looks OK, it passes all tests and all the bots are happy, so you hit that merge button? Yet, soon after, the unexpected happens and bug reports start showing up. 🙈

Even though you manage to find the issue and solve it, one question remains: “Why did the checks pass?” The answer isn’t easy to hear: “The tests need to be improved”.

TLJH operates on a “rolling release” model, where people just install from master, so it’s very important to have strong tests. However, our issue wasn’t exactly with the tests themselves, but rather our CI didn’t match the production installation workflow, and that small difference between the workflows left TLJH in an uninstallable state. Merging that PR in the first place along with finding the divergence point was the key to solve the problem.

Bonus point: whenever someone wants to upgrade TLJH, they do it by running the installer again. Thus, the changes we’re adding by merging a PR, don’t just need to pass the tests, but also they must not break the upgrade process. So, we also added an upgrade test to our CI to solve any potential future upgrade problems.

🚩 Three repositories in the JupyterHub organization are now being watched by a support bot

The Jupyter community currently uses GitHub, Gitter and Discourse (the newest of them) to connect to one another. Having three communication channels might be confusing if there isn’t any clear distinction between them in terms of their scope. So we needed to find a way to organize our discussion to make the contributing experience better for everyone (users, core developers, and everyone in between).

The Discourse forum was already the place where most of our general discussions happened. So why not make this place even more popular and encourage everybody to share their ideas and questions there? This way, the forum becomes the place where we can help and inspire each-other, and GitHub issues are where we talk about cool new features, find bugs, and figure out how to solve the issues.

The support bot will help spread the word about the forum and will make the transition easier. Currently, it is plugged into the jupyterhub, binderhub and tljh repositories and will act each time the “support” label is added to a GitHub issue that should be on Discourse.

We kindly encourage everybody to use Discourse to start or participate in awesome discussions or ask questions of any kind. A great place to start is the Introduce yourself thread where you can just stop by and say “Hi”.

Struggles and solutions

Working on little tasks sounds simpler and easier than starting a big, complex project from scratch. Well, this is not entirely true… Those small tasks are happening around big projects that are used by a lot of people and are constantly evolving. So, even a small, not very complicated task, can be hard when you’re still figuring out the “hows” and “wheres” of a project.

Struggle: distributing focus across multiple repositories and issues

When working on a bigger project, I used to switch between tasks as a means to unwind when running out of ideas and things seemed impossible. I guess one could call this multitasking. However, switching between tasks of different projects that you know little about is challenging enough to cause stress, resuscitate that impostor syndrome, and ultimately turn into burnout.

Solution: Although I don’t have a solution to this and I’m still having troubles mastering this skill, I find inspiration and hope among the people in the Jupyter community. Even if maintaining lots of projects at once, they manage to successfully juggle between tasks, sometimes even when this isn’t their main job, so they deserve a huge shout out ❤️

Struggle: keeping track of little things

The difference between working on a big project and solving small tasks here and there (besides the obvious size distinction) is that solving smaller tasks makes you prone to loosing a sense of progress. Though not a project in the traditional way, the CIR is a concept that would be worth extending if successful, so evaluating its progress is a must and can help determine and improve those weak points — and why not — even point out when it’s time to stop. But keeping track of little things is tricky.

Solution: GitHub issues, comments or PRs can automatically be retrieved through GitHub’s API (Chris Holdgraf has a great tool for this called github-activity). However, to get a sense of accomplishment and progress at the end of the day, I tried to keep my own list of todos and tasks done. I experimented with two approaches in this sense: keeping a HackMD “journal” and creating a GitHub project. After these three months I think I like the GitHub project better because it allows me to keep a todo list with small enhancements I’d like to work when that notification bell doesn’t ring that often and it’s less copy and pasting links.

Struggle: getting out of the comfort zone

Implementing that cool new feature or solving that annoying bug when feeling adventurous are part of the fun of software development. However, being a CIR is more than writing code. Things like writing blog posts and reviewing PRs are core responsibilities as well. But these are all new territory, so much that, I think up until last year I could count on the fingers of one hand the number of blog posts I had written or the number of PRs I had reviewed. So, the only thing left to say is “Hello new challenges! Farewell comfort zone 👋”.

Solution: Accept the challenge and practice every day until things don’t feel that scary anymore. Patience is key.

Plan 2.0

Version 1 of the plan was solid enough not to have to modify it completely.
So, things like tracking down the bugs and failing tests won’t be halted.
Instead, during these next two months, the plan will only suffer some little focus-related adjustments.

Specifically, I want to focus more on:

Getting more familiar with some of the repositories that didn’t get enough attention during the first phase of the project (zero-to-jupyterhub-k8s, jupyterhub-deploy-docker, binderhub, dockerspawner, kubespawner, repo2docker), potentially improving the documentation along the way. No more dipping the toes in the water, just dive in 🌊.
Writing more blog posts about the superpowers of TLJH.
Have more bots 🤖 helping the community.

In the process, I hope I can check the boxes for some of the items on my “learn and improve” wish-list:

technical writing
Kubernetes
UI/UX
courage

Stay tuned, because the next blog post about the JupyterHub and Binder Contributor in Residence adventures should be out in two months time.

Introducing TraefikProxy — a scalable and highly available proxy for JupyterHub

2019-03-06T11:58:00+00:00

In the JupyterHub context, the proxy is the unit in charge of directing the user requests to their notebook servers.

The proxy manages a list of [user : notebook] mappings (the proxy routing table) in order to decide which request is sent where. The routing table must be continuously updated as users start and stop their servers without disrupting the requests being processed. The following drawing illustrates the proxy functionality in a JupyterHub deployment.

Why the need for a new proxy?

Currently, the default proxy implementation for JupyterHub is configurable-http-proxy (CHP), which is a single-process nodejs proxy, that stores the routing table in-memory. CHP is easy to install and run, and thus in most of the cases it’s a fine option. However, because you can only run a single copy of the proxy at a time, it has its limitations when used in dynamic, large scale systems.

What makes this new proxy special?

JupyterHub 0.8. opened the way towards allowing users to create custom proxy implementations based on their deployment needs. JupyterHub Traefik Proxy leverages this feature to offer an alternative to the default proxy. It is an implementation of the JupyterHub Proxy API based on traefik, an extremely lightweight, portable reverse proxy implementation, that supports load balancing and can configure itself automatically and dynamically. JupyterHub Traefik Proxy comes in two flavors, depending on how traefik stores the routing table:

TraefikTomlProxy — for smaller, single-node deployments
TraefikEtcdProxy — for distributed setups

How does it work?

Both TraefikTomlProxy and TraefikEtcdProxy use a toml file for the global configuration. This file contains information about how to set up the connections to the routing table provider (the unit that stores the routing rules like “/user/mary” should be sent to Mary’s server at http://10.0.1.5:12345) and to the network entry points into Traefik (listening port, SSL, traffic redirection). However, the two proxies go in different directions when it comes to the provider used for storing the routing table.

TraefikTomlProxy uses a toml file to store the routes and keeps an in-memory copy of it for a faster access to the routes. This is appropriate for smaller-scale deployments. For example, the Littlest Jupyterhub (the single-node JupyterHub distribution, for a small number of users) just switched from using two proxies (traefik as an edge proxy with letsencrypt support and CHP for routing) to a configuration with just TraefikTomlProxy that serves both requirements. ❤

TraefikEtcdProxy uses etcd, a distributed key-value store to persist the routing table. This implementation aims to benefit Zero to JupyterHub with Kubernetes because it allows having multiple proxy replicas, making the proxy highly available and thus improving the scalability and stability of the system.

Adding the information about TraefikProxy to the first diagram , the drawing below presents the two proxy flavors with their common parts (the global configuration file — traefik.toml) and their differences (the mechanism used for storing the routing table — a toml file vs. etcd)

Another cool Traefik feature is the Web UI dashboard which lists all of the registered frontends (the set of rules that determine how incoming requests are forwarded) and backends (the notebooks), the routing rules, some useful metrics, and other configuration elements. The port on which TraefikProxy’s api will run, as well as the username and password used for authenticating, are all configurable.

Here’s what the dashboard looks like:

How to enable Traefik Proxy?

These instructions help you enable one of the Traefik proxies on your JupyterHub.

Install it:

python3 -m pip install jupyterhub-traefik-proxy

Install traefik and etcd:

$ python3 -m jupyterhub_traefik_proxy.install --output=/usr/local/bin

This will install the default versions of traefik and etcd, namely traefik-1.7.5 and etcd-3.3.10 to /usr/local/bin specified through the --output option.

Configure JupyterHub to run with TraefikProxy through jupyterhub_config.py, using the proxy_class config option.

from jupyterhub_traefik_proxy import TraefikEtcdlProxy
c.JupyterHub.proxy_class = TraefikEtcdProxy
# will configure JupyterHub to run with TraefikEtcdProxy

from jupyterhub_traefik_proxy import TraefikTomlProxy
c.JupyterHub.proxy_class = TraefikTomlProxy
# will configure JupyterHub to run with TraefikTomlProxy

As there are many other proxy configuration options, please check out the project documentation for more info and some example configurations.

What’s next?

TraefikEtcdProxy is soon to be integrated into zero-to-jupyterhub-k8s, the Helm Chart for deploying JupyterHub on Kubernetes. By using the Traefik proxy with etcd, we can eliminate the downtime while the proxy restarts or upgrades, as well as have an all-in-one proxy that can be used for both routing and HTTPS (Let’s Encrypt) support.

Also, there’s a performance analysis of all the three proxies on the way that will help us understand the advantages and limitations of this new JupyterHub Proxy implementation.

The story behind

JupyterHub Traefik Proxy was developed as part of my Outreachy internship. I am very grateful to have had the chance to work with the Jupyter community and I’m happy and proud to say that I’ve learned a lot and the experience has been invaluable. There were a lot of questions and a ton of fears along the way, but the encouragements and guidance I received, helped me move forward and finish this great project. So, a big thank you to everyone for their support and to NumFocus and Berkeley Institute for Data Science for sponsoring this Outreachy round! ❤

Simpler authentication for small scale JupyterHubs with NativeAuthenticator

2019-02-26T16:31:00+00:00

In this post I’ll tell you about the new JupyterHub authenticator I implemented in the last couple of months, not only technically but also the context in which it was built. If you have no idea what JupyterHub is and how it works, I recommend this short talk by the awesome Carol Willing 🙃.

But first things first, let me introduce myself! My name is Leticia and I’m Brazilian 🇧🇷. Last year (2018) I saw an opening for an internship program called Outreachy. I am a woman, a self-taught programmer with a non traditional background (I’m an oceanographer) and I had no job at this time. The program would fit me perfectly. Lucky me, Jupyter was one of the projects available! Thus, everything you’ll see here is the result (so far) of everything I’ve been doing with the internship and the help of my mentors.

Before going any further I would also like to thank the Berkeley Institute for Data Science and NumFocus for jointly sponsoring my Outreachy internship. This was a huge opportunity for me ❤️

Why do we need another authenticator?

JupyterHub authenticators determine how users on a particular installation of JupyterHub can log in. For example, here are a few common authenticators already available to JupyterHub:

PAM Authenticator: Any whitelisted user with an account and password on the system will be allowed to login;
OAuthenticator: An authenticator that uses the login of other services (OAuth) to authenticate on the Jupyterhub (such as Azure, Github or Moodle);
First Use Authenticator: A new user is created when the user logs in for the first time.

Large installations usually rely on a third party service (using OAuthenticator), while smaller ones maintain user accounts by hand (PAM Authenticator). However, maintining user lists by hand can be a lot of effort, especially as the number of users grows. Native Authenticator was created to supply smaller JupyterHub installations with a more convenient authentication system instead of maintaining user accounts by hand, without the overhead of needing an external third party service.

Native Authenticator features

Native Authenticator was added to PyPI this week and we are super excited about this! The default features available on the authenticator are:

A signup page that creates new users
Username sanitation (avoiding spaces, commas and backslashes)
“Administrator” privileges that can be given to specific users
The ability for users to change their passwords
A panel for controlling access to the system on a user-by-user basis

Some optional features are available and can be activated through the configuration file and will be discussed further on this post 🙃

Installing and using Native Authenticator

To use the authenticator, you can install it through pip :

$ pip install jupyterhub-nativeauthenticator

You can configure JupyterHub to work with Native Authenticator by adding the following line to the configuration file:

c.JupyterHub.authenticator_class = 'nativeauthenticator.NativeAuthenticator'

and that is it 😉

Default authenticator workflow

Once you’ve configured your JupyterHub installation to work with Native Authenticator, no new user can enter the system. To enter the system, a new user must sign up. To do so, the operator of the Hub should point users to/hub/signup and create a new username and password. After submitting the information, the user receives a message indicating that the information was sent to an admin.

By default, all users that Sign Up are unauthorized and can’t login to the system. The admin will need to enter the Authorization Area dashboard at /hub/authorize and authorize the users into the system. The panel looks like this:

Authorized users are shown with a green background while unauthorized users have a white background. Each has a button to change the authorization status. Once a user is authorized, it will be able to go to the / endpoint and login.

The default workflow of the Native Authenticator is like this:

Default workflow of Native Authenticator

A quick remark on this workflow: this only works for users that are not listed as admins. Any username that is listed as an admin in the JupyterHub configuration file will automatically have authorization to login when they sign up. You can add admins by adding this line to the configuration file:

c.Authenticator.admin_users = { 'johnsnow' }

Admins still need to sign up in order to define a password, but they already have authorization to login.

Although the default workflow can be useful for avoiding unwanted people to enter the system, it can lead to a bottleneck if you have a lot of users that need to quickly enter the system. So, one option is to allow everyone in, and later block any users you don’t want to access your system.

To do this, you need to activate the Open SignUp feature on Native Authenticator, which is done by adding the following line to the JupyterHub configuration file:

c.NativeAuthenticator.open_signup = True

Now the workflow is much simpler:

Workflow of Native Authenticator when you configured an Open Signup

Increasing password security

We wanted to give an option to increase security, if needed. So, with Native Authenticator you can add verification of strong passwords. But, what is indeed a “strong password”? We followed the NSIT guidelines described beautifully on this post.

The first thing is that we added an option to see the password while typing. As said on the post:

Typos are common when entering passwords, and when characters turn into dots as soon as they’re typed, it’s difficult to tell where you went wrong. This motivates users to pick shorter passwords that they’re less likely to mess up

So, we added this on SignUp and Change Password (and soon to the login as well 🙃).

Example of input box for passwords with hidden password (left) and open text password (right)

Also, you can add a verification for a minimum number of characters in a password by adding the value in the configuration file:

c.NativeAuthenticator.minimum_password_length = 10

This will prevent users to signup if the password is less then 10 characters. An error message appears in case the password is not good enough:

Error indicating the password is not long enough

Additionally, you can add a verification to avoid common passwords. Instead of asking for users to add special characters or upper case letters, the guide suggests to check for common passwords instead. Passwords such as 123456 or qwerty are common and not nearly secure enough. We used a list of 10 thousand common passwords that will be checked when this option is activated. To activate this verification add to the configuration file:

c.NativeAuthenticator.check_common_password = True

Another thing that increases security is to block users when they’ve had a number of failed logins. In Native Authenticator you can activate this behavior. The system will add a count for every consecutive failed login attempt. If the user exceeds the number of attempts that is permitted, it will check if some time has passed since the last attempt. If not enough time has passed, the system will keep them blocked until they wait a while longer before attempting to sign-in again.

As you can imagine, both the number of attempts permitted and the wait time are configurable.

c.NativeAuthenticator.allowed_failed_logins = 3
c.NativeAuthenticator.seconds_before_next_try = 1200

How we accomplished this

To construct this new Authenticator, Project Jupyter decided to get an intern (me!) through the Outreachy Program. The description of the project was made in advance and potential interns applied by resolving a micro-task defined in a specific repository. The main workflow was:

Create a profile on the Outreachy website;
Submit a pull request for a micro-task of the project you want to participate;
Fill a more complete profile about yourself;
Wait for the selection 🤓

The full presentation of the Project Jupyter participation, a list of mentors, and more details can be seen on this post, and more technical details on the micro-task I submitted can be seen here.

Technical details and day-to-day development

There were a lot of struggles to get things going. As part of the Outreachy program, I had to write about the tasks and struggles I had in posts on my personal blog. If you would like to check the details for how the process went at each you can checkout each of the available posts:

See ya :)

Outreachy & Jupyter: Supporting diversity in open communities

2018-11-28T17:26:00+00:00

Project Jupyter has accepted 2 interns through the Outreachy program, which supports open community members from under-represented backgrounds. Our Outreachy interns will work on important problems in the JupyterHub community with help from 2 mentors from December 2018 through March 2019. This is a short post describing why Jupyter is committing to this program, as well as what we’ll be working on with our Outreachy participants.

We are grateful to the Berkeley Institute for Data Science & NumFocus for jointly sponsoring our interns. This article is cross-posted with the Berkeley Institute for Data Science blog.

Why focus on diversity and inclusion?

The tech industry isn’t doing great on diversity, and the Open Source community is doing worse . This limits the community’s ability to tackle challenging, diverse projects. The amount of responsibility and power held by open source communities is increasing, so we have a responsibility to be more representative of the diversity present in our world. Many projects grow their community by encouraging people to start making small patches on their own. However, requiring uncompensated work is a big barrier to getting more diverse representation in our open source communities. Paid internships with dedicated mentors are a great way to help people break through this particular barrier and make the open source community more diverse.

What is Outreachy?

Outreachy is an internship program coordinated by the Software Freedom Conservancy. It is run twice a year with a goal to bring people from underrepresented backgrounds in tech into open source projects. 21 open source organizations are participating in this round, and will be working with a total of 46–47 interns. Importantly, these are paid internships, which make them more viable for a much broader slice of the population. Jupyter and BIDS will both contribute funding and mentorship for two Outreachy interns.

Our Interns and Mentors

We have two amazing interns for this Outreachy round, working on important projects in the JupyterHub ecosystem. Below we’ll describe the projects that they’ll work on.

A highly Available Proxy for JupyterHub (link)

Georgiana Dolocan will be mentored by Min RK and Yuvi Panda in building a highly available & scalable proxy for JupyterHub using the Traefik project. This will help JupyterHub deployments scale more easily for thousands of active users with minimal service disruptions. Georgiana will be working from Bucharest, Romania where she likes to paint and explore the city and the villages nearby alongside her dog and camera.

Improvements to user management (link)

Leticia Portella will be mentored by Yuvi Panda and Min RK in building better native user management features into JupyterHub. Small to medium installations of JupyterHub that do not want to depend entirely on an external authentication provider will benefit greatly from this. Leticia will be working from Dublin, Ireland (although she used to live in Florianópolis, Brazil), where she likes to read a lot (especially The Chronicles of Ice and Fire), to swim, and to work on her podcast (the first Brazilian podcast specialized in Data Science topics), Pizza de Dados (Data Pizza, free translation).

About our mentors

Outreachy requires more than just funding, but also mentorship. Two members of the JupyterHub community have offered their time to help mentor our Outreachy contributors, some information about them is below!

Min RK is a research engineer at Simula Research Laboratory in Norway and has been working on the IPython and Jupyter open source projects since joining in 2006 as an undergraduate in Engineering Physics at Santa Clara University with Brian Granger, one of the founders of the Jupyter project. Min currently focuses on JupyterHub, and is excited to welcome new folks to the Jupyter team.

Yuvi Panda is an operations engineer at University of California, Berkeley. He works at the Berkeley Institute for Data Science, where he maintains the JupyterHub infrastructure for the Division of Data Sciences. He has been involved in various Open Source communities in the last ten years, spending time in the GNOME, Wikimedia & Jupyter communities. His life was changed drastically by participating as a student in Google Summer of Code 2010, and he’s excited to give back to the Open Source community.

We would like to thank everyone who applied to JupyterHub for this round of Outreachy. We received a number of robust proposals, out of which we were only able to accept two. JupyterHub mentors spent quite a lot of time mentoring candidates during the application period, in reviewing their pull requests, and giving them feedback on their proposals. We look forward to bringing these new open source contributors into our community, and hope that we can set a path that other open source projects may follow in the future. Open source works best when it is diverse and inclusive, we think this is a small step in that direction.

Thanks to Orianna DeMasi, Sara Stoudt, Chris Holdgraf & others for contributing heavily to this blog post.