Jupyter Blog - Georgiana Dolocan

From intern to mentor

2023-06-19T08:02:00+00:00

Four years ago, Project Jupyter participated for the first time in the Outreachy program. And at the end of last year, during the December round, three more awesome interns joined the project. Because I was lucky enough to be part of this program both times, first time as an intern and second time as a mentor, in this post I will share the parts that were the most impactful and contributed the most to my growth as both mentor and engineer. I hope this will inspire other interns, to start their journey, but also other community members into becoming mentors.

When I first interacted with the Jupyter community, I remember I was impressed by how welcoming everyone was, how well organized they seemed, how promptly questions got answered and how fast code got reviewed. This was especially impressive during the contribution phase when prospect interns were everywhere at the same time, we were generating crazy amount of work for mentors and the community when asking for guidance and tackling tasks.

When the internship ended I realized how much I appreciated this process compared to other internships. The contribution phase made me feel very motivated because of the opportunity to work on real tasks, with impact for the entire community.

All the work traffic, immense quantity of GitHub notifications, emails, questions, etc., were totally opaque to me as an intern, they seemed to just be magically handled by mentors. I now know this seamless interaction is made of some very special ingredients like team work, time commitment and hard work, nothing magic about it.

Scoping the project, the most challenging part

I believe scoping the project was the part that I found the most challenging as a first time mentor.

The plan I had in mind for how to create the project roadmap, was to define a few tasks that were not very tightly scoped, choose one to start with, allocate time for the intern to research about it, have synchronous meetings to collaboratively refine it, then implement it, review, and then repeat for next task. And although this was a good first plan, I failed to recognize when it stopped working and didn’t realize it’s important to pause and adapt the plan.

It took me some time and guidance from others in the community, to realize that sometimes, constraints are beneficial and allowing complete implementation freedom over a task, can be overwhelming. For finding the appropriate [freedom — constraint] balance I needed to experimented with different task sizes, until tasks had a less overwhelming learning curve. This fosters and grows confidence and improves time management skills.

I found this to be particularly tricky because this balance can be hard to achieve when creating a project roadmap. The hidden complexities of a project, existing skills, or any prior experience managing a project from scratch, are hard to know and plan around in the very first iteration.

Another challenge of being a mentor that is usually hidden is the context switch penalty between mentor time and daily job responsibilities. A mentor needs to fine tune the amount of intern interaction and involvement in the project implementation so that context switches from daily responsibilities are as soft as possible. It’s also critical that a mentor has full support and transparent communication with the daily job management so that work responsibilities don’t spill over into mentoring time, otherwise this can lead to stress and a sub-optimal experience for the intern.

Asking questions publicly, the biggest achievement

The open source comes with a challenge attached to it by default, which is being comfortable communicating in the open and asking questions publicly. The time and work that needs to go into fostering this skill needs to be taken into account from the very beginning and it’s extremely important to consider it from the project proposal and roadmap definition phase. It will help set the correct project expectations, and expose an otherwise hidden complexity of open source project maintainability.

This was a challenge that was more difficult to overcome than expected, because in our case it involved, a first time mentor, a first time intern, and a project that required diving into the codebase of lots of other different projects in the ecosystem, so guidance from the entire community and asking questions was crucial.

My intern, Sheila was very determined and courageous and she received great care and support from the entire community in overcoming this obstacle. She is now a key maintainer of the internship project she worked on, asking and answering questions without issues. Checkout out her many achievements captured in this list (not complete) and her presentation about the Reusable JupyterHub Pytest Plugin at JupyterCon. A recording of her talk should soon be available on JupyterCon’s YouTube channel.

Mentor training, mentor shadowing, co-mentoring, the best allies

I am extremely grateful to have had the support of the other mentors and the community during theses three mentoring months.

In particular, I am very grateful for Sarah Gibson, the Jupyter strategic lead and our Community Coordinator with Outreachy, for organizing for us to get mentor training beforehand through the OLS community. I found that to be very helping and ground setting and it made me feel more empowered.

I also appreciated and found really useful the mentor shadowing sessions that I did with Sarah and the demo session that Min RK hold with all the interns and mentors. They were both great opportunities for me to observe and learn from the other mentors and get feedback from them.

Co-mentoring was something that was considered during this round, but because we wanted to select all these great interns we ended up not mentoring in teams. But this appears as a great approach of introducing first-time mentors.

It was all worth it, the takeway

The JupyterHub project got to improve its documentation, accessibility and testing thanks to this program and to the three amazing interns that worked with us during this round ❤️. In addition, this internship round helped us create a place where people can learn about JupyterHub’s experience with Outreachy and a template repository to be used for quickly generating other JupyterHub Python sub-repositories that follow the standards of the organization.

On the personal side now that the internship is over, I can happily say that it was very enlightening and humbling to be on the mentoring side as well. Even if it’s challenging, you get an amazing feeling when you give back and channel all the effort and kindness that shaped you and that you received as a mentee from other people.

Lessons learned from Jupyter’s Contributor in Residence pilot

2021-03-11T20:22:00+00:00

By Georgiana Dolocan and Chris Holdgraf

Project Jupyter, like most open source communities, often lacks the time and people power to maintain existing projects. Existing maintainers are often overloaded with work, while new volunteer contributions are often driven by the excitement of new feature development. Although on-boarding new maintainers is a sustained long-term team effort, it is something necessary and beneficial not only for the project, but also for existing maintainers and new contributors alike.

For this reason, we decided to pilot a Contributor in Residence program that was generously funded for a year through CZI’s Essential Open Source Science program. Last year we ran the first iteration of the JupyterHub Contributor in Residence role. This post is a short report about what we learned, in the hopes that it guides other projects in pursuing similar programs.

tl;dr

The Contributor in Residence program is a great way to:

Introduce new members into your community
Create pathways for others to get more experience, and grow in their role and leadership
Get a lot of work done that is hard to motivate volunteers to do
Spend time thinking at a “meta” level about how to improve dynamics in the community
Have a paid opportunity to support individuals who can’t contribute via volunteerism alone

However, doing this well requires planning ahead of time and attention throughout the program. It is crucial that you develop a plan and pathways for integrating the CIR into your community, and ensuring that they have the right perspective to support the community. If you do this, the CIR program can be a fantastic win-win situation for everyone involved.

What is the Contributor in Residence?

The Contributor in Residence (CIR) carries out a variety of technical and community-focused actions, looking for opportunities to keep momentum moving forward in conversations, improving infrastructure to minimize human toil and streamline information, and making improvements to the codebase that are focused more around sustainability, stability, and understandability.

For more information about our CIR program, check out these blog posts:

The JupyterHub welcome bot image, hand-drawn by our first Contributor in Residence, Georgiana

How management and planning worked

We didn’t define explicit “metrics-based” outcomes for this Contributor in Residence pilot, as we were not sure the behaviors that would be most beneficial in the long-run. Instead, Georgiana kept track of the activities that she worked on throughout the year, and reflected with us often in identifying challenges to overcome, and opportunities to improve the work she was doing.

In practice, Georgiana did not have a sole formal ongoing mentor providing her regular guidance throughout the project. She received mentorship from a variety of JupyterHub Team members across the repositories, and via participation in issues in GitHub (more on that later).

Georgiana kept track of her actions at this GitHub project. This contains a collection of issues, to-do items, and pull-requests throughout the JupyterHub community, and is a public record of much of the work that she focused on. While we scoped the CIR to cover all of the JupyterHub repositories, these efforts had a general focus on The Littlest JupyterHub, where she is now the primary maintainer and community liaison for the project. In addition to these development-focused efforts, Georgiana also spent a lot of time improving infrastructure and team processes at the JupyterHub-wide level, generally via the JupyterHub Team Compass.

Tips for running a CIR program

The following are a few best-practices that we picked up during our pilot.

Define an explicit work plan and system for managing tasks

The biggest challenge Georgiana faced was in identifying specific projects to work on, and working through the ambiguity of a position that explicitly had no single clear workstream. To get around this, Georgiana sought out guidance and advice from others in the community, and over time gained an understanding of ways that she could contribute. This will be a challenge for any Contributor in Residence moving forward, and we will need to think of new ways to support this role with concrete suggestions. Georgiana wrote a bit about this experience in her CIR update blog post.

Dedicate management and mentorship

A related challenge was in ensuring that the CIR felt like they had a clear idea of their expectations, to-do items, and progress. In future iterations, we would like to dedicate more resources towards providing mentorship and management of the person. This role shouldn’t be a “task giver,” but rather someone to help guide, triage, and prioritize for the benefit of the community. They should have regular touch-points with the CIR to make sure they are supported in their role.

Have an onboarding session to help them navigate your project

Regardless of ongoing mentorship, you should definitely have an onboarding session. Many open source communities have multiple projects, varying stages of contributor documentation, different spaces for conversation and interaction. Moreover, the technical infrastructure may have underlying architectures and organization that is hard to learn on your own. Have a core maintainer (or the mentor if you’re using one) go over your technology’s structure and layout, and double-check the things the CIR is unsure of, and introduce them to various discussions and action points to learn from. This may require multiple meetings in the first few weeks/months.

Assess your current strategy every 3 months and adjust based on current needs

Choose a time interval when you revisit your plan. Project needs, community dynamics, and time availability will change over time, so take the time to think about what parts of the plan need to be modified to fit the situation at hand. An option could be bringing this up during team meetings to brainstorm ideas and get feedback, then write it down to put everything in order and make it real. Consider writing 3-month blog posts of your progress and status — these become public records of the experience gained from the program, and are good opportunities to reflect on how things are going.

Actively invite them into the community

Remember that most open source communities are just that — communities. Take time to make sure others in the community know who the CIR is, invite them to participate in community conversations, welcome them in gitter/slack/issues/etc. Anything you can do to make them feel welcome is great. Make sure that you keep doing this over time, not just once at the beginning of their fellowship. When someone feels that they are welcome and a part of a community, they will be much more comfortable at navigating how to support it themselves.

Tailor the role to fit the CIR as a person too

The CIR responsibilities are usually already sketched up before actually having a person to fill the role. Having an overall plan is an awesome idea and a very useful program compass. However, once the CIR is identified, it is important to re-address the overall focus and goals of the role to ensure that it aligns with the interests and skillsets of the incoming CIR. This includes things like familiarity with the project, future career directions for the CIR, and working styles for people involved.

Start slower

Jumping directly into a new project with a lot of complexity and many sub-projects is not sustainable. In our initial plan, we decided to track about 10 main repositories in the first three months of the CIR pilot. This was not sustainable, and led to an overwhelming feeling because of the constant influx of new issues, questions, etc. Especially for someone new to the community and their new role, this led to a feeling of “I’m not learning fast enough,” and difficulty concentrating on the plan. So please, start slow, take the time to acclimate to the project and go repository-by-repository.

Remember that knowing everything was never part of the role

Being a CIR for a complex project can be very overwhelming. There may be different sub-projects, technology stacks, and team processes across a community. It is crucial to consider the CIR program as a two-way street — it is primarily a learning experience and a chance to grow for the CIR, as well as an opportunity for the project to get some much-needed assistance. The CIR should not be expected to know or do everything either at the beginning or the end of their tenure — instead, celebrate the process and the learning that has come from it.

Ideas for things the CIR can work on

Here are a few things that we found particularly useful for the CIR’s workstream, that will likely be portable across other communities as well.

Identify automation and standardization action points

Observe whether a repetitive task could be performed by an automated system like a bot or a CI system, as this has the potential to ease the workload for maintainers.

For example, the JupyterHub GitHub org has about 60 repositories. Some of them had issue and pull requests templates, but most of them didn’t. Some had instructions of how to make a new release of the project, others didn’t. Some had their instructions written in a README and others had documentation, etc.

When the CIR project took off, the JupyterHub and Binder team already started the quest of automating and standardizing its sub-projects. The CIR’s job was to help the other team members propagate the changes (bots setup, templates, badges, docs) to as many repositories as possible.

Research and discover best-practices from other communities

CIRs can also act as helpful eyes and ears to watch other communities and understand the practices that they adopt. Many practices are useful across projects, and a person serving in this “meta” role is in a good position to think at a high level about opportunities.

For example — a simple “thank you” to a contributor can boost their experience a lot, especially when it is their first time contributing to a project. GitHub has a mechanism of letting you know when someone opens an issue or a PR to a project for the first time, but that little notification can be missed when under a huge load of work. For this, there is “the welcome bot” that can be set up to express the community’s gratitude towards their first time contributors.

Our CIR noticed that “The Turing Way” — an open source community-driven guide to reproducible, ethical, inclusive and collaborative data science — made excellent use of GitHub’s welcome bot, including illustrations to make the message even more pleasant and celebratory. Georgiana was inspired by this, which led to adding welcome bots across the JupyterHub repository (including our own hand-drawn images!). Every time a contributor reacts with a ❤ to the bot’s message reinforces the feeling that the bot has its desired, positive effect.

Welcome images that Georgiana drew for the “welcome bots” that she deployed across the JupyterHub repositories.

Spend time on fixing bugs and improving testing infrastructure

Having a working testing infrastructure helps maintainers and contributors alike. Various updates to the underlying infrastructure of the CI system, or even something as simple as a dependency version bump, can break it. So keeping an eye on the health of the testing infrastructure is something that should concern a CIR. This is particularly useful as the world of “maintenance automation” changes very rapidly, with new frameworks, plugins, and standards. Having a pair of eyes constantly thinking about how to improve this process is very helpful.

Having just a handful of people knowing how to to release a new version of a project is not scalable and puts an enormous pressure on that group. Instead, each project should have public instructions on how to do this, so that other contributors that have the permissions can cut a new project release.

The process of creating RELEASE.md files with these instructions for JupyterHub sub-projects, just like the standardization process, had already been started inside the JupyterHub org by the team members before the CIR started. The CIR’s job was to extend this idea to as many sub-projects as possible. This standardization and improvement quest continues to move forward thanks to the CIR and to Erik’s and Simon’s coordination efforts!

More generally — the CIR is often someone relatively new to a project, which makes them a valuable perspective to understand where team documentation is unclear to newcomers. Use the fresh eyes of the CIR to understand how to make the project more welcoming and easy to navigate, and it will clear a pathway for more to join the project in the future.

What we still haven’t figured out

While we learned a lot during this pilot phase, there are a few things we still haven’t figured out.

How to define a pipeline for new CIRs

The tenure of a CIR seems like it will be relatively short — maybe one to two years on average. This means that we need a pipeline of new individuals that are ready to learn and take on this role once a previous CIR moves on. Being effective as a CIR was only possible for someone with some context for the JupyterHub repositories and team processes, and we likely would not have had success hiring someone on a one-year contract who had never worked in the JupyterHub community before. In future iterations we will need a reliable pipeline of newcomers to the project to potentially serve as CIRs. Programs like Outreachy and Google Summer of Code are excellent sources of future CIRs, but we need to think about the “pipeline” for gracefully using them to power the CIR project.

Find mentorship that isn’t “nights and weekends”

While we may be able to find funding for mentorship and management, many people in the JupyterHub community already had full-time jobs. Having funding to pay for a part of someone’s time is helpful, but this is only possible if you can actually buy out part of somebody’s time (often not possible in certain careers and roles). It may be possible to spend extra time on mentorship in the evenings and weekends, but a sustainable model for this program will require somebody dedicating part of their “FTE time” on mentorship. We need to identify certain kinds of roles / individuals that have this flexibility in their work-load, and bake it into the strategy of the program.

Define a funding mechanism for CIRs

Finally, this CIR pilot was funded by a one-year grant from CZI. In future iterations we will need to have a strategy for funding this position in a reliable manner (or at least, in a repeatable manner). To make this sustainable, we must define a pattern to follow and require resources to do a good job. This will both reduce the amount of effort needed to plan a new round for the CIR, and will make it easier for other organizations to understand how their financial contributions could help fund another round of the program. Perhaps this can be an ongoing fundraising effort, similar to how other programs have “target funding levels” in their annual giving strategy, or perhaps it can be a coordinated effort across projects to join resources together for these positions. Either way, it will be important to have reliable funding to ensure that the CIR program isn’t in a constant state of sputtering and jumpstarting.

We hope that this post was an informative and useful set of lessons-learned from the JupyterHub team. If you’ve got any suggestions, comments, or questions for us, we’d love to chat. Feel free to open up an issue in the Jupyter Community Forum! We look forward to continuing this experiment, and hope to see other communities experimenting with the same model in the future.

The Jupyter Contributor In Residence, update 1

2020-05-07T15:22:00+00:00

This year Jupyter was awarded a one-year “Essential Open Source Software for Science” grant from the Chan Zuckerberg Initiative, which we are using to bring on our first “Contributor in Residence” (many thanks, CZI ❤). This is a short report back from Jupyter’s first “Contributor in Residence”, Georgiana Dolocan. We are now about one third of the way through her tenure as CIR. Editor’s note: Georgiana has also made all of her own GIFs, because she’s just awesome like that ;-)

The JupyterHub and Binder Contributor in Residence. The first keepalive message.

At the end of last year, I found out that the CZI grant (so nicely put together by Chris Holdgraf), was accepted. I was very happy to have gotten the chance to continue working alongside an amazing community that I grew very fond of from day one that I started contributing.

Plan 1.0

Being the first project of its kind, we had millions of ideas about what the Contributor in Residence (CIR) project should be about. We discussed and converged towards a common plan, then we decided to continuously iterate once we got some real life feedback.

See this GitHub issue with the end result of the brainstorming, and with various milestones set in place. In short, the purpose of the CIR is to work on the little day-to-day tasks that occur across a project (respond to questions, review PRs, fix bugs, write documentation, maintain infrastructure), nothing fancy, no big projects.

The biggest little achievements

Among the little day-to-day tasks, there are a few achievements that are more noticeable and worthwhile to mention, that will potentially have a bigger impact than others.

🚩 The first blog post about The-Littlest-JupyterHub is out

The superheroes and the magic wand is the first blog post about TLJH on the Jupyter Blog. It first started as a technical blog post but while writing and constantly relating to TLJH as having superpowers to explain how it works, it turned rapidly into a superhero story.

TLJH is an important project in the Jupyter ecosystem and one that I care about a lot, so stay tuned as there are more blog posts to come about it, and although technical this time, I won’t restrain myself from talking about its superpowers.

🚩 The TLJH CI tests were refactored and an upgrade test was added

Are you familiar with that feeling when a pull request looks OK, it passes all tests and all the bots are happy, so you hit that merge button? Yet, soon after, the unexpected happens and bug reports start showing up. 🙈

Even though you manage to find the issue and solve it, one question remains: “Why did the checks pass?” The answer isn’t easy to hear: “The tests need to be improved”.

TLJH operates on a “rolling release” model, where people just install from master, so it’s very important to have strong tests. However, our issue wasn’t exactly with the tests themselves, but rather our CI didn’t match the production installation workflow, and that small difference between the workflows left TLJH in an uninstallable state. Merging that PR in the first place along with finding the divergence point was the key to solve the problem.

Bonus point: whenever someone wants to upgrade TLJH, they do it by running the installer again. Thus, the changes we’re adding by merging a PR, don’t just need to pass the tests, but also they must not break the upgrade process. So, we also added an upgrade test to our CI to solve any potential future upgrade problems.

🚩 Three repositories in the JupyterHub organization are now being watched by a support bot

The Jupyter community currently uses GitHub, Gitter and Discourse (the newest of them) to connect to one another. Having three communication channels might be confusing if there isn’t any clear distinction between them in terms of their scope. So we needed to find a way to organize our discussion to make the contributing experience better for everyone (users, core developers, and everyone in between).

The Discourse forum was already the place where most of our general discussions happened. So why not make this place even more popular and encourage everybody to share their ideas and questions there? This way, the forum becomes the place where we can help and inspire each-other, and GitHub issues are where we talk about cool new features, find bugs, and figure out how to solve the issues.

The support bot will help spread the word about the forum and will make the transition easier. Currently, it is plugged into the jupyterhub, binderhub and tljh repositories and will act each time the “support” label is added to a GitHub issue that should be on Discourse.

We kindly encourage everybody to use Discourse to start or participate in awesome discussions or ask questions of any kind. A great place to start is the Introduce yourself thread where you can just stop by and say “Hi”.

Struggles and solutions

Working on little tasks sounds simpler and easier than starting a big, complex project from scratch. Well, this is not entirely true… Those small tasks are happening around big projects that are used by a lot of people and are constantly evolving. So, even a small, not very complicated task, can be hard when you’re still figuring out the “hows” and “wheres” of a project.

Struggle: distributing focus across multiple repositories and issues

When working on a bigger project, I used to switch between tasks as a means to unwind when running out of ideas and things seemed impossible. I guess one could call this multitasking. However, switching between tasks of different projects that you know little about is challenging enough to cause stress, resuscitate that impostor syndrome, and ultimately turn into burnout.

Solution: Although I don’t have a solution to this and I’m still having troubles mastering this skill, I find inspiration and hope among the people in the Jupyter community. Even if maintaining lots of projects at once, they manage to successfully juggle between tasks, sometimes even when this isn’t their main job, so they deserve a huge shout out ❤️

Struggle: keeping track of little things

The difference between working on a big project and solving small tasks here and there (besides the obvious size distinction) is that solving smaller tasks makes you prone to loosing a sense of progress. Though not a project in the traditional way, the CIR is a concept that would be worth extending if successful, so evaluating its progress is a must and can help determine and improve those weak points — and why not — even point out when it’s time to stop. But keeping track of little things is tricky.

Solution: GitHub issues, comments or PRs can automatically be retrieved through GitHub’s API (Chris Holdgraf has a great tool for this called github-activity). However, to get a sense of accomplishment and progress at the end of the day, I tried to keep my own list of todos and tasks done. I experimented with two approaches in this sense: keeping a HackMD “journal” and creating a GitHub project. After these three months I think I like the GitHub project better because it allows me to keep a todo list with small enhancements I’d like to work when that notification bell doesn’t ring that often and it’s less copy and pasting links.

Struggle: getting out of the comfort zone

Implementing that cool new feature or solving that annoying bug when feeling adventurous are part of the fun of software development. However, being a CIR is more than writing code. Things like writing blog posts and reviewing PRs are core responsibilities as well. But these are all new territory, so much that, I think up until last year I could count on the fingers of one hand the number of blog posts I had written or the number of PRs I had reviewed. So, the only thing left to say is “Hello new challenges! Farewell comfort zone 👋”.

Solution: Accept the challenge and practice every day until things don’t feel that scary anymore. Patience is key.

Plan 2.0

Version 1 of the plan was solid enough not to have to modify it completely.
So, things like tracking down the bugs and failing tests won’t be halted.
Instead, during these next two months, the plan will only suffer some little focus-related adjustments.

Specifically, I want to focus more on:

Getting more familiar with some of the repositories that didn’t get enough attention during the first phase of the project (zero-to-jupyterhub-k8s, jupyterhub-deploy-docker, binderhub, dockerspawner, kubespawner, repo2docker), potentially improving the documentation along the way. No more dipping the toes in the water, just dive in 🌊.
Writing more blog posts about the superpowers of TLJH.
Have more bots 🤖 helping the community.

In the process, I hope I can check the boxes for some of the items on my “learn and improve” wish-list:

technical writing
Kubernetes
UI/UX
courage

Stay tuned, because the next blog post about the JupyterHub and Binder Contributor in Residence adventures should be out in two months time.

The superheroes and the magic wand

2020-03-19T18:23:00+00:00

This is the first of a series of posts describing tools in the JupyterHub ecosystem, written by our wonderful Contributor in Residence, Georgiana. For our first post, we’ll share some lore of JupyterHub, and tell you a story of how it all began…

In a place far, far away, on a planet called Jupyter, magic happens every day. This land is special because it’s full of magical tools with all kind of powers, devoted to one common purpose: to help people. Everyone sympathizing with this goal either advocates, uses, or cares for these tools. So, in the blink of an eye, these people with sometimes nothing else in common than the same drive to help others, gathered together and formed a community.

This special group is known in the galaxy as “The Jovyans*” and new recruits join every day.* 🚀

This image was created by Scriberia for The Turing Way community and is used under a CC-BY licence. Zenodo record.

But somebody needed to take care of these magical tools. So, the first Jovyans decided that from then on, they will become tool-keepers. Because their greatest responsibility was to teach new Jovyans how to use the magic, shortly after, they created a set of guiding laws. Some call this “the Documentation”.

The magic wand

One greatly cherished tool on planet Jupyter is the magical wand. This wand’s very special power is to help people work together as a team and find solutions to important problems.

The wand is called “JupyterHub”.

But the wand is of vast complexity and because of this, so are the guiding laws that control it.

As a consequence, people trying to benefit from the magic of JupyterHub, spent a lot of time reading and understanding the instructions.

The superheros

The tool-keepers noticed that most of the people coming to planet Jupyter to become Jovyans were from a big city in the cloud 🌤 called Kubernetes. So they gathered together and debated what is the best way to help the people of Kubernetes.

After 3 days and 3 nights of intense discussions (also lots of pizza breaks of course) they decided that one of them needed to get special training, rent a house in Kubernetes and teach the people there the wonders of the JupyterHub magic wand.

The Chosen One gained the people’s trust, and it got better and better at anticipating and understanding their needs. So the locals started seeing it as a superhero and they even gave it a name, “Z2JH”. 👓

The news of these events started to spread far and wide, and people living in little towns outside of Kubernetes felt that they deserved the support of a superhero too. They also had great ideas and little time and needed to use the wand’s magic to do good.

The tool-keepers were inspired by the magnificent accomplishments of these little towns, taking place even without a superhero around. So the littlest of them all, volunteered to go into superhero training to help the people of the little towns do good, faster. Though it was little, in no time, it grew to become the helper these small groups needed.

The residents called it “The Littlest JupyterHub” or “TLJH” because of its stature. But everybody knew that its stature didn’t reflect its enormous tenacity, speed and ability to do great things. This ended up inspiring people all over to believe that not all superheros wear capes, nor do they need to know how to fly in the cloud to be cool.

Nowadays, people from all over join their forces and help the superheros learn new skills. Thanks to them, TLJH and Z2JH get stronger and better and have more special powers than they had when first created by the tool-keepers. Join them, become a Jovyan! ツ

Introducing TraefikProxy — a scalable and highly available proxy for JupyterHub

2019-03-06T11:58:00+00:00

In the JupyterHub context, the proxy is the unit in charge of directing the user requests to their notebook servers.

The proxy manages a list of [user : notebook] mappings (the proxy routing table) in order to decide which request is sent where. The routing table must be continuously updated as users start and stop their servers without disrupting the requests being processed. The following drawing illustrates the proxy functionality in a JupyterHub deployment.

Why the need for a new proxy?

Currently, the default proxy implementation for JupyterHub is configurable-http-proxy (CHP), which is a single-process nodejs proxy, that stores the routing table in-memory. CHP is easy to install and run, and thus in most of the cases it’s a fine option. However, because you can only run a single copy of the proxy at a time, it has its limitations when used in dynamic, large scale systems.

What makes this new proxy special?

JupyterHub 0.8. opened the way towards allowing users to create custom proxy implementations based on their deployment needs. JupyterHub Traefik Proxy leverages this feature to offer an alternative to the default proxy. It is an implementation of the JupyterHub Proxy API based on traefik, an extremely lightweight, portable reverse proxy implementation, that supports load balancing and can configure itself automatically and dynamically. JupyterHub Traefik Proxy comes in two flavors, depending on how traefik stores the routing table:

TraefikTomlProxy — for smaller, single-node deployments
TraefikEtcdProxy — for distributed setups

How does it work?

Both TraefikTomlProxy and TraefikEtcdProxy use a toml file for the global configuration. This file contains information about how to set up the connections to the routing table provider (the unit that stores the routing rules like “/user/mary” should be sent to Mary’s server at http://10.0.1.5:12345) and to the network entry points into Traefik (listening port, SSL, traffic redirection). However, the two proxies go in different directions when it comes to the provider used for storing the routing table.

TraefikTomlProxy uses a toml file to store the routes and keeps an in-memory copy of it for a faster access to the routes. This is appropriate for smaller-scale deployments. For example, the Littlest Jupyterhub (the single-node JupyterHub distribution, for a small number of users) just switched from using two proxies (traefik as an edge proxy with letsencrypt support and CHP for routing) to a configuration with just TraefikTomlProxy that serves both requirements. ❤

TraefikEtcdProxy uses etcd, a distributed key-value store to persist the routing table. This implementation aims to benefit Zero to JupyterHub with Kubernetes because it allows having multiple proxy replicas, making the proxy highly available and thus improving the scalability and stability of the system.

Adding the information about TraefikProxy to the first diagram , the drawing below presents the two proxy flavors with their common parts (the global configuration file — traefik.toml) and their differences (the mechanism used for storing the routing table — a toml file vs. etcd)

Another cool Traefik feature is the Web UI dashboard which lists all of the registered frontends (the set of rules that determine how incoming requests are forwarded) and backends (the notebooks), the routing rules, some useful metrics, and other configuration elements. The port on which TraefikProxy’s api will run, as well as the username and password used for authenticating, are all configurable.

Here’s what the dashboard looks like:

How to enable Traefik Proxy?

These instructions help you enable one of the Traefik proxies on your JupyterHub.

Install it:

python3 -m pip install jupyterhub-traefik-proxy

Install traefik and etcd:

$ python3 -m jupyterhub_traefik_proxy.install --output=/usr/local/bin

This will install the default versions of traefik and etcd, namely traefik-1.7.5 and etcd-3.3.10 to /usr/local/bin specified through the --output option.

Configure JupyterHub to run with TraefikProxy through jupyterhub_config.py, using the proxy_class config option.

from jupyterhub_traefik_proxy import TraefikEtcdlProxy
c.JupyterHub.proxy_class = TraefikEtcdProxy
# will configure JupyterHub to run with TraefikEtcdProxy

from jupyterhub_traefik_proxy import TraefikTomlProxy
c.JupyterHub.proxy_class = TraefikTomlProxy
# will configure JupyterHub to run with TraefikTomlProxy

As there are many other proxy configuration options, please check out the project documentation for more info and some example configurations.

What’s next?

TraefikEtcdProxy is soon to be integrated into zero-to-jupyterhub-k8s, the Helm Chart for deploying JupyterHub on Kubernetes. By using the Traefik proxy with etcd, we can eliminate the downtime while the proxy restarts or upgrades, as well as have an all-in-one proxy that can be used for both routing and HTTPS (Let’s Encrypt) support.

Also, there’s a performance analysis of all the three proxies on the way that will help us understand the advantages and limitations of this new JupyterHub Proxy implementation.

The story behind

JupyterHub Traefik Proxy was developed as part of my Outreachy internship. I am very grateful to have had the chance to work with the Jupyter community and I’m happy and proud to say that I’ve learned a lot and the experience has been invaluable. There were a lot of questions and a ton of fears along the way, but the encouragements and guidance I received, helped me move forward and finish this great project. So, a big thank you to everyone for their support and to NumFocus and Berkeley Institute for Data Science for sponsoring this Outreachy round! ❤