JupyterHub and nbgrader in a small multi-class lab environment

February 2022 (republished in March 2026)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

This post was written in 2022 and has not been updated since (up to fixing some typos and broken links).

Given a bare-metal GPU server in a lab environment installing nbgrader shouldn’t be too difficult. Especially, if installing the whole system including JupyterHub and GPU accelerated machine learning stuff was straight-forward. But it turns out that due to the progress in JupyterHub and JupyterLab development, nbgrader is a bit outdated and its installation requires lots of patching and tweaking.

The purpose of this blog post is to collect all the customization and patching steps required to get nbgrader 0.6.2 running in a multi-class and multi-instructor environment, including integration into JupyterHub and JupyterLab’s classical Jupyter Notebook interface. Basic installation instructions are taken from the nbgrader documentation.

Development of nbgrader

In past years nbgrader development slowed down and seems to be stuck to some extent. There is no integration into JupyterLab available, but only in the classical Jupyter Notebook interface (in JupyterLab click “Help” > “Launch Classical Notebook”). Also there are lots of open issues. Some hope for this really useful project comes from creating a new maintainer team.

This lack of progress seems to be the reason for complicated installation. Neighboring projects moved one, while nbgrader did not.

Where we start

Following software is up and running an the server:

There is an LDAP server in the background which is used for user authentication. User may login via SSH to the server and each user has its own home directory on the server. The nbgrader admin (the one who installs nbgrader and creates new nbgrader courses) has access to everything on the server. In particular, he’s able to create new local user accounts and to change file permissions.

JupyterHub installation was done very similar to JupyterHub the hard way, no Kubernetes, no docker. So there is a virtual environment in /opt/jupyterhub/ containing all hub related installation. Also nbgrader will go there.

What we want

At least the student facing part of nbgrader should be accessible through the webbrowser. For instructors as much as possible should be manageable via browser. For the nbgrader admin command line is okay, but for everyday work a browser interface is preferred.

We want to use nbgrader for managing assignments for different courses and we want to have different instructors as well as multiple instructors per course.

Autograding should be relatively save, that is, students shouldn’t be able to destroy the system or delete an instructor’s files. It’s possible to run student submissions in a docker container, but this has several drawbacks (complicated configuration, synchronization of Python environments with the outside world,…). So we have to find (and will find) a solution without docker.

Basic installation

To get nbgrader we install it using pip in the virtual environment of JupyterHub:

sudo /opt/jupyterhub/bin/python -m pip install nbgrader

There are unresolvable dependencies between nbgrader and JupyterHub due to conflicting versions of traitlets and nbconvert. Installing the most recent versions keeps JupyterHub running, but nbgrader will need some patching (see below):

sudo /opt/jupyterhub/bin/python -m pip install --upgrade 
     --upgrade-strategy eager traitlets nbconvert

Global nbgrader configuration and logging

Global (that is, relevant for all users) nbgrader configuration is read from /opt/jupyterhub/etc/jupyter/nbgrader_config.py. So we create this file with the following content:

from nbgrader.auth import JupyterHubAuthPlugin

c = get_config()

c.Exchange.path_includes_course = True
c.Authenticator.plugin_class = JupyterHubAuthPlugin
c.Exchange.root = '/home/nbgrader_exchange'
c.NbGrader.logfile = '/opt/jupyterhub/share/jupyter/nbgrader.log'

This prepares multi-course usage (path_includes_course), activates user mangement via JupyterHub (plugin_class), sets the directory used for file sharing, and activates logging to a file.

We have to create the exchange directory with appropriate permissions:

mkdir /home/nbgrader_exchange
sudo chmod ugo+rw /home/nbgrader_exchange

Further, the log file has to be created. Permissions of the log file have to be rather weak, else some components of nbgrader run by non-root users will fail.

sudo touch /opt/jupyterhub/share/jupyter/nbgrader.log
sudo chmod 666 /opt/jupyterhub/share/jupyter/nbgrader.log

Installing Jupyter extensions

Next we install extensions to Jupyter’s web interface to access nbgrader from the webbrowser:

sudo /opt/jupyterhub/bin/jupyter nbextension install --sys-prefix --py nbgrader
sudo /opt/jupyterhub/bin/jupyter nbextension enable --sys-prefix --py nbgrader
sudo /opt/jupyterhub/bin/jupyter serverextension enable --sys-prefix --py nbgrader

We disable all extensions not needed by students. Extensions used by instructors will be enabled later on for each instructor separately.

sudo /opt/jupyterhub/bin/jupyter nbextension disable
     --sys-prefix create_assignment/main
sudo /opt/jupyterhub/bin/jupyter nbextension disable
     --sys-prefix formgrader/main --section=tree
sudo /opt/jupyterhub/bin/jupyter serverextension disable
     --sys-prefix nbgrader.server_extensions.formgrader
sudo /opt/jupyterhub/bin/jupyter nbextension disable
     --sys-prefix course_list/main --section=tree
sudo /opt/jupyterhub/bin/jupyter serverextension disable
     --sys-prefix nbgrader.server_extensions.course_list

Some patching

To get nbgrader running in an up-to-date Python environment together with a recent JupyterHub version requires several patches. Some of them already seem to be integrated into nbgrader’s GitHub master branch, but not all. Here we show how to patch vanilla nbgrader 0.6.2.

Some, not all, patches listed below are discussed in plenty of GitHub issues on nbgrader. There’s also pull request 1405, which gives important hints on how to get nbgrader running.

Rename all files in /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/server_extensions/formgrader/templates. The extension tpl has to be replaced by html.j2. Else, there will be “template not found” erros. These modifications have to be made, too, wherever the renamed template files are referenced (precise listing of locations follows).

In all files contained in /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/server_extensions/formgrader/templates replace references to *.tpl files by corresponding *.html.j2.

Replace .tpl by .html.j2 in following files:

Remember that the installed version of nbconvert is newer than allowed by nbgrader. So we have to make some changes to nbgrader’s source.

In /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/server_extensions/formgrader/templates/formgrade.html.j2 and .../feedback.html.j2 replace the line

{%- extends 'basic.html.j2' -%}

by

{%- extends 'classic/index.html.j2' -%}

In /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/converters/generate_feedback.py replace

if 'template_path' not in self.config.HTMLExporter:

by

if 'template_paths' not in self.config.HTMLExporter:

and

c.HTMLExporter.template_path = ['.', template_path]

by

c.HTMLExporter.template_paths = ['.', template_path]

In the definition of build_extra_config in /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/server_extensions/formgrader/formgrader.py replace

extra_config.HTMLExporter.template_path = [handlers.template_path]

by

extra_config.HTMLExporter.template_paths.append(handlers.template_path)

In /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/server_extensions/formgrader/templates/feedback.html.j2 add the line

{{ resources.include_css("classic/static/style.css") }}

below the line

{{ mathjax() }}

Else, student feedback looks scrambled.

Adding configuration option for hiding hidden tests in feedbacks

By default, hidden tests for autograding will show up in student feedback. To add an option to remove hidden tests from feedback, we have to apply a further patch.

In /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/converters/generate_feedback.py replace the last line of

preprocessors = List([
    GetGrades,
    CSSHTMLHeaderPreprocessor
])

by

]).tag(config=True)

JupyterHub configuration

JupyterHub requires some configuration for multi-class nbgrader usage. We have to create a service running the autograding tool. One reason for this is, that this way autograding is accessible to more than one instructor. Another reason is, that student code shouldn’t run in an instructor’s user account (with access to the instructor’s files).

Configuration is described in nbgrader’s documentation (outdated). Some fixes for getting nbgrader running with recent JupyterHub versions are suggested in issue 1533. Here we adapt the proposed fixes to our setting.

In the definition of get_jupyterhub_authorization in /opt/jupyterhub/lib/python3.8/site-packages/nbgrader/auth/jupyterhub.py replace JUPYTERHUB_API_TOKEN by JUPYTERHUB_API_TOKEN_CUSTOM.

Go to JupyterHub’s Hub Control Panel, click the admin button, and create an API token. Then add the following line to your JupyterHub config file:

c.Spawner.environment = { 'JUPYTERHUB_API_TOKEN_CUSTOM':
                              '0123456789abcdef0123456789abcdef' }

This sends the API token to the spawner environment. So every (!) spawned single-user server may access it, but only servers using nbgrader require it. Not a nice solution, but it works.

In JupyterHub’s config file create a token service for this API token:

c.JupyterHub.services = [
    {
        'name': 'nbgrader_token_service',
        'api_token': '0123456789abcdef0123456789abcdef'
    }
]

c.JupyterHub.load_roles = [
    {
        'name': 'nbgrader_token_role',
        'scopes': ['read:users:groups', 'list:services',
                   'groups', 'admin:users'],
        'services': ['nbgrader_token_service']
    }
]

If there already are services or roles, you have to add the new ones to the existing lists. Lines c.JupyterHub.services = [ and c.JupyterHub.load_roles = [ should only appear once in the config file. The purpose of such a token service is to set permissions for API requests using this token (see JupyterHub documentation for details).

Promote regular hub user to instructor

Intructors are hub users with admin access (maybe not necessary) having some nbgrader Jupyter extensions enabled.

Add the username to c.Authenticator.admin_users in JupyterHub’s config file. Then log in to the instructor user’s account via SSH or JupyterLab terminal and run

/opt/jupyterhub/bin/jupyter nbextension enable 
    --user course_list/main --section=tree
/opt/jupyterhub/bin/jupyter serverextension enable
    --user nbgrader.server_extensions.course_list

Create a new course

To create a new nbgrader course we have to create a new user (the grader) running the autograding tool:

sudo adduser grader-test-course

In JupyterHub’s config file we grant the grader access to the hub by adding its username grader-test-course to c.Authenticator.allowed_users. Then we create two groups formgrade-test-course and nbgrader-test-course in JupyterHub’s config file:

c.JupyterHub.load_groups = {
    'formgrade-test-course': ['instructor', 'grader-test-course'],
    'nbgrader-test-course': []
}

Group names are prescribed by nbgrader. And it’s really formgrade..., not formgrader...! The second group remains empty.

Now the autograding service has to be configured in the hub config file:

c.JupyterHub.services = [
    { 
        'name': 'test-course',
        'url': 'http://127.0.0.1:8101',
        'command': ['jupyterhub-singleuser',
                    '--group=formgrade-test-course',
                    '--debug'],
        'user': 'grader-test-course',
        'cwd': '/home/grader-test-course',
        'api_token': '0123456789abcdef0123456789abcdef',
            # api_token can be removed (not tested)
        'environment': { 'JUPYTERHUB_API_TOKEN_CUSTOM':
                         '0123456789abcdef0123456789abcdef' }
    }
]
c.JupyterHub.load_roles = [
    { 
        'name': 'formgrader-test-course-role',
        'groups': ['formgrade-test-course'],
        'scopes': ['access:services!service=test-course']
    }
]

Remember that only one c.JupyterHub.services = [ line is allowed (same for load_roles).

Log in to the grader account and activate relevant nbgrader Jupyter extensions:

/opt/jupyterhub/bin/jupyter nbextension enable
    --user create_assignment/main
/opt/jupyterhub/bin/jupyter nbextension enable
    --user formgrader/main --section=tree
/opt/jupyterhub/bin/jupyter serverextension enable
    --user nbgrader.server_extensions.formgrader
/opt/jupyterhub/bin/jupyter nbextension disable
    --user assignment_list/main --section=tree
/opt/jupyterhub/bin/jupyter serverextension disable
    --user nbgrader.server_extensions.assignment_list

Then create the grader’s ~/.jupyter/nbgrader_config.py:

c = get_config()

c.CourseDirectory.root = '/home/grader-test-course/test-course'
c.CourseDirectory.course_id = 'test-course'

c.GenerateFeedback.preprocessors = [
    'nbgrader.preprocessors.GetGrades',
    'nbconvert.preprocessors.CSSHTMLHeaderPreprocessor',
    # remove: hidden tests from feedback:
    #'nbgrader.preprocessors.ClearHiddenTests',
    # remove tracebacks of hidden tests from feedback
    #'nbgrader.preprocessors.Execute',
]

Handling of hidden tests in student feedback is discussed in issue 917.

Finally, create the directory holding all the course assignments:

mkdir ~/test-course

Students can be added to the course via nbgrader’s web interface. But in addition the following command has to be run from the grader user’s command line:

nbgrader db student add STUDENT_NAME

This can be done from JupyterLab. Simply go to the URL https://location/of/hub/services/test-course and start a Terminal. That this additional step is necessary is described in issue 1396