Dear all,
The grading of Jupyter Notebooks is a frequently requested feature by other lectures as well.
In our discussions with Harald Schnurbusch / CLS we identified this feature be part of a third phase for Jupyter-Moodle integration:
- Phase 1: Opening Jupyter individual Notebook via Hyperlinks from Moodle (done)
- Phase 2: Multi-tenancy / Access-control to Jupyter materials via Moodle course-membership (work-in-progress)
- Phase 3: Notebook grading. Sync points/grades back into Moodle
> Are there already recommended methods to do this or can you use something like nbgrader (https://nbgrader.readthedocs.io/en/stable/) on the RWTH JupyterHub?
There is currently no recommended method for implementing it supported by the ITC/CLS or the Jupyter Admins.
The RWTH JupyterHub cluster is currently in the progress of concluding its pilot phase.
As such, it will graduate to an official ITC service. The scope of this service is yet to be defined.
So we don’t know yet to which extend the integration of Moodle and Jupyter will be part of this initial service offering.
This will be decided by the IT center.
Unfortunately, nbgrader is not supported with the new JupyterLab frontend [1] which blocks us from using it with our cluster.
But I am optimistic that there are alternative methods to handle auto grading of notebooks.
But it is a challenging task on many layers:
- How do you define your tests?
- How do execute the unit-tests without allowing the students to fake the results?
- How do you link Moodle Test to a Jupyter Notebook?
Currently, my time and the resources of the ITC are limited to continue the development of the JupyterHub cluster in this direction.
But in my opinion we should use the Jupyter mailinglist/user-group as a place to develop such a method [2].
Once we found a common approach for auto grading and the moodle integration we can approach the ITC to incorporate it into the cluster.
There is also an active group around nbgrader currently working on JupyterLab support.
Best regards
Steffen Vogel
PS: To all the new participants in this conversation: Please consider joining the Jupyter User Mailing list [2]
[1] https://github.com/jupyter/nbgrader/issues/1006
[2] https://lists.rwth-aachen.de/postorius/lists/jupyter.lists.rwth-aachen.de/
---
Steffen Vogel
Research Associate
Tel: +49 241 80-49577
Institute for Automation of Complex Power Systems
E.ON Energy Research Center
RWTH Aachen University
Mathieustr. 10
52074 Aachen - Germany
http://www.acs.eonerc.rwth-aachen.de/
From: "Persike, Malte" <persike(a)cls.rwth-aachen.de>
Date: Tuesday, 8. September 2020 at 08:53
To: "Pieschel, Lars" <lars.pieschel(a)rwth-aachen.de>, "jupyter(a)lists.rwth-aachen.de" <jupyter(a)lists.rwth-aachen.de>
Cc: "welten(a)dbis.rwth-aachen.de" <welten(a)dbis.rwth-aachen.de>, "gleim(a)dbis.rwth-aachen.de" <gleim(a)dbis.rwth-aachen.de>, "Schnurbusch(a)lists-new.rwth-aachen.de" <Schnurbusch(a)lists-new.rwth-aachen.de>
Subject: [Jupyter] Re: Automatic evaluation of jupyter notebooks
Dear all,
FYI: my colleague, Harald Schnurbusch, has attended to the issue of auto-grading Jupyter Notebooks as part of the Moodle integration of Jupyter. I took the liberty to forward him Lars Pieschel‘s mail.
Best wishes
Malte Persike
Von: Pieschel, Lars <lars.pieschel(a)rwth-aachen.de>
Gesendet: Freitag, 4. September 2020 18:31
An: jupyter(a)lists.rwth-aachen.de
Cc: welten(a)dbis.rwth-aachen.de; gleim(a)dbis.rwth-aachen.de
Betreff: [Jupyter] Automatic evaluation of jupyter notebooks
Hello everyone,
I was tasked by the chair I5 to digitalize the Semantic Web Lecture using Jupyter notebooks but have some questions regarding the evaluation of said notebooks.
We plan on using the RWTH JupyterHub as a substitute for Exercises and would like to automatically grade the jupyter notebooks with unittest on the output. Are there already recommended methods to do this or can you use something like nbgrader (https://nbgrader.readthedocs.io/en/stable/) on the RWTH JupyterHub?
Apart from that, are there ways to integrate the submission and/or grading of jupyter notebooks into RWTHmoodle?
The workflow should look like this:
The students get their assignments in form of a jupyter notebook with gaps they have to fill.
After they are done working on their notebooks, they can submit them for grading.
The notebook is evaluated using unittests.
Ideally the notebooks can only be submitted before a certain deadline and after a notebook gets graded, the grade is automatically transmitted to RWTHmoodle.
Best regards and thank you in advance,
Lars Pieschel
Hello everyone,
I was tasked by the chair I5 to digitalize the Semantic Web Lecture using Jupyter notebooks but have some questions regarding the evaluation of said notebooks.
We plan on using the RWTH JupyterHub as a substitute for Exercises and would like to automatically grade the jupyter notebooks with unittest on the output. Are there already recommended methods to do this or can you use something like nbgrader (https://nbgrader.readthedocs.io/en/stable/) on the RWTH JupyterHub?
Apart from that, are there ways to integrate the submission and/or grading of jupyter notebooks into RWTHmoodle?
The workflow should look like this:
1. The students get their assignments in form of a jupyter notebook with gaps they have to fill.
2. After they are done working on their notebooks, they can submit them for grading.
3. The notebook is evaluated using unittests.
Ideally the notebooks can only be submitted before a certain deadline and after a notebook gets graded, the grade is automatically transmitted to RWTHmoodle.
Best regards and thank you in advance,
Lars Pieschel
