Scalable Assessment Design Recipes: Concrete Examples for/from Math Courses

Posted by Martijn Boussé, on Oct. 16, 2023, 6:47 p.m.
Pedagogy
Assessment

Martijn Bousse

Faculty of Science and Engineering, Maastricht University, The Netherlands

 

Challenge and goal

Challenge: Rising student numbers put pressure on lecturers' workload affecting assessment quality and student learning. How can we make, both formative and summative, assessment scalable while maintaining quality and keeping the UM's vision on education and assessment in mind?

Goal: Implement scalable assessment design in our first year bachelor math courses (calculus and linear algebra).

Topic of the user case

Scalable assessment design recipes.

Local context (specific)

Our department has recently introduced a new bachelor, effectively doubling the number of students in the first year from 250 to 500 students. This increase was only partially expected and flanked by some additional staff. This relatively sudden increase has put enormous pressure on the staff's workload which in turn has effects on the assessment quality. For example, assessment are redesigned in such a way that the workload is decreased, e.g, extensive feedback moments are replaced by (often) less qualitatitative peer assessments, assignments are removed and a higher weight is put on the final exam, project presentations are shortened while team sizes have increased, grading rubrics are made simpler and smaller, etc. We do not yet know what the impact is on the constructive alignment on the course and curriculum level of all these changes.

Furthermore, the UM's vision on education revolves around four aspects: problem-based learning, small-scale teaching, societal relavance, and international classroom. Especially the first two aspects are under tremendous pressure by the student number increase. The vision is showing cracks due to scaling issues. Additionally, the UM's vision on assessment revolves around assessment of learning, assessment for learning, assessment as learning. The student number increase, again, puts a lot of pressure on the latter two modes and it's unclear as of yet how this impacts student learning.

The rising student numbers have (or will) put enormous stress, especially, on our first year math courses such as calculus, discrete mathematics, linear algebra, and numerical mathematics. 

Local CPD goals

Implement scalable assessment formats that align with the UM's vision on education and assessment for our first year math courses.

Needs defined in STEM-CPD Roadmap

Competences
3 choose an appropriate assessment method for their course
9 give prompt feedback and support students during learning
27 organize peer-assessment / peer-feedback in their courses
Attitudes
15 attend training for lecturers at the university.
8 discuss teaching with their colleagues.
Activities
2 attending presentations about teaching approaches.
16 getting just-in-time support on a specific teaching and learning issue.
18 giving workshops to other lecturers.

CPD activities at the local university

In this user case, we intend to organize a CPD workshop on scalable assessment design for fellow lecturers. We do this by giving a set of "recipes" with concrete examples.

We will also write an article for edUMinded to disseminate the ideas more widely within the university.

Teaching and learning materials

The teaching material comprise a slide deck with concrete assessment examples such a

  • Online Quiz System: We have developed a large databank of quiz questions in the online learning platform (Canvas) for calculus, multivariable calculus, and linear algebra that can be readily copied by other lecturers. The databanks uses randomization, parametrization, and diversification to provide meaningful formative assessment in the form of practice quizzes. The graded quizzes provide a useful way to keep students on track with their study progress. The quizzes are fully automatically graded and provide answer-dependent feedback. This approach requires only an initial set-up time by the lecturer and no further workload during the period.
  • Assignment design according to the principle "Sweat on the right back"
    • Create-your-own-assignment assignments
    • Easy-Grade Rubrics
    • Digital Grading
    • Peer feedback systems
  • "Smart" exam design and grading (ILO grading, student-generated question)

Sustainable implementation

We follow the ADDIE approach to implement this CPD activity:

  • Analysis: We perform a survey with the target lecturers to investigate their needs. The data will be analyzed and inform the design stage.
  • Design: We envision the workshop as a 1-hour onsite active and hands-on workshop with a follow-up meeting (blended approach: pre-task, workshop, mini project + support, follow-up discussion)
  • Development:
  • Implementation: We run the workshop at the departmental level (as a department CPD seminar or during the Research Team meeting). In later stages, it can be ran at faculty or university level.
  • Evaluation: Each workshop will be followed by a survey to improve the workshop further.

Expected impact of the CPD User Case

The assessment design recipes are topic-independent, but have been implemented by the workshop organizer in the context of (multivariable) calculus and linear algebra. Initially, we want to exploit these concrete examples specifically by organizing workshops for the coordinators of similar courses.

We hope to have an impact at several (increasing) levels:

  • Departmental level: Calculus and Linear Algebra is taught at 2 different programs within the department: Data Science and AI (200 students) and Computer Science (300 students). 
  • Faculty level: Calculus and Linear Algebra is taugt at the following programs by coordinators from our department: Maastricht Science Program (80 students), University College Maastricht (20 students), Circular Engineering (60 students), and Business Engineering (200 students).
  • University level: Calculus and Linear Algebra is also taught in various forms at other faculties. 

We will organize this workshop for the respective course coordinators at the departmental and faculty level to inspire them to use our design recipes. We will provide support throughout the course to help them with the implementation. At a later stage, we hope to organize the workshop as a CPD workshop via Edlab, our UM-wide teaching and learning center, so that we can reach the whole university.

Plans for eventual continuation of the CPD within the same topic

See above.

Developing confidence in sound course design, interactive teaching, and learning facilitation (type P1-1, P1-2, P1-3)

This CPD scenario describes a User case in which lecturers develop their competence in sound course design and teaching in higher education and organizing peer-feedback and collaborative learning and develop attitudes in how to stay motivated and self-regulate their continuous professional development and knowledge sharing.
The approximate duration of a User case that follows this scenario is several hours.
In this CPD scenario the participants professionalize in a close connection to their own teaching practice (at their workplace) and meet in person on location with the training staff and with other participants.

Learning environment
Workplace
Face-to-face
Time
Several hours

Creative Commons Licence Logo Creative Commons 3.0 BY SA applies to all content on Starfish.
Starfish-education support for the publishing on Starfish-education.eu does not constitute an endorsement of the contents, which reflect the views only of the authors and Starfish-education cannot be held responsible for any use which may be made of the information contained therein. Starfish-education cannot be held responsible for the content published by authors that is not conform with Creative Commons 3.0 BY SA.

See also