Process-based Intended Learning Outcomes

Posted by Peter Ruijten-Dodoiu, on Oct. 8, 2024, 3 p.m.

Peter Ruijten-Dodoiu

Industrial Engineering and IInnovation Sciences, Eindhoven University of Technology, The Netherlands.

Challenge and goal

Educators face difficulties ensuring that students are adequately prepared for exams and other assessments. Many students begin their preparation too late, which results in uncertainty about expectations and poor performance. There is often a disconnect between what students perceive as sufficient preparation and what is actually required to succeed. This user case aims to help educators design course components that offer continuous opportunities for practice and feedback. By focusing on process-based ILOs, the user case will help educators in helping students engage regularly with course content and develop better study habits over time.

Local context (specific)

The overarching aim of this user case is to address the common challenge of student disengagement with course content. Many students delay engaging with materials until right before exams, leading to surface-level learning and higher failure rates. This user case aims to equip educators with strategies to enhance student engagement throughout the course by shifting from traditional, outcome-focused learning to process-based Intended Learning Outcomes (ILOs). Educators will learn to design ILOs that foster reflection, self-assessment, and interactive learning, ensuring a continuous and meaningful engagement with course content.

Local CPD goals (ILOs)

  • Educators are able to implement process-based ILOs in their courses
  • Educators are able to organize activities in their course to help students perform a good self-reflection
  • Educators are able to argue for the beneficial effects of introducing process-based ILOs in their courses

Background of the user case

Educators often experience drops in student engagement throughout the course, which can impact their preparation for exams and overall academic success. Various evidence-based strategies have been implemented to address this challenge:

  1. Interactive Lectures
    Interactive lectures include strategies such as using clickers, live polls, and structured discussions to keep students actively engaged. Research has shown that incorporating interactive elements into lectures significantly enhances students' understanding and retention by promoting active participation. For example, in large STEM classes, active learning leads to increases in examination performance (Freeman et al., 2014).

  2. Peer Instruction
    Peer instruction is a proven technique that leverages structured peer discussions to encourage students to articulate and refine their understanding of complex topics. This method has been extensively validated in STEM fields and promotes deep learning by having students explain concepts to each other, thus reinforcing their knowledge (Mazur, 1997a). Studies indicate that peer instruction improves learning outcomes and enhances students’ ability to approach challenging problems collaboratively (Mazur, 1997b).

  3. Formative Assessments
    Formative assessments, such as low-stakes quizzes, immediate feedback, and reflective activities, help students track their progress and identify areas for improvement early in the learning process. Meta-analyses of formative assessment practices indicate substantial improvements in student achievement when feedback is timely and actionable (Black & Wiliam, 1998). Karaman (2021) found that increasing the number of different types of formative assessment practices in the classrooms would promise a considerable contribution to student learning.

  4. Scaffolded Learning
    Scaffolded learning involves breaking down complex topics into smaller, more manageable chunks, providing support at each stage. Simons and Klein (2007) found that scaffolds may enhance inquiry and performance, especially when students are required to access and use them. Scaffolding supports students’ cognitive development and guides them towards independent learning over time.

Most of these approaches are tailered to specific student populations, specific courses, or specific levels of students. In this user case, I will propose a method that can be applied to any student population, level, or discipline.

References

  • Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74.
  • Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active Learning Increases Student Performance in Science, Engineering, and Mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415.
  • Karaman, P. (2021). The effect of formative assessment practices on student learning: A meta-analysis study. International journal of assessment tools in éducation, 8(4), 801-817.
  • Mazur, E. (1997a). Peer Instruction: A User's Manual. Upper Saddle River: Prentice Hall.
  • Mazur, E. (1997b, March). Peer instruction: Getting students to think in class. In AIP conference proceedings (pp. 981-988).
  • Simons, K. D., & Klein, J. D. (2007). The impact of scaffolding and student achievement levels in a problem-based learning environment. Instructional science, 35, 41-72.

Needs defined in STEM-CPD Roadmap

Competences
4 engage students and arouse interest for the discipline in the class
8 develop critical thinking by students
Attitudes
5 be interested in their students' progress.
1 be reflective teachers and reflect about their courses / lectures.
Activities
5 following online courses / MOOC about teaching and learning.

Overview of activities

Activities within this user case will be centered around developing educators’ ability to create process-based ILOs, which provide students with clear, continuous assessment criteria. The activities are organized based on the flipped classroom approach, see picture below.

 

 

CPD activities (before)

  • Tutorial on Process-Based ILO Design: Educators will learn to structure ILOs towards a process-level (self-)assessment, while ensuring alignment between course activities and final assessments.
  • Tutorial on Designing Self-reflection Activities: Educators will learn to design course activities that encourage students to reflect on their personal development and growth on the ILOs of the course.
  • Tutorial on Teaching How To Self-reflect: A core component of courses that incoporate the process-based ILO design is an activity in which students learn how to do a proper self-reflection on personal growth and development. This tutorial shows the different ways in which that activity can be designed and executed.

CPD activities (during)

  • Presentation of the Benefits of Process-based ILOs: A short presentation in which I share my experiences and an evidence-based approach to process-based ILOs.
  • Discussion on Ways to Implement: We will discuss the possibilities for implementing process-based ILOs in your course(s). This includes the set-up of the learning objectives (including the process-based rubric), activities for students, assessment types, and evaluation moments and activities.

CPD activities (after)

  • Implementing Process-Based ILOs in your Course: After meeting and discussing, you will start implementing the ideas we discussed in your course(s). We will stay in touch about how things are going, and plan a final evaluation after the course has finished.

Teaching and learning materials

Additional to the tutorials, a checklist will be created that assist educators in making preparations for their course (following constructive alignment components).

Evaluation

Evaluation will be conducted through a combination of quantitative measures and qualitative feedback to assess both student preparedness and educator confidence in implementing the new ILO framework. Data collection points will include:

  • Student Performance Metrics: Compare exam results before and after the user case implementation to identify changes in student understanding and preparedness.
  • Teacher Self-Assessment: Reflective journals and structured interviews will be used to capture teacher perceptions of their development and any changes in teaching practices.
  • Course Review Surveys: Regular surveys of course content, student engagement, and alignment between ILOs and assessments.

Expected impact of the CPD User Case

The introduction of process-based Intended Learning Outcomes (ILOs) and reflective practices is expected to have a significant impact on both student performance and teaching practices. The anticipated outcomes include:

  1. Improved Exam and Assessment Preparation
    With process-based ILOs, students will be better prepared for exams and other assessments, reducing the reliance on last-minute cramming. This approach ensures that students build a deeper understanding of the material throughout the course, which leads to improved academic performance and greater confidence during assessments.

  2. Enhanced Reflective Skills and Self-Regulation
    Through structured reflective practices and formative assessments, students will develop the ability to self-assess their progress, identify gaps in their knowledge, and take proactive steps to address these gaps before the final evaluation. This aligns with other user cases where incorporating reflection has shown to improve students’ metacognitive skills and their ability to self-regulate learning processes.

  3. Strengthened Teacher Competencies in Course Design
    Educators will gain confidence in designing and implementing process-based ILOs that emphasize continuous assessment and feedback. This approach will encourage teachers to integrate interactive and reflective components in their teaching, resulting in courses that are more aligned with students’ developmental needs.

  4. Increased Student Engagement and Motivation
    Although the focus is on assessment preparation, a byproduct is likely to be higher student engagement due to the increased relevance and interactivity of course content. This is consistent with similar user cases where active learning strategies and well-structured formative assessments have led to sustained student motivation throughout the course.

  5. Broader Adoption of Best Practices Across Programs
    This user case envisions a gradual implementation, starting with a pilot in selected courses and expanding across all BSc and MSc programs at TU/e. This strategic scaling will create a community of practice, where educators share their experiences and collectively refine the integration of process-based ILOs.

Developing confidence in learning facilitation (type P1-3)

This CPD scenario describes a User case in which lecturers develop their competence in how to engage and motivate students and how to facilitate discipline specific thinking and how to facilitate student’s deep learning and development of higher cognitive skills and develop attitudes in supporting student development and enabling students’ well-being in a learning process and inclusivity and reflecting on own teaching practice.
The approximate duration of a User case that follows this scenario is several days.
In this CPD scenario the participants are using a very short open online course, a micro mooc (μmooc).

Competences
Engagement and motivation, facilitation discipline specific learning
Deep learning
Attitudes
Pastoral interest
Reflection
Activities
Learning facilitation (person-centered)
Learning environment
μMOOCs
Time
Several days
Creative Commons Licence Logo Creative Commons 3.0 BY SA applies to all content on Starfish.
Starfish-education support for the publishing on Starfish-education.eu does not constitute an endorsement of the contents, which reflect the views only of the authors and Starfish-education cannot be held responsible for any use which may be made of the information contained therein. Starfish-education cannot be held responsible for the content published by authors that is not conform with Creative Commons 3.0 BY SA.