MultiElo

About

Name lecturer

André Nusselder

student-assistant + Grassroots: Corneel den Hartogh

Scale

medium group (33 students)

Content

Innovation & Design Thinking

Situation before the intervention

Course consisted of 3 components

1. Individual Learning Journal with weekly reflections handed in at the end (40%)
2. Real-world Team Challenge in 'Design Thinking' from a client with a final report and presentation (40%)
3. Individual Essay about (parts of) mandatory literature handed in at the end (20%)

Problems:

• Students received no structural feedback on learning journal and some would write them all in the end, instead of on a weekly basis.
• Since there were part-time students, groups were often unable to meet outside of classes which made collaboration difficult and work fragmented (while 'design thinking' as method requires intensive collaboration)
• Individual essay was done at theend, students didn't read all the texts (they had no external incentive).During lectures times spent on the readings resulted in the teacherstalking a lot, while students were not really participating. Ideally, studentswould be more actively engaged with the readings (within and outside class).

Intervention

Two Digital Learning Environments were introduced

• FeedbackFruits (FbF) for general instructions, reading material and reflection reports
• ProjectCampus (PC) for the team challenge

This enabled the following changes:

• FbF provided the possibility to students to upload their journal weekly and for others to comment on this within the digital learning environment. A feedback-schedule was drafted to provide weekly feedback. In addition, the student-assistent provided after week 2 and 4 individual feedback to all journals. This ensured that the learning journal was a continuous process with gradual improvement.
• FbF provided a space for students to interact around the reading material, which left space during class time for group collaboration.
• The essay was replaced for a measure of 'online activity' which meant commenting on reading material, providing feedback to fellow students and the performance in 3 multiple choice quizzes on the literature (students had to hand in mc-questions weekly themselves).
• PC provided a place for online group collaboration. The clients that provided the challenge could enter this environment as well and provide feedback. Students could also view and respond to other groups. The staff could gain insight in the groupwork outside of class. This is especially important since a 'design thinking'-process is not as linear as the scientific process (see Figure 1 below).

Figure 1: Design Thinking Process. In Design Thinking the problem is not defined at the start, but the problem space is explored. In addition, there is no best solution, but a solution space. Figure from: Lindberg et al. (2011), Design Thinking: A fruitful concept for IT development?, http://citeseerx.ist.psu.edu/viewdoc/download?doi=...

Learning outcomes

• The weekly learning journal + feedback was a great success. Almost everyone regularly provided good feedback on their peers and responded well to feedback (of peers as well as student-assistant). This enabled students to really take steps in their personal thinking.
  • In addition, some students really entered modes of connective learning when discussions around learning journals arose (one journal entry got over 30 thought-through comments, some groups were constantly above 10).
• The interaction on FbF around the reading material was not as hoped. A quartile of the students were quite active, but most deemed it not worth the hassle. However, the time in class for group collaboration was used very well. Groups functioned better and were more actively engaged.
• The activity on PC was quite low. Since students gained a lot of time for groupwork in class, it might not be that necessary to gain more insight in their work outside of class. If a collaboration tool is nonetheless desired, a fluent integration with Google Drive is necessary. Almost all of the clients (responsible for the team challenges) preferred communication via mail / skype / face-to-face.
• The quizzes assured that students read the material (at least for the quizzes, still not always before the lecture about the topic). However, mc-quizzes do not really fit with the learning goals of this course. In addition, the measurement of online activity was difficult. Maybe a different format can be found to assess the understanding of literature of the students.

Figure 2: Collaboration in class. 'Design Thinking'-proponents hold that the creative solution lies in the intensive collaboration of the team. This teamwork doesn't only consist of meetings and paperwork, but 'thinking through making' is important as well. Since the use of the DLEs freed up class time for group work, students were enabled to spend time together and with materials in order to come to creative solutions. This improved their learning process.

Evaluation

A) Experience (satisfaction) of students

Full survey results available on request, the highlights:

Scale of 1-5:

• 3.6 for: 'Receiving feedback from peers on my learning journal and providing feedback on their learning journal in FeedbackFruits improved my learning journal'
• 4.2 for: 'I prefer FeedbackFruits above Blackboard as main digital learning environment '
• 4.2 for: 'The time we were given during class hours to work freely with our group was beneficial for our group collaboration

In addition, since ProjectCampus had not a strong integration with Google Drive students made a folder there (unfortunately out of sight of teachers and other students) and communicated in addition via WhatsApp. ProjectCampus was as a result not really used (since it was seen as an unnecessary second environment).

Students really appreciated the feedback on the learning journals, especially the feedback provided by student-assistant.

Student really didn't appreciated the fact that their online activity was 'graded'. Some even argued that they participated less due to the fact to this (however, whether no grading would have had resulted in more participation – or whether this was more a convenient excuse – remains the question.

B) Experience (satisfaction) of the lecturer about (re)design

The learning journals in new form were a real success. Students did them weekly and had a real reason to do so (since feedback was provided weekly by peers and regularly by the student-assistant). They improved their journals over the course. The ease of providing feedback on the contributions of fellow students is a big plus for FeedbackFruits.

FeedbackFruits was also from teachers' perspective an improvement in comparison with Blackboard. It was much more user-friendly / intuitive.

ProjectCampus is very easy to use and skillfully designed. However, a better integration with Google Drive would be necessary to make it effective for groupwork. Currently, it might be of better use for individual long-term project in which students could show a professor or fellow students their progress.

Grading is not an option in FeedbackFruits / ProjectCampus. However, this has recently become possible in Datanose itself.

In general we were quite happy with the experiment and the result.