Abstract: Context: Over the past 3 years, students in the first year common course: Engineering Modelling and Problem Solving, have engaged in collaborative discussions of engineering materials concepts through the use of an online tool called "MOOCchat". The MOOCchat tool is an online variation of the typical peer instruction protocol (Mazur, 1997) whereby students are given a multiple choice question to vote on, then are asked to discuss their answer with their peers, and then are given a revote. MOOCchats are used to facilitate online collaborative discussions that can reinforce each student's individual learning as they work through the weekly online modules.
Purpose: The aim was to investigate whether the use of the MOOCchat tool would facilitate student understanding of core weekly concepts. To substantiate this investigation, analysis of online discussion data and the development of a scale that could be used to classify chats in terms of the quality of student interactions were required.
Approach: The data from two years was used to investigate the effects of MOOCchats on short term learning by counting the numbers of students that shifted to from incorrect to correct answers as a result of the chat. The effects on long term learning were then evaluated by looking at the flow on effects of the MOOCchat shifts on exam marks. To create a scale for classifying chats, a two-step process was used. First, content analysis was performed on a subset of the 2014 chats so as to identify a set of meaningful categories that best described the data. These categories were then transformed into two simpler 'Quality of Chat' scales (Individual Interaction and Group Depth of Collaboration) that were given to the 2015 students to rate the quality of their interactions.
Results: MOOCchat experiences helped about one quarter of the students to shift from incorrect answers prechat to correct postchat. Groups who had at least one member who was correct prechat showed greater gains in the short term. In the long term, students who ended up with more correct answers postchat fared slightly better on the subsequent mid-term exam. This was most true when the concepts were easy or medium, while harder concepts showed fewer learning gains as a result of the chats. The content analysis resulted in a Depth of Collaboration scale with 3 levels: shallow, one-way (just telling or agreeing), and integrative interaction. The derived Quality of Chat scales showed that students rated themselves better than they rated their groups, and also showed benefits for groups whose collaborations were focused on understanding. However, these benefits were reduced when concepts were harder.
Conclusions: MOOCchat discussions are able to develop student understanding in an online blended learning course if the concept questions and the task goals are appropriately designed.
To cite this article: Reidsema, Carl A; Kavanagh, Lydia; Ollila, Emmi; Otte, Stephanie and McCredden, Julie E. Exploring the quality and effectiveness of online, focused peer discussions using the MOOCchat tool [online]. In: 27th Annual Conference of the Australasian Association for Engineering Education : AAEE 2016. Lismore, NSW: Southern Cross University, 2016: 552-564.
[cited 30 May 17].
Reidsema, Carl A; Kavanagh, Lydia; Ollila, Emmi; Otte, Stephanie; McCredden, Julie E;
Source: In: 27th Annual Conference of the Australasian Association for Engineering Education : AAEE 2016. Lismore, NSW: Southern Cross University, 2016: 552-564.
Document Type: Conference Paper
Engineering models; Peer teaching; Interaction analysis in education; MOOCs (Web-based instruction); Computer-assisted instruction--Evaluation; Computer-assisted instruction--Curricula;
(1) Faculty of Engineering Architecture and Information Technology, University of Queensland
(2) Faculty of Engineering Architecture and Information Technology, University of Queensland
(3) Faculty of Engineering Architecture and Information Technology, University of Queensland
(4) Faculty of Engineering Architecture and Information Technology, University of Queensland
(5) Faculty of Engineering Architecture and Information Technology, University of Queensland, email: email@example.com
Database: Engineering Collection