Abstract: Context: Many methods have been proposed in the study of linguistics for the representation of words and sentences. Most classical methods are symbolic and consist in things like dictionaries, thesauri, ontologies and syntax trees. Another approach is to represent words and sentences via the use of high dimensional vectors, which capture the distributional statistics of words and sentences. One application of representing words as vectors is to automatically evaluate text, which can further be applied to the assessment of students' text-based answers.
Purpose: This study investigated approaches to automatically analyse student responses to questions in the signal processing domain.
Approach: We investigated vector analysis approaches to capture various semantic and syntactic features of words, such that these representations can be compared and scored in a graded fashion, as distinct to simply true/false or same/different. The approaches used in this study can be trained in a semisupervised fashion, where minimal human input is typically required.
Results: The data investigated in this study consisted of student responses to short-answer questions in text form with associated metadata indicating the correctness for answers. Difficulties encountered when automatically assessing student short answers, either for correctness or knowledge gaps, were a) variations in vocabulary b) variations in grammatical structures c) precisely determining when specific concepts occur and don't occur, and d) relevant concept modifiers that may alter the assessment of the short answer. One element - important for addressing these difficulties - is how words and sentences are represented in short-answer question responses.
Conclusions: The study described in this paper focused on vector space representations for text. We recommend the development an agile methodology to be employed so that regular outputs be produced and sent for comment, which can then be used to inform further work. We suggest the best approach is to make use of a combination of methods including the many classical Natural Language Processing (NLP) techniques such as part of speech (POS) tagging, and phrase chunking.
To cite this article: Goncher, Andrea M and Boles, Wageeh. Investigating the use of vector analysis to assess students' understanding [online]. In: 27th Annual Conference of the Australasian Association for Engineering Education : AAEE 2016. Lismore, NSW: Southern Cross University, 2016: 292-299.
[cited 26 Apr 17].