The Design Space of the Services of Knowledge Assessment Systems

The emergence of the knowledge based society as well as the rapid expansion of the number of students enrolled into higher education is an unmistakable international trend. As a consequence, the input necessary for delivering teaching materials is on to wane compare to that required for proper knowledge assessment. In addition, a usual deficiency of distance learning systems is the lack of any integrated knowledge assessment component. These were the reasons that triggered the appearance of Knowledge Assessment Systems (KAS) from the 1970's and onwards. KAS's may be considered from two distinct aspects: either focusing on the services they provide or based on their implementation. Our paper aims to provide a comparative demonstration of published KAS systems using the design space tree concept. Based on relevant design aspects as well as implementation alternatives, the design space of the services of knowledge assessment systems was defined. A projection of relevant design aspects and the possible alternatives then yielded a spanned design space of KAS services, in which the correct position of all implementations documented in the literature to date were identified. An analysis of the implemented systems in the design space allows for the demonstration of knowledge assessment system generations as well as the recognition of relevant KAS development trends. The highlights of these trends include the continuous growth of the complexity of evaluated question types as well as the level of assessment automation on the on hand, and - in terms of active question types - the fact that although several solutions are already available for assessing long essays, the systems analyzed here offer no solutions for short text evaluation.