Knowledge structures and their applications in CALL systems

Thepaperdescribestwoapproaches towardsknowledgeassessment which canbe appliedwithin any CALL application. The first approachusestest itemsto definea universeof empiricalknowledgestates.We show that the relationsbetweenitems(‘areaboutequal’,‘is harderthan’)andsubjects(‘are aboutequal’,‘is betterthan’)canbeeasilycomputedwithin aCALL system, andthat theempiricalresultsgive insight into thestructureof problemsand their solutions. Given a standardizedtest procedure,a CALL systemcan usethis methodto describetheknowledgestateof a testedsubjecteitherusing a referencepopulation,a knowledgestructuregeneratedby experts(see below), or by computinga knowledgestructureusingthegivensample. The secondapproachis basedon queryingfour expertsaboutthe skills minimally necessaryto solve problemsof the item setof thefirst approach. The resultsof thesequeriescanbe transformedinto theoreticalknowledge structures,and the empiricaldatacanbe usedto test the predictionof the experts.Theresultsshow that Theexpertsdisagreeremarkably, Mostof theexpertsdid notexpressevensimplerelationshipswhichare observableby theempiricaldataanalysis, Situationalfactorssuchastheappearanceof learningtasksin thesame lectureareobservablein the empiricaldataaswell asin theopinions of theexperts. Equalauthorshipimplied 1 Structural Background and Tools We considera set of subjectsand a set of problemsas well as a relation suchthat if andonly if subject solvesproblem (1) Theset ! " # $ &% is calledthe (knowledge) state of subject s, andthecollection ' ( ) * + $%-, /. % is calleda knowledge structure . Eachknowledgestructure' is partially orderedby setinclusion . We cansay thataproblem is a prerequisite of problem 0 if every statewhichcontains0 also contains ; in this sense,we alsocall 0 harder than . In otherwords, 0 is harder than , if thereis no subjectwhich solvesproblem 0 but fails to solve problem . Similarly, we cansaythatsubject1 is better thansubject2 , if 354&67358 . A consolidatedline diagramof thesetwo partial orderssuchasFigures1 or 2 can be obtainedvia the formal concept analysis approachpresentedin Wille (1982). Supposethat 3 is a setof skills. A function 9 : <; =?>A@ is calleda skill function , if 9CB 0 D is anonemptysetof nonempty, pairwiseincomparablesubsetsof 3 for each0 . We interprettheelementsof 9CB 0 D asexactly thosesetsof skills which areminimally sufficient to solve problem 0 ; in otherwords,eachE 9CB 0 D is a setof skills sufficient to solve 0 , while eachpropersubsetof E is not. If 9CB 0 D containsonly oneelementE , theexpertstatesthateachstrategy to solve problem 0 mustcontaintheskills whicharespecifiedby E . If 9FB 0 D containsG subsetsof 3 , thereare G essentiallydifferentstrategiesto solveproblem0 . In Düntsch& Gediga (1995)we have shown that eachskill function uniquelydeterminesa knowledge structureon . In otherwords,anexpertwhospecifiesaskill functionat thesame time builds aknowledgestructure. Knowledgeassessment via knowledgestructuresandskill functionshasbeen usedsuccessfullyin otherareas(seeFalmagneet al. (1990);Düntsch& Gediga (1995)),andbelow we describean applicationwithin the context of a language learningenvironment. 2 Empirical Knowledge Structures