Infer.NET user guide : Tutorials and examples

Student Skills

This example is a model of how particular skills allow students to answer questions correctly on a test. The model allows you to reason backward to determine what skills a student has by looking at which questions they answered correctly. This model was used in the paper “Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory” by Junker and Sijtsma (Applied Psychological Measurement 2001). You can run this example in the Examples Browser. The source code is in StudentSkills.cs.

There are two variants of the model, named DINA and NIDA. The example code runs DINA by default; you can change this to NIDA by changing the value of the variable useDina. The example code first generates random true skills for the students, then answers questions according to the student model, and finally uses Infer.NET to reason backward from the answers to estimates of the skills. The example prints out the percentage of correctly estimated skills and compares the estimated parameters of the model (pSkill, slip, and guess) to the true parameter values (used to sample the data).

In this model, there are multiple students who answer multiple questions, and each question requires multiple skills. The data consists of responses and skillsRequired. The responses matrix gives a boolean value for each student and question, indicating if the student’s answer was correct or not. The skillsRequired array lists the skills required for each question. Skills are numbered by integers, so this is represented by an array of integers for each question.

To model the responses, we hypothesize a hasSkills array that gives a boolean value for each student and skill, indicating whether the student has that skill. At this point, we could say that a student answers a question correctly if and only if they have the required skills. However, we also need to account for guessing and mistakes. The two variants (DINA and NIDA) model this differently. In the DINA version, if a student doesn’t have all the skills, then they must guess, and every student has the same guessing ability (regardless of how many other skills they have). In the NIDA version, a student who lacks a particular skill can still exhibit that skill by chance. Thus a student with more of the required skills is more likely to answer correctly. These two variants are not exclusive; in fact you could make a model that incorporates both kinds of guessing (at the skill level and the question level).