lang/en_utf8/help/workshop/comparisonofassessments.html

   1 <h1>Comparison of Assessments</h1>
   2
   3 <p>In a workshop it is common for the same piece of work to be assessed by the
   4     teacher and the students. If examples are used then the teacher first
   5     assesses them before the students assess a selection of them. The work from
   6     the students may well be assessed by the teacher, at least in part, and
   7     very possibly by a number of students. A workshop allows the teacher to award
   8     a proportion of the grade to the student's assessments, the remainder of
   9     the grade is allocated to the assessments of the work itself. (The
  10     proportions of the grade given to these two areas is set towards the end of
  11     the workshop.) A student's assessments are given a grade based on how well
  12     they match the corresponding assessments make by the teacher. (In the
  13     absence of a teacher assessment then the average of the peer assessments is
  14     used).</p>
  15
  16 <p>The degree of agreement between the student's and teacher's assessment is
  17     based on the differences between the scores in individual elements
  18     (actually the squared differences are used). The mean of these differences
  19     must to converted into a meaningful grade. The &quot;Comparison of
  20     Assessments&quot; option allows the teacher a degree of control on how
  21     these comparisons are converted into grades.</p>
  22
  23 <p>To get some idea on what effect this option has, take the (fairly simple)
  24     case of an assessment which has ten Yes/No questions. For example the
  25     assessment might use questions like "Is the chart correctly formatted?",
  26     "Is the calculated profit $100.66?", etc. Assume there are ten such
  27     questions. When the &quot;Very Lax&quot; setting is chosen, prefect
  28     agreement between the student's and teacher's assessment gives a grade of
  29     100%, if there is only one question which does not match the grade is 90%,
  30     two disagreements give a grade of 80%, three
  31     disagreements 70%, etc.. That might seem very reasonable and you might be thinking
  32     why is this option called a &quot;Very Lax&quot; comparison. Well, consider
  33     the case of a student doing a completely random assessment where the
  34     answers of the ten questions are simply
  35     guessed. On average this would result in five of the ten questions being
  36     matched. So the &quot;monkey's&quot; assessment would get a grade of around
  37     50%. The situation gets a little more sensible with the &quot;Lax&quot;
  38     option, then
  39     the random assessment gets around 20%. When the &quot;Fair&quot; option is
  40     chosen, random guessing will result in a zero grade most of the
  41     time. At this level, a grade of 50% is given when the two assessments agree
  42     on eight questions of the ten. If three questions are in disagreement then
  43     the grade given is 25%. When the option is set to &quot;Strict&quot; having
  44     two questions out of sync gives a grade of 40%. Moving into the &quot;Very
  45     Strict&quot; territory a disagreement in just two questions drops the grade to
  46     35% and having a single question in disagreement gives a grade of 65%.</p>
  47
  48 <p>This example is sightly artificial as most assessments usually have elements
  49     which have a range of values rather than just Yes or No. In those cases the
  50     comparison is likely to result in somewhat higher grades then the values
  51     indicated above. The various levels (Very Lax, Lax, Fair...) are given so
  52     that the teacher can fine tune the comparisons. If they feel that the grades
  53     being given for assessments are too low then this option should be moved
  54     towards the &quot;Lax&quot; or even &quot;Very Lax&quot; choices. And
  55     alternatively, if the grades for the student's assessments are, in general,
  56     felt to be too high this option should be moved to either the
  57     &quot;Strict&quot; or &quot;Very Strict&quot; choices. It is really a
  58     matter of trial and error with the best starting point being the
  59     &quot;Fair&quot; option.</p>
  60
  61 <p>During the course of the workshop the teacher may feel that the grades given
  62     to the student assessments are either too high or too low. These grades are
  63     shown on the exercise's Administration Page. In this case, the teacher can
  64     change the setting of this option and re-calculate the student assessment
  65     grades (the &quot;Grading Grades&quot;). The re-calculation is done by
  66     clicking the &quot;Re-grade Student Assessments&quot; link found on the
  67     administration page of the workshop. This can be safely performed at any
  68     time in the workshop.</p>
  69