An important part of how most administrators of Brilliant Assessments build the feedback report is based on scores and ratings. Sometimes the score and rating are hidden from the respondent, but it still drives the process of personalizing the feedback report.
Ratings are based on Scores, so we will start with Scoring.
Every Question has a Potential Score and every Available Answer has a score. The score is calculated at each level (Assessment, Section, and Subsection (and Segmentation, if used)) by dividing the score for the answers selected by the potential score for the questions answered.
- The assessment score is based on the calculation above, it is not the average of the sections. Likewise, the section's score is not the average of subsections. This means that a section with fewer questions will affect the overall score less than a section with more questions. You can counteract this (if you want to) by weighting the questions in smaller sections (see below).
- You can weight the importance of certain questions by increasing the potential and available answer scores. You may do this for the reason in point 1, but the more common consideration is the relative importance of the question.
- If a question is not answered, which may be because
- the question was not asked because it was a conditional question
- the respondent skipped it
then the questions potential score will not be taken into the calculation. Normally, this behavior is what you would want, however, you can override this behavior by choosing a default answer for a question. This is a checkbox in the Advanced Scoring Tab of the Answer dialog in the Assessment Builder. (Read more about the Answer dialogue box) The score for the answer selected as the default and the Potential Score for the question will be taken into the calculation.
- If you have an available answer of "Not Applicable" you may want to stop the system using the Potential Score of the Question in the calculation. To do this select the Suppress Scoring checkbox in the Advanced Scoring Tab of the Answer dialog in the Assessment Builder. (Read more about the Answer dialogue box). The Answer Points for this answer should be set to zero.
If you have complex scoring requirements you can test how the system is calculating the score at each level. You would normally only use this feature while the system is being tested, and suppress it when you make the assessment available live.
- Select the Show Scores In Assessment checkbox in the Assessment Add/Edit (Read more about the Assessment Add/Edit)
- Select the Section Menu checkbox in the Settings Screen (Read more about the Settings Screen)
then a hamburger menu will appear alongside the section name in the response. Selecting that menu will allow Show Scores menu item to be selected, and the Response Scores screen displayed (below). Note that if Advanced Ratings are in use, any additional Rating types will also be shown as additional columns on the Response Scores screen below (Read more about Advanced Ratings). If Segmentations are in use they will also be shown as an additional section to the screen, under subSections. (Read more about Segmentation)
Ratings are used to band scores. Text on the feedback report is selected based on the rating rather that the score, which makes it more manageable.
Ratings are held separately for every assessment. You can have any number of ratings for an assessment. On each Rating, there is a maximum score that it applies to. The system finds the lowest rating band that is above the calculated score. This calculation is done at the Assessment, Section, Subsection and, if applicable, the segmentation level.
The feedback report typically shows
- Rating text, based on the overall rating, and
- Subsection Rating text, based on the subsections rating. This provides (along with answer text) the bulk of the detail of the assessment.