Configuring Assessment Scoring

Scoring and Ratings are core concepts in Brilliant Assessments and are used to personalize feedback reports. Even when scores or ratings are hidden from respondents, they still determine how feedback text, charts, and results are generated.

Scores are calculated first, and Ratings are then derived from those scores. This article explains how scoring works, common configuration rules, and how ratings are applied in reports.

Scoring

Every Question has a Maximum Score, and every Available Answer has a score. Scores are automatically calculated at the following levels:

• Response
• Section
• Subsection
• Segmentation (if used)

Important notes:

• Questions with a Maximum Score of zero are ignored in all scoring and charts.
• For Multiple Choice (multiple selection) questions, answer scores are accumulated up to the question’s Maximum Score.
• Weighting questions by increasing answer scores is valid only for percentage scoring and does not apply to Likert scoring.

Viewing Score Calculations:

You can view exactly how the system calculates scores at each level.

As a System Administrator, when you view a response, a menu icon appears next to each section name. Selecting this menu and choosing Show Scores opens the Response Scores screen.

The Response Scores screen displays:

• Scores at the Response, Section, and Subsection levels
• Additional columns for each Advanced Rating Type, if configured (Read more about Advanced Ratings)
• An additional Segmentation section below Subsections, if Segmentations are in use (Read more about Segmentation)

Common Scoring Edge Cases

In most assessments, the default scoring behavior is sufficient. The scenarios below cover common edge cases and advanced configurations.

1- Unanswered questions

• If a question is not answered because it was conditional or skipped (and not required), its Maximum Score is excluded from calculations.
• This behavior can be overridden by assigning a default answer in the Scoring Options tab of the Answer dialog. The score for the answer selected as the default and the Potential Score for the question will be taken into the calculation. (Read more by going to the question dialogue and scrolling down to the Answer dialogue box section) 

2- “Not Applicable” answers

• If an answer such as Not Applicable should not affect scoring, enable Suppress Scoring in the Answer dialog.
• The Answer Points should be set to zero.

Read more by going to the question dialogue and scrolling down to the Answer dialogue box section

3- Totals vs averages

• By default, overall scores are based on totals, not averages. (i.e. the total of the answer scores divided by the total of the question potential scores)
• Sections with fewer questions contribute less to the overall score than sections with more questions.
• While this is considered by many to be more mathematically correct, you can counteract this if you want to, by setting the subsection weightings (see below).

4- Subsection weightings

• Subsection weightings can be used to control how much each subsection contributes to section and overall scores.
• If weightings are used, every subsection must have a weighting. Any subsection without a weighting is excluded from scoring.
• To calculate a true average, assign the same weighting to all subsections.

Read more about setting SubSection Weightings - see the Weighting tab.

Using weightings to calculate averages:

If you want the system to calculate the Section score or the overall Response score as the average of all subsections, assign the same weighting to every subsection.

The actual numeric value of the weighting (for example, 10) is not important. What matters is the proportion of each weighting relative to the total of all subsection weightings.

Using weightings to emphasize certain subsections:

If some subsections should have more influence on the score, assign them a proportionately larger weighting.

For example, a subsection with a weighting of 20 will contribute twice as much to the score as a subsection with a weighting of 10.

Where to manage subsection weightings

A convenient place to review or configure subsection weightings is the Advanced Ratings screen, which is accessed from the Ratings form. For the Rating Type you are using (typically Base), click Subsections. All subsections and their assigned weightings are displayed together in a single list.

5- Advanced scoring

• Using Calculations and Segmentations, you can implement custom scoring logic using your own formulas.
• Calculations have access to all of the assessment, section, subsection, and other segmentation scores.  You can then use your own formula to calculation a segmentation score which you can then report using all of the charting options and have Rating Text based on.

Ratings & Report

Ratings are used to band scores and simplify feedback reporting. Feedback text is typically selected based on the rating rather than the raw score.

Ratings are defined per assessment and may include any number of rating bands. Each rating applies up to a specified maximum score. The system selects the lowest rating band above the calculated score.

Ratings are calculated and available at:

• Response level
• Section level
• Subsection level
• Segmentation level (if applicable)

Feedback reports commonly include:

• Overall Rating text
• Subsection Rating text (the primary source of detailed feedback)
• Optional Section and Segmentation Rating text

Read more about Ratings 

Read more about Feedback Reports

You can read more about displaying scores here:  Displaying scores – Brilliant Assessments