An important part of most feedback reports is based on scores and ratings. Sometimes the score and rating are hidden from the respondent, but it still drives the process of personalizing the feedback report.
Ratings are based on Scores, so we will start with Scoring.
Scoring
Every Question has a Potential Score and every Available Answer has a score. The score is calculated at each level (Response, Section, and Subsection (and Segmentation, if used)) in two different ways:
- by dividing the total score for the answers selected by the total potential score for the questions answered to calculate the percentage score.
- by dividing the total score for the answers selected by the number of questions answered to calculate the average score (often referred to as Likert scoring).
Pro Tips:
- You can weight the importance of certain questions by increasing the potential and available answer scores reflecting the relative importance of the question.
- If a question is not answered, which may be because
- the question was not asked because it was a conditional question
- the respondent skipped it (and it wasn't required)
then the question's potential score will not be taken into the calculation. Normally, this behavior is what you would want, however, you can override this behavior by choosing a default answer for a question. This is a checkbox in the Advanced Scoring Tab of the Answer dialog in the Assessment Builder. (Read more about the Answer dialogue box) The score for the answer selected as the default and the Potential Score for the question will be taken into the calculation.
- If you have an available answer of "Not Applicable" you may want to stop the system using the Potential Score of the Question in the calculation. To do this select the Suppress Scoring checkbox in the Advanced Scoring Tab of the Answer dialog in the Assessment Builder. (Read more about the Answer dialogue box). The Answer Points for this answer should be set to zero.
- By default, the response score is based on the total of the calculations above (i.e. the total of the answer scores divided by the total of the question potential scores.) It is not the average of the sections. Likewise, the section's score is not the average of subsections. This means that a section with fewer questions will affect the overall score less than a section with more questions. While this is considered by many to be more mathematically correct, you can counteract this if you want to, by setting the subsection weightings (see below).
- You can set subsection weightings to specify the weight of each subsection in the calculation of the section score and the overall response score. If this is done at all you must specify a weighting for every subsection, if any subsection is not specified it will be given a zero weighting and not included in the calculation at all. (Read more about setting SubSection Weightings - see the Weighting tab.)
If you want the system to calculate the section or the overall response score as the average of all subsections, then specify the same weighting on every subsection. (The weighting itself (eg 10) is irrelevant, it is the proportion that weighting is of the total of all of the weightings specified.
If you want some subsections to be given more weight in the calculation, specify a proportionately larger factor - ie a subsection with a weighting of 20 will be given twice the weighting of a subsection with a weighting of 10.
A good place to check or set up weightings is the Advanced Ratings form (accessed from the Ratings form). For the Rating Type you are using (normally Base), click Subsections. All of the Subsections and Weightings are shown in a single list.
- Using calculations and segmentations you can calculate scores using your own methodology. Calculations have access to all of the assessment, section, subsection, and other segmentation scores. You can then use your own formula to calculation a segmentation score which you can then report using all of the charting options and have Rating Text based on.
- You can check how the system is calculating the score at each level. As a System Administrator, when you view a response a hamburger menu will appear alongside the section name in the response. Selecting that menu will allow the Show Scores menu item to be selected, and the Response Scores screen displayed (below). Note that if Advanced Ratings are in use, any additional Rating types will also be shown as additional columns on the Response Scores screen below (Read more about Advanced Ratings). If Segmentations are in use they will also be shown as an additional section to the screen, under subSections. (Read more about Segmentation)
Ratings
Ratings are used to band scores. Text on the feedback report is selected based on the rating rather than the score, which makes it more manageable.
Ratings are held separately for every assessment. You can have any number of ratings for an assessment. On each Rating, there is a maximum score that it applies to. The system finds the lowest rating band that is above the calculated score. This calculation is done at the Response, Section, Subsection and, if applicable, the segmentation level.
The feedback report typically shows
- Rating text, based on the overall rating, and
- Subsection Rating text, based on the subsections rating. This provides (along with answer text) the bulk of the detail of the assessment.
- Segmentation and Section Rating Text are also available.
Read more about the Feedback Report
Comments
0 comments
Please sign in to leave a comment.