Setting Up Calculations for Assessments

Overview

Calculations are an advanced feature that let you create extra values or scores based on answers already given in an assessment.

Calculations are only available if the Enable Calculations checkbox in the Assessment Settings is selected.  As Brilliant Assessments calculates scores automatically, most assessments do not need calculations.  

Calculations allow you to:

• calculate answer values for reporting, for example; you may wish to calculate the return on Investment based on data collected in the assessment or
• to calculate a score based on previous input, for example, you may want to calculate a score based on a number entered into the assessment.  A score calculated in this way will be added to subsection, section, assessment, and potentially a segmentation score as if it was a standard question and answer.
• both answer values and scores can use previous response answers, scores at all levels (at that point), and literals as input to the calculation. 

Validations are available to inform, warn, or stop the user from entering invalid data.  They use much of the structure of calculations but are documented separately.  

How calculations work in assessments

Calculations can occur anywhere in an assessment but are hidden in the response.  The results of calculations can appear in subsequent sections, however, as text in a question or instructions, using question piping;  or they can be used to drive conditional questions.  The system treats them as a question and an answer so all of the feedback reporting options work with calculations - either as a calculated value or a score.

Calculations are always calculated when the responses to the section they are in is saved, either by hitting Next or Save Progress. For an answer value, you can specify whether the calculation is done before or after scoring for that page is calculated.  For score values, the system will always calculate the scores both before and after calculations.

Advanced functions and logic

Calculations use the power of Microsoft's Transact-SQL, meaning that as well as regular operators like +-*/ with or without brackets, advanced users can access the following functions:

• Abs (absolute - to get a positive value regardless of whether the input is positive or negative)
• Cast (used to convert a numeric value to a character string ore vise versa)
• DateDiff (used to find the difference between two dates) 
• GetDate (used to get today's date - often used in conjunction with a DateDiff function)

Advanced users can also use Case statements for logic. 

The functions are limited for security reasons but contact support if you would like a function to be considered for addition.

Setting up a calculation

This Video shows setting up the example explained in detail below:

Modify Calculation pop-up screen

Jump to:

• 6–8 Result Calculation area
• 9–12 Result output format
• 13–15 Score Calculation area
• 16–20 Other settings and documentation

1. Calculations are denoted in the assessment builder with an = before the Calculation name.
2. Calculation Name is used for identification purposes only.
3. Calculation Number is used for reference and is interchangeable with a question number.  For example, you can refer to a question number in a merge string in the feedback report to display the result of a calculation.
4. Calculation Type specifies whether this is a calculation or a Validation.    
5. Potential Points should only be specified if a score is being calculated.  The Potential Points will be treated in the same way as a Question's Potential Points.

Result Calculation area (6–8)

6. Result Calculation – Selectors 
   The selectors give you access to all components used to build a calculation. Each component appears as a tile you can insert into your formula. Selectors include:
   • Operators - Common mathematical and logical operators: + - * / = % ( ) < >
   • Functions - SQL style functions you can call within a calculation, such as: Abs(), Cast(), DateDiff(), GetDate() 
     Each function requires parameters depending on what you are calculating.
   • Logic - Build conditional logic using: Case, When, Then, Else, End, As. 
     Useful for branching calculations or assigning values based on thresholds.
   • Answer Text - Inserts values the respondent entered. 
     - For Text and Numeric answer types, the exact input is used. 
     - For Single Choice or MultiChoice, the Answer Report Name is used. 
     Note: If the Answer Report Name represents a numeric value, enter it without formatting (for example 300000 not $300,000).
   • Free Text - Enter literal values directly into a formula: 
     - Numbers as plain numeric, for example 10 or 0.25 
     - Text values inside single quotes, for example 'High'
   • Assessment Score - Access the overall assessment score as a percentage, returned as a decimal (for example 0.5 represents 50 percent).
   • Section Score - Choose a section and use its score as a percentage (returned as a decimal).
   • Subsection Score - Choose a subsection and use its score as a percentage (returned as a decimal).
   • Segmentation Score - Choose a segmentation and use its score as a percentage (returned as a decimal).
7. Result Calculation - Shows the calculation being built as separate tiles. Tips:
   • To start a new calculation first click the New Line button.
   • Use the New Line button for every new part of your statement. For example, each When in a Case Statement should be on a new line.  This doesn't affect the running of the calculation but makes it more readable, and makes the Line Copy more useful.
   • Copy the line when you have very similar lines. 
   • You can delete a tile by clicking the x in the top right of the tile. 
   • When you add a tile, it will always be placed at the end but you can drag and drop it to its correct position. 
   • If you have numerics that are user input, the Answer Type should be Numeric or Decimal rather than Text.
   • When scores are used in a calculation they are returned as a percentage.  e.g. 0.6 = 60%.  if you want the Likert equivalent you need to multiply it by 5 (assuming your Potential Points = 5). 
   • Case statements can be used to decide whether subsequent questions / subsections / sections should appear based on the current score.  
     For example, the answer to: "Case when{RatingScore.Score}>.60 then 'true' else 'false' end" could be used as a condition for subsequent questions to be asked. (I.e. we only ask further questions if they have scored 60% or more so far.  If the score is < 60% the answer will appear unselected for a condition.)
   • Support will help build more complex calculations for you.
8. Builder Buttons:
   • Test - tests the formula using inputs from the Test Response. (Tip: populate the Test Response with answers to facilitate this testing)
   • New Line - adds a New Line to the calculation. should be used for separate parts of the calculation.
   • Copy Last Line - helps build a calculation by copying the previous line.  You will normally need to delete some tiles, create their replacement, and drag the replacement into position.

Result output format (9–12)

9. Calculate after Scoring - You should select this checkbox if (and only if):
   • you have a score of any type as input to this calculation or
   • you have an answer as input that is output by another calculation that has a score as input. 
10. Result is Numeric - if the result should all be converted into a numeric value, select this checkbox.  Additional fields (Format and Number of decimals) will be displayed.
11. Format for numeric fields.  You can pick from General (no formatting), Thousand separated (with commas or as appropriate for the culture code of the Language of the assessment set in the assessment settings or en-US by default), Currency (again using the language), or percentage.  Note that if you want to use the output as a numeric in a subsequent calculation or validation, you must output it in the general format.
12. Number of Decimals this field is only displayed for numeric fields. You can specify the number of Decimals will be displayed.  The result of the calculation will be rounded to this number of decimals. 

Score Calculation area (13–15)

13. Score Calculation - Selectors - work in the same way as the Result Calculation selectors
14. Score Calculation - Follow the same rules and Tips as the Result Calculation.
15. Builder Buttons: 
    • Follow the same rules and Tips as the Result Calculation
    • Copy Result - copy the calculation from the Result Calculation.

Other settings and documentation (16–20)

16. Documentation - unlimited documentation of calculations can be entered.  They do not affect the calculation or appear anywhere but here.
17. Conditions - a calculation can be conditional like a question.  The system will treat it as unanswered and no calculations will be done.
18. Options - Segmentations - Calculations can be linked to segmentations in the same way as questions.  Only relevant when a score calculation is used.
19. Report Text. - can be used on the Feedback report when a score is calculated in the calculation. 

Troubleshooting calculations

As a system administrator, you can view the workings of Calculations from within a response, for diagnostic purposes.  The  Section menu (to the left of the Section name on the response header) provides access to the Calculations form.  

NOTE: Before displaying Calculations on the current page of the response, Hit Save Progress.  This will cause the system to do the calculations as part of the Save Process.  The same occurs when the user hits Next.

1. Calculation Name
2. Section / Subsection / Question Number of the calculation
3. The value calculated and the value in the format specified in the calculation form.  The value is calculated to the number of decimals specified, except if the format is a percentage; in which case it is calculated to an extra 2 decimals.
4. If a score is calculated the calculated score is shown here.  This value will be added to the answer scores for the subsection, and any Segmentations specified on the Segmentations tab( if used).  The Potential Points will be added to the same Question scores.
5. The calculations are shown how they are passed to SQL for calculation.
   • The system pre-resolves all of the merge strings - so you can see the current values.
   • SQL has a quirk that if it is only passed whole numbers it will always return a whole number.  For that reason, we multiply numbers by 1.00 to give the numbers decimal places.
   • If there is an error in the calculation an error message will be shown.  That error is from SQL and is usually quite cryptic.  Most errors are caused by unmatched brackets or missing operators.  Support can help you with these if you can not find them.