Using Assessor Functionality

Assessor Functionality Overview

Cohort-Level Assessor

• Purpose: Allows assessors to manage and assess groups of respondents collectively.
• Key Features:
  • Assessor can initiate cohorts, invite respondents, and monitor progress.
  • A landing page dashboard provides a quick snapshot of cohort results.
  • Using Results dashboards within their response, assessors can analyze cohort data, provide feedback, and add commentary/recommendations to cohort reports.
• Use Case Example: An assessor running a cohort, can monitor progress via a summary dashboard in the cohort landing page. Once they are ready to assess the cohort, they will enter the assessor response and fill out the assessor questions which are accompanied by summary dashboards in the assessment such as a stacked bar chart.  Using the dashboards as a reference, the assessor adds their insights and recommendations for input in the final report.

Single Respondent Assessor

Assessor functionality for individual respondents supports three main scenarios:

• Training Assessments: Assessor scores text-based answers (e.g., essays) and adds comments.
• Risk/IT/Cybersecurity Assessments: Assessors evaluate complex responses, provide feedback, and assign scores before the report is produced.
• Request for Information Assessments: Assessors may need to request additional details from respondents. Responses can be exchanged back and forth until satisfactory.
• Other Use Cases: Assessors can perform actions, such as validating or scoring responses, before generating the final report.

It is expected that assessor functionality will be used in other scenarios too when an assessment manager has to perform some action when a response is completed.

Setting up Assessors in the system

Step 1 - Assessor Permissions

• Assessors must have the Administrator or Assessment Manager role in the system.
• For individual response assessors that are managers in the system:
  • Assign assessors via the organization to set them as a default assessor and to control access and permissions.
  • Enable the Assessor checkbox on their user profile to designate them as an assessor.
  • Cohort assessors do not need to be designated as an assessor in the system.

Tip: If the assessor completes the entire assessment, marking them as an assessor is optional. Use the Assessor Only Answers checkbox on the assessment form for this scenario.

Step 2 - Assessment Settings

• Go to your Assessment Setting > Options > Tick the Open Response checkbox. (See Open Assessment below for more details
  
   

• In Assessment Setting > General > Show Completed:

  • This field will only be displayed if Open Response is selected.
  • You can select who will be given the option to indicate an open response is Completed. (An extra button is shown, Finish and Complete.)
  • In most cases, Assessor is selected. This means only Assessor can see and select the Complete button

  

Setting up your Assessment for an Individual Response Assessor

Create a Hidden Assessor Question: set up a single choice question which establishes whether the response is currently being answered by an assessor or a respondent (includes anyone who is not an assessor).  The system uses a Smart Question Short Name for this.

Usually, this would be a hidden question (indicated using a checkbox on the Options tab of the Question Dialogue box in the Assessment builder.

Link the question to Response Setting > Assessor

Tip: Test it is working before making the question hidden.

• Under the Question Options tab, the Question Report Name is set to "Assessor". 
• Under the Assessor Answer Options button, the answer Answer Report name is set to "Assessor".
• Under the Respondent Answer Options button the answer Answer Report name is set to "Respondent".

Make Questions Conditional:

All questions that are to be answered by the assessor will be conditional on the Assessor question answer and all respondent questions should be conditional on the Respondent answer.

Here is an example assessment:

1. This is the question that is all the respondent would initially see.
2. This technique would only be used for "Open" assessments (see below), and would only appear when the respondent reopened the assessment after an assessor had added comments.  It is a blank question with instructions using Answer Piping The merge sting is (in this example) colored red in the instructions.  The Question would be conditional on it being the respondent answering, and the assessor having entered something in the Assessor Comments. 
3. The score will be added to scores from other questions in the normal way.  Any scoring range can be used. The score question would be conditional on it being the assessor answering.
4. The comments can be used on the Feedback report as normal. This question would be conditional on it being the assessor answering.

Assigning Assessors to Responses

Any assessor can assess a response. When creating a response:

• You can select the assessor from a drop-down list in the response settings.
• If uploading responses via CSV, you can include the assessor's name in the upload template.
• If an organization is selected, the assessor will default from the organization table if an assessor is specified there.
• For shareable URL links, pass the organization code in the URL to assign the assessor automatically based on the organization.

Open Assessments

In open assessments, responses are not marked as completed immediately. Instead, they pass through various stages.

Setup 

To use Open Assessments with assessor functionality (see the Setup Assessor Steps above for more details):

• Enable the Open Assessment option in the assessment settings.
• Configure the Complete Button visibility (Assessor, Respondent, or both).
• Add Interim Report email type.

Assessment Stages (Assessor-only Complete Button)

When the Complete button is visible to the Assessor only, the response moves through the following stages:

Answered

• The Respondent completes all questions and clicks Finish.

• The response is saved but not completed.

• The response status changes to Answered.

• If an Interim Report email is configured, it is triggered at this point.

Assessed

• The Assessor reviews the respondent’s answers and provides feedback.

• The assessor clicks Finish to save progress without completing the response.

• The response remains open and the status changes to Assessed.

Completed

• The Assessor clicks Complete.

• The response status changes to Completed.

• This indicates that:

  • The respondent has finished answering, and

  • The assessor has completed their review.

• The completion report and completion email are triggered, if configured.

Interim Reports

In addition to the final completion report, Open Assessments can send Interim Reports.

• Interim Reports are triggered when the respondent finishes and the response enters the Answered stage.

• When configuring the Assessment Report, you specify who receives the Interim Report.

• Typically, Interim Reports are sent to the Assessor as a prompt to review the response.

• Optionally, Interim Reports can also be sent to the Respondent.

Setting Up a Cohort Level Assessor

Cohort assessors are set up differently than single respondent assessors. Here's how to configure them:

Note: adding Results Dashboards is optional. It is recommended that cohort assessors are system managers, however it's not required if they don't need access to the system.

Setting up a cohort assessor doesn’t require you to enable assessor functionality like you do for a single assessment.

1. Add an Assessor Role: Add a role to your Cohort settings called Assessor (or similar). Also, add the role to a role question linked to response setting.
   • Create an Assessor role in the cohort settings.
   • Link this role to the Role Question in the assessment builder response settings.
2. Include Assessor-Specific Questions:
   • Add questions for assessors to the assessment.
   • Make them conditional based on the assessor's role.
3. Update Reports:
   • Include merge strings for assessor feedback in cohort reports.
   • Filter the merge string by role to display the assessor-specific input.
4. Create Dashboards:
   • Landing Page Dashboard: Set up a summary dashboard for the cohort's landing page.Read more.
   • Assessor Dashboards: Add dashboards to assessor-specific questions as needed. Read more here.

How it works: 

• When creating a cohort, the assessor is also added as a respondent assigned the assessor role. When the cohort has hit a sufficient completion rate, the assessor will enter their response and complete their assessment.