Test

Summary

Test is a quiz and exam tool with a wide range of uses from informal knowledge checks to summative exams.

This guide covers how to use and set up a Test, and is primarily aimed at teaching staff.

Relevant VLE site design principles

• 3.4 Essential: Site and materials content is accessible.
• 4.1 Essential: The assessment section contains all information about module assessments.
• 4.2 Essential: Assessment instructions are clearly labelled and explain the task and requirements.

When to use Test

Test has a lot of flexible features, which makes it useful in many situations, from short practice quizzes to formal summative exams:

• Wide range of question types: most with automatic marking and feedback
• Multiple ways to add questions: manually create questions, import questions from file or reuse questions from existing banks or tests
• Randomisation: randomise the order of questions and/or answer options presented in each Test, or use calculated formula/numeric questions to automatically generate a large bank of different questions
• Question banks and pools: present a random selection of questions for each student
• Flexible grading possibilities: can set up partial, negative and extra credit

Knowledge checkPractice quizFormal exam

A short, one-off self-assessment.

Some examples:

• self assessment/knowledge check: a few questions to check understanding of weekly content. You can do this using Test or by adding questions directly within a Document.
• track completion of required content (eg. safety materials): honesty-based True/false question to self-declare completion.
• demonstrate preparation for workshops or practicals: a few questions on preparatory content or True/False to self-declare completion.

Test completion can also be used as a criteria for setting Release Conditions for other items. For example, access to laboratory session materials could be restricted until students have successfully completed a preparatory quiz.

Common features:

• Tend to be short: usually 5 or fewer questions
• Question display: usually all possible questions
• Question types: all questions should be automatically marked so no input is needed from teaching staff
• Set up effort: low, usually quick and simple to set up
• Insights to inform teaching: easy to gauge engagement and identify learning gaps

An informal quiz that can be taken multiple times.

Some examples:

• practice quizzes: providing an active opportunity to practice 'right answer' content (eg. match terminology, complete calculations)
• revision quizzes covering part of or all module content

Common features:

• Question display: to allow students to retake the quiz, it's common for practice quizzes to use question pools to display a random subset of questions drawn from a larger question bank. However, it is also possible to display a fixed set of questions each time.
• Question types: all questions should be automatically marked so no input is needed from teaching staff
• Set up effort: low to medium, depending on how questions are created and whether random subsets are used.
• Insights to inform teaching: easy to gauge engagement and identify learning gaps, question analysis can help you monitor the efficacy of your question design

Formal summative or formative exams

Some examples:

• Remote exam: students begin the exam during a given time window (eg. 10:00 - 10:30) at a place of their choosing. They then have the given time limit (or longer if accommodations are set) to complete the exam. Can't be invigilated.
• Scheduled on-campus exam: students complete the exam at a set time in an on-campus computer lab. Can be invigilated, but computer access to other tools can't be locked down. We will advise on set up, but the department must take full responsibility for delivering these exams. For example, it's not possible for central technical support staff to attend in the room.
• Formative/mock exam: to familiarise students with the online exam process.

Common features:

• Question display: to support robust assessment, formal exams generally use question pools to display a random subset of questions drawn from a larger question bank.
• Question types: preferably automatically marked (to allow non-anonymous workflows, which are significantly easier to administer), but can include manually marked questions.
• Set up effort: medium to high, as requires very careful set up and checking.

Warning

If you are setting up a summative Test, also review this more detailed guide:
Staff Help: Considerations Around & Setting up a Summative VLE Test

You must contact the Digital Education Team well in advance if you want to run a summative and/or synchronous exam (in person or online) using Ultra Test.

Find out more about how Test has been applied across the University:

Case study: Ultra Test for formative and summative assessment [Computer Science]

Tommy Yuan shares his experiences of using the Ultra test tool for formative and summative assessment. Topics include how it can save time for lecturers and administrators, and reduce the likelihood of collusion and academic misconduct.

Watch their presentation:

VLE Test for assessment (Panopto viewer) (6 mins 59 secs, UoY log-in required)

See the full case study for more details and the transcript. You can also browse our full set of case studies.

Case study: Ultra Test for practice quizzes [Language & Linguistic Science]

Ellie Rye provides a walkthrough of the 'Structure of English' Ultra site, describing how they applied the Ultra template to present teaching content, and reflects on the use of Discussions and Tests for formative practice quizzes.

Watch their presentation:

Structure of English (Panopto viewer) (8 mins 21 secs, UoY log-in required)

See the full case study for more details and the transcript. You can also browse our full set of case studies.

Question types

Question type	Description	Grading type
Multiple Choice	Pick the correct answer(s) from options given. Option order is randomised. Can give partial or negative credit.	auto graded
Fill in the Blank	Input the missing word(s) in the given text. Set if answers should be exact, match part of a specified answer or match a pattern.	auto graded
Matching	Match corresponding items from two groups. Options can be fixed or randomised. Can give partial or negative credit.	auto graded
True/False	Choose True or False in response to a given statement.	auto graded
Calculated Formula	Calculate the answer to a given formula (eg. 3x + 4y = ?). Values (x/y) are randomly generated so each student has a different question.	auto graded
Calculated Numeric	Similar to Fill in the Blank questions, but for numeric answers. Can set the answer as an extact number or within a range.	auto graded
Hotspot	Drop pin(s) on an image. Consider accessibility carefully.	auto graded
Essay	Type a response (of any length) in the answer box. Can provide a model answer for feedback.	manually graded

Accessible Test Content

As with all teaching content, accessibility is very important when building test questions and answer options. All the usual considerations around accessibility apply to tests, but it is particularly important that you take into consideration accessiblility when using tables, images or mathematical content in test questions.

• Guidance on creating accessible images, table and maths can be found on our "Ultra Accessibility" VLE page. (Don't have access? Contact us).
• Examples of quality alternative text on graphs, diagrams and other complex images can be found here.

Create a Test

Knowledge checkPractice quizFormal exam

Location

Knowledge checks are best located alongside the relevant materials, usually within a weekly content section.

Location

Locate the quiz where it will make most sense to students. This could be:

• practising specific content: alongside weekly materials.
• general revision: in the Assessment section (make sure to label clearly as a formative practice quiz to avoid confusion with formal assessment items)

You can use a Course Link to show a quiz in more than one location.

Location

Consider the location of the exam and also information about how to take the exam.

• Exam itself: usually created in a separate exam site to aid set up and for exam security.
• Details of the exam: include in the Assessment section of your module site.

1. Hover where you want the Test to appear, click the plus icon, then Create, then select Test.
2. Enter a descriptive name at the top left.
3. Click the plus icon to add questions (see Test questions section below).
4. Set the Due date and adjust other settings as needed (see Test settings section below).
5. Once confident that the Test is ready, set it as Visible to students or specify Release conditions in the top right (see our guide to Content visibility for more detail).

Tip

You can build and trial the Test in your personal Ultra sandpit site, and when it is ready use the Copy Content tool to add it to your module/exam site in the relevant location.

Add Test questions

There are various ways to add questions to a Test. Which method is most appropriate depends on the amount of questions to add, whether to display all questions or a random subset, and whether questions have already been added elsewhere in the site.

Knowledge checkPractice quizFormal exam

Method to add questions

As knowledge checks have a small number of questions that are all displayed to students, it's likely easiest to manually add questions to your Test.

Method to add questions

If all questions are displayed:

• small number of new questions: manually add questions
• large number of new questions: upload questions in a .tsv file
• use questions from another Test or Question Bank: re-use questions

If subset(s) of possible questions are displayed:

• add questions to a Question Bank or another Test and then set up a Question Pool(s) in this Test

Method to add questions

If all questions are displayed:

• small number of new questions: manually add questions
• large number of new questions: upload questions in a .tsv file
• use questions from another Test or Question Bank: re-use questions

If subset(s) of possible questions are displayed:

• add questions to a Question Bank or another Test and then set up a Question Pool(s) in this Test

Manually add questions

You may find it helpful to draft your questions in another document first.

1. Click the plus + icon.
2. Select the relevant question type.
   
3. Enter the question and answers as needed for that question type (see the linked guides in the Question Types section for more information)
4. Optional question settings (availability depends on question type):
   • set partial or negative credit for different answers
   • set the question as extra credit
   • add automated feedback
   • change the points awarded (default = 1 point)
     
5. Click Save.
6. Repeat for all questions.

Upload questions from file

You can draft questions in a spreadsheet and upload them in .tsv format to your Test.

Prepare the file

1. Make a copy of the Ultra tsv template for Tests Google Sheet
2. Enter your questions by editing the BB test tab (contains examples of the formatting required for each question type).
   
3. Download the questions in .tsv format: File > Download > Tab-separated values (.tsv)

Upload the file

1. Return to the Test and click the plus + icon.
2. Select Upload questions from file.
   
3. Select your .tsv file.
4. Once the upload is processed, review the status message for any errors.

For more details and examples of the required file format, see Blackboard's guide to uploading questions.

Optional question settings: these can't be specified in the .tsv file, so first upload your file and then manually update each question.

Reuse questions

Copy questions that already appear in another Test or Question Bank in the site. This creates copies of questions, so any edits made to re-used questions are not updated in the original question.

Tip

Reusing questions will display all of the selected questions in the Test. If you want to display only a subset (eg. 2 of 10 possible questions), use a Question Pool instead.

1. Click the plus + icon.
2. Select Reuse questions.
   
3. Select the questions to copy, using the filter options if needed: search by keyword, browse by source (Tests and Question Banks), browse by question type.
4. Click Copy questions.
5. Once the copy is processed, review the status message for any errors.

Question pools

Add a random subset of questions that already appear in another Test or Question Bank in the site. This is useful for creating robust assessments and also for practice quizzes that students may take multiple times.

Question pools do not copy questions; any edits made to questions in a pool will appear everywhere that question is used. Deleting a question from a Question pool does not delete the question in other locations.

Warning

For fairness and to create a valid assessment, all questions in the pool must be of equivalent difficulty. To include questions at different levels or points values, use multiple pools.

1. Click the plus + icon.
2. Select Add question pool.
   
3. Select the questions to add, using the filter options if needed: search by keyword, browse by source (Tests and Question Banks), browse by question type.
4. Click Add questions.
5. Enter the number of questions to display and optionally update the points awarded per question. Click Save.
   
6. In edit mode a summary of the pool is shown where you can view the questions and edit settings. Students will see the questions pulled from the pool.
   

For more details, see the BlackBoard Help guide to Question pools.

Use Question Pools in Assessments in the Ultra Course View [YouTube]

Test Settings

There are various settings possible for Tests, including;

• due date and attempt management
• randomising the order of pages, questions and answers
• how marks and feedback are presented to students

Tip

Randomising questions will display all questions in the Test in a random order. If you want to randomly display a subset of questions, use a Question Pool.

Edit settings in the Assessment settings panel:

• Click the cog icon to open the full Test settings
• The settings summary gives quick access to some settings: Due date, Mark category, Marking, Attempts allowed, Originality Report

Knowledge checkPractice quizFormal exam

Suggested settings

• Details & Information
  • tick No due date
  • leave other options unticked
• Presentation Options
  • if using LaTeX, tick Display one question at a time for more consistent rendering
  • leave other options unticked
• Formative Tools
  • tick Formative assessment
  • leave Display formative label to students ticked
• Marking & Submissions
  • Mark category: leave as Test or change to Quiz (this will change the icon displayed in the Course Content area)
  • Attempts allowed: set to Unlimited
  • Assessment mark: leave Post assessment marks automatically ticked
  • leave other options unticked
• Assessment results: no action needed
• Assessment security: no action needed
• Additional Tools: no action needed
• Description: add an optional short description to display under the item's name in the Course Content area.

Suggested settings

• Details & Information
  • tick No due date
  • leave other options unticked
• Presentation Options
  • if no randomisation needed: leave all unticked
  • if randomisation is needed: tick Randomise questions, Randomise answers or Randomise pages as desired
  • if using LaTeX, tick Display one question at a time for more consistent rendering
• Formative Tools
  • tick Formative assessment
  • leave Display formative label to students ticked
• Marking & Submissions
  • Mark category: leave as Test or change to Quiz (this will change the icon displayed in the Course Content area)
  • Attempts allowed: set to Unlimited
  • Assessment mark: leave Post assessment marks automatically ticked
  • leave other options unticked
• Assessment results: no action needed
• Assessment security: no action needed
• Additional Tools: no action needed
• Description: add an optional short description to display under the item's name in the Course Content area.

It's essential that settings are correct for formal exams. This will depend on the structure of your Test and other requirements.

Contact us to advise on appropriate settings for your specific exam.

Print or save a Test

You can print or save your Test as a PDF, either with or without answers. This could be useful to:

• deliver and mark the Test as a paper-based assessment
• save the Test for your records
• provide a copy of the Test for external examiners, reviewers etc.

Note

It is not currently possible to print Tests containing a Question Pool.

1. Click Print at the top of the Test.
2. Select Questions only or Questions with answers
3. Save as PDF or send to the printer.