2. Create Your Problem Set Content

You will use ASSISTments to create your problem set content. To get started, create a teacher account in ASSISTments

You should learn what ASSISTments looks like from a teacher's perspective. Teachers explain how they use ASSISTments on our teacher example page.

  • Learn how to select problems that have already been built and add them to a problem set
  • Learn how to create problems and them into a problem set
  • Learn how to edit a problem in a problem set you built
  • Learn how to insert an image into your problem
  • Learn how to insert a link into your problem
  • Learn how to insert a video into your problem

Basic Building

Google Presentation
Google Presentation
Google Presentation
Google Presentation
Google Presentation
Google Presentation

Basics on building your study

Your study will be in the form of a problem set (or sets) delivered to students. If you are intending to use our subject pool you will need to create this study so that you are comparing two normal instructional strategies (to pass our IRB). This section will describe the process of creating the problems in your problem set. We will also show how you can and share these problem sets with colleagues as you work.

Tutoring Strategies

As a researcher, tutoring strategies can become the focus of your experimental design. You can use these methods to help students solve a problem or to deliver an intervention. Please note that if you add more than one strategy to a problem and both strategies are enabled, each student will be randomly assigned to one or the other, they will not see both.

Hints

This option offers a simple statement for students who answered the main problem incorrectly. Hints do not require students to answer any sub-questions, they just offer guidance. The final hint should always provide students the answer – this is called the Bottom Out Hint. Providing students the answer ensures that they will reach the next problem, or finish the problem set. The following video offers a step-by-step tutorial of how to add a hint to a problem:

Building and ASSISTments with Hints Note: this video is shows a dated interface.

Scaffolds

When a student answers the main problem incorrectly, the problem can be broken down into a scaffold or a set of sub-questions. While the navigation is slightly outdated, these videos offer a succinct view of how to best use scaffolds:

Building an ASSISTment with Scaffolds part 1

Building an ASSISTment with Scaffolds part 2

As you grow accustomed to the system, you can get very creative with scaffolds and hints. You may wish to embed a scaffold within a scaffold to better design a condition. More advanced building instructions can be found at the ASSISTments advanced builder instruction site.

Controlled Vocabulary: There is a way to tell ALI how to so better reporting by naming your sections smarter.

ALI Controlled Vocabulary

ALI has a controlled vocabualcy.

If you use this vocabulary you can get ALI do better reporting for your study.

The terms are

[ignore]: it means that any time the problem appears in the log files it will be ignored. This is commonly used for message a condtion that one extra problem at the begining ('it appears you can hear video. Click 'ok' to continue

[control]

[treatment]

[posttest]

Note, they have to spelled exactly correct and inside the square brackets with no spaces and all lowercase.

(Vase: add to this page text and a few examples to show.)

You can sign up for an account at www.assistments.org using the code StartNow.

Detailed instructions are below.

Step 1. Go to www.ASSISTments.org and create an account.

Step 2. Once you have created an account you want to turn on some features that are typically off for teachers.

Click on Preferences and follow the images. Each selection is described below.

  • Click on Login
  • Click on Create an Account
  • Select I am a teacher then for school information you can either request your university or use our researcher school show in the image. Use the code FirstStep.

Important Teacher Settings

For some of the teacher settings it should not matter if you have them on or off a few of them do.

  • Do not Show Teacher in Student Reports, this is important to have off since you will want to go to your student tab and be a student and you will want to see yourself on the report. Many teachers turn this on because they do not want to see their name on the report with the students.
  • Show Google Classroom Share Icon, this is a neat way to deliver your content to students. If you are working with teachers who use Google Classroom you should use this.

Builder Settings

  • Show attempt count option when creating your own problem set. This allows a researcher to change the attempt count.
  • Show "Inputing Problem ID's" link, this allows you to build a problem set from a fresh start.
  • Show "Using a Skill Builder Template" link, you will use this if you use variablized templates.
  • Show “Problems” link, This will allow you to see all the problems you have ever built.
  • Can add sections to problem sets. This will allow you to create a randomized controlled trial. This is the key feature you need to turn on to make an experiment.
  • Show advanced skill builder setting, anyone who builds variablized templates will need this feature to create the problem sets.
  • Show "Share Links" (Details page), this will allow you to distribute your content using ASSISTments Direct.
  • Create only textbook problems, If you are using the build your own problems link you can make things quicker for creating textbook problems.
  • Show External Problem Types, When you build these will be available if you wantt o add an external problem type.
  • Show Variablized Problem Types Option, This allows you to create variablized problems.

Getting Started

Once you are ready to create a study you may have questions, feel free to email us at assistments-research@wpi.edu. You will need an assistments account and of course your research idea. Section 3, Collecting Data, reviews how you can use your own students or the existing ASSISTments subject pool.

There are two types of studies

Embedded: All a teacher needs to do is assign the problem set and the study will run. Most teachers will not know a study is running. We encourage researchers to create their studies as a Skill Builder so they have a better chance of being used.

Orchestrated: This type of study involves the teacher and the researcher working together to run the study. For example there may be two groups of students and they need to be in the same condition for three problem sets over the course of a week and then everyone takes a post test the following week. The researcher still creates the problem sets but then must work with the teacher to get them delivered in the correct way.

Working with the ASSISTments.org subject pool

If you would like to use our subject pool you will need to submit a structured abstract to Korinn Ostrow (assistments-research@wpi.edu) for review. Download this form, fill out as much as you can, and e-mail it to Korinn to start the process.

Take a look at our list of Skill Builders and find one that has not yet been used for a study. We have over 300 Skill Builders currently available in the system, most do not have studies in them. You can see what is in the skill builder by going to the Builder tab in your account, inputting the ID and clicking on View Problems. Include the Skill Builder you choose in your application form.

We support OpenScience so we ask you preregister your study somewhere (A good place to do that is https://cos.io/prereg/).

iFrame - Outside Learning Resources

Neil's iframe page

Pre-Register Your Study

Options for Pre-Registration

One of our terms of use is that researchers pre-register their experimental designs, hypotheses, and analyses with a reputable repository. The promotion of open science is critically important in increasing the reliability and validity of research findings. These practices keep researchers from cherry picking significant results, keep null results from ending up hidden away in the 'file drawer,' and promote accessible and transparent data sharing practices. Please follow one of the links below to learn more about pre-registration.

Open Science Foundation - https://osf.io/

SREE Randomized Control Registry - https://www.sree.org/pages/registry.php

The If-Then-Else Section Type

The "IfThenElse" is a section type that allows for adaptation.

We will use this screenshot to walk through how this works. You can also login to your ASSISTments account and from Build edit a problem set or Assemble a problem set to follow along. On the left we will describe the example, on the right the generic terms.

In this example we are showing one section from a separate section, This is just to let you know you can put an IfThenElse wherever you want.

Example

  1. Problem Set Type: IfThenElse
  2. Variables. Each IfThenElse starts with a "Conditional" object. In this example it is a "Linear Order" which is a "Problem Set". In this example the Condition will be True if the student gets 100% of the problems correct. (the setting was Saved using the "Save Variable" button).
  3. Problems: In this example "I will put three objects in the Problem section" was selected so there must be three parts in the problems section.
    • The "Conditional Part" in this example is a linear order problem set. Students will do these problems at the end they will either have them all righ (reached the 100% setting) so it will return True, if they don't get them all right it will return False.
    • The "Then Part" In this example if the the student gets all the problems right they will do the linear order section called "StuffToDoIfYouDoReallyWell" Once they finish the IfThenElse section will be over for this student.
    • The "Else Part" in this example if the student gets even one problem wrong in the conditional part they will do the problems in the section called "ElseCondition-posttest" Skipping the then part.

General Terms

  1. Problem Set Type: You start by setting the section to IfThenElse
  2. Variables: There are three main choices for a conditional object. A Standard Problem Set (linear or random), a Skill Builder (standard or random), or a Problem on its own. The Skill Builder and the Problem on their own do not need a set variable.
    • The Problem is true if it is correct and false if it is wrong.
    • The Skill Builder is true if it is completed and false if it is exhausted (use up all the problems without completing).
    • For a Standard Problem Set you can set a percent correct to determine when it designates true and false. (be sure to "Save Variable")
  3. Problems: You have a choice to have three parts or to let one or to let the then or the else be empty. If you select the empt option you will only put two objects in. There are always three parts in the problems section.
    • The "Conditional Part" This problem set or problem will return True or False. The above section on variables describes how this is determined.
    • The "Then Part" This is what students do if the Conditional Part returns True. (this can be empty)
    • The "Else Part" This is what students do if the Conditional Part returns False. (this can be empty)

The Condition Part is the key to all IfThenElse activity. Here are some things to think about.

  1. If Conditional Part is just a problem the problem may not just be a simple right wrong problem.
    • If it has more than one main problem it will return an average of the correctness of the two main problems. The only way for it to be "true" is for a student to get both main problems correct.
    • If there are scaffold questions they are ignored. If a student sees the scaffolds that is only because they got it wrong so it will return "false"
    • Open Response questions are considered incorrect so be careful using them in the "Condition part". You can use them as part of a problem set but be careful.
  2. If you put an open Response question in a problem set remember a student will NOT be able to get 100% since for IfThenElse they are considered wrong.

Examples of how the IfThenElse has been used.

  1. As a video check. Ask students to watch a video with a code word in it. If they type in the code word then the problem returns True and they go into your study which is in the Else Part. Otherwise tell them to type in something esse and if that is the input they will get it wrong and they will go into the "Else Part" which is just content but no study because your study relies on video access.
  2. IfThenElse can be used for an adaptive study, one where students get easier content if they do poorly (either one problem wrong not right or they don't reach a certain percent correct on a problem set), then they get some remedial work (Else Part) or an announcement of how they get it (Then Part). Finally everyone goes on to the next section.
  3. Be Creative!

Tagging: Section Naming Details

Tagging section names will let our automatic system help you with preliminary analysis. Once your study goes live, you will get weekly email to google docs with your information. If you tag your sections, the weekly email will give you (among other things) the count of students per section.

This is a screenshot for a tagged experiment. One of the powers of tagging is that multiple sections can be collapsed into one tag.

For more information about how to build problem sets with complicated structures, visit our advanced builder website. The instruction and specification on how to tag your sections can be found on section 04b of the advanced builder website.