Completed Studies

E-TRIALS has been used in numerous published randomized controlled experiments to investigate methods to improve student learning

Selected works are provided below

ASSISTments has been used in over 27 published randomized controlled experiments to investigate different ways to improve student learning.

13 were completed with 17 other universities, 8 Heffernan in collaboration with others, and 5 without Heffernan as an author.

Studies Without Heffernan Authorship

  1. Andres-Bray, M., Hutt, S., Zhou, Y, & Baker K. (submitted). A Comparison of Hints vs. Scaffolding in a MOOC with Adult Learners. Submitted to AIED 2021. Retrieved from

  2. Duquennois, C. (2019). Fictional Money, Real Costs: Impacts of Financial Salience on Disadvantaged Students. (Revise and Resubmit at American Economic Review). Dr. Duquennois has told Neil that this looks like they are close to publishing this journal article. See her website for the status.

  3. Fyfe, E. (2016). Providing feedback on computer-based algebra homework in middle-school classrooms. Computers in Human Behavior 63: 568-574. Retrieved from

  4. Harrison, A., Smith, H., Hulse, T., & Ottmar, E. (2020). Spacing out!: Manipulating Spatial Features in Mathematical Expressions Affects Performance. Journal of Numerical Cognition. 6 (2): 186-203. DOI: 10.5964/jnc.v6i2.243

  5. Jiang, Y., Almeda, M. V., Kai, S., Baker, R. S., Ostrow, K., Inventado, P. S., & Scupelli, P. (2020). Single Template vs. Multiple Templates: Examining the Effects of Problem Format on Performance. In Gresalfi, M. & Horn, I. S. (Eds.), The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (ICLS) 2020, Volume 2 (pp. 1015-1022). Nashville, Tennessee: International Society of the Learning Sciences.

  6. Koedinger, K. & McLaughlin, E. (2016) Closing the Loop with Quantitative Cognitive Task Analysis. In Barnes, Chi & Feng (eds) The 9th International Conference on Educational Data Mining. Pp 412-417. Retrieved from

  7. McGuire, P., Tu, S., Logue., M., Mason, C., Ostrow, K. (2017) Counterintuitive effects of online feedback in middle school math: results from a randomized controlled trial in ASSISTments. Educational Media International. 54:3, 231-244, DOI: 10.1080/09523987.2017.1384161. Retrieved from

  8. Smith, H., Harrison, A., Chan, J. C., & Ottmar, E. (2020). Dynamic vs. static: Which worked examples work best? Poster submission to the 2020 meeting of The Mathematical Cognition and Learning Society. [pre-registration]

  9. Walkington, C., Clinton, V., & Sparks, A. (accepted pending minor revisions). The effect of language modification of mathematics story problems on problem-solving in online homework. Instructional Science.

  10. Hurst, M. A., Cordes, S. (2018) Labeling Common and Uncommon Fractions Across Notation and Education. Proceeding of Cognitive Science. 1841-1846 Retrieved from

Studies Comparing Feedback to "Business as Usual"

Studies Comparing Types of Feedback

Studies Comparing Socio-Emotional Interventions

    • Kelly, K., Heffernan, N., D'Mello, S., Namias, J., & Strain, A. (2013). Adding Teacher-Created Motivational Video to an ITS. Florida Artificial Intelligence Research Society (FLAIRS 2013). pp. 503-508. With University of Notre Dame.

    • Ostrow, K. & Heffernan, N. T. (2014). Testing the Multimedia Principle in the Real World: A Comparison of Video vs. Text Feedback in Authentic Middle School Math Assignments. The International Educational Data Mining Conference. Retrieved March 10, 2014, from

    • Ostrow, K. & Heffernan, N. T. (2015) The Role of Student Choice Within Adaptive Tutoring. In Conati, Heffernan, Mitrovic & Verdejo (Eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer. pp. 752-755.

Studies Assessing Parental Intervention

Assessing the Automatic Reassessment and Relearning System (ARRS)

    • Soffer, D., Das, V., Pellegrino, G., Goldman, S., Heffernan, N., Heffernan, C.,& Dietz, K. (2014) Improving Long-term Retention of Mathematical Knowledge through Automatic Reassessment and Relearning. American Educational Research Association (AERA 2014) Conference. Division C - Learning and Instruction / Section 1c: Mathematics. PDF, Poster Nominated for the best poster of the session. With University of Illinois.

    • Soffer-Goldstein, D., Pellegrino, J., Goldman, S., Stoelinga, T., Heffernan, N., & Heffernan, C. (submitted) The Effect of Automatic Reassessment and Relearning on the Retention of Mathematical Knowledge and Skills. Submitted to Journal of Applied Research in Memory and Cognition (JARMAC). Elsevier. University of Illinois

    • Xiong, X. & Beck, J. (2014) A Study of Exploring Different Schedules of Spacing and Retrieval Interval on Mathematics Skills in ITS Environment. In Stefan Trausan-Matu, et al. (Eds) The Proceedings of the International Conference on Intelligent Tutoring 2014. LNCS 8474. Pages 504-509.

    • Xiong, X., Wang, Y., & Beck, J. B. (2015, March). Improving students' long-term retention performance: a study on personalized retention schedules. InProceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 325-329). ACM.

Studies Comparing Instructional Approaches

Links to an older website show the experimental desings.