PUBLICATIONS

E-TRIALS has been used to conduct numerous published randomized controlled experiments to investigate methods to improve student learning.

Researchers have used ASSISTments to produce more than 33 published randomized controlled experiments to investigating methods to improve student learning. Of those studies, 8 were collaborations between WPI and researchers from other universities, 23 included Neil Heffernan as an author, and 10 were conducted without Heffernan's authorship. Citations and PDFs are provided below. You can also learn more about the experimental designs of these studies.

Studies Without Heffernan Authorship

  1. Andres-Bray, M., Hutt, S., Zhou, Y, Ostrow, K. & Baker K. (2021). A Comparison of Hints vs. Scaffolding in a MOOC with Adult Learners. AIED 2021. [PDF]

  2. Jiang, Y., Almeda, M. V., Kai, S., Baker, R. S., Ostrow, K., Inventado, P. S., & Scupelli, P. (2020). Single Template vs. Multiple Templates: Examining the Effects of Problem Format on Performance. In Gresalfi, M. & Horn, I. S. (Eds.), The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (ICLS) 2020, Volume 2 (1015-1022). Nashville, Tennessee: International Society of the Learning Sciences. [PDF]

  3. Harrison, A., Smith, H., Hulse, T., & Ottmar, E. (2020). Spacing out!: Manipulating Spatial Features in Mathematical Expressions Affects Performance. Journal of Numerical Cognition. 6 (2): 186-203. DOI: 10.5964/jnc.v6i2.243. [PDF]

  4. Smith, H., Harrison, A., Chan, J. C., & Ottmar, E. (2020). Dynamic vs. static: Which worked examples work best? Poster submission to the 2020 meeting of The Mathematical Cognition and Learning Society. [pre-registration]

  5. Duquennois, C. (2019). Fictional Money, Real Costs: Impacts of Financial Salience on Disadvantaged Students. Working Paper. New version under revise and resubmit at American Economic Review. [PDF]

  6. Walkington, C., Clinton, V., & Sparks, A. (2019). The effect of language modification of mathematics story problems on problem-solving in online homework. Instructional Science. 47, 499-529. [PDF]

  7. Hurst, M. A., Cordes, S. (2018). Labeling Common and Uncommon Fractions Across Notation and Education. Proceeding of Cognitive Science. 1841-1846. [PDF]

  8. McGuire, P., Tu, S., Logue., M., Mason, C., Ostrow, K. (2017). Counterintuitive effects of online feedback in middle school math: results from a randomized controlled trial in ASSISTments. Educational Media International. 54:3, 231-244, DOI: 10.1080/09523987.2017.1384161. [PDF]

  9. Fyfe, E. (2016). Providing feedback on computer-based algebra homework in middle-school classrooms. Computers in Human Behavior 63: 568-574. [PDF]

  10. Koedinger, K. & McLaughlin, E. (2016) Closing the Loop with Quantitative Cognitive Task Analysis. In Barnes, Chi & Feng (eds) The 9th International Conference on Educational Data Mining. 412-417. [PDF]

Studies Comparing Feedback to "Business as Usual"

  1. Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, G. & Soffer, D. (2013). Estimating the Effect of Web-Based Homework. In Lane, Yacef, Motow & Pavlik (Eds) The Artificial Intelligence in Education Conference. Springer-Verlag. pp. 824-827. [PDF] With the University of Illinois

  2. Kehrer, P., Kelly, K. & Heffernan, N. (2013). Does Immediate Feedback While Doing Homework Improve Learning. In Boonthum-Denecke, Youngblood (eds) Proceedings of the Twenty-Sixth International Florida Artificial Intelligence Research Society Conference, FLAIRS 2013, St. Pete Beach, Florida. May 22-24, 2013. AAAI Press 2013. 542-545. [PDF]

  3. Mendicino, M., Razzaq, L. & Heffernan, N. T. (2009). Improving Learning from Homework Using Intelligent Tutoring Systems. Journal of Research on Technology in Education (JRTE). 41(3), 331-346. [PDF] With West Virginia University

Studies Comparing Types of Feedback

  1. Singh, R., Saleem, M., Pradhan, P., Heffernan, C., Heffernan, N., Razzaq, L. Dailey, M. O'Connor, C. & Mulchay, C. (2011). Feedback during Web-Based Homework: The Role of Hints In Biswas et al (Eds) Proceedings of the Artificial Intelligence in Education Conference 2011. 328–336. [PDF]

  2. Razzaq, L., Heffernan, N. T., Lindeman, R. W. (2007). What Level of Tutor Interaction is Best? In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press. 222-229. [PDF]

  3. Razzaq, L. & Heffernan, N.T. (2006). Scaffolding vs. hints in the Assistment system. In Ikeda, Ashley & Chan (Eds.). Proceedings of the Eight International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. 635-644. [PDF]

  4. Razzaq, L. & Heffernan, N. (2010). Hints: Is It Better to Give or Wait to be Asked? In Aleven, V., Kay, J & Mostow, J. (Eds) Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS2010) Part 1. Springer. 349-358. [PDF]

  5. Shrestha, P., Wei, X., Maharjan, A., Razzaq, L., Heffernan, N.T., & Heffernan, C., (2009). Are Worked Examples an Effective Feedback Mechanism During Problem Solving? In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. 1294-1299. [PDF]

  6. Kim, R, Weitz, R., Heffernan, N. & Krach, N. (2009). Tutored Problem Solving vs. “Pure”: Worked Examples In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. 3121-3126. [PDF] With Seton Hall University, Carnegie Mellon University, and Albert Einstein College

  7. Pardos, Z., Dailey, M. & Heffernan, N. (2011). Learning what works in ITS from non-traditional randomized controlled trial data. The International Journal of Artificial Intelligence in Education. 21, 47-63. [PDF]

  8. Razzaq, L. & Heffernan, N. (2009). To Tutor or Not to Tutor: That is the Question. In Dimitrova, Mizoguchi, du Boulay & Graesser (Eds.) Proceedings of the 2009 Artificial Intelligence in Education Conference. IOS Press. 457-464. [PDF]

Studies Comparing Socio-Emotional Interventions

  1. Kelly, K., Heffernan, N., D'Mello, S., Namias, J., & Strain, A. (2013). Adding Teacher-Created Motivational Video to an ITS. Florida Artificial Intelligence Research Society (FLAIRS 2013). 503-508. [PDF] With the University of Notre Dame

  2. Ostrow, K. & Heffernan, N. T. (2014). Testing the Multimedia Principle in the Real World: A Comparison of Video vs. Text Feedback in Authentic Middle School Math Assignments. The International Educational Data Mining Conference. [PDF]

  3. Ostrow, K. & Heffernan, N. T. (2015). The Role of Student Choice Within Adaptive Tutoring. In Conati, Heffernan, Mitrovic & Verdejo (eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer. 752-755. [PDF]

Studies Assessing Parental Intervention

  1. Broderick, Z., O’Connor, C., Mulcahy, C., Heffernan, N. & Heffernan, C. (2011). Increasing Parent Engagement in Student Learning Using an Intelligent Tutoring System. Journal of Interactive Learning Research, 22(4), 523-550. Chesapeake, VA: AACE. [PDF, Longer Version]

Assessing the Automatic Reassessment and Relearning System (ARRS)

  1. Soffer, D., Das, V., Pellegrino, G., Goldman, S., Heffernan, N., Heffernan, C.,& Dietz, K. (2014). Improving Long-term Retention of Mathematical Knowledge through Automatic Reassessment and Relearning. American Educational Research Association (AERA 2014) Conference. Division C - Learning and Instruction / Section 1c: Mathematics. Nominated for the best poster of the session. [PDF, Poster] With the University of Illinois

  2. Soffer-Goldstein, D., Pellegrino, J., Goldman, S., Stoelinga, T., Heffernan, N., & Heffernan, C. (submitted). The Effect of Automatic Reassessment and Relearning on the Retention of Mathematical Knowledge and Skills. Submitted to Journal of Applied Research in Memory and Cognition (JARMAC). Elsevier. With the University of Illinois

  3. Xiong, X. & Beck, J. (2014). A Study of Exploring Different Schedules of Spacing and Retrieval Interval on Mathematics Skills in ITS Environment. In Stefan Trausan-Matu, et al. (Eds) The Proceedings of the International Conference on Intelligent Tutoring 2014. LNCS 8474. 504-509. [PDF]

  4. Xiong, X., Wang, Y., & Beck, J. B. (2015). Improving students' long-term retention performance: a study on personalized retention schedules. InProceedings of the Fifth International Conference on Learning Analytics And Knowledge. ACM. 325-329. [PDF]

Studies Comparing Instructional Approaches

  1. Koedinger, K.R. & McLaughlin, E.A. (2010). Seeing language learning inside the math: Cognitive analysis yields transfer. In S. Ohlsson & R. Catrambone (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. 471-476. [PDF] By Carnegie Mellon University

  2. Sao Pedro, M., Gobert, J., Heffernan, N. & Beck, J. (2009). In N.A. Taathen & H. van Rjin (Eds.) Comparing Pedagogical Approaches for Teaching the Control of Variables Strategy. Proceedings of the 31st Annual Conference of the Cognitive Science Society Austin, TX: Cognitive Science Society. [PDF]

  3. Ostrow, K., Heffernan, N.T. & Heffernan, C. (2015). Blocking vs., Interleaving: A Conceptual Replication Examining Single-Session Effects within Middle School Math Homework. In Conati, Heffernan, Mitrovic & Verdejo (Eds) The 17th Proceedings of the Conference on Artificial Intelligence in Education, Madrid, Spain. Springer. pp. 388-347 (acceptance rate 28%). [PDF] With Carleton College

  4. Lang, C., Heffernan, N., Ostrow, K., & Wang, Y. (2015). The Impact of Incorporating Student Confidence Items into an Intelligent Tutor: A Randomized Controlled Trial. In Santos, Boticario, Romero, Pechenizkiy, Merceron, Mitros, Luna, Mihaescu, Moreno, Hershkovitz, Ventura, & Desmarais (eds.) Proceedings of the 8th International Conference on Educational Data Mining (EDM 2015). Madrid, Spain. June 26-29. 144-149. [PDF]