Sixth SPLICE Workshop Proceedings
Proceedings Citation
P. Brusilovsky, K. Koedinger, D.A. Joyner, T.W. Price. Proceedings of SPLICE 2020 workshop Building an Infrastructure for Computer Science Education Research and Practice at Scale at 7th ACM Conference on Learning at Scale, Aug 12, 2020, Virtual EventOrganizers
Peter Brusilovsky, University of Pittsburgh
Ken Koedinger, Carnegie Mellon University
David A. Joyner, Georgia Institute of Technology
Thomas W. Price, North Carolina State University
Peer Reviewed Papers
Title: Using Discussion Board Data to Hire Teaching Assistants
Authors: Arnaud Deza, Haocheng Hu, Vaishvik Maisuria, Michael Liut, Andrew Petersen and Bogdan Simion
Slides: Available Here
Abstract: Teaching Assistants (TAs) fill critical roles in instructional teams, spending one-on-one time with students, providing feedback, and serving a critical mentoring role. As a result, the process of selecting TAs is important, yet we often have little information when making hiring decisions. We used data collected from a course discussion board to identify which statistics are indicative of participation and which might be useful for hiring TAs. We then analyzed our past TA hires and found that there are opportunities to use discussion board metrics to increase the number of potentially qualified applicants for TA roles and better inform instructors about the engagement of past students as they select candidates for interviews.
Title: Custom "Caring IDE" for Online Offering of CS1
Authors: Mohsen Dorodchi, Mohammadali Fallahian, Erfan Al-Hossami, Aileen Benedict and Alexandria Benedict
Abstract: The role of an integrated IDE capable of several different key features in teaching and learning programming is very clear to everyone. In this work-in-progress paper, we present the new version of our Caring IDE, a cloud-based IDE system integrated with a Learning Management System (LMS), an autograder, databases for storage, and dashboard prototypes to (1) deliver a smoother programming learning experience for students and (2) enhance the instructor's ability to informatively perform student success interventions quickly and early. Here, we report and extrapolate on the design and implementation of the Caring IDE. We also demonstrate the value of the Caring IDE in promoting student-self learning in an online CS1 summer course during the COVID-19 pandemic. Finally, we showcase preliminary IDE-based analytics to promote student success in CS courses.
Peer Reviewed Lightning Talks
Title: Recommending Personalized Review Questions using Collaborative Filtering
Authors: Zain Kazmi, Wafiqah Raisa, Harsh Jhunjhunwala and Lisa Zhang
Slides: Available Here
Abstract: This paper presents the work in progress towards a tool where CS1 students receive personalized review questions to prepare for their term tests. Specifically, the tool recommends multiple choice and coding questions. The recommendations are generated using collaborative filtering, based on students' past performance on these questions. We test recommendation engine models based on last year's student data, and present offline experiments that show the promise of this approach.
Title: CS1 Programming Feedback with Bug Localization
Authors: Lucas Roy, Haotian Yang and Lisa Zhang
Abstract: This paper presents the work in progress towards generating automatic feedback to student solutions to CS1 coding questions that highlights regions of the code that may be problematic. We use a Recurrent Neural Network to generate such feedback. We use a data-driven approach to train the model by re-purposing past student submissions and student corrections to their own code. We present preliminary results of a model that works on one problem. We hope to eventually integrate this kind of feedback in a CS1 setting.
Title: TYPOS: A Computer Science Exercise Platform
Authors: Adam Gaweda and Collin Lynch
Slides: Available Here
Abstract: Computer Science has a number of exercise types available for learning. However, it is unknown when the appropriate exercise type should be given to students on their path to learning CS. This paper describes TYPOS, a Computer Science Exercise Platform that hosts a variety of exercise types. These CS exercises range in complexity and interactivity based on the ICAP framework. As part of this paper, we provide a brief overview over each exercise type and their respective complexity. Finally, we present considerations for future research on using TYPOS for activity sequence mining and suggested next practice activities for students.
Title: Learnersourcing at Scale for Introductory Programming: Longitudinal Data Collection on the Python Tutor Website
Authors: Philip Guo, Julia Markel and Xiong Zhang
Abstract: The Python Tutor website (pythontutor.com) currently gets over 10,000 daily active users executing around 100,000 pieces of code daily. We have been experimenting with collecting large-scale data about learners' thought processes while coding. For instance, we created a learnersourcing system that elicits explanations of potential misconceptions from learners as they fix errors. We have deployed this system for the past three years to the Python Tutor website and collected 16,791 learner-written explanations. By inspecting this dataset, we found surprising insights that we did not originally think of due to our own expert blind spots as programming instructors. We are now using these insights to improve compiler and run-time error messages to explain common novice misconceptions.
Title: Database Query Analyzer Integration
Authors: Ryan Hardt and Kamil Akhuseyinoglu
Abstract: This paper describes updates to Database Query Analyzer (DBQA) that increase its interoperability with other learning tools using the Learning Tools Interoperability (LTI) protocol. As a result, DBQA has been integrated with Mastery Grids and allows for integration with learning management systems.
Title: Live Catalog of Smart Learning Objects for Computer Science Education
Authors: Alexander Hicks, Kamil Akhuseyinoglu, Clifford Shaffer and Peter Brusilovsky
Abstract: We present the initial version of a "live catalog" of LTI enabled smart learning objects that instructors and educators are able to preview and test before deciding whether to integrate these tools in their own courses. The catalog is available on the public Instructure Canvas site and currently showcases LTI tools from multiple educational institutions.
Title: Making it Smart: Converting Static Code into an Interactive Trace Table
Authors: Zak Risha and Peter Brusilovsky
Abstract: This paper introduces a new type of smart learning content, an automatically generated trace table, that can easily integrate and adapt to existing curriculum and learning systems for computer science education. In addition to current features of the software, we describe how this tool constructs trace tables using only source code as an input. The potential of this tool is also explored by examining future opportunities in adaptation, feedback, and learning specifications. Last, we report a pilot integration into an existing system to demonstrate interoperability with a tangible use case.
Title: Runestone: An Open-Source Platform for Interactive Ebooks
Authors: Barbara Ericson and Bradley Miller
Abstract: The Runestone platform is an open-source platform for interactive ebooks. It served over 100,000 registered learners and an average of 800 thousand page views per week during the 2019 - 2020 academic year. There are ebooks for secondary computer science as well as for undergraduate computing courses: CS1, CS2, data science, and web programming. There is even an ebook for a course on Linear Algebra. The platform supports executable and editable examples in Python, Java, C, C++, HTML, JavaScript, Processing, and SQL. It also includes code visualizers/steppers for Python, Java, and C++ code. The ebooks contain typical instructional material: text, videos, and images. They also include practice problems with immediate feedback such as multiple-choice, fill-in-the-blank, and matching questions. Runestone also has unusual features such as audio tours of code, clickable areas, adaptive Parsons problems, and a unique practice tool. This paper highlights some of the unusual features, explains the log data that is collected and is available for analysis, and describes plans for future development.
Invited Lightning Talks
Title: Open Learning Initiative, Echo Authoring, and Community Development for Computer Science Education
Authors: Erin Czerwinski
Abstract: The Open Learning Initiative (OLI) is an online learning environment for the delivery of effective online courses in a variety of domains. OLI offers an authoring tool, Echo, that allows anyone to create OLI courses. These tools each provide learning analytics for monitoring student performance, course health, and effective design. OLI and Echo are part of a larger educational technology ecosystem at Carnegie Mellon University, the OpenSimon Toolkit. These tools are available and open-sourced for computer science (CS) educators to take advantage of, and to part of a larger community of computer science educators and educational researchers.
Title: PEML Update: Progress, Latest Plans, Help Needed
Authors: Stephen H Edwards
Abstract: PEML, the Programming Exercise Markup Language, is the focus of the working group aimed at a shared representation for programming exercises/problems. This update will summarize PEML's current state, and describe plans for a simple micro-service to make PEML easier to adopt (for tool developers) and explore (for everyone). Help is needed in creating a simple/clean authoring guide and a small set of example exercises of multiple scales that the community can use as a reference, and in devising the structuring scheme for collections of PEML artifacts that authors wish to share in smaller communities.
Title: Code Snapshots Working Group Update
Authors: Thomas W. Price, David Hovemeyer, Kelly Rivers, Austin Cory Bart, Ge Gao,
Ayaan M. Kazerouni, Brett A. Becker, Andrew Petersen,
Luke Gusukuma, Stephen H. Edwards, David Babcock
Slides: Available Here
Abstract: This presentation introduces the Code Snapshots working group, its goals and its recent accomplishments. Our goal is to enable collaboration among computing education researchers by helping them to collect and share data, analysis code, and data-driven tools to support students. We aim to do so through a standardized format for logging programming process data, called ProgSnap2. The presentation will give an overview of the format, including how events, event attributes, metadata, code snapshots and external resources are represented. It will also present a case study to evaluate how ProgSnap2 can facilitate collaborative research. We investigated three metrics designed to quantify students' difficulty with compiler errors - the Error Quotient, Repeated Error Density and Watwin score - and compared their distributions and ability to predict students' performance. We analyzed five different ProgSnap2 datasets, spanning a variety of contexts and programming languages. We found that each error metric is mildly predictive of students' performance. The presentation will close by inviting researchers to contribute to the work through shared datasets and analyses.