Faculty members, please use the following form to submit project descriptions for this page: https://forms.office.com/r/QH4QnYr8Hq

Summer term: May 6-August 6, with final presentations during August 8-15 approximately

This listing is being updated until the start of the summer term.

Course: EECS4480/EECS4080

Supervisor: Jonatan Schroeder

Supervisor's email address: jonatan@yorku.ca

Project Description: Unit testing platforms like Java's JUnit and Python's unittest provide a simple interface for evaluating the correctness of individual functions in a large project. These platforms can also be used in an academic environment to automatically test student-submitted code in programming assignments and generate a grade based on if these tests pass or fail. However, given that these platforms were originally developed for running code that is expected to be trusted, this practice can lead to a potential risk if students are able to provide code that causes the test to pass without resulting in the expected value (see https://www.seas.upenn.edu/~hanbangw/blog/hack-gs/). While most modern autograding platforms introduce security practices to avoid this kind of code from receiving a valid grade, some vulnerabilities still exist.

For this project you will strengthen the security of an autograder process for either Python or Java code for the PrairieLearn platform. You will start by creating possible attack vectors in the form of code that is expected to cause the autograder to pass a test without actually returning the expected results. Examples of attack vectors include code that saves or outputs well-formatted values that are interpreted by the autograder as a success, code that is able to identify secret information from the autograder code, and/or code that crashes the original autograder process. Then you will implement safeguards that ensure student-submitted code is unable to bypass container sandbox limitations, and that ensure that malicious student code does not result in a successful grade.

You will work in coordination with the supervisor and the PrairieLearn developer community to brainstorm possible strategies and guidelines. Your final deliverable will be a pull request to the PrairieLearn codebase with the proposed fix.

Required skills or prerequisites:

• To work on autograder for Python code, you must have completed EECS 1015 (or a similar course) with an A/A+. Must have solid programming skills in Python, including the use of unit testing.
• To work on autograder for Java code, you must have completed EECS2030 (or a similar course) with an A/A+. Must have solid programming skills in Java, including the use of unit testing
• Must be able to work independently and have good communication skills.

Recommended skills or prerequisites: EECS 3221 is highly recommended. Experience with Docker containers is helpful but can be obtained during the project. Git experience is helpful. Experience with open source software development is an asset.

Instructions: Additional information about PrairieLearn can be found here: https://prairielearn.readthedocs.io/en/latest/. A sample PrairieLearn assessment that includes Python autograded questions can be found here: https://us.prairielearn.com/pl/course_instance/136606/assessment/2351069. Please submit a brief description of your experience with the skills listed above.