Project

Students will be required to complete a computer vision project as a part of this course. This project will allow students to gain “hands on” experience with a computer vision research topic. Projects will follow one of two streams (Engineering or Scientific) which will dictate the focus of the work and the specific grading scheme to be used. Regardless of the selected stream, all projects will consist of five graded components (percentage of final grade given in parentheses): White Paper (2%), Proposal (8%), Site Visit (5%), Demo (7%), and a Final Report (18%).

For cases in which a particularly extensive project is proposed, students may request to work in groups of at most two students. However, any students proposing a group project will need to speak to the instructor prior to the deadline for the White Paper with a clear plan for how the work will be divided, and the instructor will work with them to modify the expectations of the project components to reflect a larger scope.

Note that the project stream selected does not in any way reflect on whether a student is in an engineering or science program.

• White Paper - September 23rd
• Proposal - October 9th
• Site Visit - November 4th/6th (in lab)
• Demo - November 20th, 25th, 27th, December 2nd (in class)
• Final Report - original deadline: December 2nd, revised deadline: December 13th

Engineering projects focus on the implementation aspect of a project, and will emphasize open coding practices, software design, and technical documentation. To complete an engineering project, a student is expected to find a computer vision model or algorithm which has been described in a peer-reviewed publication but lacks an available open-source implementation, or one which the student would like to provide an implementation in a novel format (eg. convert a model implemented in MATLAB to C++ or Python). The goal of this project will be to provide an implementation of the chosen algorithm which matches as closely as possible the functionality of the original source material and which can be released under an open license.

Note that the selected model must have a semantically understandable computer vision component which will be implemented by the student (eg. it will not be sufficient to simply train a deep network on an available dataset). Examples and suggested algorithms are provided below.

• White Paper: Each student will select a particular model to implement and inform the instructor of their choice via a short (300 word maximum) written description, including a note informing the instructor that the student is proposing an engineering project. The final topic selection is subject to the instructor's approval.
• Proposal: Each student will submit to the instructor a brief (1-2 page) proposal for their project. This proposal should include an explanation in the student's own words of the motivation and significance for the original research the project is based on, a description of the algorithm parts which must be implemented, a set of any foreseen challenges or components of the model which are unspecified in the original documentation, and a plan for testing the final product to ensure it meets the expected functionality.
• Site Visit: Each student will make a brief verbal report of their progress; students are encouraged to provide preliminary demonstrations of any software that has been developed at this time.
• Demo: Each student will be expected to provide a demonstration of their project results in front of the class. This should describe to the class the goal and function of the implemented algorithm, as well as provide an example of its execution.
• Final Report: A technical report of the software (approximately 3-5 pages) should explain the purpose of the work, describe the student's implementation, note any decisions the student made which may lead to potential differences with the original work, and present the results of any tests or evaluations performed. Additionally, the student must provide the instructor with access to a version control repository (see details on the Resources link on the side bar to this page) containing the project code. A portion of the grade will be assigned based on the quality of the code (in terms of both functionality and ease of use) and its user documentation. The final written report also will be required to have an annotated bibliography which includes the original paper being re-implemented. For students enrolled in EECS 5323 only, this bibliography must also include additional references to the primary literature that is related to the project.

Scientific projects focus on exploring a novel aspect of computer vision. To complete a scientific project, a student is expected to propose a hypothesis which they will investigate. The goal of this project is to produce new knowledge about the field of vision. This work can take many forms, such as cross-model comparison or testing model performance under unexpected conditions. For a more complete set of suggested example projects, see below.

• White Paper: Each student will select a particular topic of study and inform the instructor of their choice via a short (300 word maximum) written description, including a note informing the instructor that the student is proposing a scientific project. The final topic selection is subject to the instructor's approval.
• Proposal: Each student will submit to the instructor a brief (1-2 page) proposal for their project. This proposal should include an explanation of the motivation for their project (why is this an interesting question to investigate?), a description of their planned approach (what tools will they use to complete this project? This includes how they will analyse or measure model behaviour, and any third-party software they plan on using, such as model code released by other researchers), and a brief discussion of possible outcomes and what those different outcomes might signify.
• Site Visit: Each student will make a brief verbal report of their progress; students are encouraged to provide preliminary demonstrations of any software that has been developed at this time or experimental protocols which are ready to run.
• Demo: Each student will be expected to provide a demonstration of their project results in front of the class. This should describe to the class the goal and motivation for the study, what results were obtained, and what conclusions can be drawn from those results.
• Final Report: A technical report of the software (approximately 5 pages) should explain the motivation and purpose of the work, describe the student's methods and pertinent implementation details, present the results the tests or evaluations performed, and provide a discussion and conclusion based on those results. Additionally, the student must provide the instructor with access to the project code (this is preferably in the form of a version control repository - see details on the Resources link on the side bar to this page). For students enrolled in EECS 5323 only, the final written report also will be required to have an annotated bibliography of references to the primary literature that is related to the project.

Electrical Engineering and Computer Science Department laboratory facilites will be available for support of projects. Documentation of the lab is available under the Syllabus link on the sidebar to this page.

<!-- -->

Following is a list of suggested topics for this course as well as brief descriptions. If further discussion of any of these topics is desired, then see the instructor.

Note that if a student would like to claim one of these models for their project, they are encouraged to speak to the instructor early to avoid duplication of work with another student.

• Discriminant Saliency - Gao et al., 2008, Paper Link. Code was released as a binary, and does not appear to run anymore. Project selected by a student.
• Discriminative Correlation Filter with Channel and Spatial Reliability - Lukežič et al., 2017, Paper Link. Code was released for MATLAB, and could be converted to Python or C++. GitHub Repository of MATLAB Code - Project selected by a student.
• Learning Background-Aware Correlation Filters for Visual Tracking - Galoogahi et al., 2017, Paper Link. Code was released for MATLAB, and could be converted to Python or C++. Project page with link to code - Project selected by a student.
• Remote Sensing Image Scene Classification Using Multi-Scale Completed Local Binary Patterns and Fisher Vectors - Huang et al., 2016, Paper Link. Model appears to have been released without code. - Project selected by a student.
• Compositional Model Based Fisher Vector Coding for Image Classification - Liu et al., 2017, Paper Link. Model appears to have been released without code.
• Person Following Robot Using Selected Online Ada-Boosting with Stereo Camera - Chen et al., 2017 Paper Link. Model released without code, but the dataset is publicly available: Project page. Note that this is a robotics paper; only the vision component would be expected to be completed, and evaluation on existing data would be sufficient.
• Early Recurrence Improves Edge Detection - Shi et al., 2013, Paper Link. Model appears to have been released without code. Project selected by a student.

Note that this example is not from prior coursework for this course, but nevertheless provides an example of work which is similar to the end goal of the engineering projects in this course. Keep in mind that this example is primarily an effort in machine learning rather than vision, but is nevertheless included to provide showcase the kind of technical report which might be produced.

• OpenSALICON GitHub Repository, Technical Report PDF

As mentioned above, the primary goal of a scientific project is to find out new information in computer vision. This could take many forms, but the following set of suggestions will hopefully provide some ideas, and students are encouraged to discuss specific ideas with the instructor during office hours.

• Image formation: Since in computer vision we seek to recover information about the world from images, it is important to understand how the world information was projected into the images. Correspondingly, in lecture and in the textbook, various models of image formation have been described. For this topic, the student will further explore image formation. As a specific direction for investigation, one might study the effects of the approximations that are involved for various models of image projection (e.g., a quantitative comparison of the geometric ramifications of perspective vs. weak perspective vs. orthographic projection). Alternatively, one might embark on a quantitative exploration of optical aberrations (geometric and/or radiometric). The student also is welcome to explore models of image formation that have not been covered in class (e.g., image formation using transmittance image formation, etc.). In any case, an appropriate methodology for this project would be to couple formal mathematical analysis with computer simulations of the models of concern.

• Image preprocessing: The output provided by a computer vision algorithm can be greatly improved via appropriate preprocessing of the input imagery. For example, techniques can be applied to suppress noise; alternatively, features of particular interest can be enhanced or even extracted as explicit input to a learning step. For this topic, the student will select a computer vision algorithm that takes as its input one or more images (e.g., edge detection, optical flow estimation, etc.) and quantitatively evaluate the effects of one or more image preprocessing steps.

• Adaptive stereo vision: In our study of stereopsis, we will learn that a useful strategy is to begin our estimation procedures with coarse image data (e.g., imagery with low spatial resolution) and subsequently refine our solution through systematic incorporation of more refined image data (e.g., imagery with higher spatial resolution). We will refer to this paradigm as course-to-fine refinement. An interesting question that arises in this paradigm is how to decide on the level of refinement that is appropriate for a given image or even a given image region. For this project, the student will explore methods for automatically adapting the coarse-to-fine refinement of stereo estimates based on the input binocular image data and implement as well as test at least one such procedure.

• Primitive event recognition: The combinatorics of some approaches prove challenging either from a required hardware perspective or to achieve real-time processing. For timely or efficient processing, it is critical that early operations begin to recognize significant patterns in this data. For example, when processing a video sequence, it might be desirable to make distinctions regarding those portions of the imagery that correspond to moving objects so that subsequent processing might attend to such regions. For this project, the student will suggest and investigate a set of simple patterns or events that it might be advantageous to support through early processing (e.g. for videos, what is moving, what is not, what is noise, etc.). The student will develop an analysis that shows how the patterns can be distinguished on the basis of simple vision operations as well as implement and test corresponding algorithms.

• Module combination: Typically, a solitary computer vision module provides incomplete information about the world; however, taken in tandem (two or more) modules might combine forces to provide a better understanding of the environment. For example, binocular stereo typically performs best in regions with well define features or texture patterns, but poorly in smoothly shaded regions; in contrast, shape from shading can perform reasonably in smoothly shaded regions, but less well in highly textured regions. For this project, students will select a pair of complimentary vision modules (e.g., stereo and shading, feature-based and area-based image correspondence/matching for stereo, binocular stereo and motion, etc.) and study how they can be combined in an advantageous fashion. The student will develop an analysis that shows how the modules can be combined as well as implement and test corresponding algorithms.

• Algorithm comparisons: For any given research or application topic in computer vision there is more than one possible approach. For example, many different approaches to optical flow estimation have been developed. For this project, students will consider a particular topic (e.g., binocular stereo correspondence, optical flow estimation, shape from shading, etc.) and select at least two algorithms that have been developed for this topic. The student will compare the selected algorithms both analytically (to develop a theoretical understanding of their relationships) and empirically (to develop a practical understanding of their relationships). A restriction on this topic is that comparison of algorithms for edge detection is not an allowable topic; there has been a great deal of research on this topic, which will make it too difficult for students to make a novel contribution.

• Principled probing of algorithm behaviour: It is often instructive to examine algorithm failure cases or scenarios which are particularly challenging within a given problem domain. For this project, students will consider how to design or select a dataset which will test the behaviour of a model or set of models in a new way. The student must be able to clearly explain what their chosen dataset is designed to reveal about the model, and draw conclusions from the empirical results obtained.