Below is a list of 4080/4088 projects proposed by faculty members. More projects may be added once they come in (normally in August). Please contact the supervisor directly you are in interested in a project. You can also contact faculty members individually to discuss possible projects. Click here to see a list of faculty members, their research areas and their contact information.

Course: EECS4088 (Capstone Project) Supervisor: Maleknaz Nayebi Contact: mnayebi@yorku.ca Software developers are increasingly sharing images in social coding environments such as stack overflow, GitHub, Bugzilla, and Slack. This growth is parallel to the general growth of in visual interactions in general purpose online social networks like Instagram, Facebook, and Pinterest. Developers' shared images are meaningful and provide complementary information compared to their associated text. These images extensively help in understanding the change requests, questions, or the responses submitted. Relying on these observations, we are working on automatically processing these images by first forming a database similar to ImageNet for software domain and then applying the text detection object detection and automated captioning techniques on these data. We will work with software teams and software developers to evaluate the usefulness of our methods and results. The project includes: (i) Forming and structuring a dataset of crowd labelled images for software related images, (ii) Mining content and objects within developers’ shared images, (iii) Automatic captioning the images to describe the content. Required skills: Good knowledge of Python, Good knowledge on Machine learning, Good knowledge of image processing, Ability to work independently, Ability to communicate clearly. ====== Estimating emotional state from a speech audio signal ====== Supervisor: Michael Jenkin Contact: jenkin@eecs.yorku.ca When a robot interacts with a human it can be very helpful if the robot can estimate the emotional state of the person to whom it is talking. This estimation can be made in a number of ways, from the choice of words used, the visual cues obtained from a camera pointed at the person, and through an analysis if the speech signal obtained from a microphone listening to the speaker. This project is concerned with this last task. This project will apply modern recurrent neural networks, and in particular LSTM to the problem. The project will involve three major phases (i) Background work: Understanding the problem of estimating speech emotion from the audio signal and a review of approaches to date. Identifying and obtaining access to appropriate datasets for training and evaluations. (ii) Basic implementation: Based on the results of above, implement an existing LSTM-based approach to the problem and evaluate on the dataset identified above. (iii) Integration with a social robot system (SENTRYNet) so that the robot can utilize this information in the development of dialog when interacting with speakers. Required skills: General EECS408x prerequisites. Good knowledge of Python. Ability to work independently. ====== Automatic classification of Eurasian Water-Milfoil from sonar and visual data ====== Supervisor: Michael Jenkin Contact: jenkin@eecs.yorku.ca Many lakes in Canada are infested with Eurasian Water-Milfoil (see Eurasian Water-Milfoil – Ontario's Invading Species …www.invadingspecies.com › eurasian-water-milfoil). Eurasian Water-Milfoil (EWM) is an invasive water plant that is detrimental to the health of the lake which is infected. Treating such infections requires understanding where the infections are and the state of the infection. As part of a larger robot project, this project will develop a system for the automatic classification of sensor signals from an autonomous robot boat this is being deployed to monitor EWM infestation in a lake north of Toronto. This work will concentrate around the development of an Amazon Mechanical Turk to assist in labelling visual and sonar imagery to train a CNN to label infestation levels at different locations in the lake. The project will involve three major phases (i) Assist in data collection. For the first few weeks of term (before the weather changes) you will help in data acquisition, editing and organization while reviewing the process of Mechanical Turk systems (such as the Amazon toolkit). (ii) In consultation with domain experts, develop a model for infestation level and construct a Turk to apply this to a training dataset (iii) Develop a DNN to label the multi-modal dataset and evaluate it. Required skills: General EECS408x prerequisites. Good knowledge of Python. Ability to work independently. Ability to interact with subject matter experts. Willingness to participate in data collection sessions as scheduled in the early fall. ====== A framework for VR-based vestibular assessment tools ====== Supervisor: Michael Jenkin Contact: jenkin@eecs.yorku.ca Commodity VR is already being used as part of ongoing assessment of a range of different conditions, including vestibular damage and its treatment. This project involves the development of a software library (written in Unity) to provide VR-based tools to support vestibular treatment both at the clinic and remotely. This work will build upon an existing code base but will (i) refactor the software so as to be useful as a more general library, and (ii) support a wide range of assessment and rehabilitation tools. This work will involve interaction with an off-site partner in their physiotherapy office. The project will involve (i) review of VR-based vestibular assessment and treatment and the existing system and treatment/assessment systems (ii) Understanding the existing code base and development of library structure to ensure a more device-agnostic input system (iii) Development of a small number of treatment modules (2 or 3) to demonstrate the effectiveness of the tools/libraries developed Required skills: General EECS408x prerequisites. Good knowledge of C#. Ability to work independently.