Project Overview

Project Problem Statement

        Machine learning offers a versatile toolkit for a wide range of applications. In the context of our project, our client wanted to develop a system capable of tracking the movement of an individual's pupils. The primary objective of this endeavor is to facilitate the assessment of a user's emotional state by analyzing the movement of their pupils.
        To achieve this, three algorithms has to be ran: image preprocessing, blink detection, and eye tracking. This project is a continuation of many teams, previous teams successfully executed these algorithms independently on the Kria board. Our challenge, however, is to design a comprehensive system capable of concurrently operating all three algorithms on the Kria board, along with any other supporting algorithms required.
        This task is complicated by the hardware constraints of the Kria board, which, at the moment, is equipped with a single Deep Learning Processing Unit (DPU). Both the blink detection and eye tracking algorithms rely on the DPU. On top of that, by running multiple resource heavy algorithms at the same time, other resources, such as memory and processor cores, has to be delicately managed.
        Our objective is to design a system that optimizes the distribution of these resources to achieve a target throughput of 200 frames per second (FPS). This rate is deemed the minimal requisite for the system to effectively fulfill its intended function.

Team Members

Jonathan Tan

DPU Resource Manager

Jonathan Tan is a senior in Computer Engineering and is working towards completing his undergrad and getting his master's in computer engineering. His area of interest is microarchitecture and hardware accelerator design for Machine Learning.

Josh Czarniak

Eye-tracking Algorithm Developer

Josh Czarniak is a senior in Computer Engineering and is working towards completing his undergrad. His main interests are embedded systems embedded software systems, computer architecture hardware, and machine learning.

Justin Wenzel

Blink Algorithm Developer

Justin Wenzel is currently a senior pursuing a Bachelor’s degree in Computer Engineering, complemented by a minor in General Business. His academic and professional interests primarily revolve around low-level system programming and the architecture of machine learning models on different platforms.

Kai Heng Gan

Image Pre-processing Developer

Kai Heng Gan is a senior in Cybersecurity Engineer and is a research assistant with the Department of Civil, Construction and Environmental Engineering. His current research interest includes large-scale distributed system monitoring, chip architecture, and AI in cyber defense system.

Santiago Campoverde

Profiling Developer

Santiago Campoverde is a Software Engineering senior working on completing his undergraduate degree. His main professional interests include app development, artificial intelligence, and machine learning.





Weekly Reports

Report 1
Report 2
Report 3
Report 4
Report 5
Report 6
Report 7
Report 8
Report 9
Report 10


Design Documents

Design Document - User Needs
Design Document - Requirements
Design Document - Project Plan
Design Document - Design
HAML Design Document V1
HAML CPRE 491 Presentation Slides


Lightning Talks

Lightning Talk 1 - Problems and Users
Lightning Talk 2 - User Needs and Requirements
Lightning Talk 3 - Project Plan
Lightning Talk 4 - Design Part 1
Lightning Talk 5 - Design Part 2
Lightning Talk 6 - Contexualization Design
Lightning Talk 7 - Ethics and Professional Responsibility