Project Overview
Project Problem Statement
Machine learning offers a versatile toolkit for a wide range of applications. In the context of our project, our client wanted to develop a system capable of
tracking the movement of an individual's pupils. The primary objective of this endeavor is to facilitate the assessment of a user's emotional state by analyzing
the movement of their pupils.
To achieve this, three algorithms has to be ran: image preprocessing, blink detection, and eye tracking. This
project is a continuation of many teams, previous teams successfully executed these algorithms independently on the Kria board. Our challenge, however, is to
design a comprehensive system capable of concurrently operating all three algorithms on the Kria board, along with any other supporting algorithms required.
This task is complicated by the hardware constraints of the Kria board, which, at the moment, is equipped with
a single Deep Learning Processing Unit (DPU). Both the blink detection and eye tracking algorithms rely on the DPU. On top of that, by running multiple resource
heavy algorithms at the same time, other resources, such as memory and processor cores, has to be delicately managed.
Our objective is to design a system that optimizes the distribution of these resources to achieve a target
throughput of 200 frames per second (FPS). This rate is deemed the minimal requisite for the system to effectively fulfill its intended function.
Team Members
Jonathan Tan
DPU Resource ManagerJonathan Tan is a senior in Computer Engineering and is working towards completing his undergrad and getting his master's in computer engineering. His area of interest is microarchitecture and hardware accelerator design for Machine Learning.
Josh Czarniak
Eye-tracking Algorithm DeveloperJosh Czarniak is a senior in Computer Engineering and is working towards completing his undergrad. His main interests are embedded systems embedded software systems, computer architecture hardware, and machine learning.
Justin Wenzel
Blink Algorithm DeveloperJustin Wenzel is currently a senior pursuing a Bachelor’s degree in Computer Engineering, complemented by a minor in General Business. His academic and professional interests primarily revolve around low-level system programming and the architecture of machine learning models on different platforms.
Kai Heng Gan
Image Pre-processing DeveloperKai Heng Gan is a senior in Cybersecurity Engineer and is a research assistant with the Department of Civil, Construction and Environmental Engineering. His current research interest includes large-scale distributed system monitoring, chip architecture, and AI in cyber defense system.
Santiago Campoverde
Profiling DeveloperSantiago Campoverde is a Software Engineering senior working on completing his undergraduate degree. His main professional interests include app development, artificial intelligence, and machine learning.
CPR E 492 Bi-Weekly Reports
Report 1Report 2
Report 3
Report 4
Report 5
CPR E 491 Weekly Reports
Report 1Report 2
Report 3
Report 4
Report 5
Report 6
Report 7
Report 8
Report 9
Report 10
Design Documents
Design Document - User NeedsDesign Document - Requirements
Design Document - Project Plan
Design Document - Design
HAML Design Document V1
HAML CPRE 491 Presentation Slides
Lightning Talks
Lightning Talk 1 - Problems and UsersLightning Talk 2 - User Needs and Requirements
Lightning Talk 3 - Project Plan
Lightning Talk 4 - Design Part 1
Lightning Talk 5 - Design Part 2
Lightning Talk 6 - Contexualization Design
Lightning Talk 7 - Ethics and Professional Responsibility