top of page

Projects

Introduction:

Here are my previous academic and professional projects, which has these focused areas:

VR/AR

Machine Learning 

Game

Quantitative Analysis

Software Engineering

Robotics

★ XRT: Geo-based Mobile AR Museum Platform

Technical Producer, Lead Programmer

#AR #Geo #Unity #Google Firebase

Cooperated with Andy Warhol Museum and CMU-HCII 

Spring 2023

Website

The project originally aims at iteratively design, prototype, and evaluate a virtual immersive gallery with marginalized Pittsburgh artists, which will be physically and virtually centered at the Andy Warhol Museum Pop District. However after iterations, we see much more meaningfulness to promote the idea to the general public.

Supported by Google AR Core and Firebase features, the project allows the real-time, multi-user museum experience, both for content creators and viewers. Our product also serves as a platform to strengthen the connection in art community.

64060b51c4552b1701c136f8_c705c6ae-ad49-4191-a3ff-c0f7ef761f6b.png

Final Demo (Credit to: Jiayu)

★ Branching Narrative Editing Tool in Roblox

Producer, Lead Programmer

CMU academic project cooperated with Roblox

Fall 2022

Website

This is a Roblox Education sponsored project, aiming to discover the potential of Roblox in education area. In the project, we learnt from Twine - a widely used narrative building tool, and deliver a similar application that supports real-time editing and story building within Roblox, the leading content creation platform. 
 

As the lead programmer and producer, I am responsible for the key function development: dialogue system, and implemented the frontend and backend including server-side programming with Lua. We also communicate with the Roblox, participating in weekly meetings and reporting on progress during the delivery timeline of 13 weeks.

unnamed.png

Final Demo 

★ Machine Learning Supportive UI Tools

Unity Technologies, Machine Learning Engineer Intern

ML Artistry Team, San Francisco

Summer 2022

Along with the tremendous growth of virtual worlds in games and movies, scene arranging has always been a time-consuming issue that takes a group of technical artists months to complete a high-quality scene.

Our goal is to implement optimized layout(set-dressing) tools that greatly boost this process, by providing real-time prototype solution, including the placement tool, the physic-based transform tool, and the intelligent assets palette. The project is also compatible with future XR devices interaction since it's in Unity platform. Meanwhile, I align with team regularly about the machine learning techniques in scene editing, including terrain generation and smart brushes in the natural scene playground.

As a Machine Learning Engineer Intern, I started by collecting critical needs and started by making system designs, then followed by prototyping UI features and implemented them. Also we experimented with different physics methods for one of the UI features as well.

unity-symbol-1100x825.jpg
tool.gif

Usage Demo (Part) 

pasted image 0.png

Asset Explorer

Screen Shot 2022-08-05 at 12.06.39 PM.png
rocks.gif

Physic tools

★ WebGL Self-driving Car Demonstrating AI Concepts

Lead Programmer

Collaborated with CMU Machine Learning & Computer Sciences Departments

Spring 2022

Website

This is a project whose client is Boys and Girls Club of America and is currently pushed in use.

trAIn gAIn is an interactive WebGL game to assist 14-16-year-old AIPI (Artificial Intelligence Pathways Institute) students understand the concepts of neural networks through interactive visualization of the self-driving car. 

cropped-cropped-cropped-WeChat-Image_20220407160357-1.jpg
spreadsheet.png
lilith

Scene Art

Flyers

★ 16-726: Learning-Based Image Synthesis

This course taught by Jun-Yan Zhu start from Computer Vision basics and then go further into content generation with machine learning methods. The primary focus for this course is Generative Adversial Network(GAN), but it take advantage of deep learning methods and digital image processing as well. 
 

See the course page and my project writeups: proj1 | proj2 | proj3 | proj4

★ Meta-human: Motion Capture and Emotion Model

Algorithm Engineer Intern

Lilith Games, Summer 2021

 

Our algorithm team researched making the automatic workflow from a normal video containing human dancing to an avatar with the same body movement. 
 

Besides, we have implemented a virtual pet prototype with RESTful APIs backend by Python Flask.

6821b602-8d1d-4a70-91d1-3e74249adffc.png

★ Cotton Picking Robot

Research Assistant

ShanghaiTech School Lab & Xinjiang Esquel, 2020-2021

After college graduation, I worked as Research Assistant in the SMART Lab at ShanghaiTech, and worked in a research team with Xinjiang Esquel Group. We aim to develop a mobile cotton-picking platform, with the robot arm and precise cotton sense with the 3D camera and computer vision object detection algorithm. My main focus is the suction device on the end of the robot arm. We altogether iterated several versions of the suction device and I coded on the control part of the suction device. The device is a prototype based on Arduino and Raspberry Pi, and is connected to the ROS robot platform. Besides, I helped with developing and deploying detection methods on the robot.

e763cdbe-cc6f-4851-bcef-65ed5be9d430_rw_600_edited.jpg

★ Venture Capital Investment Data Analysis

During my senior year in ShanghaiTech, I became a student Research Assistant in School of Entrepreneurship and management. My contribution is to analysis venture capital investment data collected by pitchbook.com and run machine learning methods on them, aiming to find mechanism of how it is related with an important index: Economy Policy Uncertainty Index. Besides, we found this vary between investors depends on their education level, gender and some other personal attributes. This is also my graduation paper as well.

snapshot.png
dataset.png
stata.png
bottom of page