Product designer

Dancebattle | GUI/UX App

A social dance game using computer vision




In July 2018, I attended Angel hack Hackathon and discovered this new technology, pose estimation. We decided to use this interesting technology to create a fun experience to motivate people to get up and move! We come up with Dancebattle, a social dance game. You can challenge your friend with a dance. You can get scored based on how well you dance.

We ended up winning the grand prize of the hackathon and accepted to the Hackcelerator program(only 2% hackathon attendees make it through). We are one of the selected 9 teams to pitched in front of 100 investors on Global demo day and received a chance to interview with most prestigious accelerator programs

Most Importantly, I had this unique opportunity to design a completely new way of interaction between human and computer



Researching, Ideating, User flow, Wireframing, Branding, Presentation design, User Testing


Sketch, Principle, After effects, Premier Photoshop, Illustrator



Patrick Carrol (Software Engineer), Gia Cheng (UI/UX Design + Presentation), Lily Anna Li (Visual design), Laura Cheng( Illustration)


July 29th 2018 to March 2019



  1. Conducted a user research to find user pain points

  2. Prioritized users’ needs based on feedback to create clear information architecture.

  3. Designed how users navigate through the app. Due to time constrains, we had to keep our redesign to a specific area.


I designed and built a simple prototype on Keynote that link to a website, which turned on your webcam and you could see yourself with the filter overlay on you. We asked users a series of questions about social media and dance games . Then, we showed them the prototype and asked them to complete the following tasks. At the end we asked them what would they think about the idea.

  1. Pick dance video from the website

  2. Watch the dance

  3. Complete your dance challenge

  4. Share your video


User Testing

We tested on 2 groups of potential users. We facilitated 6 user testing sessions over a week, each 45 minutes long.

  1. Generation Z people are in their 10S and 20s who are in school

  2. People who love dancing, interested to learn more about dance

These 2 groups had very different opinions about the idea. We analyze the result, with the time and technical constrains that we had, we decided to focus on the first group which is the younger generation. 

We color coded each task based on how successful users were at various tasks

We color coded each task based on how successful users were at various tasks


Testing Result

There were 11 usability issues that we needed to fix. They ranged from simple changes in UI  to revamp the flow.

We simplified the game as much as possible. We determined the priority and feasibility of the design changes we required to implement.

Screen Shot 2019-04-28 at 8.32.29 PM.png


We created a storyboard to portray how a user would interact with this app through 4 different stages (illustrated by Lily).


Start a dance challenge from a friend invitation or pick a challenge online


Set up

Accept camera access and dance invitation; Move back from the screen; Pick filters

Setting up.jpg

Dance battle

Copy the dance; Get a score



Share your video to your friend, Try again or another dance


Information Architecture

Dancebattle IA.jpg


Because of new technology, We could create a very different experience from all the other games in the market. People could interact with their whole body in a computer-generated environment on their screen without the need for any special equipment but your computer.

From the usability test, we found that most people are confused during the dance battle part. This is is the most important part of the MVP.

How could we introduce users to this new environment?

Most people never played games like this. This product experience was probably people’s first exposure to this technology. There was no established interaction paradigm. We would need to build many things from scratch. In this case study, I would like to talk about one specific problem which was how could users navigate the game? 🤔



Gesture controlled UI



Since Dancebattle used your camera phone or webcam to track your body.  , It gives us 2 options for user interaction. One is the traditional mouse and keyboard. The other one is using gesture or motion. We looked into Kinect games which offer an advanced gesture-based user experience. The experience wasn’t invented by Kinect. Similar experience like the Myron Krugger’s video place was published in 1985 at a cutting-edge academic conference. However, Kinect is the first company made the gesture-based user experience available to millions of people for a  $150 Xbox add-on.

kinect gui.jpg

The takeaways from Kinect GUI

  • Visibility problems: On-screen alerts are easily overlooked because the user's attention is focused elsewhere.

  • Consistency problem: That there are no universal standards for gestural interactions yet is a problem in its own right, because the UI cannot rely on learned behavior.

  • Accidental Activation: It uses confirmation alerts users that something is about to happen, and prevents unintentional gestures from initiating actions.

  • Unnecessary Gesture control. Don’t use gesture controls unless it is necessary. Clicking a button is sometimes easier and more familiar than a gesture. Use it only when it is natural to users or more convenient than Clicking a button.



In order for the computer to recognize your movements, you would need to back up so your whole body is in the frame. So the problem was when people were backed up from the camera. They were far away from their keyboard and mouse. It would be cumbersome for the users to go back and forth from their computer to select something. So gesture controls became necessary in this case.




Confirmation alert that informs users, text prompt or icons instant recognize

Familiar sounds for feedback, Sound Metaphors



we picked a dark background because we wanted to portray a nightclub environment. It made people feel mysterious and anything can happen. We used neon color for the button and the filters because it provided a better contrast. Users could have a mix of colored background. One single color could easily blend into the background. We found that bright colors are the best for this scenario. We chose neon because it also fits into the night club theme.

color system.png


Version 1

In the MVP that we presented in the global demo week, we used “Jump” as the gesture to start recording. Because it is a one-word command that most people understand compare it to other gestures like”circle your arm”. People can interpret in many different ways without a visual demonstration.  Therefore, we used to jump as our first gesture control.



we tested the MVP with 14 people. We realize that the text instruction “ Jump to start recording” is often overlooked because users attention is focused somewhere else. And as the game became more complex, we would need users to make more choices like pause, try again, or change a song. Jump wasn’t the best gesture for selecting or navigating. Users would also be tired of all these jumping.

Version 2


We came up with another idea. People can use their hands to touch an icon. We used icons instead of text because icons are understood universally. Texts are sometimes too small to read when you all the way backed up from the screen.

We had to make a simpler version of the game to test on selection because Dancebattle had more complications in development. We used 1 week to design and developed Christmas dance game and released it before Christmas. People could use it to make a Christmas card and send it to their family and friends.

  • We used 2 hands to select because we thought people would accidentally select something.

  • We used visual cues like half of the icon would light up when you touch it with 1 hand. If you touch it with 2 hands the whole icon would light up.

We test it on 10 people in person and posted the game on Reddit, Facebook, and Twitter.

After evaluating the public and in person testing data, we realized that 2 hands selection is not an easy adaption for most people.

Version 3

  1. It was not as natural as 1 hand selecting because people were used to using their dominant hand.

  2. There was also a technical difficulty with 2 hands selection. Because how fast the computer recognized your movement depends on how fast your computer ran. It would take more time for the computer to recognize 2 hands than 1 hand. So 2 hand selection would have a longer activation time than 1 hand. We didn’t want to people think the gesture control was broken.

We quickly iterated on our Christmas game. It would use 1 hand selection instead of 2 hands. We also changed the onboarding process. Instead of everything came to you at once, we introduced them step by step.


We evaluated the testing data. We got a 50% increase in retaining the users to record a video. They also took less time to get to the other part of the game.

Screen Shot 2019-04-05 at 5.23.50 PM.png
Screen Shot 2019-04-05 at 5.25.56 PM.png
Screen Shot 2019-04-05 at 5.55.53 PM.png

Final Prototype



Due to the time and technical constraints, We had to solve this problem by making a simpler flow. However, we learned the most after this compromise. Because we could test on random users on the internet which is a more realistic scenario.

I think we can improve on giving people a better purpose, reason, and role to interact with the app. Even technology is new that can give the user a novelty experience, it will only keep the users for a few minutes. Design a game with a better story and engaging loops that is how to make a great game.

This experience helped me become a better product designer. I would think of the interface and experience not only regarding features and tasks but also story and loops. How to gamify the product and increase user engagement.