MFA Thesis, Human Computer Interaction UX Research / ML

Client

Graduate Thesis, Human Computer Interaction Program (Future of Wellness and Fitness)

Scope

2021—2023

Skills

UX Research, Human Factors, Prototyping, Information Architecture, User Flows, Data Visualization

Categories

UX/Interaction, Design Research

Keywords

Biomechanics, Metadata, Computer Vision, Cloud-based Services, Perception Neuron Motion Capture Systems, and Axis Neuron

Team

UX Researcher and Engineer

OVERVIEW

Personal Experience

Movement is a fundamental component of our everyday lives. When we move through a space, we perceive and appropriate environmental features in relation to our bodies. This generates an awareness of not only the three-dimensionality of the environment, but also the capabilities of our own bodies (Proulx, M. J. et al., 2016). Most of us develop this awareness over time without much effort when we perform daily activities. When it comes to movements that we don’t perform daily, or that our body fails to adopt as a norm, do we still know the capabilities of our body and how to move it as the way we want in our mind?

I encountered this question when I was practicing yoga steps — the movements were completely new to my body. In my learning process, the trainer often divides the asanas into multiple motion segments based on the flow, demonstrating each segment for the students to observe and imitate. However, most of the information my senses gathered was about the end state of each segment, which was a static pose. I often had a hard time comprehending what happened to the body between each end state because the movements happened too quickly. So what if I slow down a yoga video and map my body in space according to the pose presented in the video for every second? I realized that while I am able to see what the trainer is doing and control my body to the specific trajectory in space, I don’t know how my body should feel like and act like when I want to achieve proper form alignment in a certain way: I don’t understand how to move my body from one location to another over time in order to master an asana in a smooth way.

Intent

I started this project because the health benefits of yoga physical postures (asanas) fascinate me. I myself am an athlete and am always amazed by the complexity of the technique conveyed by yogis through their body movements. When I learned that there was a lack of effective tools that help experienced yogis find and understand movement qualities in solo learning environments, I was further motivated to investigate the topic.

I address my personal experience that inspired the intents of this thesis, which are:

(1) exploring new ways of representing movement-related information

(2) helping yogis enhance body awareness and control

(3) creating tools that can inspire research in other motor learning fields

Scope

In motor theory, this understanding of the “how” is defined as an understanding of movement quality, that is, the way humans execute movements with respect to time and space (Alaoui, S. F. et al., 2012). One way to understand movement quality is to extract and represent movement-related information. Existent methods range from a simple video recording, to position mapping of specific body parts, to animations of polygonal imageries.

While these approaches are effective in communicating the spatial trajectories of movement over time, they provide little information about the relations between yogis body movement and the movement quality the yogi produces. For this thesis, I approached the issue of “understanding movement quality” through bodily awareness. I was interested in exploring new ways of representing movement-related information that allow yogis connect their body with movement qualities in an easy and natural way. I aimed to help yogis enhance their awareness of their body capabilities and develop the ability to precisely control their own body with their mind. I also created tools that can inform other movement-quality-related studies or be implemented to other motor learning fields.

Goals

The primary research and design focuses of this thesis are below. I will discuss how I decided to focus on the topic of movement quality in yogic learning with literature review.

(1) finding target user group among yogis

(2) confirming and updating audience’s needs in relation to movement qualities

(3) designing representations of movement-related information in a meaningful way

(4) designing accessible and delightful user experiences for the proposed tool

LITERATURE REVIEW

15+ papers and articles that are related to yogic learning, movement study, and mixed reality technologies.

Motor Learning

In this stage, I discuss my literature review process on movement and motor learning, and address the increasing interest and need in exploring movement qualities in many motor-learning fields such as physical health. Then I discuss the concept of movement quality in the context of yoga, and point out the significance of understanding “movement quality” in yogic education.

Significance

I started my thesis with extensive literature reviews on movement and motor learning. What I found is that there is an increasing interest and need in exploring movement qualities in many fields such as physical health, etc. Labanotation is a notation system developed by Rudolf Von Laban in 1928. Within the system, the Effort Graph is used to visualize eight basic elements that constitute the quality of movement. While the graph symbols can be useful in clarifying and communicating the quality of a given movement, its complexity makes it very difficult to understand and be used by the general audience.

Movement Quality in Yoga

In the field of yoga, having a healthy range of movement quality in all joints allows the body to move and adjust its position efficiently. Functional movement is movement that helps us perform daily life tasks better. Many people consider "pretzel"-like yoga poses, deep ranges of motion, and passive stretching to be non-functional. However, yoga supports and enhances functional movement. Functional movements are integrated movements that require many muscles across multiple joints to work synergistically.

Movement quality in yoga is often used interchangeably with the term “style”. Throughout history, some yogis have created distinguished and systematic combinations of movement patterns and qualities, which later became different asana style categories, such as “Hatha Yoga” or “Slow Flow Vinyasa”.

PRIMARY RESEARCH

Sample Study Group

After the literature review process, I became particularly interested in movement qualities in the context of physical education. In order to understand exactly how movement quality is playing a role in the current learning process, I worked with a group of Yogi's from yoga studio as a sample group for asana learners.

Research Methods & Questions

I chose “observation study” and “in-depth interviews” as my research methods in that observation study helped me to uncover Yogi’s unconscious attitudes toward movement quality, and in-person interviews can reveal Yogi’s opinions and desires in a more intimate and direct way. I consider the following four research themes to guide the creation of my research questions to be used in the observation study and interviews with Yogi learners:

• Learning Expectations

• Awareness of Movement Quality

• Means of Understanding Movement Quality

• Body Execution of Movement Quality

Competitive Analysis

I conducted competitive analysis to confirm the originality of my proposed design, which comprised of comparison with current yoga practice venues, as well as comparison with innovative motor learning products that use mixed reality technologies. In summary, the proposed tool can help target yogis fulfill the following three needs all at once in a movement quality exploration process: discover various movement qualities, compare between two movement qualities in real-time and practice movement quality in a solo environment. And most of the product competitors cannot achieve in doing so.

Part I: Comparison with Current Yoga Practice Venues

The chart shown below illustrates the advantages of the proposed tool in detail: while in-person yoga classes are good at providing real-time feedback, these feedback are momentary and lacked tools to keep track of; digital learning venues provide the right environment to practice movement quality, but they fail to provide real-time comparison between yogis movement quality and the reference quality.

Part II: Comparison with Relevant Innovative Works

From literature review I learned that the evolution of learning paradigm from traditional methods toward computer vision has eventually led people to adopt mixed reality (MR) as one of the promising directions for learning technology. Emergence of body tracking technologies such as Microsoft Kinect (motion tracking) or Athos Suit (muscle activity tracking) has shown great potential in integrating with MR to enhance yogic learning experiences. From the competitive analysis I found that many MR tools have been developed to learn postures and for artistic/entertainment purposes, and few of them focus on finding and understanding movement qualities.

The three works shown below are attempts that use MR to explore movement qualities:

A tool for comparative visualization between takes of motion capture data (Malmstrom, C. etal, 2016). Although this tool is helpful in showing details in movement differences, it cannot show those differences in real time. Yogis need to find additional tools to capture motion and stop asanas to review the data.

This system visualizes movement qualities in 3D shapes and motion trails in real-time (Tsampounaris, G. et al, 2016). Although the system inspires yogis to explore different ways of moving in a self-reflection approach, it does not provide additional motivation or goal (like guiding prompts) for yogis, who might be lost in the sea of unrelated visualizations. Also, the system cannot show differences between current and previous movements, which makes it hard to reflect on yogic practice.

The proposed design envisioned an augmented mirror that provides visual guidance on physiotherapy exercises (Tang, R. et al, 2015). Since the system is designed in a physical therapy context, it does not provide a variety of movement qualities but one correct way of moving.

SECONDARY RESEARCH

Observation Studies & Interviews

I observed two yoga classes with more than twenty learners, and conducted seven in-depth interviews. Below is photo of the observed yoga class and some sample interview questions:

• Why you decided to join this yoga class? What is your learning goal?

• What are the different components in Hatha yoga you think you need to learn?

• Have you tried other ways of learning yoga other than yoga class? What do you like/dislike about it?

• Do you see any differences between how your body moves versus how the instructor’s body moves? Can you describe them?

From observation studies and interviews, I synthesized main insights:

Insight 01 - Studied yogis wish to find a variety of ways to execute movement in order to expand the expressive range of their bodies.

As studied yogis become more experienced and gain more interest into asanas, they no longer just want to perform flows arbitrarily. Instead, they want to discover as many asana variations as possible so that they know how to move their body with precise technique when they want to target tension release in certain poses. They believe that by increasing exposure to unfamiliar movement qualities, they will have a clearer picture of the relationship between the physical body’s technique, the yoga flow and the quality of their movement.

On the one hand, this insight aligns with what I found in literature review about the significance of movement quality in yogic learning, on the other hand, this insight revealed the group of audience that I targeted for my project: once beginner yogis have memorized the steps to yoga flows, they start to shift their focus from sequence of movements to the execution of the sequence, and raise a series of needs that are specifically relating to movement quality. Therefore this group of yogis can be said to reach an intermediate level in yogic learning.

Insight 02 - Existing ways of finding and understanding new movement qualities are very limited.

The studied yogis have directly or indirectly expressed the difficulty of discovering new movement qualities. In a typical yoga studio learning environment, the instructor can only provide a few ways to perform a certain move, and students have to watch online videos to see more possibilities. However, it is very hard to understand new movement qualities from these venues. Obstacles include singling out a specific yoga move, copying and repeating the desired movement quality, etc. Below are some quotes from interviews that support this insight:

• “I see trainers demonstration, but then I don’t quite remember it. When I practice on my own, I don’t know what to do.”

• “I go to yoga expos often to see different flow styles. But it is almost impossible to follow. People will improvise with other yoga asanas.”

• “I don’t want to copy the trainer because his moves are very muscular. I watch tons of videos to see more possible ways of doing a move.”

• “I need to pause, play and rewind the video over and over to understand what is going on. It is really tedious.”

Insight 03 - Yogis find it hard to practice movement qualities in group learning environments.

Yogis also said that when they perform in a group, they find it uncomfortable practicing their own movement qualities in front of others because the in-process movements are not presentable. In addition, if there is an instructor, his or her movement quality is often presented in a fragmented and distorted way due to demonstration purposes. Therefore, they wish to have non-disruptive access to the desired movement quality as it is performing in solo learning environments.

Below are some quotes from interviews that support this insight:

• “I prefer to practice my movement qualities at home alone. I feel more comfortable and can focus on myself.”

• “It is embarrassing to practice hip motion in class. I prefer to practice certain asanas at home alone. I feel more comfortable and can focus on mastering proper technique.”

• “When a trainer breaks down a movement by demonstrating an asana, their movement quality is a simplified version. I don’t gain much knowledge of how to correct form alignment in relation to actual movement quality.”

DEFINE

HMW Question

Based on the insights collected from observation studies and interviews, I determined the design goal of my thesis:

How can we help yogis track and improve form alignment when exploring a variety of movement qualities in solo learning environments?

Identifying Users & Needs

The diagram below illustrates this identified user group: beginner-to-intermediate yogis who have a high degree of familiarity with the sequence of yoga steps. The diagram also lists some of this user group’s subgoals, which all lead to one larger goal: enhancing the technique of body movement through discovering new movement qualities and knowledge of body to master form proper form alignment.

IDEATE

Test Existing Theories & Methods

Step 1: Search Relevant Theories & Methods

With this design goal in mind, my next step was to search for existing theories and methods of movement quality exploration and test them with users to find out what works and what does not, what are the expectations and concerns.

Somaesthetics:

• This methodology emphasizes the importance of bodily awareness. This theory’s main contributor, Richard Shusterman, argues that a heightening of somatic consciousness would not only increase the perceptual awareness of meanings and feelings, but also enhance artistic appreciation and creation (Shusterman, R., 2012). This theory offers two ways of viewing the body: viewing the body as an external machine that works independently from the mind, the other one is viewing the body as an internal vehicle that functions as the extension of the mind.

Making Strange:

• This methodology (Loke, L. & Robertson,T., 2013) offers various analytical and experiential ways to de-familiarize the movements of the body. For example, “Finding Pathways” is identifying the specific part of the body that yogis use when they move, and changing the moving sequences of those parts. And “Imagery” is using descriptive metaphors to discover new movement qualities, such as “perform Dancers Pose like you are a growing tree.” This methodology argues that these methods enable: an investigation into the experience of movement, and can open up new spaces in the design of artifacts and technologies.

Step 2: Design the Workshop

Based on this research, I came up with the activity framework for an exploratory user workshop, which is illustrated below. I grouped variables that could effect movement quality based on the found concepts and methods, and created prompts that ask participants to sequentially experience the changes of these variables while performing asanas, and experience the resulting movement qualities with and without seeing themselves.

The major goal of the workshop was to test the effectiveness of these prompts as a way to heighten yogis somatic consciousness and thus discover new movement qualities. Participants were asked to follow a series of analytical and experiential prompts that identify some of the variables summarized in the framework, then move their body accordingly. For example, an analytical prompt would ask participants to change the kinematic parameter of a specific body part, and an experiential prompt would ask participants to perform a yoga asana with an imagery in mind (perform Dancers Pose as if your are a growing tree).

Step 3: Conduct the Workshop

I recruited five yogis to participate in this exploratory user workshop. During discussion, participants are positive about the helpfulness of using different types of prompts to increase yogis body awareness and discover new movement qualities during form alignment. Compared to the conventional way of discovering new movement quality through other yogis, this approach embarks on a journey of self-discovery, which offers greater possibilities in understanding the relations between movement quality and the body, the mind and thus expanding the range of body expressiveness when mastering asanas.

Design Principles

From this workshop, I also derived three design principles:

Principle 01 - Multi Modes

• In addition to learn proper form alignment from trainer, it is promising for the proposed design solution to provide other pathways to discover new asanas.

Principle 02 - Comparative Feedback

• Yogis need to compare the motion of different asana styles so that they can understand the relations between body and movement qualities when correcting their form. The solution needs to provide visual feedback that makes this comparison process easier.

Principle 03 - Seamless Experience

• Yogis wish to spend more time on yoga itself rather than wasting time on the logistics. The solution needs to provide integrated functions and smooth interactions to ensure a continuous flow.

Synthesis of Findings

In this stage of the process I cover the steps I took to develop research findings into the final design, which include:

• (1) Speed dating of feedback typologies with graphic and interactive prototypes, which results in 3 types of feedback to be included in my proposed system “live + video split view”, “real-time visualization” and “motion difference data.”

• (2) Testing system design with storyboards and initial user experience design.

• (3) User testing with mid-fidelity prototypes and model. Here I discovered mismatches in the designed experience with user’s mental model and confusing interactions on the interface.

• (4) Design iteration into final design, which is illustrated by high-fidelity animated prototypes and model.

• (5) Development of a functional visual notation prototype using JIRA, GitHub, and Unity.

• (6) Competitive analysis with both current yoga practice venues and relevant innovative works to confirm design originality.

Since one of the design principles I found was the importance of comparative feedback, the first step I took was to determine what types of feedback should be provided to aid movement comparison. I created graphic and interactive prototypes and tested them with yogis to hear their thoughts.

Speed Dating of Feedback Typologies

From conducting speed dating of feedback typologies, I found that 3 types of feedback are particularly useful for my target users:

Live + Video Split View:

• The first feedback type is a split screen that allow yogis to see both themselves live and a video simultaneously.

Real-time Visualization:

• The second type is real-time visual overlay augmented on the screen, which gives prominence to body motion itself.

Motion Difference Data:

• The third type is data about motion differences, which can be recorded and reviewed later for further reflection.

Speed Dating of System Concepts

The next step I took was to design the overall system that helps yogis explore a variety of movement qualities in solo learning environments through a technique-based educational tool. I generated three concepts and illustrated my ideas using storyboards. I presented the storyboards to yogis and gathered their feedback.

What I learned was that yogis are concerned about the usefulness and cost of the proposed multimodal systems. The usage of AR glasses would allow for haptic or three-dimensional feedback, but their cost might be very high, and they might also impede yogis movements. Using a digital mobile application with mixed reality features is a good idea since yogis wish to see feedback in real-time in an accessible platform. Another consideration was a motion controlling gaming console as feedback is in real-life scales.

DESIGN

System & Experience Design

Based on the result of concept speed dating, I decided to design a feature for novelty motion controlling games such as the Nintendo Switch to master movement qualities in solo learning environments. The yogi first purchases the product from online channels and receives the digital package. The yogi installs the plug-in feature, which can be connected to other output devices to see themselves in real-life scale. The televisions webcam will track the yogis movement and provide feedback. The yogi can use the joy-cons to remotely control the application and practice movement quality.

The application offers a mode of movement quality exploration: Create Mode. Under the create mode, yogis can choose to increase bodily awareness by investigating relations with the body itself, then follow prompts from interested categories to discover new yoga asanas by mastering their form alignment. During exploration, yogis can use augmented visual overlays to compare style differences between themselves in real time and a recorded video of either themselves or avatar trainer. I mapped out the user journey for the proposed system, sketched out the application interfaces, then refined the sketches into mid-fidelity wireframes.

• User Journey of the Create Mode

Computational Design Prototype

My research allowed me to draw parallels between machine learning and data visualization by making use of graphical notations when collaborating with an engineer to intelligently combine the biomechanics of movement with sensor data, video capture technology, and cloud-based software services. 

The form tracker will inform the ways in which users unlock new levels of yoga programs through their ability to match the correct and desirable form alignment over time. The logic behind the documentation tool is based on a line being indicative of the origin of gesture and how it moves over a timed duration. Where as the sphere represents how long one holds that gesture. Accelerated movements are represented in smaller sphere formations and decelerated movements form larger spheres.

To confirm the viability of the proposed tool using current or near-future technologies, I produced a functional prototype to realize the motion tracking and visual overlay display features integrated in the exploration process.

TESTING

Revisions

Usability testing was performed for the mid-fidelity prototype of computational design. Testing was done by observing participants learn how to do Dancer Pose (Natarajasana). Once testing was completed of the proper alignment and technique, helping users safely move into Dancer Pose I synthesized results. The results determined pain points, frustrations and routes towards improvements.

Low-Fidelity Designs

Dancers Pose: Improved Balance

Dancers Pose: Improved Flexibility

Dancers Pose: Improved Strength in Legs & Ankle Muscles

ITERATION

Mid-Fidelity Designs

Modifications were made to the code and settings to adapt designs with the prototype shown below. The necessary changes were informed by the user’s goal to track and master the Dancers Pose while utilizing the system for the logic of the tool. This process resulted in the visual tracking of body postures in real-time to provide feedback with performance insights on yogis form alignment with mastery of asana training programs. This included differentiating critical values in the coding which were used to create an in-depth understanding about the type of movement and how much movement is being enacted. Feature development innovates the way people coach, train, analyze, and improve fitness-specific training programs.

Pose Instructions: (1/3) Bringing bent knee down in line with standing leg

Pose Instructions: (2/3) Begin to tip forward and reach lifted foot up behind you

Pose Instructions: (3/3) Let go with free arm, turn chest toward the front of mat lifting elbow up

LAUNCH

Reflections

Through this experience I learned myself to be a highly motivated practitioner with a passion for creating and supporting great products. I thrive on collaboration, working side by side with people of all backgrounds and disciplines with strong verbal and written communication skills. Working within a research development environment founded an ability to learn new skills quickly and use the skills efficiently.

Points of Improvements

The following are aspects in the proposed tooling that I wish to further refine, including further developing the user experience of the pathways proposed in the system, improving the movement comparison functionality and experience. I also wish to point out aspects that I believe hold potential in aiding yoga education and indicate future research opportunities in other motor-learning fields, which include offering a self-discovery approach to find new movement quality, offering features to compare different movement qualities and record motion data to visually track movement changes.

Opportunities

The proposed design offered features that can compare different movement qualities and record motion data to keep track of movement changes. This aspect of tooling holds exponential potential for expansion of key offerings in sports training or physical therapy sectors, particularly using the following labwork to create biomechanical data with the aim of helping to improve physical performance and rehabilitation from injuries. Patients who are going through the rehabilitation process need tools to track their movement progress in order to self-reflect and communicate with physicians to receive the most suitable treatment. This need is also true for athletes. This might encourage other researchers to adapt my proposed design into these fields and develop new tools for use.

Key Learnings

• Passion for improving Health, Fitness and Sports performance fields.

• Greater understanding of how to apply data-driven technology suggestions and effectively contribute to discussions as part of a cross-disciplinary team within a research environment.

• Interpreting research insights and building understanding of users and context.

• Defining user experience goals and design principles.

• Designing the architecture of the functions and information of the experience.

• Designing the interaction patterns between functions/information and users/customers.

• Designing pathways of navigation through an experience; demonstrating understanding of content, functions, and relationships within the system.

• Supporting and planning for variation of user contexts (e.g. culture, environment, and activity).

• Converging and translating technological, operational, and functional constraints into human-centered opportunities.

• Facilitating and designing for the relational aspects across every part of the experience.