The majority of school districts predominantly rely on Chromebooks for student use, making them the most prevalent devices in our dataset. However, we've received feedback indicating that many teachers and students also utilize their personal smartphones for video capture.

After analyzing the findings from our research and studies, the team initiated the project planning phase. Our primary focus was to improve the mobile user experience, making green screen effects more efficient and user-friendly. To achieve this, we researched various approaches and technologies related to auto-background replacement, commonly found in many social media apps. Meanwhile, our developers explored available technical resources, conducting a comparative analysis to assess the advantages and limitations of each option.

I invited other designers to brainstorm the ideas of improving the experiences of capture content in classroom environment. Our primary objective is to streamline the greenscreen setup process, ultimately enabling students to unleash their creativity and achieve greater success with their video assignments while saving time and effort.

Idea 1:

‍ Before recording, setup the default background is the solid green, the user can replace the background if needed. Once open the video in the editor, separate the foreground and background content.

Open the media in the web editor

Idea 2:

After recording, apply the background remove, it’s optional to add a new background or color.

Open the media in the web editor

Idea 3:

After recording, apply the background remove, the file has a transparent background, the user can choose background in the editor.

Open the media in the web editor

• Apple Vision Framework: To achieve this, we harness the power of the Apple Vision framework, available on iOS 15 and newer versions.
• Android version: Google SegmentationMask will allow us to manipulate the data for background removal.
• Background Options: Users can replace the background with various choices, including solid colors, individual media clips stored on their device, clips uploaded to their cloud account, or our extensive library of stock media.
• Output: For the Minimum Viable Product (MVP), the output will be a single rendered clip. This clip can be downloaded to the device, sent to the WeVideo cloud, or incorporated into a mobile timeline. It's important to note that WeVideo does not currently support transparent layers in its editor.

The current limitations and capabilities have enabled me to explore ideas in a practical manner. Sharing my initial concepts with developers and PMs, along with using mockups, greatly facilitated our discussions. We successfully aligned on our direction and the MVP scope, which is focus on making the background replacement feature intuitive and instantly accessible to users when they are capturing with the phone. Students no longer require physical green backdrops; they can record content anytime, anywhere.

Final direction:

The users can replace background before recording, so they can see the final result in real time. The output of the recording will be a background embed media file, so the users don't can import it to the video editor as one clip.

Open the media in the web editor

Usability Test:

I created the V1 flow prototype and test it with the coworkers and some volunteer users. I got some feedback and potential issues based on the tests:

• Camera entrypoint: The camera entrypoint is hidden under the creation flow. If the user only want to capture media within the app, it's not intuitive and not expected.
• Replace background button: The "+" button is not clear to trigger the media library, even not clear as an action.
• Media library: The selection on media library is not clear that there is only ONE selection. Some users tried to select second photo and it feels like a bug.

• Entrypoint: I redesigned the app dashboard so we can emphasize the capture tool and media uploader.
• Default background: I decided to use a default background image to help the user understand the feature more easily so they can replace their own background or keep the default image as a placeholder.
• Background library: Because the scenario is to record the person and replace the background later, we assume the solid color will fit the current user flow better, we prioritized the colors rather than the device library and stock content.
• Pre-trim: After finished recording, the user can playback and pre-trim the clip.
• Edit: Once the recording is added to the video editor, the user can use the keying tool to remove the background and continue their editing.
  

• After the user recorded their video, they can choose to start editing in the mobile app, upload the file to the web, and edit the video on their computer.
• Media uploader also can be a tool helping users to transfer their existing media files on their phone to the web editor. Furthermore, they can add the file to a shared folder in WeVideo, so other users can use the files too.

• The monthly usage of the mobile capture tool has increased by about 90% in 2023 compared to the previous year.
• The number of mobile and web app cross-platform usage has increased by about 10% in 2023 compared to the previous year.
  

• AI-powered features come with specific limitations and capabilities, which are influenced by the technology and tools we select. Unlike some UI development projects, these limitations might not be as easily overcome. For a designer, maintaining continuous communication with developers and exploring possibilities simultaneously is crucial. Our foremost concern is advocating for our users' needs in our decisions, and we must be ready to adjust our design directions as soon as we have a clear understanding of the technology.
• At times, our initial understanding and assumptions about our users' needs may differ from the actual product usage. It's essential to center our attention on genuine user needs, which can be discerned from user feedback and data. We should be prepared to iteratively refine our designs based on what we learn from these insights.