Gal Shaya
Creative Technologist
Walmart & NEtflix - The hidden world of waffles + mochi - in store AR experience (2021)
The Hidden World of Waffles + Mochi app allows families to access games, recipes, and clips from the Netflix show Waffles + Mochi. By pointing a camera at foods and scanning them, product recognition AI unlocks unique content related to each featured food item.
Using AI to inform product recognition enables this app to work with thousands of scanned products. While most augmented reality experiences respond only to graphic cues (like packaging, logos, or QR codes), this app can recognize items without graphics. The app knows the difference between a stack of apples and a stack of tomatoes. It can also launch the same content whether you scan a can of tomatoes or a fresh tomato.
Role
Creative Technologist
Responsibilities
Ideation, Research, ML Training, Testing
Technologies Used
ARCore (iOS), ARKit (Android), App Clip (iOS), Instant App (Android), AI/ML, 3D Design, Unity
Eko Volumetric Fashion Prototype (2022)
Volumetric capture is one of the most fascinating emerging technologies in the media space. While working on numerous fashion campaigns, we realized the legacy production workflow is a huge hurdle for a field built around high-quality content and rapid delivery.
We teamed up with our long time collaborators Real Motion VFX, who created a patented technology that unifies the data captured by the volumetric capture system to an editable 3D mesh, to create a prototype of an interactive redress fashion experience that uses exclusively volumetric captured footage.
ROLE
Creative Technologist
RESPONSIBILITIES
Ideation, Research, Post Production
TECHNOLOGIES USED
Volumetric Capture, Mesh Unifying Tool, Eko Studio (Interactive Video), 3D Design, Front End Development (JavaScript, CSS)
Audio Tester - Original Prototype (2021)
Every healthy company has a secret passion (read: obsession) that consistently arises in the conversations between the morning meeting and the evening grind to make a tight deadline. For Particle3, these conversations center around music and audio.
After making dozens of projects in the e-commerce space (including XBOX and tens of others that live on Walmart.com), we realized that interactive video could completely overhaul the way audio products are experienced online. Since audio quality is dependent on the monitoring system through which it is played, it’s hard to reliably judge the quality of an audio product. With the option to toggle between A/B samples in realtime, viewers maintained a consistent reference point that allows them to better understand the qualities of a certain product, regardless of their playback system.
Additionally, we have made the products explorable which teaches the viewers about different features and their capabilities.
ROLE
Creative Technologist , Creative Director
RESPONSIBILITIES
Ideation, Research, Videography, Post Production, Motion Graphics, Implementation, UX
TECHNOLOGIES USED
Eko Studio (Interactive Video), Front End Development (JavaScript, CSS)
Walmart KidHQ (2019)
We were asked by Walmart to create an online, interactive experience that functions as a safe space for kids to test out their favorite toys in a fantastical visual environment.
The experience was built from hundreds of individual videos, seamlessly connecting UI elements, live action shots, and CGI. In some cases, all three elements appeared at the same time.
Previously, I took a key role in this project’s predecessor (2018’s Walmart ToyLab) in which I created solutions to enable the biggest interactive experience on the eko platform at the time. For KidHQ, I was given multiple important roles to create this experience. My responsibilities varied from ideation and pre-production, to supervising a team of editors to create more than 4 hours of edited material in less than 6 weeks. I managed these elements all while composing the music and participating in a cross-functional team responsible for making daily adjustments to optimize the technology and video for this project that was unprecedented in scale.
The experience won 2 Silver Telly awards and a Digiday award for “Best Interactive Content Series or Piece”.
Role
Creative Technologist, Editing Supervisor, Audio Post Production Supervisor, Music Composer
Responsibilities
Ideation, Research, Video Editing, Prototyping, Workflow Design, Music Composition, Interactive Supervision (on set)
Technologies Used
Eko Studio (Interactive Video), Front End Development (Javascript, CSS, Lottie), Back End Development, 3D Design
Stok x Lil jon - stok yourself web experience (2018)
We were approached by the production companies Fourclops and Prettybird to join a wild project for Stok Cold Brew Coffee featuring Lil Jon. We took ownership of the interactive and tech aspects of the experience.
The project was a website in which the user entered a name and selected an activity to get hyped for (for instance, call your mom). The viewer would then get a customized hype song that included their name and chosen activity, all sung in a AI generated voice.
We teamed up with VocalID to create this unique singing AI voice, and created a unique back end muxing system to create a personalized video within seconds.
Role
Creative Technologist, Interactive Producer
Responsibilities
Ideation, Research, Prototyping, UX, Testing
Technologies Used
Text To Speech, AI/ML, Front End Development (Javascript, CSS, Lottie), Back End Development, API Integration
Thank you, and feel free to explore the rest of the website!