We live in an increasingly digital world.
If you've been paying attention, this shouldn't be much of a surprise. Even before the pandemic, technologies like virtual reality and augmented reality were attracting considerable interest. Amid lockdowns and quarantine measures, that interest has only grown exponentially.
By now, you've probably heard all the hype surrounding the Metaverse. At best loosely defined and at worst little more than a marketing buzzword, there's at least one concept at the core that shows considerable promise: the notion of one day connecting with each other through virtual reality not just as customizable avatars, but as digital vivids. representations of our real selves.
The idea of connected online environments populated by realistic, simulated, dynamic avatars of our friends and family.
This may sound like something straight out of the annals of science fiction. But is not the case. Rather, it's something that Chinese tech company OPPO has just made more feasible than ever with its AI-guided 3D body reconstruction technology.
Creation of virtual bodies
When it comes to 3D body reconstruction, one of the biggest challenges has historically been the realistic representation of clothing, especially those that are baggy. The wireframe itself isn't the problem - we've been mastering realistically proportioned virtual bodies for quite some time. The problem is that the clothes react like… well, real clothes.
Without getting too technical, traditional digital avatar reconstruction methods tend to result in blurry or just plain fake clothing. The problem is that there is a significant complexity gap between the different body poses. Ensuring that clothing realistically accommodates these movements takes inordinate effort.
Until recently, there was no real solution to this problem other than "send more engineers there". And that's where OPPO comes in. As demonstrated at the 2022 Computer Vision and Pattern Recognition Conference (opens in a new tab), the company has finally tackled what may well be one of the oldest problems in the field.
As described in this article (opens in a new tab), OPPO's process began by training a refined algorithm to understand how fabric forms and deforms with movement. Initially, he focused on identifying and understanding small details like logos, hems, and textures. The key to success here lies in treating these tissues as a separate layer from an avatar's skeletal movements, something traditional rendering methods still fail to achieve.
Along with this training, the algorithm leverages a series of predefined moving points attached to the body model of an SMPL model (essentially the standard 3D model of the human body (opens in a new tab)). These points are then combined with structured local radiation fields, processes that simultaneously model appearance and geometry to generate more compelling images (opens in a new tab).
Based on its new method, OPPO can generate three-dimensional dynamic models one by one through simple video analysis, an innovation with very interesting applications.
OPPO Virtual Avatar Creation Process: A Step-by-Step Guide
As we established earlier, digital human modeling has traditionally been an extremely complex task. Scanning the model is only the first step. Costs aside, success typically requires a series of manual processing and engineering-intensive tasks.
As seen in the video above, OPPO's process essentially throws these difficulties out the window.
1. First, the model body data is collected through OPPO's Digital Avatar Lab.
2. After the smartphone software collects enough data about the subject, the AI starts to generate the 3D avatar.
3. A professional dancer acted as a model of the human body and the movements were recorded. The first step in the rendering process is to generate a skeleton with a series of predefined anchor nodes.
4. The algorithm continues to analyze the movements of the skeleton to differentiate it from the shape and movement of any clothing applied to the avatar.
5. Finally, a few hours later, rendering the neural scene generates a fully clothed digital character with realistic clothing physics and movements.
Do you have no rhythm and are you ashamed not knowing how to dance? Your avatar won't have this problem as you twist and turn, your clothes float around you like they would on the dance floor. While the algorithm is still under development in the lab, it will gradually become more efficient and effective as it receives more training data, ultimately making it applicable to limitless real-world scenarios beyond dance.
Visualize a digital world with OPPO
It is a big step towards virtual reality in total immersion. As the technology behind OPPO's algorithm improves, digital and physical will be more and more combined. Unsurprisingly, this has some very interesting applications.
First, malls and other retailers can provide more immersive online experiences through technologies like virtual locker rooms and food courts. This can be combined with lifelike, realistic custom avatars that allow people to interact with the virtual world on a level that was previously impossible. These avatars would also allow friends and loved ones to see the "real important ones" and behave as we wish, even at great distances.
There are also many other applications.
More compelling and dynamic characters based on the actual movements of people in video games combined with reduced development costs. Remote work supported by virtual reality meeting rooms in total immersion. AI-guided personal training and fitness classes.
Realistically, the sky is the limit here.
Believe it or not, technology like this is nothing new for OPPO – the company has been at the forefront of artificial intelligence for several years. He initially started his research on AI development in 2015, since then he has deepened his focus considerably. Today it operates the Institute for Intelligent Perception and Interaction, with dedicated research and development teams focused on a range of different AI technologies and applications.
In addition to 3D body reconstruction, OPPO's technology has also been used in products such as CybeReal real-time spatial AR generator (Opens in a new tab), OPPO Air Glass (Opens in a new tab), and Omoji (opens in a new tab), which uses facial recognition to create 3D emojis.
All this to say that, as exciting as it is, this innovation only scratches the surface of OPPO's research. The company designs new technologies with each passing day. And there's no doubt that next year will have something even more exciting to share.