Researchers have managed to create a new AI technology which possesses the capability to bring a human subject within a photograph “alive.” The tech, which looks like something from a futuristic sci-fi movie, is able to make lift a person from the photo, and have them walk, run, sit, and even jump forward out of the photo in 3D.
Details of the development feature within a paper about the new AI, which comes from scientists at the University of Washington and Facebook. Entitled “Photo Wake-Up: 3D Character Animation from a Single Photo,” it details the advancements involved in bringing such technological capabilities to fruition.
The AI can read a photo in order to detect any human subjects featured. From there, it “turns the person’s silhouette into an animatable mesh after identifying and labeling the various body parts.” Now for the fascinating part: the AI is able to guess a weight of the subject, from which it can gauge how the subject would move and navigate in a 3D space. Of course, it also manages to fill the "hole" left in the background of the photo, caused by the ‘removal’ of the subject. The process is similar to the Content-Aware Fill tool in Photoshop.
UW professor Ira Kemelmacher-Shlizerman said:
The big challenge here is that the input is only from a single camera position, so part of the person is invisible. Our work combines technical advancement on an open problem in the field with artistic creative visualization.
While the tech isn’t of particularly high quality of that, its researchers are keen to note this is a huge breakthrough – and it’s also just the beginning for what this kind of AI can potentially do in future.