New research from Italy leverages Transformers to offer a more intelligent and acerbic method of restoring damaged archival video – a goal that could help VFX practitioners that need to acquire better historical data of actors and other potential subjects for whom archive material is often not in the best state.
Generative AI very often produces badly-cropped images, because the models are trained on non-square images which have been automatically (and, usually, quite randomly) cropped by the training process. Though it’s possible to devise methods for better cropping, many of them are compute-intensive, and would radically raise the cost for training hyperscale datasets in systems such as Stable Diffusion. But now, researchers from Japan propose a more economical yet intelligent way of performing salient crops on large amounts of data.
Putting dots on faces has been a staple of computer vision and visual effects processes for decades. It may seem old hat in the age of transformers and Stable Diffusion, but, along with CGI-aided neural rendering processes, it offers a level of control that is missing in dazzling new generative AI systems – and a new system from China proposes a landmark method that could cover all species, instead of needing individual systems for faces, hands, animals and other sub-categories.
If you’re old, you wear glasses. If you’re smiling, you’re probably a woman. These and many other assumptions and biases in datasets lead to generative systems that can produce biased images with unwanted elements. But new research from Canada and the US offers a way to disentangle these ‘spurious associations’, and may help to develop better systems to produce synthetic data for VFX and other purposes.
Neural synthesis has a harder time depicting a person sitting at a desk than CGI does, because CGI has legs ‘waiting in the wings’ if they’re needed, whereas the neural pipeline only knows about what it can see. For this reason, new research is offering a better method of handling occlusion for AI-generated humans, albeit in an uneasy partnership with CGI itself.
Understanding what facial expressions mean is going to be essential in neural facial synthesis in the coming years. But in many cases, it’s extremely difficult to correctly guess an emotion unless you can see more of the context than just the face (one example being ’embarrassment’, which of necessity cannot be felt or studied without understanding the context). Now, researchers from Canada are proposing a more intelligent annotation pipeline, using Large Language Models such as GPT3, in order to bring more intelligence to Facial Expression Recognition (FER).
Films such as ‘Here’ and ‘Indiana Jones and the Dial of Destiny’ are using advanced machine learning technologies to age and de-age characters. But it’s still a pretty ‘manual’ and painfully laborious process. Now, new research offers a potential pipeline that could shave off or add years to actors in a more systematic way, with the use of Stable Diffusion.
It’s hard to guess what people look like from the side if you only have frontal views of their face; and the chronic lack of profile views in popular datasets makes this a stubborn data problem that’s standing in the way of 360-degree facial synthesis. Now, researchers from Korea are offering a method that might alleviate this traditional roadblock.
New research from France and Switzerland uses Generative Adversarial Networks (GANs) to create extra examples of races and genders that are under-represented in historical face datasets, in an effort to offset controversies such as the tendency for facial recognition systems to fail to recognize (or to over-recognize) particular types of people.
Getting your face into Stable Diffusion has been a relatively complicated affair since the text-to-image system launched in August of 2022 – but a new offering from China and Singapore proposes a method of embedding your own image into Stable Diffusion with only a single photo, instead of needing to train a model every time.
