AI ML DL

LayGa - Source: https://arxiv.org/pdf/2405.07319
AI ML DL

Editable Clothing Layers for Gaussian Splat Human Representations

While the new breed of Gaussian Splat-based neural humans hold much potential for VFX pipelines, it is very difficult to edit any one particular facet of these characters, such as changing their clothes. For the fashion industry in particular, which has a vested interest in ‘virtual try-ons’, it’s essential that this become possible. Now, a new paper from China has developed a multi-training method which allows users to switch out garments on virtual people.

Read More »
A film grain effect applied to a stock image - source: https://pxhere.com/en/photo/874104
AI ML DL

The Challenge of Simulating Grain in Film Stocks of the Past

Hit shows like The Marvelous Mrs. Maisel and WandaVision use some cool tricks to make modern footage look like it was shot in the 1960s, 70s, and various other eras from film and TV production. But one thing they can’t quite pull off convincingly is reproducing the grainy film stocks of yesterday – a really thorny problem that’s bound up with the chemical processes of emulsion film. With major directors such as Denis Villeneuve and Christopher Nolan fighting to keep the celluloid look alive, it would be great if AI could lend a hand. In this article, we look at the challenges involved with that.

Read More »
Paint-bu-inpaint
AI ML DL

Better Stable Diffusion Inpainting by Learning to Remove Real Objects

Inserting novel AI-generated objects into images only through text-based instructions is a tricky task, and many of the best current models have used synthetic data to generate the necessary datasets. However, a new work from Israel attacks the problem from another angle: by REMOVING objects from training images, and telling the training system that the original images are actually the modified/edited images. Tests indicate that this use of real-world imagery obtains superior results.

Read More »
AI ML DL

Trying out New Clothes in Stable Diffusion-Based Videos

The fashion industry has been investing in virtual try-on systems heavily over the last 5-6 years, but to date has not produced a system capable of projecting customers’ diverse appearances into the latest fashions in an actual video. Now, a new system from China has used the generative power of Stable Diffusion to make that facility a reality, in a new project titled Tunnel Try-On, that’s capable of projecting individual items of new clothing convincingly into existing videos.

Read More »
RHanDS
AI ML DL

Repairing the Nightmarish Hands Produced by Stable Diffusion

Stable Diffusion has captured the imagination of the world since its release in 2022, but retains a notable difficulty in rendering human hands – one of the most difficult anatomical challenges also for human artists. A new wave of ‘hand repair’ architectures is appearing in the literature of late, the most recent of which is this complex but effective new post-processing framework from China.

Read More »
emotion-gan
AI ML DL

Improving Facial Expression Synthesis Through Gan-Based Frontalization

It is difficult enough for modern Facial Expression Recognition (FER) systems to accurately identify even the minimal 6-8 emotions defined by FACS, from front-facing mugshots; when the person is depicted at a hard angle, the problem is severely compounded. Though this is a problem most addressed by security researchers, it has implications also for the VFX sector, which is increasingly being asked to provide expression-editing tools for faces, no matter where the faces are pointing. A new paper from French researchers claims to have made a breakthrough in this regard, with a novel method of ‘frontalizing’ obscure face angles.

Read More »
SMIRK FER
AI ML DL

Dealing with Unconventional Facial Expressions in Neural Synthesis

The much-used Facial Action Coding System (FACS) has a very narrow range of core expressions – 6-8, depending on which version you’re using. However, film and TV directors interested in using neural tools to adjust facial performances in post are going to need a more precise toolkit, which falls outside the FACS paradigm. Now, researchers from Greece and Germany are at least offering an expression recognition and synthesis system, titled SMIRK, which can address a wider range of facial expressions, and hopefully advance the state of the art in this under-served aspect of human synthesis.

Read More »
AI ML DL

High Resolution (And High Accuracy) Stable Diffusion With a Relatively Simple Hack

Most new innovations in upscaling and text-prompt accuracy emerging from the research sector tend to be tortuous affairs involving the gathering of new datasets, expensive and extensive training, and/or the use of heavyweight adjunct systems. By contrast, this new approach to improving the resolution and fidelity of Stable Diffusion involves only a small change to the internal process – yet offers native upscaling without the traditional Lovecraft-style horrors when one tries to generate an image above the native resolution of images on which Stable Diffusion was trained.

Read More »
AnyV2V
AI ML DL

Multitask Video Synthesis Without Fine-Tuning

Most image-to-video systems require some degree of fine-tuning of publicly released models, which is damaging to the weights of the model, time-consuming, and frequently requires model duplication and subsequent destruction after inference. Instead, a new system uses a novel method of feature extraction and re-injection to provide a potential framework for other systems to operate within.

Read More »