AI ML DL

Main image derived from https://unsplash.com/photos/mens-blue-and-white-button-up-collared-top-DItYlc26zVI
AI ML DL

Detecting AI-Generated Images With Inverted Stable Diffusion Images – and Reverse Image Search

A new system for the detection of AI-generated images trains partially on the noise-maps typical of Stable Diffusion and similar generative systems, as well as using reverse image search to compare images to online images from 2020 or earlier, prior to the advent of high-quality AI image systems. The resulting fake detector works even on genAI systems that have no public access, such as the DALL-E series, and MidJourney.

Read More »
Illustration developed from 'AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control' (https://xbpeng.github.io/projects/AMP/index.html)
AI ML DL

Powering Generative Video With Arbitrary Video Sources

Making people move convincingly in text-to-video AI systems requires that the system have some prior knowledge about the way people move. But baking that knowledge into a huge model presents a number of practical and logistical challenges. What if, instead, one was free to obtain motion priors from a much wider net of videos, instead of training them, at great expense, into a single model?

Read More »
Sources: https://unsplash.com/photos/a-man-in-a-blue-shirt-is-doing-a-yoga-pose-ak3bgRHt3zY | https://huggingface.co/spaces/hysts/mediapipe-pose-estimation
AI ML DL

Improving Pose Estimation for Generative AI

Turning human poses into skeletal stick-figures and back into new images via generative AI is fraught with pitfalls, particularly when the pose in question is uncommon, taken from an unusual angle, or in some way ‘out of distribution’ for what the target generative system is expecting. Among systems that perform these tasks for Stable Diffusion, ControlNet’s openpose module has become very popular in the last year or so – but new research has improved on it, bringing us nearer to the dream of purely generative video generation.

Read More »
NPGA: Neural Parametric Gaussian Avatars - https://arxiv.org/pdf/2405.19331
AI ML DL

Better Human Facial Synthesis With Gaussian Splatting and Parametric Heads

Gaussian Splatting has taken the VFX scene by storm over the last 6-8 months, and new research backed by Synthesia has produced some of the most impressive Splat-based human avatars ever seen, overtaking the state-of-the-art in tests. But is the burden of complexity, in adding neural systems, too high a price to pay for the improvements?

Read More »
GSDeformer. Source: https://arxiv.org/pdf/2405.15491
AI ML DL

Deforming 3D Gaussian Splat Models With Cages, the Easy Way

After years of struggling to get NeRF and GANs into a more user-friendly configuration, the advent of 3D Gaussian Splatting (based on an older technology formerly used for medical imaging) has enlivened the AI synthesis research scene, which is hoping to obtain a new and more user-friendly version of older and less effective CGI technologies. One example is this new project, which allows the user to deform 3D Gaussian Splats with an external cage – just as Hollywood has been doing with CGI for over three decades.

Read More »
LayGa - Source: https://arxiv.org/pdf/2405.07319
AI ML DL

Editable Clothing Layers for Gaussian Splat Human Representations

While the new breed of Gaussian Splat-based neural humans hold much potential for VFX pipelines, it is very difficult to edit any one particular facet of these characters, such as changing their clothes. For the fashion industry in particular, which has a vested interest in ‘virtual try-ons’, it’s essential that this become possible. Now, a new paper from China has developed a multi-training method which allows users to switch out garments on virtual people.

Read More »
A film grain effect applied to a stock image - source: https://pxhere.com/en/photo/874104
AI ML DL

The Challenge of Simulating Grain in Film Stocks of the Past

Hit shows like The Marvelous Mrs. Maisel and WandaVision use some cool tricks to make modern footage look like it was shot in the 1960s, 70s, and various other eras from film and TV production. But one thing they can’t quite pull off convincingly is reproducing the grainy film stocks of yesterday – a really thorny problem that’s bound up with the chemical processes of emulsion film. With major directors such as Denis Villeneuve and Christopher Nolan fighting to keep the celluloid look alive, it would be great if AI could lend a hand. In this article, we look at the challenges involved with that.

Read More »
Paint-bu-inpaint
AI ML DL

Better Stable Diffusion Inpainting by Learning to Remove Real Objects

Inserting novel AI-generated objects into images only through text-based instructions is a tricky task, and many of the best current models have used synthetic data to generate the necessary datasets. However, a new work from Israel attacks the problem from another angle: by REMOVING objects from training images, and telling the training system that the original images are actually the modified/edited images. Tests indicate that this use of real-world imagery obtains superior results.

Read More »
AI ML DL

Trying out New Clothes in Stable Diffusion-Based Videos

The fashion industry has been investing in virtual try-on systems heavily over the last 5-6 years, but to date has not produced a system capable of projecting customers’ diverse appearances into the latest fashions in an actual video. Now, a new system from China has used the generative power of Stable Diffusion to make that facility a reality, in a new project titled Tunnel Try-On, that’s capable of projecting individual items of new clothing convincingly into existing videos.

Read More »
RHanDS
AI ML DL

Repairing the Nightmarish Hands Produced by Stable Diffusion

Stable Diffusion has captured the imagination of the world since its release in 2022, but retains a notable difficulty in rendering human hands – one of the most difficult anatomical challenges also for human artists. A new wave of ‘hand repair’ architectures is appearing in the literature of late, the most recent of which is this complex but effective new post-processing framework from China.

Read More »