Blog
Controllable Deepfakes With Gaussian Avatars
Could Gaussian Splatting become the hottest new deepfake technology since 2017? The massive surge of interest from the research sector suggests it might – and the latest innovation not...
Badly-Compressed Images Affect CLIP’s Performance, New Research Contends
CLIP is the new darling of the computer vision research, and of image-based generative AI, with wide uptake of the image/text analysis framework across the sector. However, new research...
Low-Cost Deepfake Video Detection With H.264 Motion Vectors
Though it's possible to detect deepfake video by studying movement in a deepfaked face, it takes a lot of resources to do it, and it's not easy or cheap...
Detecting When ‘Fake’ Images Are Actually Real
A new generation of fake image detectors hope to discern AI-generated photos from real ones. But how do they cope when real photos exhibit qualities normally found in fake...
Improving Deepfaked Eye Contact via Sound Analysis
Bad rendering of eye direction in neural facial synthesis can make the difference between an image looking photoreal and looking like bad CGI. We are experts in eye contact,...
Restoring Facial Expressions with CycleGAN
Researchers from Germany have developed a new method of restoring obstructed faces using only Generative Adversarial Networks (GANs), instead of needing to undertake expensive and time-consuming fine-training of existing...
Detecting Stable Diffusion Deepfake Faces
New research specifically targets deepfake faces created by latent diffusion models such as Stable Diffusion, in a security sector that seems stuck on GANs and 2017-era autoencoders.
CGI-Style Object Control With Stable Diffusion
Though Stable Diffusion is an impressive generative system, it has difficulty performing the same operation twice, which makes it difficult to render the same subject consistently across frames -...
A Dedicated Loss Function for Neural Face Training
Loss functions, which determine the ways that a machine learning network should develop during the course of training, are at the heart of generative AI and computer vision systems....
Improving Stable Diffusion With Better Captions
New work from Google Research rewrites the terrible alternative captions that are often found in images on the web, and which subsequently get used as pertinent information for generative...
Restoring Archival Video with CLIP
New research from Italy leverages Transformers to offer a more intelligent and acerbic method of restoring damaged archival video - a goal that could help VFX practitioners that...
The Struggle for Salient Image-Cropping in Generative AI
Generative AI very often produces badly-cropped images, because the models are trained on non-square images which have been automatically (and, usually, quite randomly) cropped by the training process. Though...
One Landmark Estimator to Rule Them All
Putting dots on faces has been a staple of computer vision and visual effects processes for decades. It may seem old hat in the age of transformers and Stable...
Better GAN Disentanglement Could Facilitate Better Synthetic Data
If you're old, you wear glasses. If you're smiling, you're probably a woman. These and many other assumptions and biases in datasets lead to generative systems that can produce...
Uncovering a Body With AI
Neural synthesis has a harder time depicting a person sitting at a desk than CGI does, because CGI has legs 'waiting in the wings' if they're needed, whereas the...
Improving Facial Expression Recognition by Studying Context and Environment
Understanding what facial expressions mean is going to be essential in neural facial synthesis in the coming years. But in many cases, it's extremely difficult to correctly guess an...
Controlling Age With AI
Films such as 'Here' and 'Indiana Jones and the Dial of Destiny' are using advanced machine learning technologies to age and de-age characters. But it's still a pretty 'manual'...
Solving the ‘Profile View Famine’ With Generative Adversarial Networks
It's hard to guess what people look like from the side if you only have frontal views of their face; and the chronic lack of profile views in popular...
Repairing Demographic Imbalance in Face Datasets With StyleGAN3
New research from France and Switzerland uses Generative Adversarial Networks (GANs) to create extra examples of races and genders that are under-represented in historical face datasets, in an effort...
Stable Diffusion Deepfakes and Stylizations With a Single Image
Getting your face into Stable Diffusion has been a relatively complicated affair since the text-to-image system launched in August of 2022 - but a new offering from China and...
Replacing LoRA With a Generic Style Adapter in Stable Diffusion
For creating personalized Stable Diffusion art, Low Rank Adaption (LoRA) models are all the rage this year. But a new academic offering from China is proposing a system that...
Detecting Cheapfakes With Deepfakes
You don't need a powerful GPU to deceive the public - just a mischievous turn of phrase, applied to photos or videos that don't really support the caption. Now,...
Editing Porn, Bias, Objects and Artists Out of Stable Diffusion Models
New research from the United States and Israel offers a more discrete and less destructive way of editing access to contested material in the Stable Diffusion text-to-image model.
Using ChatGPT and CLIP to Augment Facial Emotion Recognition (FER)
Labeling facial expression data could be helped by the use of large language models such as ChatGPT, and by text/image encoder frameworks such as CLIP. However, these tools are...
Better Open Source Facial Emotion Recognition With LibreFace
Open source Facial Emotion Recognition (FER) frameworks are thin on the ground - and what there is, is rather outdated. Now, researchers from USC are proposing a better and...
Native Temporal Consistency in Stable Diffusion Videos, With TokenFlow
Research from Israel has found an innate quality in Stable Diffusion that may lend itself to producing more temporally consistent video - unlike the usual run of such projects,...
Mixed Emotions: Compound Facial Expressions Will Be Important in Image Synthesis
New research from Australia investigates novel ways to teach AI systems how to recognize a far greater range of human facial expressions than just the six basic expressions widely-used...
Generating Temporally Consistent Stable Diffusion Video Directly in the Latent Space
In the latest contender for stabilizing video output from Stable Diffusion, researchers from China attempt to 'de-flicker' generated imagery in the latent space of the system, and achieve some...
Mapping the Mysteries of the Latent Space With Class Activation Maps
It's not easy to understand what happens when you make a request to a trained neural network. Whatever the resulting output, if you have no way of understanding what...
Temporally Coherent Stable Diffusion Videos via a Video Codec Approach
Everyone wants temporally coherent Stable Diffusion videos, but the solution has proved elusive. In this later offering, a research team uses the principles of video encoding as a method...
Creating Hyperscale Face Datasets via ControlNet and Stable Diffusion
Making hyperscale datasets that are both realistic and abundant is essential to the development of the computer vision and image synthesis research sectors. But it's not easy to get...
Better Deepfakes by Ripping Out Skip Connections
Researchers have proposed what may be the most radical change in deepfake software since it emerged in 2017. By removing what was formerly considered one of the most fundamental...
High Quality Deepfaking With Mobile Phone Scans
A new project from Tel-Aviv University offers a superior method of facilitating deepfake puppetry, as well as editing key facial components (such as expressions) in the latent space of...
Protecting Neural Videoconferencing From Deepfake Puppeteering Attacks
One day, video-conferencing is likely to be a low-bandwidth, neurally-powered affair, with participants only sending minimal information about their facial movements, which will power locally-built, on-the-fly avatars at the...
Editing Neural Radiance Fields with DreamBooth
A new research paper proposes to edit the usually rigid contents of a Neural Radiance Field using text-to-image technologies, and the controversial DreamBooth method. But does this bring us...
Context Matters in Facial Expression Recognition (and Synthesis)
New research into Facial Effect Recognition (FER) seeks to understand facial expressions in the broader context of their environment. In much the same way we judge whether or not...
Personalized Protection Against Stable Diffusion Deepfaking
Adversarial watermarking has been proposed many times as a preventative measure against deepfaking, but usually with the intention to entirely disrupt the generative process. But a new method proposes...
Combating ‘Identity Bleed’ in Deepfakes
Deepfake video is possible through an act of last-minute vandalism, in terms of neural architecture, where the two trained identities are wired into each other, long after training has...
Using EbSynth to Create Better NeRF Facial Avatars
EbSynth is a non-AI system that lets animators transform video from just a handful of keyframes; but a new approach, for the first time, leverages it to allow temporally-consistent...
Improving Human Pose Extraction With Transformers
A new initiative from UC Berkeley uses Transformers to achieve a new state-of-the-art in human pose extraction efficiency and accuracy, paving the way for a variety of improvements across...
ChatFace Offers Better Disentangled Neural Expression-Editing
A new system from Peking University improves on the state-of-the-art for neural face-editing, offering more faithful expression manipulation and more disentangled editing of facets such as hair and eye...
Faking Depth Occlusion for Better Augmented Reality
New research could improve the ability of augmented reality (AR) systems to convincingly insert synthetic objects into scenes, by studying the currently complex ways that they are matted and...
Bringing Temporal Coherence to Stable Diffusion with Flow Maps
A new project offers a way to cleanly transfer motion into synthesized video, promising a new level of realism and verisimilitude - if the code is ever actually released.
NVIDIA Offers Real-Time Neural People Through a NeRF and GAN Pipeline
A new collaboration between the University of California, Stanford, and NVIDIA has proposed a system capable of turning people into neural representations in real time, using Neural Radiance Fields...
Achieving More Realistic Deepfakes by Making the Image ‘Worse’
Deepfakes and AI-edited faces are often too 'perfect', which can help deepfake detection systems to identify them. But a new academic collaboration has devised a way to inject real-world...
Generating Temporally Coherent High Resolution Video With Stable Diffusion
A new text-to-video system backed by NVIDIA can create temporally consistent, high-res videos by injecting video knowledge into the static text-to-image generator Stable Diffusion - and can even let...
Adobe’s DreamBooth Clone Is 100X Faster and Obtains Better Results
Adobe has invented a way of injecting people's identities into Stable Diffusion as custom characters that out-competes former methods such as DreamBooth and Textual Inversion, while running at 100x...
The Challenge of Using AI to Change Facial Expressions
Across people, and particularly across cultures, facial expressions are not always accurate indicators of how we are feeling. How then can we create AI systems that can accurately change...
VIVE3D: Meta’s GAN-Based Deepfake and Video-Altering Framework
A new method uses Generative Adversarial Networks (GANs) to not only offer high quality facial editing, but even to convincingly change the orientation of the subject's face in respect...
The Small Faces That Vex AI Surveillance Systems
A new system offers better recognition of faces that are very small in a surveillance video or CCTV stream, which could increase the likelihood of more accurate and reliable...
GigaGAN: Stable Diffusion for Generative Adversarial Networks
A new text-to-image generative system based on Generative Adversarial Networks (GANs) offers a challenge to latent diffusion systems such as Stable Diffusion. Trained on the same vast numbers of...
Detecting Deepfakes by Studying ‘Disconnected’ Facial Expressions
A smile affects many other parts of the face, and this tendency of expressions to be 'holistic', and to have a broader effect on a wide range of facial...
Using the Internet as a ‘Live’ Dataset for AI Training
Some of the most influential computer vision datasets are so old that many modern images - such as the latest iPhones - are completely absent. But curating and training...
Correcting ‘Selfie’-Based Facial Distortion, for Psychological and AI Development Purposes
New research offers a way to correct the extreme facial distortions that occur when taking hand-held selfies on smartphone cameras. Since AI is increasingly reliant on web-scraped data, such...
Obtaining Editable Neural Humans From Short Video Clips
Re-training Stable Diffusion models can allow users to compose scenes with accuracy, addressing one of the core weaknesses of latent diffusion models.
New System Offers Superior Layout Composition for Stable Diffusion
Re-training Stable Diffusion models can allow users to compose scenes with accuracy, addressing one of the core weaknesses of latent diffusion models.
NeRF Breaks Free From Being an ‘Animated Photo’
A new research project brings NeRF nearer to Stable Diffusion in terms of explorability and versatility.
Real-Time, Photorealistic Hands for Neural Environments
Researchers from the Max Planck Institute lead a new paper that offers personalized and high-quality live hand rendering in real time, aimed at VR/AR, teleconferencing, and other neurally-enhanced environments.
A Call to Legislate ‘Backdoors’ Into Stable Diffusion
A new MIT paper proposes not only a new adversarial method to prevent images being 'abused' by Stable Diffusion users, but a fairly drastic increase in the level of...
Preventing Stable Diffusion ‘Copyright Infringement’ by Poisoning The Source Data
A new paper proposes a method to protect the work of artists from being incorporated into Stable Diffusion, by using adversarial data perturbations to adversely affect generated results. Though...
Dataset Abuse Is Rife in Computer Vision – But the Solutions May Be Drastic
A new paper from researchers at Sony AI levels broad and scathing criticism at standards and practices in dataset re-use and abuse - but the cure might stop the...
New System Enables Tight Compositions and High-Res Output in Stable Diffusion
A new system from Spain offers Stable Diffusion users the chance to truly control their compositions, by orchestrating multiple models into a single image inside the latent space.
Stable Diffusion and Imagen Can Reproduce Training Data Almost Perfectly
New research indicates that Stable Diffusion, Google's Imagen, and other latent diffusion systems and GANs are capable of replicating training data almost exactly. The findings may have implications in...
AI is coming to Hollywood with Metaphysic
Metaphysic has been named the sole AI VFX provider for the highly-anticipated major motion picture Here, produced by Miramax, and directed by Robert Zemeckis. Starring Tom Hanks and Robin...
Image Synthesis Has an SEO Problem
Images uploaded to the internet are being scraped at scale for ingestion into AI datasets. But the captions associated with the images were written for SEO purposes, and not...
Creating Better Avatars with a Dual-Domain Approach
A new academic collaboration, including contributors from Microsoft, has developed an improved technique capable of fitting user-submitted images into a 'deepfake puppetry' workflow in only thirty seconds, with notably...
The ‘Cheap’ Decisions That Can Affect Image Synthesis
A new academic collaboration, including contributors from Microsoft, has developed an improved technique capable of fitting user-submitted images into a 'deepfake puppetry' workflow in only thirty seconds, with notably...
Temporally Coherent Human Video Deepfakes Via Diffusion
A new research collaboration between Poland and the UK may offer the first effective method to obtain a much-cherished 'holy grail' of deepfake image synthesis research – the ability...
Muse: Google’s Super-Fast Text-To-Image Model Abandons Latent Diffusion for Transformers
Google Research has revealed a new type of framework for text-to-image synthesis, based on the Transformers architecture, rather than latent diffusion.
Entanglement in Image Synthesis
In the field of image synthesis, entanglement is the enmeshing of data properties with data other properties, which can make it difficult or impossible to isolate a particular...
High-Quality Deepfake Puppetry in Thirty Seconds
A new academic collaboration, including contributors from Microsoft, has developed an improved technique capable of fitting user-submitted images into a 'deepfake puppetry' workflow in only thirty seconds, with notably...
Using Diffusion Models to Create Superior NeRF Avatars
A new research paper from the Hong Kong University of Science and Technology (HKUST) and Microsoft Research offers a rational and less resource-intensive way to generate 3D representations of...
Research Proposes ‘Moral’ Sanitization for Text-To-Image Systems Such as Stable Diffusion
New research from Korea and the United States has proposed an integrated method for preventing text-to-image systems such as Stable Diffusion from generating 'immoral' images – by manipulating the...
Creating State-of-the-Art NeRF Head Avatars in Minutes
If time were no object, Neural Radiance Fields (NeRF) might by now have made greater inroads into potential commercial implementations – particularly in the field of human avatars and...
InstructPix2Pix: Accurate, AI-Based Image-Editing With GPT-3 and Stable Diffusion
New research from the University of California at Berkeley improves notably on recent efforts to create AI-powered image-editing procedures – this time by combining the considerable calculative forces of...
Generative AI to transform Laboratory Testing
In recent years, hyperreal media has gained widespread attention for its ability to transform the entertainment industry, pop culture and visual effects. A new area of transformation is within...
Editing Out the Real World With ‘Diminished Reality’
In the 2014 Christmas special of the dystopian sci-fi anthology series Black Mirror, writer and series creator Charlie Brooker envisaged the possibility of cybernetic augmentations capable of 'blocking out'...
A New Social Image-Sharing System Deepfakes All People by Default
A new collaboration between Binghamton University and Intel offers a novel take on the problem of the unauthorized use of social media photos for facial recognition purposes, as well...
The Future of Using Public Images for AI Research
Besides their capacity to rip off the style of popular real world artists, the new breed of latent diffusion- based image synthesis systems promises a revolutionary ease-of-use not only...
NVIDIA’s Implicit Warping Is a Potentially Powerful Deepfake Technique
Over the past 10-20 years, and particularly in recent years, the computer vision research community has produced an abundance of frameworks capable of taking a single image and using...
Creating Authentic Human Motion Synthesis via Diffusion
New research from Tel Aviv University may prove capable of bringing authentic human motion to text-to-video synthesis, videogames, motion capture architectures in VFX pipelines, and also function as a...
Custom Styles in Stable Diffusion, Without Retraining or High Computing Resources
A researcher from Spain has developed a new method for users to generate their own styles in Stable Diffusion (or any other latent diffusion model that is publicly accessible)...
The Road to Realistic Full-Body Deepfakes
It's nearly five years since the advent of deepfakes released into the public realm the ability to alter people's facial identities; at first, in recorded video, and now even...
Stable Diffusion: Is Video Coming Soon?
For an excited public, many of whom consider diffusion-based image synthesis to be indistinguishable from magic, the open source release of Stable Diffusion seems certain to be quickly followed...
Vote for Metaphysic – AGT 22 Semi finals
We have reached the semis, which is fantastic. This is true. If you want Metaphysic to go on and reach the finals of AGT, you just need to vote for...
Are We Heading For ‘Deepfake CAPTCHA’ Challenges in Video and Voice Calls?
A new paper from Israel proposes the institution and ongoing development of a 'deepfake CAPTCHA' protocol, to challenge audio or video callers that may be using deepfake technologies to...
Detecting Deepfakes Through Personality Assessment
In the original, and the various cinematic incarnations of Jack Finney's 1954 science-fiction novel The Body Snatchers, the fact that aliens are 'taking over' the earthly population is signaled...
To Uncover a Deepfake Video Call, Ask the Caller to Turn Sideways
There is an interesting vulnerability in video deepfakes that, to date, has been generally overlooked by the security research community, perhaps because 'live', real-time deepfakes in video calls have...
Defeating Deepfaked Video Calls by Monitoring Electrical Fluctuations
2022 has seen the emergence of 'live' deepfakes in videoconferencing as an acknowledged security threat. In June, the FBI issued a warning to businesses about the increasing use of...
The Future of Generative Adversarial Networks in Deepfakes
Excluding 'traditional' CGI methods, which date back to the 1970s, there are currently three mainstream AI-based approaches to creating synthetic human faces, only one of which has attained any...
How I Became The Fake Tom Cruise
Actor Miles Fisher had it all: talent, charm, and the face of a movie star. Problem was, it was a particular movie star. After years of resenting the resemblance,...
Deepfakes Go High-Res – But Can Deepfakers Handle It?
Towards the end of our new feature on the future of autoencoder-based deepfakes, we preview a new capability in FaceSwap, set for release this Saturday...
The Future of Autoencoder-Based Deepfakes
The way we refer to visual effects (VFX) work may be changing soon. For instance, at the time of writing, the relatively new technology of Neural Radiance Fields (NeRF)...
NeRF: An Eventual Successor for Deepfakes?
A recent study from Lancaster University found not only that most people cannot distinguish deepfake faces from real faces, but also that we tend to trust synthesized faces more....
Synthetic 101 – VR as an extension to normal life
This is another point on a road that is leading to VR experiences being as realistic as the real world. To be truly immersive, VR needs to be a...
Why the Metaverse must be hyperreal
The Metaverse is a nebulous concept. Questions abound as to what it will look like and what we will look like within it. We believe it has to be...
Metaphysic’s America’s Got Talent experience, with Chris Ume and Tom Graham
In this post, Chris Ume and Tom Graham give us a behind-the-scenes look into the audition process for America's Got Talent, the incredible reaction of the live audience and...
The Future Will Be Synthesised – E2: Deepfakes for disinformation
Metaphysic are pleased to announce the release of the next episode of the BBC Radio 4 and BBC Sounds documentary called “The Future Will Be Synthesised”, presented by our...
The Future Will Be Synthesised – E1: Deepfake Image Abuse
At Metaphysic we are very proud that our own Head of Policy and Partnerships, Henry Ajder, is presenting a BBC Radio 4 and BBC Sounds documentary called “The Future...
Building a safer synthetic media landscape: what needs to be done
Non-consensual image abuse is aimed almost entirely against women, remains the most prominent malicious uses of deepfakes and synthetic media.