WEEK 1
Deep Fakes
What are they?
Deepfakes (a portmanteau of "deep learning" and "fake") are synthetic media in which a person in an existing image or video is replaced with someone else's likeness. While the act of creating fake content is not new, deepfakes leverage powerful techniques from machine learning and artificial intelligence to manipulate or generate visual and audio content that can more easily deceive.
Deepfakes use a type of technology called 'machine learning' to create a digital version of someone, mapping a person's face and mouth movements so that it can then copy them. De-aging is a visual effects technique that uses AI technology to erase wrinkles, trim jawlines, and synthesize past actors' appearances to create their young appearance without using a substitute actor.
Examples:
Star Wars fandom exploded at the sight of Luke Skywalker in the season two finale of The Mandalorian. Once the space dust eventually settled though, viewers were quick to point out what they saw as flaws in the digital recreation of a younger Mark Hamill. Once again, YouTuber Shamook(opens in new tab) had a go at deepfaking a Return of the Jedi-era Luke Skywalker with very impressive results.
In fact, it was later confirmed that Shamook had been hired by none other than Industrial Light and Magic, the legendary visual effects company that help bring the Star Wars galaxy to life. Deepfake technology is now being used to shape the galaxy far far away.
Many of the most convincing deepfake examples have been created with the help of impersonators that mimick the source’s voice and gestures, just like this video produced by BuzzFeed and comedian Jordan Peele using After Effects CC and FakeApp. Peele’s mouth was pasted over Obama’s, replacing the former president’s jawline with one that followed Peele’s mouth movements. FakeApp was then used to refine the footage through more than 50 hours of automatic processing.
Politicians and celebrities are often the subjects of deepfakes. Less than a year before the above video, University of Washington computer scientists used neural network AI to model the shape of Obama’s mouth and make it lip sync to audio input(opens in new tab).
The Camera Never Lies (Truth Claim)
The Capture (2019), a British mystery thriller series from BBC, aims to answer the question: Can we believe what we see?
By highlighting how the usage of deepfakes can manipulate and deceive people, this new series explores the theory of truth claim. As Tom Gunning explains in ‘What’s the Point of an Index? or, Faking Photographs' (2004), truth claim that can exist in several contexts is based on the value given on the visual accuracy of a photograph, on its combination of indexicality and iconicity. But as long as there is a desire to fake it, there will always be a value placed on visual accuracy. Gunning argues that the potential of lying implies the truth.
Unreal: The VFX Revolution (Podcast)
Paul Franklin, the narrator of Unreal: The VFX Revolution podcast, explores the history of visual effects and highlights how they have affected the film industry. Robert Blalack, one of the founders of Industrial Light & Magic, describes how compositing, the process that seamlessly integrates two or more images to make the appearance of a single picture, has transformed over time.
With his background in experimental cinema, he was able to combine an enormous number of distinct visual elements that would be produced using motion control technologies. Blalack explains that the most complex shot when working on Star Wars(1997), included the composite of 4 different aircraft and the background plate. Each ship was a composite of multiple elements, therefore the shot resulted in combining over 40 elements. They were all added to a compositing device called optical printer. However, every time images were combined, they had to be copied into a new piece of film which was softening or degrading them. In 2001, this problem was avoided by using camera composites, where layers were added to a single piece of film by reexposing it while it was still inside the camera’s magazine. A much cheaper solution found at that time and adopted by ILM was to use a larger film negative than the standard of 35mm to give sharper images. In other words, they were starting with a high-resolution element so that there was still a larger film negative than the standard of 35mm to give sharper images.
Nowadays, compositing only requires specific software such as Nuke.
Harvard Reference:
Franklin, P. (2021) Unreal: The VFX Revolution - A Long, Long Time Ago... [Podcast]. 06 July. Available at: https://www.bbc.co.uk/sounds/play/m000xltg (Accessed: 08/10/2022).
WEEK 2
Developing ideas, case studies and project examples
This week, we looked at case studies and analyzed several previous assignments in order to come up with ideas of research for our investigative study essay.
We followed the task below:
In pairs take an essay assignment from a previous year. Have a read through the essay and see if you can summarise what it was about:
- What is the research question, can you outline the aims and objectives?
- Can you describe the method used for the practical research?
- What were the findings of the practical and/or theoretical research?
- Was there an analysis of the findings?
- What was the student's argument?
- Are the sources credible?
OR you can try to list the Pros and Cons of the essay, what was done well, and what could be improved.
Deep fakes
Research Question:
Deep fakes:
->the history of deep fakes
->positive and negative aspects of deep fakes
->are deepfakes valuable to the film industry? how?
Method used for practical research
->investigating deepfake apps such as Reface app and MyHeritage
->creating a deepfake
->analyzing the process and the result
Findings of practical/theoretical research
->Reface app: ''The Reface app was launched in 2020 and allows people to swap faces in the videos, GIFs and memes with just one selfie (hey.reface.ai).''
->an image of Chris Evans and Lucy Liu was used in order to compare the face of a random person, found on google, to a celebrity who looked similar to him, as well as a celebrity that has another gender and race -> the results were undetectable only when swapping the photo of Chris Evans
->MyHeritage.: ''This app was downloadable on any app store, again like for the Reface app you can pay a monthly subscription for more effects. This app allows you to build your family tree and animate an old picture of someone who has passed in your family or friends.''
->By using pictures of family members, it was demonstrated that the results were only detectable because the animation only focused on the eyeballs and eyelids, ''so there was no movement in the eyebrows making the subjects look unnatural and creepy'' and because the image was getting distorted when the subject was not looking straight forward
->the app was provoking a feeling of unease
Was there an analysis of the findings?
->I believe that the practical part as well as the theoretical one were well connected to an analysis.
For example, the positive aspects of deep fakes were well explained using examples. It is shown how CGI can bring actors back to life in order to preserve the story without replacing the actor. The given example was from Star Wars: Carrie Fischer and Peter Cushing
Student's argument:
-> It is highlighted the positive aspects such as the developing of VFX industry as well as the negative aspects as how some studio executives use images with deceased people in order to make money while they have no say in the matter
Are the sources credible?
Yes
Developing Ideas
Potential areas of investigation for deepfakes
Deep Fakes
Uncanny Valley
How did it start vs how AI, deep fakes are created nowadays
Effects on VFX industry and jobs
Ethical repercutions
Flaws of AI, how close it can imitate reality, detectability
Believability in what we see
Looking more into different topics related to deepfakes:
Close ups
WEEK 3
In the third week of the module, we learned how to design a good essay question. Then, we had to start working on our proposal draft (500-600 words)
What does a proposal draft include?
-
A clear, focused title or research question
-
A list of searchable Keywords
-
An Introduction to the Investigative Study
-
Include the aims and objectives
-
A Methodology or list of five key sources (References) – these should be annotated
-
Any important images as figures
Apply activity:
We learned the importance of questions.
A good question can inform our reflection and write up what we have found out as well as give the reader a clear idea about the text
In teams, we aimed to answer the following question:
Should invisible effects always be invisible? Make a case for this statement
- they should be invisible in order to maintain story coherence and support the diegetic world
-absolutely concealed effects adapt to the narrative set by using matte paintings to extend location or change the weather
- invisible effects are intentionally undetectable and visual effects artists recognize that if they can be noticed, they unsuccessfully meet the criteria.
Example:
Having re-created Heathrow in the 1960s for the third series of The Crown, Framestore has returned to Netflix’s royal drama, providing VFX work on 230 shots throughout Season 4. The creative studio delivered a wide range of VFX work on the latest series of Peter Morgan’s streaming mega-hit, including digital environments and crowd work as well as a photoreal CG stag that the production team at times didn’t believe was digital.
Alongside the rest of Framestore’s work, bringing the stag to life is some of the most complex and detailed work the studio has done for the series. The show’s fourth season, which is set during the 1980s and sees Charles and Diana’s early courtship, includes a fully-digital creature even while the series has used digital effects mostly in a supporting role – something Framestore still needed to bear in mind when animating the animal.
Then, we start working on the draft proposal. We thought about what would be best to cover in our proposal and looked for references. I came up with some articles, papers, and books that I believe will help in writing the draft.
Areas that I want to cover:
-uncanny valley
-image inpainting (Image inpainting is the art of reconstructing damaged/missing parts of an image and can be extended to videos easily)
-the role of a prompt engineer (example: Improving system prompts to reliably accomplish language generation tasks(s))
Harvard References:
Kalpokas, I. (2022) Deepfakes: a realistic assessment of potentials, risks, and policy regulation. Place of publication: Publisher
Stadler, J. (2019). 'Synthetic beings and synthespian ethics: Embodiment technologies in science/fiction' Projections, 13(2)
Jan Kietzmann Linda W. Lee Ian P. McCarthy, Tim C. Kietzmann, (2020), Deepfakes: Trick or Treat?, Plac of publication: Business Horizons
Helmus, T. (2022), Artificial Intelligence, Deepfakes, and Disinformation, Perspective EXPERT INSIGHTS ON A TIMELY POLICY ISSUE, available at: https://apps.dtic.mil/sti/pdfs/AD1173672.pdf
Christian Rathgeb Ruben Tolosana Ruben Vera-Rodriguez Christoph Busch, (2022), Handbook of Digital Face Manipulation and Detection From DeepFakes to Morphing Attacks, Springer, available at: https://link.springer.com/content/pdf/10.1007/978-3-030-87664-7.pdf
Adjer, H. et al (2019), 'Deeptrace, the state of Deepfakes: Landscape, Threats and impact states.' Available at: https://regmedia.co.uk/2019/10/08/deepfake_report.pdf
Proposal(draft)
Investigate deepfakes and to answer the question of whether deepfakes are valuable to the film industry or whether their usage is rather unsafe, immoral, or controversial
The purpose of the essay is to investigate deepfakes and to answer the question of whether deepfakes are valuable to the film industry or whether their usage is rather unsafe, immoral, and too controversial. Coming up with new techniques to distort reality and make viewers doubt the veracity of what they see has always been a vital part of visual effects. Deepfakes are hyper-realistic videos that barely reveal the manipulation, generated with face swaps, by combining, replacing, and projecting images and videos.
The foundation of deepfakes originated in the 19th century. A paper written by Christoph Bregler, Michele Covell, and Malcolm Slaney in 1997 exposed what some film studios could achieve. Audio output could be used by the Video Rewrite Program to create unique facial animations. It was based on earlier work that analyzed faces, created sounds from text, and modeled lips in 3D space, but it was the first to combine these techniques and effectively animate them.The conclusions from this 1997 study are succinct yet very persuasive.
Computer vision advanced in the field of facial identification throughout the early 2000s. Motion tracking, which helps to increase the credibility of today's deepfakes, has significantly improved.
Deepfakes' scope, complexity, and level of technological expertise are game-changing characteristics. Deepfakes are able to accomplish convincing representations of actors' younger selves or reintroduce the ones who are no longer alive. For instance, Grand Moff Tarkin was brought back in Rogue One(2016): A Star Wars Story, which generated controversy by using a digital recreation of Peter Cushing.
When it comes to post-production and reshoots, deepfake technology for the film industry might have a tremendous positive impact. Filmmakers might utilize deepfake technology to modify dialogue without having to reshoot scenes since it uses a staggering amount of data and video to recreate a person's look. To replace some lines, it could all be done with a small department and deepfake technology rather than gathering the complete set and crew.
However, deepfakes can also represent a risk to our society, and national security since they influence elections and spread false information, undermine public trust in a federal information and raise cybersecurity concerns for both individuals and businesses. Nowadays, the majority of deepfakes on social media sites like YouTube or Facebook can be viewed as playful humor or creative works utilizing famous people, both alive and deceased. Some examples of the negative side of deepfakes include celebrity and revenge porn, initiatives of political persuasion, and more. Celebrities, politicians, and business executives are frequently the subjects of deep fakes because there are so many images and videos of them online that can be used to create the massive picture caches needed to use an AI deep fake system.
Deepfake still has a lot of potential in the film industry, despite the numerous issues it has raised over the last few years. How to utilize technology to its maximum potential without intruding on anyone's privacy or intellectual property is the concern. Deepfake can increase creativity and have both beneficial and negative effects.
The beginning of deepfake technology and one of the first attempts of creating a deepfake.
WEEK 4
This week we looked into literature reviews and annotated bibliographies as well as how to design methodologies.
What is a literature review?
-
A review of the literature! (Approx. a 1,000-word ch.)
-
A comprehensive look at the existing research on a particular (often quite narrow) topic.
-
An overview of some of the major theories and practices within a particular discipline.
Where does a literature review go?
STRUCTURE:
-
Introduction
-
Literature Review
-
Methodology
-
Analysis Chapter 1
-
Analysis Chapter 2
-
Conclusion
Annotated Biblios:
-
Should contain a summary of the findings
-
Could also contain description of other relevant sections of research, or perhaps methodological approaches relevant to research
Content Analysis
- a research technique for the objective, systematic, and quantitative description of the manifest content of communication’ (Berelson, 1952: 18)
Apply task:
-
Locate a source that pertains to your research question
-
Active note-taking (as you read make notes on the themes and topics covered)
-
Write up a 150-word annotated bibliography for that source on your digital sketchbook
By looking at the sources located in the previous weeks and exploring new ones, I decided to write my annotated bibliography on ‘What’s the Point of an Index? or, Faking Photographs’ by Tom Gunning.
Images have resembled reality more accurately than ever in recent decades, which raises the question, "Can we trust them?"
Tom Gunning assimilates in 'What's the Point of an Index? or, Faking Photographs', the truth claim of traditional photography with Charles Pierce’s term "indexicality". While the traditional photograph is obtained by the light reflecting of the subject photographed, passing through the lens and diaphragm, and finally, the transformation of the light-sensitive emulsion; the digital image is based on data about light which is achieved by numbers. The indexicality of the photograph depicts the physical relation between the subject photographed and the final image created. The truth claim can be disrupted due to the manipulation of digital photography. As Tom Gunning states, it is utterly important when an image is digitally transformed, to preserve a significant part of the “original image’s visual accuracy and recognizability”(2004, p.41) because the attention accorded to the visual accuracy of a photo develops the basis of the truth claim. I believe that VFX processes also make use of the truth of the image as computer algorithms can now generate unique individual portraits of people that don’t exist or add facial movements on existing footage, called deepfakes. However, although truth claim made people be more observant, it can also be considered alarming because we rely on images for information, for example, the news, where "the real world has been converted into the language of television"(James Fox).
Harvard References:
Gunning, T. (2004) ‘What’s the Point of an Index? or, Faking Photographs’, Plenary Session II, Digital Aesthetics, 1(25), pp. 39-48
True Lies, Perceptual Realism, Digital Images and Film Theory
Stephen Prince
1996
Similar words, keywords
-Realism
-Truth claim
-Index
-Photorealism
-Digital
Similar phrases
-Assumptions about realism in cinema are related to concepts of indexicality
-Cinema does not restore reality to us, but rather that it offers us an impression of reality.
Sutured Reality: Film, from Photographic to Digital
Francesco Casetti
2011
Similar theories, concepts
-Cinema's relationship with physical reality has changed with the development of digital images. 1 The story goes that we are no longer dealing with an image that is only dependent on a direct record of items placed in front of the camera, establishing the crucial connection between the reality and its representation. Thanks to the development of an algorithm, the digital picture has the capacity to provide us with a representation of things without ever needing the items in question.
- It is not essential what sustains the realistic effect of a film—whether formal devices or even a “sense of medium,” a perceptual pattern, etc. What is important is the presence of a multilayered discursive strategy that links the chain of speech and creates an impeccable feeling of reality, even if it is only illusory.
WEEK 5
Proposal
Deepfakes: Benefits and Threats
Deepfakes illustrate a type of synthetic visual media that seems genuine, but it is created by artificial intelligence (AI) and it’s not portraying the actual world. They barely reveal the manipulation generated with face swaps, by combining, replacing, and projecting images and videos. As a visual effects student, my objective is to answer the question of whether deepfakes are valuable to the film industry, but I will also be discussing whether their usage is rather unsafe, immoral, and too controversial. In order to do this, I explore the history of deepfakes, their advantages and disadvantages.
According to the Greek roots, there was no separation between art and technology. The term tekhnē assigns art as a technical process, by connecting the usage of technology such as databases and computers to the process of creation. Laurence Bertrand explains that '’starting from the prehistoric caves, artists have remained connected to their technical environment, whose tools they always appropriate to invent new forms''(2018). Artificial intelligence is based on the idea of artifice, which in its Latin origin symbolizes art in addition to its present meaning of deception and trickery.
The approach of "face replacement" in film industry was created in 1990s. A scene from Jurassic Park(1993) depicts a stuntwoman performing a fall, but ILM added Ariana Richards to the shot using digital face replacement. It required shooting the actress individually, rotoscoping the footage of her moving her head, and then matting Richard's face over the face of the stuntwoman.
Needless to say, deepfakes brought revolutionary changes to the film industry. They can be utilized in audio-visual content like movies, television shows, and advertisements to overcome language difficulties For example, David Beckham is shown speaking multiple languages in a film for the Malaria Must Die campaign. Not only language became unrestricted when it comes to content creation, but also physical existence. Deepfakes have made the existence of actors easily adaptable, for example bringing deceased actors back to life.
Another achievement of deepfakes is de-aging. In The Mandalorian Season 2(2020), Mark Hamill appeared as
young Luke Skywalker. The process of making him look younger was realized by merging his and Max Lloyd-Jones' faces. With the aid of images and video of the young Hamill, Lola employed deepfake technology to construct a CG composite of his face after first working on skin smoothing and shape warping in 2D compositing.
However, deepfakes challenged certain beliefs about the human self. Despite that visual manipulation techniques, such as photography, have already questioned people’s beliefs in the past, artificial face generation can go to extremes by devaluing the human face as a representational object in favor of database-driven generation. The European Parliamentary Research Service claims that ‘’The most worrying societal trend that is fed by the rise of disinformation and deepfakes is the perceived erosion of trust in news and information, confusion of facts and opinions, and even truth itself’’(2021).
Moreover, main application of deepfakes is the production of fake pornographic videos using the appearance of a celebrity or other person with the aim of silencing or defaming. Journalists are another group worth mentioning. If a journalist were to be deceived by a deepfake, the consequences may be severe, from the immediate propagation of disinformation to more general issues like the loss of faith in sources of information.
By using DeepFaceLab, a software that has supplied the internet with over 95% deepfake videos, and then an application that detects deepfakes, I aim to explore the workflow of face replacement by creating my own deepfake. My objective is to investigate to which extent deepfakes have developed, how they could bring value to the film industry, and what their downsides are.
Harvard References:
Kalpokas, I. (2022) ‘Deepfakes: a realistic assessment of potentials, risks, and policy regulation.’ Place of publication: Publisher, pp.3-36
Zylinska, J. (2020) ‘AI Art Machine Visions and Warped dreams’, Media: Art: Write: Now, pp. 30-35
Desowitz, B. (2021) How Mark Hamill Was De-Aged as Luke Skywalker for ‘The Mandalorian’ Season 2 Finale — Exclusive. Available at: https://www.indiewire.com/2021/08/the-mandalorian-season-2-mark-hamill-interviews-de-aging-1234660159 (Accessed: 26/10/2022)
Failes, I (2021) VFX Firsts: What was the first digital face replacement in Film? Available at: https://beforesandafters.com/2021/05/26/vfx-firsts-what-was-the-first-digital-face-replacement-in-a-film (Accessed: 26/10/2022)
Westerlund, M. (2019), ‘The Emergence of Deepfake Technology: A Review’, Technology Innovation Management Review, pp. 39-47. Available at: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://timreview.ca/sites/default/files/article_PDF/TIMReview_November2019%20-%20D%20-%20Final.pdf (Accessed: 26/10/2022)
David Beckham speaking multiple languages in a film for the Malaria Must Die campaign.
Jurassic Park(1993)
First face replacement
Deepfake created with DeepFaceLab 2.0
WEEKS 7-10
I got positive feedback for my proposal, with the suggestion of adding more academic sources.
In the next weeks, we spent time working on the sample chapter (1000 words) and on the practical research
By exploring different academic sources and film case studies, my aim is to highlight how deepfakes can bring value to the cinema, and what are their advantages and disadvantages.
I will compare and contrast between the youtube genre of deepfakes and the VFX "professional" usage. The vfx deepfake practice appears to be both serious and digitally complex, while the youtube deepfake video can be both amusing and requires a much smaller production team.
Academic Research and Film Case Studies to help me complete the sample chapter:
Corridor Crew:
Keanu Reeves stops a robbery (2019):
The platform VFX company, Corridor, makes use of the networked attention economics on YouTube. In Keanu Reeves STOPS A ROBBERY (2019), deepfakes are used to replace the face of performer Reuben Langdon with Keanu Reeves’s face. How We Faked... videos (2018-2022) by Corridor Crew show how to employ VFX tools and methods that are now publicly accessible, reasonably priced, or even free in the case of open-access deepfake generation software.
Bode, L. (2021), ‘Deepfaking Keanu: YouTube deepfakes, platform visual effects, and the complexity of reception’, Convergence: The International Journal of Research into New Media Technologies, pp.920-932
Deepfakes:
For the last few Star Wars (2016-2019) films, in order to preserve the storyline, Carrie Fischer and Peter Cushing were brought back to life. This, on the other hand, brings up moral issues regarding permission and the existence of the digital afterlife.
De-aging:
X-Men: The Last Stand (2006) is largely acknowledged as presenting digital de-aging for the first time in a mainstream film. Both Ian McKellen's Magneto and Patrick Stewart's Professor X have had their appearances digitally enhanced to make them appear younger. McKellen's hair has been darkened, age spots have been eliminated, and wrinkles have been computerized out.
Joanna Zylinska presents her perception of AI in Photography Off the Scale: Technologies and Theories of the Mass Image (2021). She claims that humans have always been technical, which means that we operate on algorithms, from DNA to behavioral guidelines created by different cultures. According to her, all artistic expressions have required cognitive expansion and extended forms of intelligence. In contrast, the author explains that even if algorithms are a part of everyone's lives, individuals need to pay attention to the way they use them, the images they generate, who views them, under what conditions, and at what cost.
Zylinska, J. (2021), Photography Off the Scale: Technologies and Theories of the Mass Image, Edinburgh University Press, Edinburgh
Planning my Practical Research:
For my practical research, I will use DeepFaceLab to create my own deepfake. My objective is to investigate to what extent deepfakes have developed, and how they could bring value to the film industry.
Finding tutorials on how to produce a deepfake and image sources to work on was easy and free. Therefore, everyone who has access to the Internet can create a deepfake
WEEK 13
Investigative Study Essay
Deepfakes: Benefits and Threats
In my essay, I addressed the question of whether deepfakes are valuable to the film industry. The essay also discussed ethical concerns, whether the use of deepfakes is rather unsafe, immoral, and too controversial. To do this, the history of deepfakes, their advantages, and their disadvantages was investigated.
For my theoretical research, I explored books such as Photography of the Scale: Deepfakes: a realistic assessment of potentials, risks, and policy regulation (2022), Technologies and Theories of the Mass Image (2021), AI Art Machine Visions and Warped dreams (2020), and articles like ’True Lies: Perceptual Realism, Digital Images, and Film Theory’(1996) or ‘What’s the Point of an Index? or, Faking Photographs’ (2004). I also looked into case studies for different films published on the Internet.
Practical Research
For my practical research, I used DeepFaceLab which is a free and very popular software among YouTube users in order to swap the faces of Leonardo di Caprio and Elon Musk. The software offers a simple-to-use pipeline that does not require an understanding of the deep learning framework or any specific skills. It is best known for swapping faces, de-aging, and altering speech. By providing photographs or videos of celebrities, DeepFaceLab demonstrates that creating deepfakes is now convenient for everyone.
Part 1 (Prepare the model for training)
-add both videos in the ‘workspace’ folder and name them accordingly, ‘data_src’ and ‘data_dst’ (the software requires these names for the source and destination videos)
- extract images from the videos
- extract faces from each frame. It is realized a report on the images found and faces detected. In order to identify faces in the images, DeepFaceLab identifies facial landmarks and generates a default mask. The software aligns the faces and produces a file for each identified face with embedded metadata.
Part 2 (Training the model)
-run the training command that will access a preview window including keyboard shortcuts, a graph that shows the efficiency of training, and a preview of the new model images
-the preview needs to be updated (using the P key) which increases the number of iterations (the result will be more accurate when the number of iterations is higher)
-look at the columns that show the original faces and the potential deepfake, and save the training anytime
Part 3 (Adjusting the created deepfake)
-merge the faces using the interactive merger. This shows the preview along with a tab that includes the current frame number and settings.
-adjust settings to blur or tighten the mask, change the color of the image, resize it, etc. I used the given map of keyboard commands to play around with some settings such as erode mask, so the border around the face will contract, and blur mask which made the edge between the original footage and the deepfake less visible.
-convert each frame of the new deepfake into a video file with the destination audio.
The image above depicts my deepfake model and the tab with the settings I adjusted in order to achieve a better result
The interactive merger makes training the subject convenient even if one doesn't have editing skills, there are easy-to-understand options like 'blur' or 'face scale', while options that seem more complex are explained on the DeepFaceLab website.
Conclusion (Practical Research)
I consider the result satisfactory, but not perfect. By closely analyzing each frame of the video, I came to the following conclusions. Even though increasing the blur mask made the expressions of Leonardo Di Caprio fit better on Elon Musk’s face and matched their skin shades, it made the face more out of focus than the rest of the body. Another fault can be easily noticed when the profile of the model comes up in the video. The final image distorts because the performer's nose and eyes extend past the limits of the deepfake.
The result is better when the subject looks straightforward. I could have achieved a more natural and undetectable outcome, if I had increased the number of iterations when training the model, according to deepfake tutorials. Because having more iterations takes more time and depends on the hardware of the computer, I terminated the training at 2,227 iterations, while most renowned deepfakes used around 40,000-80,000 iterations.
The most straightforward way to make improvements is to get better input data, personally adjust the program, and train it longer. Moreover, DeepFaceLab can be used in combination with other programs for analyzing images and videos. A more photorealistic outcome can be obtained by using image-enhancing tools, and audio modification.
Therefore, I consider my deepfake more as an eye trick, rather than a mind trick. However, my aim was not to create the perfect deepfake but to explore the process of generating one, how accessible it is nowadays, and to compare the YouTube genre of deepfake to VFX professional usage. YouTube deepfake videos require a smaller production team, less money, and zero specific skills, but they are also amusing. In contrast, VFX deepfake practice is both serious and digitally complex.