A.I. – Fan Film Factor

How I’m helping MARK NACCARATO make THE ROMULAN WAR…using ARTIFICIAL INTELLIGENCE! (feature-part 2 of 4)

Last time in Part 1, I discussed a little about how Artificial Intelligence (AI) is becoming an increasingly common tool among fan filmmakers—especially Star Wars but also Star Trek and other genres—and I provided links to some recent examples in the Trek realm. Then I moved onto how MARK NACCARATO, who is finishing up work on the sequel to THE ROMULAN WAR, PART I and also releasing some new short films in the NX-era, asked me to help him create a still image of a young T’Pau giving a speech to the Vulcan Council.

Looking at an overwhelming amount of work in Photoshop, I asked Mark if he’d mind if I used AI to create the image. He was fine with the idea, and ultimately I was able to create not just a still image to go under a voice-over narration but an actual 13-second video of part of her speech. Take a look…

Although I had used AI two years ago to generate AI voices for Spock and Dr. McCoy for my animated fan film AN ABSENT FRIEND, and I had created some Facebook memes and some short music clips with AI tools previously, this was the first time that I’d used AI to generate a video from an AI still image. And it was a lot of fun!

It was also an amazing learning experience, and I wanted to learn more. There’s nothing like having a project to focus your efforts, and so I asked Mark if he had any other clips that he’d like to try to create using AI. I was certainly game!

Mark’s next request was simply for a still image. Four fans had gotten together to stage a “toast” in front of a green screen. Here’s a still image of them…

My mission, should I choose to accept it (and, of course, I did!), was to do the following:

Put all four of these fans into NX-era Starfleet uniforms like those from Enterprise.
Trim them down a little to look more like physically fit Starfleet officers.
Change the blue Romulan Ale to just beer (as this was taking place during the Romulan War—the Romulans were still mysterious—so Romulan Ale wouldn’t be an Earth beverage at the time).
Have them sitting in the 602 Club (from the Enterprise episode “First Flight”), and Mark supplied me with some reference photos…

How I’m helping MARK NACCARATO make THE ROMULAN WAR…using ARTIFICIAL INTELLIGENCE! (feature-part 1 of 4)

Yeah, Artificial Intelligence (AI) will probably steal away jobs, destroy the environment, and eventually turn all humans on the planet into nothing but organic batteries while muscular cyborgs with Austrian accents hunt us down. But until then, AI is doing some truly unbelievable things! Many folks in the fan film community are beginning to use it in their projects. And a couple of months ago, I decided to jump down that rabbit hole. What I’ve managed to create since then is blowing my mind!

I originally began dabbling with AI two years ago when I produced the animated fan film AN ABSENT FRIEND and used AI to recreate the voices of the late LEONARD NIMOY and DeFOREST KELLEY as Spock and McCoy (I got a tacit thumbs up from the Nimoy estate, and Kelley’s voice is no longer owned as he had no children). What absolutely amazed me was that the AI-generated voices could actually act and take direction! And in the time since then, AI’s capabilities and quality have evolved even further. AI still hasn’t achieved “perfection” yet, but acceptance of it as a creative tool is spreading very quickly…even among some of those whose livelihoods were initially threatened by it.

Case in point: my longtime friend ADAM “MOJO” LEBOWITZ—two-time Emmy-winning CGI animator on Babylon 5, DS9, Voyager, and Battlestar Galactica—was staring down the AI “career abyss” a couple of years ago. But instead of just worrying and complaining about AI making his VFX skills obsolete, he decided that he would become an AI expert. He studied the expanding capabilities of countless different AI generators for images, video, audio, etc., applying his own knowledge and experience in the visual effects field to learn how to make AI do what a film producer or director actually wanted rather than what AI thought they wanted. Here’s a sample of some of what Mojo created as he was teaching himself AI. Since then, Mojo has found rewarding employment in the AI pre-viz VFX industry on an upcoming film from Lion’s Gate working as what I like to call the “AI Whisperer.”

And Mojo isn’t the only one. Many fan filmmakers have begun to use AI in their projects in various ways. Don’t even get me started on how many Star Wars fan films are being made entirely using AI—there aren’t enough minutes in the day to report on them all! But some of the Star Trek fan filmmakers who have begun to use AI in their projects include…

How I used A.I. technology to bring the voices of SPOCK AND McCOY back to life in “AN ABSENT FRIEND” (part 3: legal and moral questions)…

In part 1, I discussed how I used Artificial Intelligence (A.I.) to turn a Star Trek inspired fan script that I wrote back in 2010 into an audio drama featuring the voices of Spock and McCoy. I utilized ElevenLabs‘ voice synthesis algorithm to convert sound clips captured from a variety of sources into a series of back-and-forth dialog between the two characters. Ultimately, I wound up with an approximately 15-minute long audio drama.

Part 2 covered how I managed to take the audio drama and turn it into an animated fan film with the help of an amazing illustrator by the name of MATT SLADE, music composer MATT MILNE, and my longtime childhood friend MOJO. Indeed, in the end, I was the only person involved in this production whose named didn’t start with the letter M! The finished product came out looking like this…

And now the moment that I am certain many of you have been waiting for: the legal and moral questions of “Can Jonathan legally do this?” and “Should Jonathan ethically do this?” These are both complex subjects to tackle. But let’s dive in…!

How I used A.I. technology to bring the voices of SPOCK AND McCOY back to life in “AN ABSENT FRIEND” (part 2: the art, animation, and music)…

In part 1, I explained how I had used Artificial Intelligence (A.I.) to synthesize the voices of Spock and McCoy to speak the words of a script I originally wrote as a short story back in 2010. In it, Captain Kirk has just “died” saving the U.S.S. Enterprise-B (in the feature film Star Trek Generations), and Bones is getting angry-drunk mourning the untimely passing of his longtime friend. Spock finds McCoy sitting alone in an unnamed bar and offers to join him. It’s the last thing the doctor wants—a Vulcan with no emotions—but, ironically, the thing he most needs. And Spock needs McCoy, as well, even though the Vulcan would never openly admit it.

The story was always intended to be a sort of a two-actor stage play, simple but poignant, giving a glimpse into how these two longtime friends and colleagues deal with the loss of AN ABSENT FRIEND. But the story/script sat quietly on my hard-drive for more than a decade, unused and mostly unpublished, waiting for its “moment.”

That moment came when I discovered the ElevenLabs website (thank you, RAY MYERS!), which can generate synthesized speech from any decent voice sample of a minute or two. Assembling verbal snippets from LEONARD NIMOY and DeFOREST KELLEY from various sources, I set out to create an audio drama of my script. The project took me a couple of months, and I explained the nuances of how I did it in part 1.

Now in part 2, I’ll explain how I got from that audio drama to this completely animated Star Trek fan film…

As I mentioned in part 1, after I completed the audio drama, I was frustrated by the subtle inconsistencies between sentences (especially for McCoy) due to my generating the voices in short segments. This was a necessary evil due to the limitations of the early A.I. technology, but it still bothered me.

Then I had an idea…

How I used A.I. technology to bring the voices of SPOCK and McCOY back to life in “AN ABSENT FRIEND” (part 1: the audio)…

While some Star Trek fan films have begun to Artificial Intelligence ( A.I.) in limited ways to age and de-age characters and also to create brief visuals and short bits of dialog, my new animated fan film AN ABSENT FRIEND is the first time that A.I. speech synthesis has been used to generate ALL of the voices. And what’s more, two of those voices are sampled from the late LEONARD NIMOY and DeFOREST KELLEY, allowing the beloved characters of Spock and McCoy to live on.

The obvious question that might come to mind for some people would be: Is this legal? The short answer is “yes…at least for now.” That could change in the not-too-distant future, but no law governing or restricting the A.I. generation of a deceased actor’s voice in a fan film exists at the moment. I will dive more deeply into the legal status of voice A.I. in Part 3 of this blog series (along with the ethical considerations). However, right now in Part 1, I would like to discuss how I managed to digitally recreate the voices of these two deeply-missed actors, and in Part 2, I’ll be covering how I and my illustrator, MATT SLADE, turned an audio drama into a full animated Star Trek fan film.

First, though, let’s take a look at this groundbreaking project…

ARTIFICIAL INTELLIGENCE TAKING OVER THE WORLD?

A.I. has exploded across the planet in the last year and a half, and it’s certainly become a bit of a mixed bag. The term itself is an umbrella for a wide range of digital breakthroughs whereby computers are doing some incredible—and occasionally worrisome—things. A.I. is being used for everything from generating business presentations and news articles to writing millions of lines of computer code in seconds. A.I. digitally de-aged an 80-year-old HARRISON FORD in the fifth and final Indiana Jones feature film last year and also completed an unfinished Beatles song despite two of the original band members having died decades ago.

Law enforcement is using intuitive software to sift through endless social media postings to track down wanted suspects and potential terrorists. On the other side of the moral spectrum, however, some students are using A.I. software to write their school essays for them, while a few political campaigns have begun to generate false images and articles to spread realistic-looking fake news to unsuspecting voters. Earlier this year, a New Hampshire robocall seemingly from Joe Biden that told Democrats not to “waste their vote” in the primary was actually faked by a supporter of one of the other primary candidates. And of course, the recent Hollywood writers and actors strikes worried that A.I. would make many of their jobs all but irrelevant.

My own mind was blown last year when I saw A.I. used to create an actual Star Trek fan film! THE RODDENBERRY ARCHIVE released the following short fan film featuring a deepfaked face of Leonard Nimoy as Spock put onto the body of actor LAWRENCE SELLECK…