The Awesome
First off, let’s start by saying how frickin’ cool it is that being able to create something like this even possible! I’m still trying to wrap my head around it. And, I am VERY excited for the future. I’ve said it many times, but AI generated artwork, photos, and videos are completely transformative for me as a storyteller. I can now take my characters, worlds, and stories far beyond text on a page. I can add so much more depth, and create a much more rich and engaging experience for readers.
The Struggle 1.0
Me. My knowledge. My abilities.
I’ve only been working in the generative AI space since early December. There is so much I don’t know, and the whole process has been trial and error. I am not a dev. I am a user of software, I am a content creator, and website builder. To make good, engaging content over the last 20+ years I’ve had to pick up some skills in graphic design, photography, videography, editing, and user experience. I’m trying to bring that background to my AI generated creations, but ultimately it’s new to me and I don’t know what I’m doing.
The Struggle 1.1
Learning curve.
I started generating AI images in Adobe Firefly simply because it’s included in my subscription. I had no idea what I was doing. I tried to watch a few tutorials on YouTube, there’s a few good ones, but ultimately — no one knows what they are doing. Everyone’s just putting a bunch of prompts in, adjusting some settings, and seeing what happens. Over time, the art becomes your ability to articulate prompts.
The Struggle 1.2
Overwhelming.
There’s so many AI models out there. There’s so many cool things that come from each, and particular things that each model distinctly excels at. Again, I’m not a developer. Truth be told, I don’t care. I appreciate the work and effort in creating these models and interfaces. I have a tremendous about of reverence for their efforts. But, I’m just a guy trying to use these tools. And, for the most part, I feel lost and like I’m the lab rat. I only have so much time in the day, my world cannot be fully devoted learning all of the models out there — I simply have to get work done, and move on to the next project.
The Struggle 1.3
A new world.
Jumping from program to program, model to model, interface to interface is a big deal. It requires me to start over each time. And that’s a considerable time commitment. I left Adobe Firefly in late January for LeonardoAI — the reason was that Firefly was creating really interesting stuff for my novel, The Final Book: Gods, but it was absolutely unusable for Stevie Pearl. I just couldn’t get Stevie right. Also, LeonardoAI had just launched its motion feature, and that really, really interested me.
I dove completely in with Leonardo, signed up for the most advanced subscription package they offer, and in the first month have blown through my 60k tokens.
The Struggle 1.4
Character consistency.
Everyone’s complaint and biggest challenge: taking the same character or object with you from scene to scene, and having it look and behave within the context of that new scene. Within Leonardo, I could get Stevie pretty close. Very close. There were a few generations where I was like, “that’s her!!”
However, once I have that image of her — I need more. I need lots of imagery for the website. I need her doing different things, being in different places, and interacting with different people. I can’t do it.
Yes, Leonardo has training and datasets. I’ve tried. I’ve tried to much… If this was 10 months ago, I probably would have been blow away by what I can create from these models. But, compared to their PhotoReal V2 pipeline, anything created from my own datasets just looks like PS2 fan fiction. Using anything from my own dataset outputs would severely diminish my user experience and crossover into some creepy it-puts-the-lotion-on-the-skin Buffalo Bill territory.
Lessons Learned 2.0
Branding.
Like any artist, band, brand, or director — style is important. It’s how the world identifies you. You see Jackson Pollock, Andy Warhol, and Pablo Picasso and even if you’re unfamiliar with the work, you know they created it. Same with Tarantino or Wes Anderson.
I have my own style as well — or at least each of my stories do. Those particular worlds have a certain look. They read a certain way. All of the marketing materials I’ve created for them follow a consistent theme. I lost that early on with AI generation, especially with Stevie Pearl.
I was so excited about the possibly — I was so blown away that i could generate anything that resembled these characters that I forgot, or didn’t realize, that I could add my own style and theme to them.
Lessons Learned 2.1
Voice.
When creating Back In ’85, I started finding my voice again within AI generation. The Ballad of Stevie Pearl begins with Stevie conducting a “Barbara Walters” style interview. It’s the first time she’s gone public about a major event that happened in her life. This is supposed to be real. Like, polished, upper echelon, documentary film footage of the biggest pop star on the planet telling you about her story. This is the prologue and the epilogue of the story — the book ends.
The rest in the middle is Stevie’s fantasy. It’s her colorful love story. It’s a reenactment of her imagination, and sadly, her fading memories of Alex Nopah. This is where the creative freedom really comes in, and my ability to go beyond reality. Colors, angels, wows — visually stunning and unique.
Lessons Learned 2.2
Carry over.
From the book cover, to all the promo materials, to my website, and to the digital experience of The Ballad of Stevie Pearl, I’ve followed a pattern: unicorn puke. Pinks, purples, vibrant, vibrant, vibrant. For whatever reason, I lost that when I started creating my AI artwork. I was so consumed and focused on getting the look of Stevie correct, that I compromised away my artistic vision for the project as a whole.
Ultimately, this is one of the big reasons why this music video is a failure. It doesn’t follow the format, look, or feel of the rest of the Stevie Pearl universe. This is completely my fault, and I’m excited to say, not a limitation of the AI itself.
I can, without a doubt, create the stunning visuals I had always intended. I just failed, as an artist, to do that in this video. And that’s a major reason why I can’t release this version alongside anything officially related to Stevie Pearl.
Lessons Learned 2.3
It’s not there yet.
Adding motion to AI still and generating video is awesome. Those of us who are doing it right now are blown away because we know how difficult this is to achieve — and we see where the future is heading.
However, to the average person scrolling their feed — these AI video look like glitching video games from generations past. There’s still a little hokey, janky, eerie, and overall not something they’d take serious.
This diminishes my brand and reputation. While we, those who tinker with AI, think it’s beyond cool — these struggling clips are a blemish to the disengaged real world. Their brains are inherently comparing them to 4k Marvel movies, and it makes my story look like a low-quality grocery store romance novel from 1990.
The same reason I wouldn’t publish a book with random different sized fonts throughout, page breaks for no reason, and spelling errors — the same reason I don’t feel this video is ready to be published.
Lessons Learned 2.4
As a director.
Working with AI generated video, I have no control over what kind of motion and movements the AI is generating. Each generation is a roll of the dice. This removes a lot of the creative vision, or how you want to tell a story.
This is also time and resource consuming. Almost every clip I used in Back In ’85 required 3 or 4 (or many more) generations before I ultimately accepted an output and moved on to the next.
Further, if you do achieved a good output, the resolution is very low. This actually hides a lot of the AI mistakes, but increases the workflow. Anyone viewing this videos outside of a phone are not going to put up with it.
Back In ’85 was up scaled to 1080p with After Effects. I purchased Topaz yesterday for a different project for a client, so eventually any new Stevie Pearl clips will be run through that. We’ll see how it does.
The Future 3.0
It’s Bright!
While this version of Back in ’85 is a “failure” — meaning, I won’t be including it in The Experience, or alongside anything officially related to the novel, it was a tremendously valuable experience.
Hell, it’s pretty damn cool!
I’m still blown away that I can even attempt something like this. I need to learn how to use my tools better, the tools need to evolve a bit — but the secret sauce is here. It’s happening. I’m so excited to bring the world of Stevie Pearl to life, and I truly believe I’ll be able to do it in a way that does the story justice.
The Future 3.1
Out now!
My novel, The Ballad of Stevie Pearl, released yesterday. That’s pretty exciting in my world! You should be able to buy digital and physical versions of the book everywhere online. I know for sure Amazon, Lulu, and my website have copies for sale.
I’m also in the middle of developing The Experience — a highly visual and immersive version of the story, including way more character backstories and supplemental content. All of this is being done with the help of AI. It’s really rough right now, just a landing page, but follow me and I’ll give you updates as this fills out.
https://www.theballadofsteviepearl.comThe Future 3.2
Your help & inspiration.
With all of that said, what have your experiences been while trying to generate AI videos and short films? I’d love some advice, tips, and tricks. I plan on doing a lot this type of content for my novels, and I’m interested in learning more. I want to make my content the best it can be – your help is instrumental in accomplishing that!
And, finally, please share your creations with me! I’d love to watch, talk, and gain more and more inspiration! Thanks for reading.
Sean
