Reallusion’s Cartoon Animator Versus PixVerse Image to Video and Lip Sync AI

PixVerse’s Show Muscle Effect is a real flex for other AI generative video sites.

AI generated digital avatars have improved greatly over the last 12 months, to the point where you can replace yourself with a digital, talking head, clone avatar that can say anything using your AI generated voice clone, and casual viewers wouldn’t even notice.

This is one of Hedra’s better generations of video using my TET Avatar as a source.

One area that has been slower to catch up is using AI to create 2D cartoon style avatars, from very specific character designs, that don’t have realistic human proportions. I put my TET Avatar through Hedra, one of the best talking head avatar generators currently available, and got a complete fever dream of a talking head character as output (see image).

I set about trying to find another AI image to video site that could also handle lip syncing audio, that would do a better job with my 2D avatar.

Enter PixVerse.AI

PixVerse has five animation styles you can apply to your creations.

After much frustration with various sites, I discovered PixVerse through a video by YouTuber, The Zinny Studio, titled, Best FREE AI Video Generator? FIND OUT (Generative AI). I’d highly recommend watching this video for a run down on how to use PixVerse. I also recommend The Zinny Studio’s channel if you’re at all interested in AI animation, AI Avatars, and creating a faceless YouTube channel.

PixVerse seems to specialize in animation styles, and gives you just enough free credits to test out their image to video and lip sync features in one go. 

Video generations of up to eight seconds are ready in seconds. If you run out of credits, wait 24 hours for them to be topped up. Unfortunately free credits don’t accumulate.

My initial trial was so impressive, I decided to sign up for one month of a Standard account (USD$10.00). Note, PixVerse does accept PayPal but you may get an error that looks like your payment failed. Their FAQs, state if you give it about an hour and refresh the site, your credits should appear.

PixVerse also includes preset AI effects such as turning any object into a ‘squishy’ object, or ‘Lego Blasting’ objects into pieces which are fun to play around with. The first image on this post is using their ‘Muscle Surge’ effect, where you can start with any character image and have them rip their shirt off, revealing a completely jacked body (I don’t think it works with female bodies – but I haven’t tried).

However, for my Cartoon Animator versus PixVerse AI avatar battle, all I used was their image to video feature, with no added effect or style, followed by their lip sync feature.

Reallusion’s Cartoon Animator Versus PixVerse AI – Battle of the Talking Head Avatars

So I decided to compare the process of animating an almost two minute long, talking head monologue, by my TET avatar in Cartoon Animator, with animating the same thing using PixVerse image to video, lip synced clips.

This article would be too long if I gave you a complete step by step guide for what I did in each application. So instead I’ll give you an overview of each process, with my thoughts, and then the finished video.

Both versions use the same recorded audio of my own voice (not AI cloned). Both were also taken into my video editor (Shotcut) to put the animation together and to add any additional video effects.

The Cartoon Animator Version

Animating my avatar in Cartoon Animator. To duplicate the first minute of the animation I collected all the motions into a single clip and saved it to my avatar’s Action Menu.

This version was animated in the space of an afternoon, going on into the evening. I have no doubt, if I were to animate another monologue of this length using this character, I could get it done much faster but, I hadn’t animated a G2 legacy character for quite some time, so I spent some of my time relearning the process. G3 characters are so much easier and quicker to animate.

The entire 1:55 minute scene was animated in one project file. I used the auto lip sync for the mouth with no manual adjustments so you can compare Cartoon Animators lip sync with PixVerse’s lip sync.

Once I got the hang of things it was relatively quick to create a chain of upper body animations by Frankensteining bits and pieces of pre-made motion clips into a sequence for half of the animation. Then I simply duplicated the sequence for the second half.

It’s not the perfect syncing of body motions to link to what the character is saying but it’s passable. No hand gestures are completely incongruent with what is being said. 

The point was to get the animation done as quickly as possible.

I also added facial expressions using the face templates in the Face Editor, and eye blinks and head movement using the Puppet Editor.

From there I exported the animation to my video editor. Replaced the green screen background with my AnimLife Video background, and added in my lower third titles and credits.

Honestly, If I wanted to animate a new monologue using the same character and scene I could pretty much just update the audio, make a few adjustments to the body and face animations, and I’d be done.

The PixVerse AI Animated Version

Lip syncing in PixVerse is as simple as uploading your video and audio, and then clicking a button.

My PixVerse starting point was with a couple of still frame captures of my avatar sitting at the desk with a green screen background, taken from my Cartoon Animator project.

I used these still images as the base image for generating 13 five second animations of my avatar sitting at the desk talking and making appropriate hand gestures. I could’ve generated eight second clips but I’ve found, the longer you let the AI run with a prompt, the more likely it is that it’ll start to introduce elements you didn’t prompt for.

Even with my five second clips I got three that were unusable, as my avatar magically introduced props that weren’t in the scene, or performed actions that moved props in the scene around (such as grabbing the nearby laptop and moving it closer – messing up continuity between clips).

While I didn’t prompt for any camera movement, PixVerse’s AI several times decided my avatar was too far away and had him move closer. Which is why, in the final version you’ll see the character move forward and backwards. I had to settle for these clips because, on a standard, paid plan, I only had 1200 credits to play with (each five second generation used 30 credits), and I needed to save some for the lip sync, which I had no idea on how many I’d need. Also I was generating at the lowest resolution to save on credits too.

Once I had my clips, I downloaded them and brought them into my video editor along with my audio track. I then proceeded to piece together a sequence of clips where the movements and gestures loosely matched with what my avatar was saying.

Sometimes I’d use the full five second clip. Sometimes I’d follow this with the same clip played in reverse. Sometimes I’d use just part of a clip. I’d even slowed down the speed of a clip in one case.

Once I’d done this for half the audio, I copied the entire sequence to use for the remaining half. I did tweak the second half a little but, for the most part, it’s an exact replay of the first minute.

PixVerse will only lip sync 30 seconds of video at a time, so I exported the animation as five clips under 30 seconds, along with the matching audio files (separately). Each file used around 140-160 credits to lip sync, so I ended up buying 550 more credits (USD$5.00) to finish the last two clips.

From there I downloaded everything and reconstructed it back in my video editor. I replaced the green screen background with my AnimLife Video background, and added in my lower third titles and credits.

Unlike Cartoon Animator, if I wanted to change the audio monologue on the whole clip, I’d have to redo the whole lip sync process again, giving Reallusion an actual win on time saving over AI generated animation.

Who Wins, Cartoon Animator or PixVerse?

Overall, Cartoon Animator wins and is the better option for 2D, animated, talking head videos. You have a lot more character control, and I think you’ll agree Cartoon Animator’s auto lip sync is far ahead of what PixVerse produced in this comparison.

Plus, once you have you’re initial project in Cartoon Animator, using it as a template for new audio is quicker than having to regenerate all the lip sync in PixVerse – not to mention the cost. 

PixVerse could definitely get expensive if you make a lot of videos. Cartoon Animator only costs the initial purchase price of the software. I can tell you, I burned through my monthly credit, and the extra credit I purchased, in a day on PixVerse.

However, I am very impressed by PixVerse. Particularly that it was able to animate my specific avatar character without changing the art style at all. Even when I applied that muscle flex effect, it still was consistent with the art style of my image.

There are still a few issues to iron out, such as the usual AI deformed hands problem. One of my initial base images clearly shows my character has four fingers and a thumb but, if the base clip didn’t show the hands, then the AI would produce three fingers and a thumb. Partly I expect because this is common for hands in 2D animation.

I wasn’t overly impressed with the lip sync. There are times when the avatars lower teeth look more like its lower lip. Sometimes the voice isn’t synced at all, and sometimes the mouth disappears altogether momentarily.

That said, if you want to turn your avatar into Batman… you can do that in seconds on PixVerse.

Be yourself. Except if you can be Batman. Be Batman! Animated entirely with PixVerse’s Batman Effect.