In Rebellion, you play as Mandella, colonel in the Confederation, a powerful military force. At the beginning, you explore the mothership as you're asked by admiral Spade to attend your briefing. You'll discover that you're part of the first attack wave sent to storm Tor29, a little colony lost in the Nephele system. Accompanied by your best friend, major Threepwood, you'll face an oddly weak resistance. Why does the Confederation unleash its wrath on a colony that is clearly of no match? What hides behind this conflict driven by doubtful justifications? These obscure reasons are not to please you and Threepwood. Will you carry on following blindly an authority fouled by power and arrogance? Or will you break your former links and embrace a cause that seems fairer to you, however desperate it may be? Maybe these are favourable times for... Rebellion.

Report article RSS Feed Faceposer II - Lips synchronization

In this Tutorial, you'll learn about : * New Wav : Add a Soundscript to your "dialog" channel * Lips Synchronization : Synchronize the movement of the lips with the Wav file

Posted by RaphaelG on Feb 4th, 2009
Advanced Animation.

Tutorial Description

In this Tutorial, you'll learn about :

  • New Wav : Add a Soundscript to your "dialog" channel
  • Lips Synchronization : Synchronize the movement of the lips with the Wav file

Requirements :
- Adding custom voice files
- Faceposer (1/5) : Setting up the workspace

New Wav

Our first and easy step

1) Right click on the timeline of our "dialog" channel > New Wav...

news post image

2) Select your Soundscript, name it and press "OK"
news post image

Note : If you don't know what I'm talking about when I say "Soundscript\", make sure you've been through my previous tutorial Adding custom voice files

Lips Synchronization

Lips synchronization works by associating what we call "phonemes" to each syllable of the Sound.
A phoneme correspond to a specific movement of the lips, which linked together with other phonemes makes your NPC "speak" the sentences
And when you see that, you'll feel great satisfaction. I promise

1) Select your Sound, then double click on the "Phoneme Editor" tab, in the bottom left of the window
news post image

2) A new window with the spectrum of your Sound appears
news post image

Now, we'll split the work in 3 parts :
- synchronize words with the sound
- create phonemes for each words
- synchronize phonemes with the sound

Synchronize Words with the sound

1) Press the "Re-Extract" button, a window appears
news post image

2) Write down all words spoken in the Sound

Note : Here, no need for punctuation, all that interrests us are the words that are spoken.
news post image

3) Sometimes, Faceposer automatically creates and synchronizes phonemes for the words inserted.
Forget about this, it's just crap.

Note : If you do have phonemes proposed by Faceposer :
- Right Click on the Sound > Cleanup words/phoneme
news post image

Now your workspace should look like this :
news post image

4) Time now to synchronize our words with the sound
Understand : Put them at the right place.

Note : How to manipulate the words (select it first with Left Click) :

    a) Move left / right
    SHIFT + Left Click on the word

    b) Widen / Shrink
    CTRL + Left Click on the border of the word

    c) Unselect a word
    Left Click right above the words

    d) Separate two words
    - Select the word on the right (Left Click)
    - Right Click on it > Select All Words After...
    news post image
    - SHIFT + Left Click > Move mouse to the right, to move all words
    news post image

Note : In a normal spoken sentence, you don't need to seperate two words. Do this only when you have a clear and audible blank between two of them.

Now that you know how to manipulate the words, move around the green time cursor to know exactly where the intersection between two words lie.

news post image

Make sure each word is synchronized exactly with its spoken word.
Take the time to do this, it's the base of the upcoming work. If you're not precise at this point, the rest won't be precise either.
This is what my sound looks like :
news post image

5) When it's done, Click on the "Save" button
news post image

Create phonemes for each words

To see what a specific phoneme looks like, you can open the tool tab "Expressions"
news post image
As you click on one expression, you model in the 3D view will show you how it looks like.

Now that you know what every phoneme looks like, let's get back to our "Phoneme Editor" tab.

1) Select a word

2) Right Click on this word > Add Phoneme to 'word'
news post image

2)Select all phonemes needed in the word, then press OK
news post image

Note : Because you have previously synchronized the words, the phonemes now automaticaly adapt to the size of the word.

3) Check your freshly added phonemes on the 3D model
It's a little bit tricky to have the correct phonemes for our words, so you better check them out after you choosed them. Move with the time cursor to do that. At this point, don't worry about phonemes synchronization with the sound, we'll worry about that later.
news post image

Note : If you're not happy with one of them :

    a) Select a phoneme
    b) Right Click on it > Edit 'phoneme'
    news post image

    c) Select the new one

Tip : If you want to add more phonemes to a word that already has phonemes assigned, you better delete all existing phonemes for that word, and recreate the phonemes for you word.

Do that for every words now, and don't worry about the synchronization of these phonemes. That's our next step.

Here's how it should look like :

news post image

4) Save your work

Synchronize phonemes

The movement of the lips might not be (and certainly won't be) synchronized yet, so let's do this, as we did it with the words previously.

1) Listen to each syllable of the sound with the time cursor

2) Compare the syllable of the sound to the phonemes you've just created

3) If it's not synchronized, widen / shrink the phonemes

4) Do that for each phoneme

5) When you're done, press "SPACE" to play the sound, check for the movement of the lips of your model in the 3D View

Tip : Now that's the most accurate way of doing this. But obviously, it takes time.
If you don't want to spend too much time on this (and thus accept not to have a perfect synchronization), on step 2), here right above, you can just compare the movement of the lips of your 3D model with the sound and see if it looks good enough.
Some scene, that take place during combat for example, don't need to much polishing since the player probably won't be paying too much attention to the talking NPC. On the opposite, if the player is alone in a room with a talking NPC, the player will certainly focus on the NPC only. That's when Lips Synchronization should be perfectly accurate.

Here's the final result :
news post image
See, all phonemes in one word don't have the same size. They are synchronized to each syllable of my sound.

That's it guys !
Now that was a very big part of scene creation. If you've done that, first, you can be proud of you. Second, you did almost half of the work on your scene. The other steps go way faster, you'll see in the next tutorials.

Until then, enjoy and have fun !

Cheers

Kyl'

Post comment Comments
Varsity
Varsity Feb 4 2009, 6:19pm says:

Could you post this on VDC? Developer.valvesoftware.com

+1 vote     reply to comment
RaphaelG Author
RaphaelG Feb 6 2009, 11:27am says:

Hey Varsity :-)

I never really thought about that, but yeah, that would be a good idea !

Cheers mate.

+1 vote   reply to comment
KieranG
KieranG Aug 13 2011, 1:55pm says:

For some reason it crashes whenever i try to move a word or resize it. Any idea whats causing it?

+1 vote     reply to comment
Post a Comment
click to sign in

You are not logged in, your comment will be anonymous unless you join the community today (totally free - or sign in with your social account on the right) which we encourage all contributors to do.

2000 characters limit; HTML formatting and smileys are not supported - text only

Icon
Half-Life 2 Icon
Platform
Windows
Developed By
RaphaelG
Contact
Send Message
Official Page
Rebellion-source.net
Release Date
Released Jan 3, 2009
Mod Watch
Track this mod
Tutorial
Browse
Tutorials
Report Abuse
Report article
Related Mods
R e b e l l i o n (Half-Life 2)
R e b e l l i o n Half-Life 2 - Single Player First Person Shooter
Related Games
Half-Life 2
Half-Life 2 Single & Multiplayer First Person Shooter