LGM's Learning to Talk Tutorial

Learning to Talk

By Nathan Dunlap (LGM)

Welcome to my "Learning to Talk" tutorial. Here, I will try to teach some basic lipsync animation techniques.

Be aware of 3 things:

  • 1. There are many ways to animate, and consequently many ways to lipsync. This is a way that I have found works very well and results in nice animation.
  • 2. It is manual lipsyncing, without the use of plugins.
  • 3. This tutorial assumes a basic knowledge of and a familiarity with Blender. There are some really basic commands which I have put in, while I assume you know how to do the others. It also assumes that you have already completed my "Learning to Walk" tutorial. This is because walking is more important than lipsync, and also so I don't have to explain basic details all over again. In addition, you must know how to download and install a program onto your computer.
  • We will be using Jason Pierce's Ludwig model for this tutorial. Download it now. It is a very, very nice rig.

    You will need two more files to follow this tutorial completely:

  • Magpie (the free version) is a simple program that allows you to map out where the various mouth shapes go during a dialogue piece. (Note: I know of two other lipsync programs that are similar to Magpie: Papagayo and JLipsync. While I will be using Magpie for this tutorial, you can use another similar program just as easily since we're not going to use a really detailed lipsync program with lots of software-specific functions.)
  • The audio sample that we will be using throughout this tutorial. The dialogue is a recording I made that runs, "My name is Ludwig, version X6." I made it to test Ludwig's lipsyncing capabilites back at version X6. Go figure, hey?
  • (One more note, in case you're having trouble figuring out which image goes with which words, each image corresponds to the paragraph(s) above it.)

    All right, install Magpie and open it up. Just click on the flash screen that appears.

    Now we're in Magpie. Go to the File menu, then to open and find the audio clip which I've named ludwig_X6lipsync.wav, click on it and open it.

    You'll notice a few buttons on the top. I should explain them now. Left to right, they are "open file," "save file," "copy to clipboard" (no idea what it does), "play all," "play selected frames," "play audio from selected frame to the end," "play audio from the beginning to the selected frame," "zoom in," and "zoom out."

    Now we must make sure we're in the same FPS (which stands for Frames Per Second). This is the rate at which the frames flash by in a movie or animation. Animations are usually at 24 fps, so go to options, then set the FPS to 24. Incidentally, this is the frame rate of most (if not all) films in made in America. You may have noticed subtle differences if you have seen movies made in America and movies made in Britain between the image quality. Not that one's better than the other, there's just something different. This is the fact that American films are filmed at 24 fps, while British films are filmed at 30 fps.

    Now we get to use that very useful feature in Magpie. As you can see the audio is divided up into frames. Click on the first frame in the wavform and drage the mouse to the right for about four frames. This will highlight and select those four frames. As you can see, nothing's happening during those four frames. So double-click on the word "closed" in the left-hand column. As you can see this adds the words "Closed" to the spreadsheet under the column labelled "Mouth."

    Ah, but something does happen next. You can actually see a tiny jump in the wave form on frame 5. So click on that frame that has that tiny jump, and drag to the right for a few frames. Hit the "play selected frames" button (the one with the arrow that's between two red lines). Only the audio in those few frame that you selected is being played back. You can hear a tiny sound that happened when I opened my mouth and breathed in. We'll mimick that (mark, it actually) by double clicking on the "E" in the left-hand column.

    Next, we'll mimick the idea of a mouth opening by selecting the next few frames (right up to the point where the audio actually starts doing something) and double-clicking on the "A" in the left-hand column.

    Now select all the frames that you have just entered values for (the "values" being the phenome shapes, closed, "E," and "A"). Now press the "play selected frames" button. Meh, not much yet, but we've started.

    Now we're going to start getting into the fun stuff. Starting on frame 15, select the next 3 frames and press the "play selected frames" button (which will be referred to from now on as the PSF button). By the way, to do this you'll notice that you can't select more than one frame at a time on the spread sheet. So just select frame 15 from the spread sheet, that will automatically select frame 15 in the waveform above. Then click on the already selected frame 15 in the waveform and drag to the right for 3 frames. You can hear the beginning of the word "my." But at this point I must state that you are not lipsyncing to the words of the dialogue, you are lipsyncing to the sounds in the dialouge. So rather think that you hear the "M" sound and the beginning of an "iiee" sound. Since it's mostly "M," mark it by double-clicking on the M in the left-hand column.

    By using the PSF button, find the frame where the "M" sound stops and the "I" sound starts. I found that to be frame 17. Select the next few frames and press the PSF button. Add the I sound to the spreadsheet I assume you know how to do that by now.

    Now, you may have noticed that the "I" sound doesn't last all that long. I usually over shoot, making sure I get the beginning of each sound in the right place. So the next step is to find where the "I" sound stops and the "N" sound starts. I found this to be frame 18. So select that frame and the next few frames and add "N" to the spread sheet. As you can see, the "I" on the spread sheet was simply overwritten with "N."

    Continue in this vein. The next sound is "A," then comes "M," then comes "E." Wow, why "E"? I hear you say. The word that we're at is "is." Shouldn't it be an "I"? Well as I said before, we're syncing to the sounds, not the words. Select frames 23 and 24, hit the PSF button, and listen to the sound. The sound I hear is "ezz." Besides the "I" shape is the same as the "A" shape and that's too big for this small word. Put it down as "E."

    Next (page2) ->