Cubism SDK‎ > ‎Live2D Library‎ > ‎Motion Manager‎ > ‎

Lip-sync

There are mainly three types of Lip-sync.
The followings are presupposed that the Parameter "PARAM_MOUTH_OPEN_Y" is made to control the Lip-sync.

1.Get the real-time volume, and specify the range to open and close the mouth directly


model->setParamFloat("PARAM_MOUTH_OPEN_Y" , xxxx ) ;//.....(A1)
You can control the range of opening and closing the mouth by setting the value from 0 to 1 directly for the second parameter of setParamFloat() before model->update().


You can get the real-time volume playing with Android 2.3 or later versions(*).
You can lip-sync it by setting the order above with the value which is manipulated in the range from 0 to 1 based on the playing volume acquired.
(This is because Mouth Open is created with the Motion Parameter from 0 to 1 as the standard Motion Parameter settings.)
It will not be resulted in an error if the setting value is less than 0 or greater than 1, lip-sync may not be executed correctly.

(*):You cannot get the real-time volume with Android2.2 or before.
It depends on the library of audio play if you can get the real-time volume or not with other platforms (like PSP).



2.Use the volume data for Lip-sync
You can use this method by extracting the information for Lip-sync somehow and make settings same as in no.1.

*You use this method even when you use the data exported with clearing the "Export as .mtn" setting with "Volume Extraction Tool".


3.Use the motion (.mt) with the information for Lip-sync
Create an .mtn file which includes Lip-sync in the Animator tool or "Volume Extraction Tool" to play a motion to control Lip-sync.
This will basically play a normal motion and a motion for Lip-sync simultaneously.
Generate a MotionQueueManager instance to manage only Lip-syncs to distinguish Lip-syncs from other motions.
You can mount it by copying the normal motion play section, as a library will be treated equally as long as it is an mtn file.

*If you use this method, you need to comment out if there is any processing to update "PARAM_MOUTH_OPEN_Y" directly like above (A1).
It is recommended to search "PARAM_MOUTH_OPEN_Y" for the whole source related to Live2D for checking just in case.
There is a possibility that Lip-sync will be overwrote if you do not update after the normal motion.


Pseudo code like Java is as follows;

---Add a management class of motion-------------------------------------------------------------
  1. private MotionQueueManager mainMotionMgr = new MotionQueueManager () ;//For normal mtn
  2. private MotionQueueManager mouthMotionMgr= new MotionQueueManager ();//For mtn for Lip-sync

---Add update---------------------------------------------------------------------------------
  1.         
  2. mainMotionMgr.updateParam( model ) //Set the main motion.         
  3.  
  4. /* Update the motion for Lip-sync */
  5. mouthMotionMgr.updateParam( model );
  6.  




---Play the motion.-------------------------------------------------------------------------
  1. //Play the motion loaded from an mtn file for Lip-sync. Same as normal motion.
  2. mouthMotionMgr.startMotion( motion , false ) ;