Operation Process
Step 1: Select the videos for editing and add them to the video track.
You can upload videos from your local device or just choose from the free Library Avatar, which offers hundreds of avatars spanning various ethnicities, ages, and styles.
Step 2: Add audio to the audio track and assign it to the corresponding face in the video.
- You can add audio to the audio track and assign a face from the video to the track. The AI algorithm will automatically perform facial recognition and lip-syncing.
- Multi-track support! Effortlessly handle multi-person video conversations by assigning unique voices to each individual in the video.
Audio sources include two options:
- Option 1: Import audio files.
- Option 2: Enter your prepared text and use the Text-to-Speech feature. The voice library offers over 1,000 high-quality voice models. If none of the provided voices meet your requirements, you can also clone a custom voice to use.
Step 3: Additional configurations
- AI model
- V1.3 and V1.4 are AI models used in older app versions 1.3 and 1.4
- V2.0 is the latest model
- V1.3 - Fast with good performance
- V1.4 - Fast with better performance
- V2.0 - Slower, but offers the best performance
- If you have sufficient computing power (such as a high-end Nvidia or AMD graphics card), we recommend using the latest V2.0 AI model.
- Teeth enhancement
- Enhances teeth clarity, but may slow down processing.
- Lip sync intensity
- Controls how widely the mouth opens during speech.
- Lip sync smoothing
- Controls the smoothness of mouth movements during speech, enabling this reduces jitter or shakiness in lip motions.
- Face orientation
- If the face is heavily tilted, it may affect lip sync accuracy.
- In such cases, selecting a large-angle processing mode can improve results.
- For best performance, use a frontal face orientation when possible.
- Audio noise reduction
- Perform lip sync after reducing background noise from the audio.
When there is only one video on the video timeline, the application allows you to use the "Save as My Avatar" function. This saves the video along with its currently linked voice model as a quick preset. Note: Audio files on the audio timeline will not be saved. Once the preset is successfully saved, you can find it in the Avatar Library - My Avatar section of the application.
Important Notes
Video Guideline
Video Requirements
- Format: Import MP4 video files with a recommended resolution of at least 360p to ensure good final output quality.
- Lighting: Avoid strong light or shadows; ensure the subject's face is clearly visible.
- Content: Use stable footage with minimal shaking or rapid movements.
- Subjects: The subject's facial expressions in the video should be natural and easy to adapt.
Legal and Copyright Requirements
- Usage Authorization: Ensure that all video and audio materials used have proper legal authorization to avoid violating copyright or image rights.
- Privacy Protection: If the video involves real people or their voices, obtain prior consent or confirm the legal use of the material.
Operational Notes (App)
- Videos on the track can be split, trimmed, deleted, undone, or restored.
- When importing multiple videos onto the track, ensure they have the same resolution and aspect ratio. Additionally, using videos with a consistent frame rate is recommended to prevent stuttering or lag during output.
Common Issues and Solutions
- Issue 1: Shadows too dark around the face can lead to unstable lip-syncing.
- Issue 2: Poor clarity, obstructions, or side profile angles exceeding 20° may result in subpar output.
- Issue 3: Rapid body movements or continuous head shaking can negatively affect the output.
- Issue 4: Subjects with heavy facial hair may experience reduced lip-syncing accuracy.
- Issue 5: Subjects with very thick lips may result in poor lip-syncing performance.
- Issue 6: The application currently has limited support for cartoon or animated characters.
- Issue 7: The application does not currently support lip-syncing for animal characters.
Audio Guidelines
Audio Material Requirements
- The application supports common audio formats such as WAV, AAC, and MP3.
- For user-uploaded audio, ensure the content is free from noise, distortion, and noticeable background interference. Avoid unclear pronunciation as well as excessively fast or slow speech.
- If usingText-to-Speech to generate audio, take note of the following:
- Each voice is designed for a specific language. To achieve the best results, input text that matches the voice's language.
- Some voice support adjustments for emotion, vocal pitch, and pace. Users can configure these settings as needed.
- The Text-to-Speech module generates results with some variability. If unsatisfied with the output, you can regenerate and select the preferred result.
Operational Notes (App)
- Audio on the track can be split, trimmed, deleted, undone, or restored.
- Clone voice. With just 3-10 seconds of clear audio, you can generate voice clones that capture nuances such as tone, rhythm, and emotion. Our cutting-edge AI voice synthesis technology delivers natural and fluent results.
- The quality of Text-to-Speech output using cloned voice models is closely tied to the quality of the 3-10 seconds of original audio and the accuracy of the text-audio match. If the synthesis results remain unsatisfactory, consider selecting a different audio clip to clone a new voice.
Other Notes
- After adding audio to the track and select the face, the system will automatically match the audio with the appropriate character avatar in the video. For videos include multiple avatars, please confirm the audio-to-avatar mapping.
- When adjusting audio positions, align them as closely as possible with the video content to ensure optimal lip-sync results.