Today I'm going to teach you how to use AI to generate two people talking to each othervideo, this video production, the lip-sync and the subtitles match better and are more authentic.
AI Tools
Tencent Yuanbao: https://yuanbao.tencent.com/
Beanbag: https://www.doubao.com/chat/
i.e. Dream: https://jimeng.jianying.com
Screening: PC Professional
The following is a direct explanation of the practical steps:
1、Generate copy
For two person dialog copy, you can use the following method:
1, drawing on some of the video in the conversation copy, the method explained earlier
2. Write your own
3. Use a large model
I'm using the 3rd one here, with Tencent Meta, so it's up to you to choose the right tool for you. Extract your favorite text by way of the conversation below.
Input Prompt Words:I want to make a short video of a young 30 something intellectual woman and man, discussing a topic.Could you please provide me with a few topics that are more appropriate for discussion at the moment, and present them in a dialog. Discuss them in terms of the woman, and the man, respectively. Do not exceed 30 words per paragraph
In the end, I chose the 1st text:
The following short video conversations on the topic of "Lifelong Learning and Skills Enhancement" are concise and powerful, highlighting different perspectives and points of resonance:
WOMAN (smiling, arms folded)
"Technology iterates so quickly now that skills from 3 years ago may be obsolete next year - do you think we should keep learning?"
Man (tapping his fingers on the table, thinking)
"Learning is a must, but it has to be strategic. It's better to follow blindly than to focus on core competencies such as AI logic or cross-disciplinary thinking."
WOMAN (nodding gently)
"Right! Like you said before about digital transformation, fragmented learning is not enough, it has to be systematic."
Man (leaning back in his chair, with a gentle gaze).
"Also, don't underestimate social learning. I've been in industry communities lately and have learned a lot of real-world examples."
Woman (eyes light up, pulls out cell phone)
"I'll friend you into the group! By the way - if you're short on time, do you recommend remediating your major or soft skills first?"
Man (raises his hand and makes a "two").
"Fifty-fifty! Professionalism is the foundation, but communication and collaboration is the long term advantage. Like I am now ...... (pause) still have to learn how to make things easier!"
WOMAN (nods with a light smile, screen fades)
"Lifelong learning = evolution chant. See you next issue!"
2、Designing dialog character roles
1, open the beanbag, the above selected text, copy to the beanbag dialog box, ask the beanbag according to the following dialog, design two interview characters.
Beanbags are designed that way:
2, step 2 to convert the dialog box mode to image generation
Then enter the prompt word: I want to make a short video, please design the two characters above into an interview program. Put the two characters in a scene to generate a picture. Please provide the content of the screen description that matches the dialog.
Set your ratio, I chose 16:9 here
If you're not satisfied, you can continue to optimize, like below:
The last question is very important, the distance between the two people needs to be kept, and it will be needed later when editing in the cutscene to use the mask.
After a series of operations, it was finally decided to use the first generated image.
3、Generate lip-sync videos
Note: For this step, I also generated the video in Doubtbag first, following the previous anime lip-sync video method, and generating separate clips of men talking and women not talking.
It is to use the [Counterpoint] function in an instant dream.
【AI LearningWorkplace hitman trolling video production guide: three steps to learn, with detailed step-by-step instructions
1, before generating the lip-sync video, I also did this one operation, respectively, to erase the faces of the 2 characters in the original picture. Hopefully, when generating the lip-sync video clip, only one person is talking.
Original image:
Image 1 after erasure:
Image 2 after erasure:
2、Organize the content of character dialogues
To save on production costs and time, start by organizing the entirety of the conversation between the two people, like below:
Female:
Now that technology iterates so quickly and skills from 3 years ago may be obsolete next year, do you think we should keep learning?
Right! Like you said before about digital transformation, fragmented learning is not enough, it has to be systematic.
I'll friend you into the group! By the way, if you don't have enough time, do you recommend fixing your major or soft skills first?
Right, need to keep up with lifelong learning, see you in the next issue!
Male:
Learning is a must, but it has to be strategic. It's better to follow blindly than to focus on core competencies like AI logic or cross-disciplinary thinking. Also, don't underestimate social learning. I've been in industry communities lately and have learned a lot of real-world examples.
Fifty-fifty! Professionalism is the foundation, but communication and collaboration is the long term advantage. As I am now, I still have to learn how to speak more simply!
3. Production of lip-synching video clips
I'll do the girls speech video here first and just upload and erase image 1:
Once the 1st video is finished, paste in the remaining text and continue to generate the video.
Similarly, make the boys' dialog the same way, just replace the picture with the 2nd one after the erasure.
Note: After the video clip was generated, it was found that erasing the image played little effect, and the 2 individuals were both recognized and talking at the same time.
That only depends on the later cuts.
4. Cutting and mapping clips (using masks)
Open the cutout, import the video clip, add masks and stickers.
1. Import all video clips and one original picture. It is best to name the video clips according to the number of times the girls talk and the boys talk to make it easier to add videos.
2, add the 1st video clip, here is the girl speaks first. As the video generated earlier, 2 people are moving, so add an original picture on top to cover the video.
3, selected images, select [Mask], select [Linear
4, Rotate the mask to 90 degrees. Mask the man in the video.
5. Play the video of the girl's speech first. According to the dialog text above, after the girl finishes the 1st sentence, it should be the man's speech. Split the video here and put the rest of the video into the 2nd conversation.
6, then, add the first video of men talking, the same add [Mask], select [Linear], the girls to cover.
Also as above, split the video when it's time to split it and put it on the back burner.
The masking section, too, doesn't have to be added every time.
The front of the girl's dialog and the boy's dialog has been set up a good mask, copy and paste to the back can be used. Specifically need to see the actual effect of adjustment.
Separately, add the dialog video behind it and make it in the same way.
Note: But the last video, the boys' hand movement is too big, using the mask effect is still a bit affected. So the front in the design of the original picture, the two people need to keep a distance before. Or in the lip-synching part, draw the card a few more times and choose the satisfactory clip. But the use of points is also considerable.
7. Recognize subtitles.
Select all video clips, video subtitles. Adjust subtitle formatting, add stickers to cover watermarks, add captions.
Feelings:
1, this article, the process is slightly more complex, but you can learn a lot of knowledge, but also can be applied to other scenarios by example.
2. The shortcoming is that when one person is talking, the other person's expression stays motionless and unnatural. However, using the previous method of dialoguing in anime characters, it is difficult to keep the mouth shape consistent.