HOW COME SOMETIMES THE HINTS ARE WRITTEN IN DETAIL, AND AI PRODUCES IMAGES THAT DON'T MATCH EXPECTATIONS
When the spider was asked to stay in front of the person, he ran into his eyes; when he was asked to hit the tail of the ship, the place of each explosion was different; and when the person was able to flee, he suddenly made an unjustified move。
In these cases, the first reaction of many is to continue to modify the hints and to add new adjectives and action descriptions。
But the problem may lie not in the lack of detail, but in the fact that we have not been able to identify the reasons for the deviation。
On different issues, different means of amendment are needed. There are some issues that need to be redefined, some that need to be locked through reference maps and others that need to be supplemented by the back-to-back movement of people。
This section begins in these three directions by learning how to position problems and modify the hints to make each adjustment more relevant。
I. JUDICIAL QUESTIONS BEFORE REVISION OF PRESENTATIONS
Do not rush to regenerate before it is changed。
We need to be clear: what's wrong with the current picture
Common generation deviations can first be divided into three categories。
Category I: Inaccuracy in the picture。
WHAT YOU WANT IS A SPECIFIC SHOT, BUT AI UNDERSTANDS IT AS ANOTHER IMAGE. FOR EXAMPLE, YOU WANT THE AUDIENCE TO SEE THE SPIDER IN FRONT OF THE PERSON, BUT THE SPIDER APPEARS ON THE PERSON'S EYE。
Such questions require a priority examination of lens perspectives and action descriptions。
Category II: Inaccuracy of space position。
The object, person or effect in the image appears but does not appear in the given position. For example, the explosion took place in the middle of the hull, not the tail you requested。
Such questions require a clear location relationship between objects, using reference icons where necessary。
Category III: Unreasonable behaviour。
The character did the move, but it did not match the current situation. For example, the danger has emerged and the person has not fled, but has acted without reason。
Such problems require an examination of the motivation of the person and the relationship between the action and the action。
These three categories of issues cannot all be addressed in the same way。
In the case of a lens perspective, adding emotional terms is not useful; in the case of spatial positioning, the constant emphasis on accurate images is not necessarily a solution; in the case of behavioral logic, it is equally difficult to achieve stable results simply by requiring people to be more natural。
The first step in the amendment of the hint would be to determine the type of problem before choosing the corresponding modification。
II. METHODOLOGY I: Take the action apart if the camera is not in line with expectations
A COMMON PROBLEM WITH AI VIDEO GENERATION IS THAT EACH REQUIREMENT IN THE MESSAGE IS NOT PROBLEMATIC IN ISOLATION, BUT WHEN IT IS COMBINED, THERE IS A DEVIATION。
This is usually related to camera expression and movement arrangements。
1. Do not make a shot take too many tasks
Suppose we're trying to create a picture of a character who found spiders。
The idea might have been:
The person opened his eyes, saw a spider in the first view, then waved out the spider and quickly pulled out the weapon。
It's a full-on move, by the way. But from the point of view of image generation, it contains many different tasks。
First of all, people can be seen through facial features。
Secondly, the first person claims that the perspective needs to present the person's immediate environment。
The spiders then approach, the person waves and pulls out the weapon, each in a different state of movement。
WHEN ALL OF THIS IS IN THE SAME SHOT, AI MAY NOT BE ABLE TO DETERMINE EXACTLY WHICH ACTION SHOULD BE GIVEN PRIORITY, OR MAY CONFUSE THE RELATIONSHIP BETWEEN THE EYES OF THE PERSON AND THE SUBJECTIVE VIEW。
Thus, the more action the picture is, the less complete it is; only by identifying the main tasks of each shot will it be easier to produce as expected。
2. First determine the lens perspective
The first step in modifying such hints is not to add more action, but to determine where the camera was taken。
If the audience wishes to see the characters open, external facial features are used。
If the audience is expected to see the spider in front of the person, the first person ' s subjective view is used。
Both images can appear in the same video, but should not be confused in the same shot。
For example, the original action can be split into two lenses。
Camera one: People open their eyes。
Prompt words:
Multi-censor shoots over time, with the camera switching only at specified nodes, and the camera not tailoring itself. 0.0 to 4.0 seconds -- camera 1, CU 29°, reverse image: the person is hanging on the spider foreground, with a clear focus, a slow rotation, and the front leg bends in the direction of the boy's nose. Behind it is @char_jungle4's upside-down face, with his eyes open, staring at spiders. In 2.5 seconds, the focus quickly moved from the spider to the boy's face — the moment the focus fell on him, he screamed, his eyes were wide open and his cheeks were shaking. In the middle of the clip, there is no focus deviation。
Camera two: See the spider。
Prompt words:
The first frame is pure black -- completely closed. The whole shot is full of 21:9 on every edge. The eye-opening visual effects are presented in the form of a soft, black radiant radiance: two smooth, unspecified dark blocks are separated from the centre of the picture to the side — a clean eye fading, with no anatomical details, but only a sense of open eyes. As soon as the blinds were opened, a narrow, dark green shadow crossed through the centre of the picture, and a cloud of dark spider-shaped objects stopped on its equally vague web, located at a third of the left of the picture, at a fixed distance, and the dark upside-down jungle fainted into a soft, blurry piece behind it。
The difference between these two hints is that the first shot shows the person, the second the person sees what he sees。
ONCE THE CAMERA MISSION IS CLEAR, AI DOES NOT HAVE TO DEAL WITH CONFLICTING VISUAL REQUIREMENTS AT THE SAME TIME。
3. Re-decide how the actions should be linked
Once the camera is clear, it's action。
If the person has to wave after seeing the spider, the action can be placed in a follow-up shot。
For example:
Scene 2, WS 63° Panorama: He hangs upside down — a big green vine is wrapped up in his ankles, and the entanglement is tied to his boot drums, without any contact; the calves, knees, thighs and torso above his ankles are completely untouched by the vines and completely suspended; and a moored vine with a rough green plant skin extends up from the entangled boots to a fixed point two metres high. His upside-down head was close to the jungle floor, screaming, shaking and his body shivering and shaking uncontrollably, with his already hanging arms moving indiscriminately. Cobs force position: The same @char_spider hangs on his own spiders, flats his face and head, 10-15 cm in front of his face — not too high or low. In 6.5 seconds, he fired on the back of his left wrist - with a sharp back-hand shot at the height of his face, leading by the finger section and the outside of his wrist - from the spider line: The 50-70-gram spider flew out of the drop track, falling arc to the ground, and then crawled fast towards the bushes on the left edge of the picture, and went missing in the cracks of the leaves; the small, light spider passed through the whole way down, and the bush lines remained intact, and each leaf and branch was not touched. The boy shivering, breathing heavy, looking at the direction where the spider disappeared, and the green Fujiman continued to squeak under his body weight, bending a little, until the end of the lens. No deviation in the lens。
Attention, there is no need for any more after-actions, such as the drawing of knives, turning over and running away。
If these actions are indeed part of the story, they can continue to be divided into separate footage。
4. How does it be judged whether the action needs to be divided
One simple way could be to ask yourself three questions after reading the hint。
What's the most important image of this shot
Where should the audience look at it
Does this shot also require people to perform multiple different actions
If a single camera is tasked with multiple tasks, such as the reaction of the person, the movement of the object, the alternation of the camera and the subsequent attack, the split should be considered。
However, splitting does not mean that all actions must be generated separately。
If multiple actions are continuous, simple and do not create confusion of perspective, they can be kept in the same lens. Only when there is a clear shot difference between the actions, a space conflict or a disagreement is there to be further split。
Core principle: To clarify the current camera and then set up the next one。
Method II: always error in space position, with reference icons
The second common problem is that the picture as a whole does not appear to be clearly wrong, but the position of the key object remains inaccurate。
For example, we need to generate a picture of a naval war: a sailboat is sailing on the surface of the sea, a shell is heading towards the boat and it hits the tail。
Once the results have been produced many times, the location of the explosion may be in the centre of the hull or on the sea side of the hull。
At this point, it is often not the most effective to continue to repeat that “a shell must hit the tail”。
1. Why is it not always sufficient
Because space is not only a name, it also contains specific visual relationships。
“The ”tail” describes the broad area, but does not directly show the specific coordinates of the target's location in the picture。
IN PARTICULAR, AI MAY HAVE A DIFFERENT UNDERSTANDING OF THE TARGET AREA THAN WE DO WHEN THERE IS A CHANGE OF PERSPECTIVE, A CHANGE OF LENS, OR WHEN THE HULL ITSELF HAS A COMPLEX STRUCTURE。
Location requirements are more easily disturbed by other information if the hints include both the person, the ship, the shell, the explosion and the camera movement。
So we can use the reference diagram to mark the target position。
2. Step 1: Prepare a clear reference map
Select a reference map with a clear hull structure and a clear view of the bow and tail。
To select as many pictures as possible that are close to the final scene。
If it is ultimately intended to generate a rear view of the ship, priority is given to the use of a reference map that clearly shows the tail of the ship; if a side photograph is required, a picture with a clear side structure is selected。
The role of reference maps is not simply to provide a ship's appearance, but to provide a basis for subsequent space positioning。
3. Step 2: Direct marking on the map
This can be done using any image editing tool。
Mark the target position of the shell with a red circle。
Marks do not need to be complicated and can be easily understood。
For example, the red circle represents the target of the explosion, the blue dot indicates the position of the person and the arrow indicates the direction of the shell。
Do not write a lot of explanatory text on the picture, nor use too many similar labels. The clearer the mark, the easier it is to establish a correspondence with the text phrase。
4. STEP 3: ALLOW AI TO REORGANIZE THE HINTS ACCORDING TO THE MARK
WHEN THE TAG IS COMPLETED, THE PICTURE IS HANDED OVER TO AN AI TOOL WITH IMAGE UNDERSTANDING CAPABILITY, INDICATING WHAT EACH MARK REPRESENTS。
The following phrase may be used:
Please analyse the ship reference charts and the markings in them I have provided. The red circle represents the position of the shell, the blue mark represents the position of the person and the arrow represents the direction of the shell. Please reorganise the video to generate the hints based on the actual spatial relationship in the diagram. The direction of the shell, the location of the explosion, the distance between the person and the location of the explosion and the relative position of the various parts of the hull are clearly stated。
Retain the ship ' s structure in the reference map in relation to space and do not change the location of the explosion to the central part of the ship or to the sea surface around the ship. Don't add other blast sites。
OUTPUT A PARAGRAPH THAT CAN BE USED DIRECTLY FOR AN AI VIDEO GENERATION IN CHINESE。
THE FOCUS OF THIS STEP IS FOR AI TO TRANSLATE VISUAL INFORMATION FROM IMAGES INTO MORE EXPLICIT TEXTUAL EXPRESSIONS。
IT NEEDS TO BE NOTED THAT THE DIAGRAM DOES NOT MEAN THAT AI WILL FOLLOW 100% OF THE POSITION. ITS ROLE IS TO REDUCE THE AMBIGUITY OF SPATIAL UNDERSTANDING AND, ULTIMATELY, TO COMBINE RESULTS-GENERATION TESTS。
5. Step 4: Inclusion of position relationship in final hint
The modified introduction would also require an explicit retention of the most important space constraints。
For example:
Bullet-tracking angle, 63°, real time: the enemy deck of the opening scene - – Black iron cannons mounted on wooden cannons have been launched, with guns pointed to the left side of the picture, and a large sailboat with light white sails parked far away. The latest gun opened fire in an instant — when the gun came out of its mouth, and the gun was slamming back on the liner group — and the lens then turned and moved with it: it was fixed on the chords next to the round cannon, and it was so close to the water, and the bottom of the sea was dragged long and blurry by intense motion. The shells remained in the far sky on the left edge of the picture throughout the flight, presenting a gruesome green in the fog. The target is getting clearer: it's the back wall of the Eduardo ship -- a tall wooden cliff wall extending from the waterline to the top decks with bars. It has gradually expanded from a vague contours to a huge wall full of images, and the aircraft is heading towards the first half of it。
CLIP 3 - HARD TO SLOW MOTION, MS 47°, BACK TO THE STAGE, WITH THE IMAGE EXACTLY THE SAME AS CLIP 1: THE CAMERA IS BEHIND THE REAR OF THE SHIP AND SLIGHTLY ABOVE - THE WHEEL IS ON THE RIGHT SIDE OF THE PICTURE, EDUARDO IS ONE STEP AWAY ON THE LEFT SIDE OF THE WHEEL, A MACAW ON HIS SHOULDER, TWO PEOPLE ARE WET, DRIPPING, CROSS-POSTS AT THE BOTTOM OF THE PICTURE, AND THE SEA IS ON BOTH SIDES. THE SHELL HIT THE LOWER SIDE OF THE DECK, STANDING BEHIND IT, AND WAS OUTSIDE OF SIGHT — IMMEDIATELY FOLLOWING WHICH THE EXPLOSION EXPLODED FROM BEHIND THE CREW, BEHIND THEIR FAR EDGES ON THE DECK: WOODBOARDS, LONG PIECES OF WOOD AND SMOKE CAME UP BEHIND THEIR BACKS. THE SECOND AND THIRD ROUNDS HIT THE BACK OF THE SAME BOAT IN SUCCESSION AND EACH EXPLOSION LIFTED A NEW WATER COLUMN BEHIND THEM. THE LATEST EXPLOSION DIRECTLY REMOVED EDUARDO FROM THE GROUND — STRAIGHT IN THE DIRECTION OF THE LENS. FLIGHT DIRECTION IS CLEAR AND CLEAR: AN AREA OUTSIDE THE REAR OF THE SHIP MARKED WITH A REFERENCE TO THE RED CIRCLE OF IMPACT. THE EXPLOSION WAS CONCENTRATED AT A SPECIFIED POINT OF IMPACT, AND THE FIRE AND SMOKE SPREAD ALONG THE OUTER SIDE OF THE HULL, SO HIS BODY WAS THROWN IN THE DIRECTION OF THE LENS BY A SHOCK WAVE — IN EACH IMAGE, EDUARDO'S BODY HAD GROWN LARGER, HE CROSSED THE RAILING HAND TOWARDS THE LENS AND CARRIED HIS ARMS STRAIGHT FORWARD, WHILE THE WHEEL, THE MAST AND THE DEBRIS SPRAY WERE RETREATING BEHIND HIM. HIS FACE WAS LIKE A FUNNY SHOCK MASK: HIS MOUTH WAS FULL, HIS EYES WERE ROUND, HIS CHEEKS WERE NOT SHAKING IN THE SHOCK WAVES; THE PARROT WAS LIFTED FROM HIS SHOULDER, A RED FEATHER WAS BLOWN UP AND HIS WINGS WERE THROWN AROUND HIM. THE RUDDER BOWED TO THE WHEEL, AND THE RUDDER REMAINED INTACT. THE CLIP ENDED IN HALF-EMPTY — EDUARDO WAS EMPTIED, AND THE WHOLE PICTURE CAME TO THE CAMERA。
The hint is to clarify the location of the target, the direction of the movement, the position of the person and the errors that need to be avoided。
The mere repetition of the “tail explosion” provides more conditions to be validated by the image。
If the result of the generation is still off the target, the marking position, reference map angle or movement direction can continue to be adjusted, instead of rewriting the entire paragraph from the top on each occasion。
IV. METHODOLOGY III. COMPLETE RESPONSIBILITY AND EFFECTIVENESS
The third issue is more hidden。
The role in the picture may be correct, the scene is not problematic, the movement itself is even fluid, but the whole act seems unreasonable。
For example, a man met a group of monkeys who came in fast. He was able to run away, but then he fell down on the ground, his hands around his head and he didn't move。
IF ONLY AI WAS REQUIRED TO MAKE HIM MORE AFRAID, THE RESULT WOULD NOT NECESSARILY IMPROVE。
BECAUSE THE PROBLEM IS NOT THAT THE PERSON'S EMOTIONS ARE NOT STRONG ENOUGH, BUT THAT AI DOES NOT HAVE A SUFFICIENTLY CLEAR BASIS FOR BEHAVIOUR。
1. Emotional description does not equate to action logic
“FEARING” OF “STRETCHING” “FRIGHTFUL” CAN HELP AI TO UNDERSTAND THE EMOTIONS OF A PERSON, BUT THEY CANNOT DECIDE INDIVIDUALLY WHAT THE PERSON SHOULD DO。
In the face of danger, people turn around and flee, people hide behind barriers and people fall down to protect themselves。
These acts may be consistent with feelings of fear, but they need to be judged in the context of which they are adopted。
If the hint is only:
Men are very afraid, with their hands on their heads, to beg for forgiveness on the ground. The monkeys are around。
The reasons why the person was lying down, how close the monkeys were to him, why he did not escape were not clearly stated。
AI CAN ONLY SUPPLEMENT THE MISSING BEHAVIORAL LOGIC BY ITSELF。
Therefore, to justify the person ' s actions, it is not only to describe what he has done, but also to explain what prompted him to do so。
2. Organization of actions by triggers, reactions and results
The behaviour of the person can be broken down into three parts。
Step 1: Clear triggers。
What makes people react
For example, several monkeys suddenly ran out of the woods and ran in the direction of the person。
Step 2: Clarify the response of the person。
What did the character see? Why would he do that
For example, when men discovered that the monkeys were passing by themselves at high speed, fearing to be hit, they immediately lowered their weights and their hands crossed their heads。
Step 3: Clarify the outcome of the action。
What happened to the scene after the character finished the action
For example, the monkeys ran through the characters and the men remained in a low position until calm was restored around them and their heads were raised slowly。
In this way, the person is no longer isolated。
He didn't fall down because of the “fear” in the hint, but because the danger was approaching and he needed to protect himself。
3. Inclusion of causality in a hint
You can use the following:
The camera was placed at a height of 4 metres, with an angle of 47 degrees of vision, and the camera was tilted downwards, and the mist was photographed in a small path. The explorers came to the bottom of the lens, with their arms moving naturally. His boots tripped to the edge of a slab — he fell forward, fell heavily on the floor and slipped on a rock. At the moment when he fell, the camera fell down quickly, in a series of shaking arms, slowly from a high angle down to the top of his body, and eventually in a close view. His hands beat him hard in the back of the head - with his right hand behind his right ear, with his left hand behind his left ear, with his fingers tied to his hair, with a clear gap between his hands - with his elbows close to his body, his eyes closed and his whole body constricted into a protective position. And as soon as he hit a rock, the monkeys came over: the first monkeys were whistling from both sides of his body, and the tan of a blurry tan over the picture, and one of them jumped on his back while he was begging. The monkeys swarm like a tide, and his face was still buried in stone, and his voice resounded in the camera, and he screamed and cried, “Well, I've had enough!” I do! You win! Please stop!"
The camera first shows the close environmental relationship of the monkeys, then switches to the low movement of the characters, and ends with the characters looking at the immediate surroundings. The person's movements are natural and consistent。
It needs to be noted that the addition of causality does not add much to the story。
THERE IS NO NEED TO DESIGN COMPLEX BACKGROUND STORIES FOR PEOPLE, NOR TO ADD NEW ROLES AND EVENTS. IT IS ALREADY POSSIBLE TO HELP AI UNDERSTAND THE BEHAVIOUR OF A PERSON BY ADDING THE DIRECT CAUSE OF THE CURRENT ACTION AND THE STATE AFTER THE ACTION HAS BEEN COMPLETED。
4. Do not write all actions into emotional instructions
A lot of people like to add similar words over and over again when they change their characters:
More fear, more panic, more stressful and more disturbing。
These descriptions are not completely useless, but they cannot replace specific actions。
If the person should flee, the direction, physical posture and route of the escape should be clearly written; if the person should hide, the place where the danger arises, how the person should hide, and where the person should hide。
Emotions are responsible for supplementing the status of the person, actions are responsible for the performance of the act, and causality is responsible for explaining why the person does so。
When the three components are clearly linked, the behaviour of the person is more likely to fit the scenery logic。
V. THREE ISSUES, WITH THREE METHODS OF AMENDMENT
After learning the first three cases, we can organize them into a more user-friendly methodology。
The difference between these three approaches is that they address the lack of information at different levels。
What needs to be addressed is “what the audience should see”。
What needs to be addressed is “where an object should be located”。
What needs to be addressed is “why people act like this”。
If these problems are not distinguished, it is easy to see a situation in which the movement is constantly modified, even though the space is inaccurate, and the emotional description is constantly increased, even when the person lacks the motivation to act。
The longer the hint is written, the less significant the result is produced。
Therefore, each change should be preceded by a determination of the category of current deviations。
Conclusion
AI GENERATED IMAGES THAT WERE NOT IN LINE WITH EXPECTATIONS AND DID NOT MEAN THAT THEY COULD ONLY BE INCREASED。
Sometimes we need to redefine the lens
In some cases, spatial location needs to be determined through reference maps
There are also times when it is necessary to complete the causal relationship behind the action of the person。
These three approaches address the issues of image expression, spatial positioning and behavioral logic, respectively。
When you are able to accurately judge the deviations and choose the corresponding amendment, there is no need to rely on a large number of repetitive descriptions。
THE KEY TO THE OPTIMIZATION OF THE HINT IS NOT TO MAKE EACH SENTENCE MORE COMPLEX, BUT TO LET AI KNOW EXACTLY WHAT THE PICTURE SHOULD BE, WHERE THE OBJECT SHOULD BE AND WHY THE PERSON SHOULD DO IT。