Introduction
When creating videos with AI, have you encountered these problems? The visuals look amazing, but you need to separately produce all audio elements, laboriously splice audio tracks, adjust timing, and often end up with unsynchronized audio and video. Environmental sounds, special effects, and voices remain isolated from each other, breaking the immersive experience.
Now, "SeaArt UltraVision" brings a breakthrough "audio visual integration" feature: generate everything in one go. Voice, sound effects, and ambient sounds are all handled automatically, with rhythm and emotion naturally aligned, truly achieving "what you see is what you hear."
Let's explore how to easily create viral videos with this tool!
Japanese Official Guide >> https://www.seaart.ai/articleDetail/d4trn45e878c73ad3cf0
Chinese Official Guide >> https://www.seaart.ai/articleDetail/d4tpk8de878c73fi4pq0
I. What Changes Does "SeaArt UltraVision Model" Bring?
1. Core Capabilities
● Audio Visual Synchronization: Voice rhythm, ambient sounds, and visual actions are synchronized, avoiding disconnection between visuals and audio.
● Audio Quality: Supports human voice, sound effects, and ambient sounds with cleaner quality, richer layers, approaching real mixing.
● Semantic Understanding: Can understand conversational language and complex plots, accurately grasping creator intent to deliver more harmonious audio-visual content.
➢ In summary, "SeaArt UltraVision" only requires text or images to automatically complete synchronized generation of visuals and sound. One click voiceover, synchronized sound effects, lower creation barriers, significantly reduced production time, and professional-level results at your fingertips!
2. Two Efficient Creation Paths
#1 Text-to-Audio-Visual: Input text to generate video with voice, sound effects, and ambient sounds.
Steps：
1. Select SeaArt UltraVision model
2. Choose text-to-video, input prompts
3. Set parameters (duration, size)
#2 Image-to-Audio-Visual:Upload image/text to generate audio-video with one click, suitable for expanding static images into audio-visual content.
Steps：
1. Select SeaArt UltraVision model
2. Choose image-to-video, input prompts and reference image
3. Set parameters (duration)
II. Prompt Tutorial and Case Studies
Prompt General Structure: [Scene] + [Subject] + [Motion] + [Audio] + [Others]
(A) Basic Tutorials
To get the model to produce satisfying videos, the key is to help it understand what you want. This tutorial follows a "simple to complex" logic for writing prompts, starting from the simplest [pure background sound effects], gradually transitioning to [music videos], and then to [complete plot performances], allowing you to progressively master key video creation concepts.
1. Background Sound Effect Narratives: From Soothing Animal Videos to High-Energy Fight Scenes
Let's start with the lowest threshold approach that produces the most intuitive results: simple visuals primarily relying on rich detailed sound effects for immediate immersion. Examples include cats washing dishes, fight scenes, tool impacts, etc. These scenes have simple visuals, but when sound and action are properly synchronized, they become compelling videos. With powerful SeaArt UltraVision, you can easily create various styles of "visuals + sound effects" content.
● Cute Category: Cats washing dishes in the kitchen, dogs eating ASMR, small movements accompanied by dish clanking and chewing sounds, very soothing.
● Realistic Category: Factory assembly lines running, repair workers grinding metal, roadside stall frying sounds, everyday scenes with detailed sound.
● Intense Category: Boxing gloves colliding, metal swords scraping, explosion debris rolling on the ground, suitable for satisfying battle segments.
● Suspense/Horror Category: Wooden stairs creaking, wind in corridors mixed with footsteps, faint collision sounds behind doors, instantly creating atmosphere.
● Sci-Fi/Future Category: Energy chamber low-frequency humming, mechanical arm rail sounds, metal cabin doors slowly folding, paired with cold-toned sci-fi chamber visuals.
Prompt Template:
[Scene]:
One sentence describing "where the person is & what the atmosphere is like"
[Subject]:
Who is the main sound-producing character in this video?
[Motion]:
What "sound-producing" actions does the main character perform?
[Audio]:
Break down "all sounds" into two layers:
● Nearby action sounds:
Clarify [object/material] + [action] + [onomatopoeia/sound description]
Ceramic plate being scrubbed "shh-shh", stainless steel counter being tapped "clang", wood breaking "crack", leather friction "rustle"
● Environmental background noise:
Clarify [what continues to sound in the environment] (oil pan on low heat, distant talking, air conditioning, exhaust fan...)
Low frequency rumbling in the kitchen, forest insect sounds and wind, faint traffic in the night city, hollow echoes in the basement, distant explosion echoes on the battlefield
[Camera]:
To show who is the visual protagonist, simply specify using "wide shot / medium shot / close-up".
Case Study:
Prompt:
Late night kitchen, cool lighting, spacious kitchen with neatly arranged stoves and workstations, chefs busy stir frying and chopping vegetables in the distance, bubbling foam in the sink, atmosphere of a serious working kitchen. An orange and white kitten wearing a small chef's hat stands on a stool, paws holding a bubbly ceramic plate, seriously "working." The cat uses its paws to rub back and forth on the plate, pats water splashes, and finally gently taps the plate rim, water droplets splashing. Ceramic plate being rubbed—shh-shh-shh, water splashing—splash-splash, plate being tapped—ding, water dripping into sink—drip-drop. In the background, continuous low humming of exhaust fans, overlaid with rhythmic collisions of kitchen utensils in the distance, with the metallic reverb unique to back kitchens. The camera captures the cat and sink in a wide shot, panning past a chef stir-frying nearby.
Prompt:
Dusk on a wasteland battlefield, half collapsed broken buildings, black smoke billowing in the distance, air filled with dust and flames, tense and oppressive atmosphere. A scrapped military truck stands in the center of sandy ground, surrounded by oil drums and shattered sandbags and stones, the main "protagonists" of the explosion. In front of the camera, a high explosive shell hits the bottom of the military truck, instantly igniting the fuel tank, a fireball shooting skyward, shock wave overturning sandbags, metal fragments and stones scattering in all directions. Military truck fuel tank hit by high-explosive shell igniting—"BOOM—!" (low frequency heavy explosion sound) Metal truck shell blasted open，"crack, crash!" Debris and shrapnel hitting the ground，"ding, bang" (stones and metal hitting sandy ground and metal sheets). The distant battlefield still has sporadic explosions, with intermittent "rumbling" echoes; the air filled with "sizzling" sounds of burning flames, accompanied by slight wind sounds and echoes of the open terrain, creating a chaotic battlefield atmosphere overall.
2. Music-Oriented Videos: Rap, Instrumental Music, MVs Directly Achieved
After learning to tell stories with "visuals + sound effects," many people want to take the next step: can I directly have the model help me create a complete music video, producing melodies that match the visuals? The answer is: absolutely. With powerful SeaArt UltraVision, you can easily generate music-oriented videos in various styles.
● Song Performance: Girl softly singing a love song, boy sadly singing folk music, etc.
● Street Rap: Rapper delivering a compelling rap with beats perfectly matching lip movements.
● Instrumental Music: Electronic music, piano pieces, string music, etc.
Prompt Template:
[Scene]
One sentence describing "where the music is playing & what the atmosphere is like."
[Subject]
Clearly describe "who this person/musician" is: gender/age/clothing/identity
[Motion]
Actions while singing/rapping/playing:
[Audio]
● For singing (with lyrics):
Write: "Lyric content" + singing style + instrumental description + emotion
[Female, clear voice] Singing gently: "From the day I met you, my world began to have meaning..." Pop singing style, accompanied by acoustic guitar, emotion deep, slightly melancholic
● For rap:
Write: "Rhyming sentences" + rhythm style + emotion
[Young rapper, deep and powerful] Rapping to the beat: "City lights, long nights, I'm on my grind, no goodbyes." Boom Bap, confident, slightly provocative
● For instrumental music (no lyrics, just atmosphere):
Write: Instrument type + music genre + emotion
Audio: Piano solo + Classical/Ambient, quiet, slightly melancholic, suitable as BGM for night city distant view
[Camera]
Singing/Rapping: Medium close-up focused on face and upper body, appropriate close-ups of mouth, gestures, instruments;
Instrumental music: Camera can slowly push, slowly move to match the rhythm and emotion of the music, such as slowly flying over city night views, scanning from raindrops outside the window to figures inside.
Case Study:
Prompt:
In a garden, soft evening sunlight illuminates the grass and flower beds, with blurred shrubs and several blooming rose bushes in the background. A long haired girl wearing a light colored dress stands in front of the flowers, singing softly to the camera, her body swaying slightly from side to side, hands naturally hanging at her sides. [Female, clear voice, moderate pace] Singing gently: "When the daylight fades and the stars appear, I still feel your heartbeat, whispering in my ear." Accompaniment is soft piano with slight strings, romantic and quiet emotion. Camera uses medium close up on the girl's upper body, background slightly blurred, preserving the garden's colors and light halos, overall image beautiful and soft, like a simple English love song MV.
Prompt:
Twilight piano room, sunset spilling over black and white keys. A young girl sits at the piano, her dress falling gently, fingers tenderly landing on the keys, expression calm and focused. The girl maintains her sitting posture, hands gently rising and falling, occasionally glancing out the window, the pedal beneath the keys lightly pressed, rhythm steady. Piano solo, classical genre, melancholic emotion, with slight tender vibrato, accompanied by faint wooden resonance in the room. Camera captures the girl and grand piano, slowly pushing in toward her raised fingers and sliding piano keys.
3. Plot Narratives: From Solo Vlogs to Multi-Character Short Dramas
After solving "sound effects" and "music" in the previous two sections, the next step is to make videos truly carry storylines. This means higher requirements: lip movements need to match dialogue, actions and gestures must convey subtext, and tone needs to carry subtle emotional changes. Now, with SeaArt UltraVision's powerful understanding and generation capabilities, you only need to clearly specify in your prompts: who appears, what happens, how emotions fluctuate, and it will directly generate narrative segments with rich storytelling.
The stories can be simple:
● Single-person narratives: Travel vlogs, immersive eating shows...
● Multi-person interactions: Couples saying goodbye at subway stations, colleagues confronting each other in break rooms, friends exchanging secrets in cafes...
Prompt Template:
[Scene]
One sentence describing "where people are & what the atmosphere is like."
[Subject]
● Single character:
Clearly describe "this speaking person's" character: gender/age/clothing/identity
● Multiple characters: Label each person
[Character A: office worker in white shirt], [Character B: colleague with glasses]
Briefly describe their current emotional relationship (bantering/arguing/discussing)
[Motion]
Accompanying body language while speaking
For multiple characters, write separately: A is pointing at paper, B has arms crossed, etc., to help the model know who is performing actions.
[Audio]
Character label + emotion + speech rate + tone + trigger word (says/says with a laugh/complains/etc.) + dialogue sentences
● Single character example:
[Girl, sweet and clear, natural pace] says with a smile: "Hey guys, check out this view，sunset vibes are unreal today!"
● Multiple character example:
[Male voice, suppressed and low, medium pace] complains: "You promised you'd call before making the decision!"
[Female voice, excited and firm, slightly faster pace] raises her voice to retort: "I'm tired of waiting for you to act—this is my life too!"
Remember to include audio trigger words like "says with a smile, says quietly, says excitedly, says helplessly, complains" to help the model create appropriate lip movements and emotional fluctuations.
[Camera]
Single characters typically use close-up/medium close-up: Focused on face and upper body to see both mouth movements and expressions. Background slightly blurred, only setting the atmosphere without stealing focus.
- For multiple characters, write: "Medium close-up focused on both people's faces and upper bodies, switching between A/B with simple shot-reverse-shot when necessary."
Case Study:
Prompt:
Seaside, sunset casting golden light on the water, waves crashing on the shore in the distance. A travel blogger girl is holding her phone for filming, gently swaying her body, finally pointing toward the sunset. [Girl, sweet and clear, natural pace] says, "Hey guys, check out this view,sunset vibes are unreal today!" Close up captures the girl's face and gestures, background slightly blurred showing the seascape and sunset, creating a relaxed vlog feel.
Prompt:
Evening apartment living room, neon lights beginning to glow outside, warm indoor lighting mixing with cool window light, sofa against the wall, a floor lamp creating an intimate home atmosphere. A young couple stands in front of the sofa, the man with arms crossed and furrowed brow, the woman with hands on hips and slightly flushed cheeks, just one step away from each other, tension in the air. The man leans slightly forward, the woman raises her chin, tone excited but maintaining standing posture. [Male voice, suppressed and low, medium pace] says, "You promised you'd call before making the decision!" [Female voice, excited and firm, slightly faster pace] says, "I'm tired of waiting for you to act,this is my life too!" Camera is close-up front view.
(B) Key Tutorials
1. Multi-Character Dialogue Scene Prompt Considerations
When a video contains multiple subjects or characters, clear prompts are key to generating natural dialogue, requiring clear definition of each character's identity, lines, and interactions.
P1. Fixed Character Naming
Core Principle: Use fixed labels for each speaker, avoid ambiguous terms like "he/she".
Correct Example: [Character A: Reporter in red], [Character B: Candidate]
Incorrect Example (Model Likely to Fail): [Reporter] says…[He] says again… (Model can't distinguish who is who)
P2. Actions Bound to Characters
Core Principle: Describe actions first, then dialogue, so the model knows who is doing what.
Correct Example: First describe action, then dialogue: Reporter in red widens eyes, waves hand to follow up. [Reporter in red, anxiously] "What's really hidden behind this?"
Incorrect Example (Model Likely to Fail): [Reporter in red]: "What's really hidden behind this?" (Without specifying actions, dialogue might be randomly assigned)
P3. Clear Audio Details
Core Principle: Give each character unique voice characteristics and emotion labels.
Correct Example: [Candidate, calm and deep, slightly slower pace] says: "I will explain in detail." [Reporter in red, rapid and clear] asks: "Then why the delayed response?"
Incorrect Example (Model Likely to Fail): [Candidate] says…[Reporter] says… (Without clear voice distinction, model may blend them together)
P4. Control Time Sequence
Core Principle: Use "immediately after," "then," "at this point the other responds" to control rhythm.
Correct Example: Candidate frowns, [Candidate] says, "I haven't agreed yet." Immediately after, [Reporter in red] says, "Then when do you plan to give an answer?"
Incorrect Example (Model Likely to Fail): [Candidate]: "I haven't agreed yet." [Reporter in red]: "Then when do you plan to give an answer?" (Model might have one person say both lines at once)
2. Cinematic Fight Scene Prompt Template
Those impactful, rhythmic fights in movies can also be recreated by AI models. The following "cinematic fight scene" template teaches you how to quickly write high-energy fight prompts.
Prompt Template:
[Scene]
The battle takes place in [location/environment, such as abandoned warehouse, rainy alley, sci-fi ship corridor], overall lighting is [bright/dim/strong backlight], creating a [tense/realistic/cinematic] atmosphere.
[Subject]
The frame shows [Fighter A: appearance + clothing + demeanor] and [Fighter B: appearance + clothing + demeanor], positioned [left-right/front-back] facing each other, starting with [poses like clenched fists, raised defensive stance, lowered center of gravity], giving a [calm and experienced/impulsive and ruthless] first impression.
[Motion]
● Combat process:
Both sides use [e.g., extremely precise rhythm, skilled combat training] for defense—through [blocking with both arms, gently pushing to deflect, nimble wrist rotation] to dismantle opponent's offense, but the [left/right/one side] warrior's movements are clearly more experienced, with clean and efficient moves like a true master.
● Battle escalation:
As the attack defense rhythm accelerates, the camera begins [circling, rising, moving close to limbs] movement, [the advantageous warrior] blocks each attack with [incredible speed, almost effortlessly], punches and kicks passing by just [millimeters] away, followed by [short, sharp] counterattacks precisely hitting the opponent's vital points.
● Environmental interaction:
[Elements like dust, gravel, rainwater] are constantly stirred up at their feet, camera [low flying close to ground, suddenly rising] passing over these details, freezing the frame at the moment of [renewed clash/critical strike], with [the dominant warrior] always maintaining rhythm control.
[Audio]
Each heavy hit brings a "thud" sound, the whooshing of punches and kicks cutting through air, friction of shoes against ground, rapid breathing, plus environmental background noise (like warehouse echoes/raindrops hitting ground/distant machine humming), together creating a textured combat audio experience.
[Camera]
Camera uses [low-angle wide/overhead/close up] positions, filming close to the [number] fighters, and always moving around the [location/environment] space, making viewers feel as if they're at the battle scene.
Prompt:
Dim abandoned warehouse, ceiling lights weakly flickering, overall atmosphere tense and oppressive. Two fighters stand facing each other: the warrior on the left with wild movements, strong desire to attack; the warrior on the right with stable form, calm and experienced demeanor. The two fighters suddenly rush toward each other. The first clash brings intense impact, making the frame shake. The camera quickly follows their hand movements, capturing a series of rapid punches and kicks. Both perform extremely precise blocking with both arms, palm deflections, wrist rotations, but the warrior on the right moves like a master. The camera circles them at high speed, the right warrior effortlessly neutralizing each attack, cleanly pushing strikes away. The left warrior unleashes a series of wild attacks. The camera suddenly rises, weaving between their arms, the right warrior intercepting each strike at incredible speed, attacks missing by mere millimeters, followed by short, sharp counterattacks hitting precisely. The audio emphasizes each heavy punch hitting body or wall with dull "thud" "boom" sounds, occasionally interspersed with slight whooshing of punches through air and subtle ground echoes, overall sound feeling heavy and impactful. Camera uses low-angle wide shot.
(C) Common Audio Trigger Words
To help the model generate the audio content you want more accurately, you can choose appropriate descriptive words for different types to precisely trigger corresponding sound effects, music, or voice styles.
Audio Type: Voice
● Expression Method: says, asks, tells me, explains, sighs, recites, monologue, narration, whispers
● Emotion/Attitude: says quietly, says softly, says excitedly, says seriously, says tenderly, says formally, complains, says hesitantly, says calmly, says sarcastically, says encouragingly
● Voice Quality: hoarse, clear, trembling, sweet, deep, very fast pace, very slow pace, intermittent
Audio Type: Dialogue
● Interaction Form: asks, answers, continues, responds, argues, discusses, negotiates, comforts, persuades
● Action Sounds: shouts, complains, teases, jokes, mutters, exclaims, cries/sobs, screams, laughs/giggles, sighs
Audio Type: Singing
● Technique/Style: a cappella, sings softly, hums, sings out loud, bel canto, pop vocal style, vibrato, falsetto, harmony
● Emotion/State: sings emotionally, sings tenderly, sings melancholically, sings happily, sings off-key
Audio Type: Rap
Professional Terms: rap, hip-hop, rhythmic, rhyming, flow, fast, slow, strong rhythm, improvisation, heavy bass, fast-mouthed
Audio Type: Sound Effects
● Life Actions: opening lid, pouring water, turning pages, knocking, dropping, tearing, picking up, putting down, clicking, chewing, swallowing, footsteps, rapid running sounds, door opening/closing sounds
● Material Collision: ding, pop, click, thud, bang, crisp sound, friction sound, scratching sound, glass breaking, metal collision
● Natural Sounds: splash (water), whoosh (wind), crackle (fire), gurgle (bubbling), thunder, raindrops, storm, snow crunching
● Mechanical Sounds: rumbling, beep, buzz, click, startup sound, alarm sound, braking sound, mechanical operation/gear sound
● Instrument Sounds: piano sound, guitar strumming sound, violin, drum beat, bass
Audio Type: Environmental Sound
● City: traffic, crowd murmurs, subway, mall announcements, street wind sounds, construction sounds, airplane passing sound, honking, market bustle
● Nature: waves, wind sounds, bird calls, insect sounds, stream, waterfall sound, animal roars, night insect sounds, rainforest
● Indoor: air conditioning sound, keyboard sound, paper friction, slight reverb, bar/cafe background music, quiet atmosphere of hospital corridor, library silence, fireplace burning sound
III. FAQ
Q1: What languages does SeaArt UltraVision currently support for voice output?
SeaArt UltraVision currently supports English and Chinese voice output. You can input text in any language, and the system will automatically recognize English or Chinese content and generate corresponding voice dialogue. Other languages are not currently supported for voice generation.
Q2: Why do my prompts seem inadequate, with visuals and sound often mismatched?
The key is "layered description." We recommend prompting by separate points [Scene/Subject/Action/Audio/Camera/Style], rather than piling all elements into one sentence. For example, when describing sound effects, use "glass window gently closing，ding" rather than vaguely saying "door closing with sound." Clear prompts help the model understand faster.
Q3: I want to include more elements in my creation, but it ends up messier?
"Less is more" works better for AI. Focus on 1 or 2 core elements per creation (like a dialogue segment or a significant sound effect), describe them in detail, then gradually add audio-visual layers through iterations. Complex scenes can be broken into multiple short clips generated separately, then combined.
Q4: Audio triggers keep failing, resulting in poor effects?
Please refer to our compiled "audio trigger words" table (such as "says," "says excitedly," "piano sound," "rumbling," etc.) and explicitly include trigger words in your audio sections. For example, to express an angry dialogue, write "[Male voice, says angrily]..." Such cues significantly improve audio recognition rates.
Q5: What if I don't know how to write prompts?
First clarify your desired creative direction (scene, atmosphere, audio type), then use the creative assistant or reference example prompts to generate a draft. After obtaining structured content, you can fine-tune details to quickly produce usable prompts.
