SentiPulse Open-Sources SentiAvatar 3D Digital Human Framework

SentiPulse, in collaboration with researchers from the Gaoling School of Artificial Intelligence (GSAI) at Renmin University, released SentiAvatar as an open-source framework for interactive 3D digital humans. The release bundles the SUSU character model, the SuSuInterActs multimodal motion dataset, a pre-trained Motion Foundation Model, and a streaming architecture designed to align speech, gesture, and facial expression for live conversation. The dataset contains 21,000 clips and 37 hours of annotated, synchronized recordings; pretraining data for the motion model is reported as 200K+ sequences in one source and 676 hours in another. The project is available on GitHub for developers and researchers to use and extend.

The release tackles three core technical gaps in digital human research: scarcity of high-quality conversational motion data, semantic composition of actions, and temporal synchronization between speech and motion. SuSuInterActs is centered on a single persona, SUSU (age 22), annotated for behavioral text, full-body motion, facial expression, and aligned audio. The authors pre-trained a Motion Foundation Model on a much larger motion corpus to bootstrap generative performance beyond the 37-hour annotation set.

• •The distribution includes the full framework, the SUSU character model, the SuSuInterActs dataset, and a streaming inference stack.
• •Real-time performance is emphasized, with a reported latency of about 0.3 seconds to generate a six-second motion sequence for live interaction.
• •Multimodal alignment models map speech prosody and semantic cues to composite gestures and facial micro-expressions to avoid the uncanny valley.

Delivering naturalistic nonverbal behavior remains the primary bottleneck for believable digital humans despite advances in rendering. SentiAvatar's combination of a curated persona dataset, a foundation motion model, and an engineering-ready streaming pipeline is notable because it moves beyond isolated motion synthesis toward coherent conversational agents where gesture, facial expression, and speech are produced jointly. Open-sourcing these assets lowers the entry cost for research labs, game studios, and product teams building virtual assistants, social VR avatars, or customer-facing agents. The emphasis on a single, consistent character with rich annotations is also a pragmatic design choice; it reduces variability during training and yields more stable behavior than multi-character datasets with inconsistent labeling.

Modelers can reuse the Motion Foundation Model weights to fine-tune on domain-specific personas or languages. Animators and engineers can integrate the streaming stack into real-time engines to evaluate latency and perceptual quality. Researchers gain access to a high-quality Chinese conversational motion corpus, addressing a geographic dataset gap that has slowed non-English progress in this area.

Adoption will hinge on community validation: reproduced quality metrics, perceptual user studies comparing SentiAvatar-driven agents to existing baselines, and how easily the streaming stack integrates with common engines and runtime constraints. Also watch for forks that expand beyond the SUSU persona into multi-persona or multi-language datasets, and for hybrid pipelines that combine SentiAvatar motion outputs with lip-sync and photorealistic rendering layers.

SentiAvatar is a pragmatic, open-source step toward escaping the uncanny valley by aligning multimodal signals in real time. For teams focused on conversational agents and realistic avatars, this release provides ready-to-use data, model weights, and an inference pipeline, enabling faster iteration and more consistent behavior across live interactive scenarios.