ceetuber
Star
Fork
Watch
ceetuber docs AVATARS.md
9.0 kB · 127 lines
Markdown
at main

Build an avatar module #

Ceetuber separates input from character behavior. Every avatar receives the same seven pose values; it owns the geometry, rig mapping, secondary motion, and behavior settings. The default rectangle is a small procedural example. Cee is an optional, lazily loaded GLB example.

Load a local model #

Use Load GLB… beside the avatar picker. Choose a self-contained glTF 2.0 binary (.glb, up to 100 MB) with embedded textures and buffers. The file is read in the browser, never uploaded. It remains in this tab's memory; reload requires selecting it again. Imports and their rig mappings are not persisted or shared with a separate OBS browser source. To make an imported model available in OBS, register it as an avatar module as described below.

The loader retains materials, skinning and morph targets, scales the model to fit the preview, and centers it above the floor. It does not play authored animation clips. Draco/meshopt geometry compression and KTX2 textures need decoders that this loader does not include; export without them. Blender files and VRM extensions are not supported by this importer.

Open Rig mapping to choose the head, left eye, and right eye nodes. Bone and object nodes both work. It automatically matches these names, ignoring case, spaces and punctuation:

Role Recognized names
Head Head, MixamoRigHead, CeeHead
Left eye LeftEye, EyeLeft, Eye_L, MixamoRigLeftEye
Right eye RightEye, EyeRight, Eye_R, MixamoRigRightEye

Use anatomical left and right. Model coordinates are Y-up, facing +Z; the Face the other way toggle rotates the whole model 180 degrees. Head motion is applied relative to the node's rest quaternion, around its local X/Y/Z axes. Gaze rotates eye nodes around local X/Y. An unusual bone axis orientation may need correction in the authoring tool. Names alone do not guarantee compatible joint axes. Without a head mapping, the entire model rotates about its center; unmapped eyes remain still.

Blink shape keys are recognized as eyeBlinkLeft / eyeBlinkRight, blinkLeft / blinkRight, blinkL / blinkR, or a combined blink / eyeBlink. Case and punctuation are ignored. With only a combined blink, both eyes close together. When no corresponding blink shape key exists, a mapped eye mesh/object compresses vertically; eye bones are never scaled to fake a blink.

These names are a Ceetuber convention, not a glTF skeleton standard. glTF describes meshes, node hierarchies, skins, materials, and morph targets without assigning VTuber roles to bones. VRM adds humanoid bone mappings, expressions, gaze and other avatar semantics. A future VRM adapter could use those directly; simply parsing a VRM file as GLB would miss its material and behavior rules.

Cee's cloth, ornaments, hat, and surface-sliding eyes remain custom behavior. A generic GLB import cannot infer that simulation from the mesh. Use the module interface below to add character-specific behavior.

Files to start with #

  • rectangle.mjs: no assets, head rotation, bounded sliding eyes, independent blinks, follow delay, and squash.
  • ghost.mjs: asset loading, a specialized rig, materials and reflections, with the same interface.
  • avatar-definitions.mjs: picker metadata, behavior fields, defaults, validation.
  • avatars/index.mjs: trusted local module loaders.
  • avatar-host.mjs: selection, asynchronous replacement, update and disposal.

The contract #

Export an async createAvatar(context) function:

import * as T from '../vendor/three.mjs';
import { disposeObject } from './dispose.mjs';

export async function createAvatar({ renderer, scene, studio }) {
  const object = new T.Group();
  // Create meshes or load your GLB, then add them to object.
  // Return a detached root: the host adds/removes it from the scene.
  return {
    object,
    update(pose, dt, settings) {
      // dt is seconds. settings contains only this avatar's declared fields.
      // Apply head angles, eye direction, blinks and your own behavior here.
    },
    reset() {
      // Clear velocities and return to the neutral pose.
    },
    dispose() {
      disposeObject(object);
      // Also release your render targets, listeners or other owned resources.
    },
    // Optional camera hints, fractions relative to neutral bounding-box height:
    framing: { faceHeight: 0.65, faceSpan: 1.8, fullScale: 1 },
    // Optional work after drawing (e.g. reflection probes):
    render(timeMilliseconds, detailedReflections) {},
    features: [], // Include 'reflections' to reveal the existing reflection toggle.
  };
}

Use Y-up coordinates, face toward +Z, and place the neutral base near Y=0 so the shared camera and floor behave predictably. The host measures neutral bounds to frame the avatar.

The context's renderer, scene, and studio belong to the app. Do not dispose them or their shared environment maps. The helper disposes model geometry, materials, and owned texture maps once each; it excludes envMap because the studio is shared. Dispose your own reflection targets separately.

The host keeps the previous avatar visible while another loads, disposes superseded results, and prevents a slow old load from replacing a newer choice. A load failure leaves the previous avatar alive. Modules must clean up partially created resources if their own factory throws.

Register it #

  1. Add your module, for example dist/avatars/my-avatar.mjs, and its assets.
  2. Add an entry to AVATARS in avatar-definitions.mjs, with a unique id, name, description, and settings array.
  3. Add its lazy loader to avatars/index.mjs:
'my-avatar': () => import('./my-avatar.mjs'),

You do not need to edit the renderer, tracking sender, relay, or settings-page HTML. The picker and behavior controls come from the manifest. Modules are trusted application code; this is not a sandbox for uploading arbitrary JavaScript.

Example behavior field:

{
  key: 'bounce',
  label: 'Bounce',
  min: 0,
  max: 2,
  step: 0.05,
  default: 0.4,
  unit: '×',
  help: 'How strongly head motion affects the body.',
}

Use type: 'boolean' and a boolean default for a checkbox. Numeric fields get a slider and exact number entry. Values are finite-checked, clamped, and filtered against the manifest in both Node and the browser. Keep ids and field keys stable if you want saved presets to survive updates.

Pose input #

Head yaw/pitch/roll are degrees; gazeX/gazeY are centered direction values; blinkL/blinkR range from 0 to 1. Head/eye multipliers can expand input beyond the initial manual ranges. Decide the appropriate bounds for your character. dt is elapsed seconds capped by the host render loop; use time-based smoothing or fixed simulation steps.

Use anatomical left/right consistently. A conventional eyeball can rotate, a drawn-on eye can translate, and a 2D face can change sprites. No particular skeleton or material names are imposed by the template. The ghost adapter retains its own specialized rig contract.

Saving and OBS #

Tracking settings (gains, smoothing, direction, calibration) remain separate from avatar behavior.

  • Save avatar saves the selected id and all current avatar behavior presets.
  • Revert behavior restores the visible avatar's saved behavior; it does not change the selection.
  • Default behavior previews that avatar's factory defaults.
  • Local persistence: data/avatar-settings.json, ignored by Git.
  • Hosted demo: ceetuber.avatars in that browser's localStorage.
  • First local migration: existing saved ghost hat/pom values seed its new preset. Tracking values are preserved.

Local pages receive avatar state through the shared SSE stream. An OBS URL without an avatar query follows the current local selection. Add ?output=1&avatar=rectangle or ?output=1&avatar=ghost to pin a source. The output link in the tester pins the visible avatar. A URL's avatar parameter selects/pins its initial page view; choosing an avatar manually resumes following shared selection. Invalid ids fall back to the default.

A shared link such as https://ceetuber.cee.wtf/?avatar=ghost opens that example without making it the default for new visitors. The default rectangle needs no Blender file, model download, or rig.

Check a module #

Verify neutral appearance, each head axis, extreme gains, independent blinks, reset, rapid switching, save/reload, and disposal. The existing avatar tests cover the rectangle and stale-load protection. Add meaningful behavior checks for your own module. Run all suites with node scripts/test.mjs.