Other providers & encoded audio

Prefer a dedicated guide when you can

For any other vendor, convert to mono pcm_s16le (or pass encoded bytes / a CORS-open URL) and use the helpers below.

Generic turn helpers

Map your vendor’s turn-start / audio-chunk / turn-end / barge-in callbacks onto these functions.

generic-turn-map.ts

type TurnState = {
  utterance: ReturnType<typeof experience.speech.createUtterance>;
  writes: Promise<void>;
};

const turns = new Map<string, TurnState>();

function beginTurn(turnId, transcript) {
  const utterance = experience.speech.createUtterance({
    format: { encoding: 'pcm_s16le', sampleRate: 24_000, channels: 1 },
    queue: 'replace-active',
    // Optional seed — pass spoken text when available (helps lipsync)
    ...(transcript ? { transcript } : {})
  });
  turns.set(turnId, { utterance, writes: Promise.resolve() });
}

function writeTurn(turnId, chunk) {
  const turn = turns.get(turnId);
  if (!turn) return;
  const u = turn.utterance;
  turn.writes = turn.writes
    .then(() => u.write(chunk))
    .catch((error) => {
      console.error('Unable to write speech audio', error);
      void u.cancel();
    });
}

async function endTurn(turnId, transcript) {
  const turn = turns.get(turnId);
  if (!turn) return;
  turns.delete(turnId);
  await turn.writes;
  await turn.utterance.close({ transcript, history: 'none' });
}

async function cancelTurn(turnId) {
  const turn = turns.get(turnId);
  turns.delete(turnId);
  if (turn) await turn.utterance.cancel();
}

One-shot PCM

await experience.speech.play({
  audio: {
    data: pcmS16leBytes,
    format: { encoding: 'pcm_s16le', sampleRate: 24_000, channels: 1 }
  },
  // Optional but recommended when you have the spoken text — usually improves lipsync.
  transcript: agentReplyText,
  queue: 'append' // append | replace-active | replace-all
});

MediaStreamTrack

Capture runs in the host page; the player only receives PCM.

// Live track — captured to PCM in the host SDK, then streamed into the player
await experience.speech.play({
  audio: { track: audioMediaStreamTrack, sampleRate: 24_000 },
  queue: 'replace-active'
});

// Or keep the utterance open while the track runs
const utterance = experience.speech.createUtterance({
  track: audioMediaStreamTrack,
  sampleRate: 24_000,
  queue: 'replace-active'
});
// utterance closes automatically when the track ends

MP3 / WAV files

Pass encoded bytes or a URL. The player decodes with Web Audio, then runs the same STA lipsync path as PCM. Api servers never download your media URL.

// Encoded bytes — decoded in the player (not on api.liforma.ai)
await experience.speech.play({
  audio: { data: mp3Bytes, encoding: 'audio/mpeg' },
  queue: 'append'
});

// Or a CORS-open URL fetched + decoded inside the player iframe
await experience.speech.play({
  audio: { url: 'https://cdn.example.com/line.mp3' },
  queue: 'append'
});

Accepted PCM rates after decode: 8 / 16 / 22.05 / 24 / 44.1 / 48 kHz.
URL fetch uses CORS from player.liforma.ai — configure your CDN accordingly.
Encoded payload size is capped (12 MB) to protect the embed.

Interrupt

await experience.speech.interrupt({ scope: 'active' });
// or { scope: 'all' } | { utteranceId } | { characterId }