Speech-to-Text API Integration 101: A Developer's Guide

Adding speech-to-text to a product looks like one API call. In practice, the call is the easy part. The work is in preparing audio, picking a provider that handles your recordings well, building a pipeline that survives hour-long files and flaky networks, and knowing when accuracy slips.

This guide walks through a production-ready integration from start to finish. It's provider-neutral: the patterns apply whether you use a large cloud vendor, a specialist speech company or a model you host yourself.

How speech-to-text APIs work

You send audio, the provider runs an automatic speech recognition (ASR) model, and you get back structured text. A typical response includes:

• Transcript text, often with punctuation and casing restored
• Word or segment timestamps, so you can sync text to the recording
• Confidence scores per word or segment
• Speaker labels if you enable diarization ("who spoke when")
• Detected language, if you let the API identify it

Some providers add extras on top: summaries, topic detection, PII redaction, custom vocabulary and profanity filtering. Treat these as nice-to-haves. The core decision is how well the base transcription handles your audio.

Step 1: Choose batch or streaming

There are two very different integration modes.

If you're not sure, start with batch. Most "real-time" requirements turn out to be "within a minute or two of the recording ending," which batch handles fine.

Step 2: Prepare the audio

Bad audio in means bad text out, and some preparation steps make a measurable difference.

• Use a supported format. WAV and FLAC are lossless and universally accepted. MP3, M4A/AAC, Ogg/Opus and WebM are widely supported. Check your provider's list.
• Downmix to mono unless channels carry different speakers. For a phone call recorded with the agent on the left channel and the customer on the right, send stereo and use per-channel transcription instead of diarization. It's far more accurate.
• Resample if recommended. Many ASR models work at 16 kHz internally. Sending 16 kHz mono shrinks uploads without losing recognition quality. Don't upsample low-quality audio, though: it adds size, not information.
• Don't over-process. Aggressive noise reduction can remove parts of speech and hurt accuracy. Test with and without it.
• Avoid repeated re-encoding. Each lossy encode adds artifacts. Convert once from the best source you have.
• Check limits for long files. Providers cap file size and duration. For very long recordings, upload via URL or split into pieces at silences (not at fixed times, which can cut words), then stitch the results back together with adjusted timestamps.

FFmpeg handles nearly all of this in one command:

That strips video (-vn), downmixes to mono (-ac 1), resamples to 16 kHz (-ar 16000) and writes lossless FLAC.

Step 3: Pick a provider with your own audio

Vendor benchmarks are measured on vendor-chosen datasets. Your audio (accents, jargon, background noise, phone-quality lines) is what matters. Shortlist two or three providers and run the same test set through each.

Here's what to compare:

Also look at boring operational details: SDK quality, webhook support, status page history and how the API reports errors.

Step 4: Use the async job and webhook pattern

For batch transcription, never hold an HTTP request open while a long file is processed. The standard pattern is:

1. Your server uploads the audio (or gives the provider a URL to fetch).
2. The provider returns a job ID immediately.
3. You store the job ID with your own record, marked as pending.
4. When processing finishes, the provider calls your webhook. If webhooks aren't available, you poll with backoff.
5. Your webhook handler verifies the request, fetches or reads the result, and saves it.

Here's a sketch in Node.js with Express. Endpoint names and fields differ by provider, so treat this as the shape, not a copy-paste client.

import express from "express"
import crypto from "node:crypto"

const app = express()

// 1. Submit a job
app.post("/transcribe", express.json(), async (req, res) => {
  const { recordingId, audioUrl } = req.body
  const job = await stt.createJob({
    audio_url: audioUrl,
    diarization: true,
    webhook_url: "https://api.example.com/stt-webhook",
    idempotency_key: recordingId, // safe to retry
  })
  await db.transcripts.upsert({ recordingId, jobId: job.id, status: "pending" })
  res.status(202).json({ jobId: job.id })
})

// 2. Receive the result
app.post("/stt-webhook", express.raw({ type: "application/json" }), async (req, res) => {
  const expected = crypto.createHmac("sha256", process.env.STT_WEBHOOK_SECRET)
    .update(req.body)
    .digest("hex")
  const received = req.get("x-signature") || ""
  if (received.length !== expected.length ||
      !crypto.timingSafeEqual(Buffer.from(received), Buffer.from(expected))) {
    return res.sendStatus(401)
  }

  const event = JSON.parse(req.body.toString())
  res.sendStatus(200) // acknowledge fast, process after

  const record = await db.transcripts.findByJobId(event.job_id)
  if (!record || record.status === "done") return // duplicate delivery

  const result = await stt.getJob(event.job_id)
  await db.transcripts.update(record.id, {
    status: result.status === "completed" ? "done" : "failed",
    segments: result.segments,
  })
})

The important details:

• Verify the webhook signature using the raw request body, so attackers can't post fake results.
• Acknowledge quickly. Return 200 first and do the heavy work in a background job, or the provider may time out and retry.
• Handle duplicates. Webhooks are usually delivered at least once. Make your handler idempotent by checking the job's stored status.
• Have a fallback. A scheduled task should find jobs stuck in pending for too long and poll them directly. Webhooks do get lost.

Step 5: Handle errors and retries

Speech pipelines fail in predictable ways. Plan for each:

A queue (such as a database-backed job table, SQS or a Redis queue) between your app and the provider makes most of this easier. It smooths spikes, enforces concurrency limits and gives you a place to see what's stuck.

Step 6: Measure accuracy with WER

Word error rate is the standard metric:

To measure it properly:

1. Pick a fixed set of 20 to 50 recordings that represent your real traffic: different speakers, accents, audio quality and topics.
2. Get human-checked reference transcripts for them.
3. Normalize both sides before comparing: lowercase, strip punctuation, and standardize numbers ("twenty five" vs "25") so you measure recognition, not formatting.
4. Compute WER with a library like jiwer in Python.

import jiwer

transform = jiwer.Compose([
    jiwer.ToLowerCase(),
    jiwer.RemovePunctuation(),
    jiwer.RemoveMultipleSpaces(),
    jiwer.Strip(),
])

error = jiwer.wer(reference, hypothesis,
                  reference_transform=transform,
                  hypothesis_transform=transform)
print(round(error * 100, 1), "% WER")

Lower is better. What counts as good depends heavily on the audio: clean, single-speaker narration scores far better than noisy phone calls with crosstalk. Compare providers on the same files, and rerun the set whenever you change provider, model version or audio preprocessing.

WER treats every word equally, but users don't. Getting a customer's name or a product term wrong hurts more than missing an "um." Keep a short list of critical terms and check them separately. Custom vocabulary features exist exactly for this.

Step 7: Control costs

Speech-to-text is usually billed per minute of audio, which makes costs easy to predict and easy to waste.

• Don't transcribe the same audio twice. Hash files or store a content ID and cache results.
• Trim silence at the start and end, and skip recordings below a minimum length.
• Check rounding. Some providers round each file up to a minimum billable duration, so many tiny files can cost more than you'd expect.
• Turn on add-ons only where needed. Diarization, summaries or redaction may cost extra. Enable them per use case, not globally.
• Use existing text when it exists. If a video already has accurate captions, fetching those is faster and cheaper than transcribing the audio.
• Set budgets and alerts on usage, especially for user-uploaded content.

Step 8: Protect privacy

Audio often contains the most sensitive data a company holds: names, health details, payment information and things people say in confidence. Before you ship:

• Read the data terms. How long is audio and text retained? Is it used to train models, and can you opt out? Which region is it processed in?
• Get the paperwork. Enterprise customers will ask for a data processing agreement. Health data in the US needs a provider that will sign a BAA.
• Redact PII in transcripts before storing or sending them on to other services, if your use case allows.
• Delete what you don't need. Remove audio from the provider after processing if their API supports it, and set your own retention policy.
• Get consent. Recording and transcribing people has legal requirements that vary by country and state. Make sure your product collects consent where needed.

Where to go from here

Once transcripts are flowing, the interesting work starts. You can chunk them for LLMs and RAG, turn them into subtitle files, or use them to make video accessible under WCAG.

If your audio lives in online videos, our API returns existing captions when a video has them and falls back to AI transcription when it doesn't, through the same endpoint, so you don't pay to transcribe audio that already has text.