How to Add Real-Time Noise Cancellation to a Voice Agent (Python Quickstart)

Arctan Eigen — TL;DR: Add Eigen to your agent’s incoming audio path before transcription and turn detection. Keep the original stream available for comparison, and measure transcription quality and response latency on representative calls before enabling processing for everyone.

In this article, noise cancellation means processing microphone or call audio to reduce unwanted sound. Voice isolation also tries to reduce competing speech so the intended caller remains prominent. Neither operation repairs clipping, packet loss, or an incorrect speech-to-text (STT) configuration.

Where should the processor sit in a Python voice agent?

The usual path is incoming audio → optional enhancement → STT and turn detection → agent response. If your agent uses LiveKit, its audio input options are one documented place to apply a processor. The Eigen product page shows a Python LiveKit integration and lists Python among the available SDK options. Check the current SDK documentation for package, key, and import names, because the exact setup depends on the versions you install.

Decide whether processing belongs in the calling client, media transport, or agent process. Choose the point where you can observe the audio that reaches STT and control whether the processor is enabled. Avoid running two enhanced cancellation models on the same stream without testing: LiveKit warns that feeding already processed audio into another model can produce unexpected results.

What should you check before connecting the SDK?

1. Identify the input format. Record the codec, sample rate, channel count, and frame duration entering the agent. Do not label transcoded audio with its former format. Google’s STT guidance emphasizes matching recognition settings to the actual audio.

2. Choose the audio problem. Constant fan or traffic noise calls for background suppression; another person talking near the caller may call for voice isolation. If both sides of a call need to remain intelligible, isolate only the intended input track.

3. Keep a bypass path. Your agent should be able to send the original audio to STT when processing fails or degrades speech. This also makes a fair A/B test possible.

4. Check the current integration contract. Confirm required credentials, supported operating systems, frame sizes, and sample rates in the installed Eigen SDK. Keep credentials in an environment variable or secret store.

What does the LiveKit hook look like in Python?

If your LiveKit agent already defines session, Assistant, and ctx, Eigen can be placed in AudioInputOptions, as shown in the published example. This excerpt adds the standard imports and reads the key from the environment. Check the installed Eigen and LiveKit versions before running it; package and parameter names can change.

import os
from arctan import arctan_enhancer
from livekit.agents import room_io

await session.start(
    agent=Assistant(),
    room=ctx.room,
    room_options=room_io.RoomOptions(
        audio_input=room_io.AudioInputOptions(
            noise_cancellation=arctan_enhancer.noise_canceller(
                key=os.environ["ARCTAN_KEY"]

Set the key in the agent’s secret environment, not in source code. Follow the current installation and key instructions, then process a short test call and verify that the agent still receives the intended speaker’s words. This example demonstrates the insertion point; it is not a complete agent application.

How do you know whether it helped?

Build a small evaluation set from consented calls that represents your real noise conditions, languages, microphones, and codecs. Transcribe each clip twice with the same STT model and settings: once from the original audio and once from processed audio. Compare word error rate against a checked transcript, plus accuracy on the fields that matter to the application, such as names, amounts, dates, or order codes. Also listen for missing consonants and dropped quiet speech.

Measure processor time, buffering delay, and end-to-end time to the agent’s first response separately. A processing-latency figure is not the full delay of a voice agent; LiveKit’s turn-taking guidance describes other stages that affect response timing. If the processed path performs worse, keep the bypass and investigate rather than assuming more suppression will help.

For a deeper test design, see why noisy audio can affect STT accuracy. For the difference between suppressing noise and suppressing other speakers, see voice isolation versus noise cancellation.

Written by the Arctan team.