Skip to content

[Feature][Relax][ONNX] Support STFT (Short-Time Fourier Transform) operator in Relax ONNX frontend #20966

Open

Description

@Chaitanya140904

Feature Description

Support the ONNX STFT (Short-Time Fourier Transform) operator in the Apache TVM Relax ONNX frontend (python/tvm/relax/frontend/onnx/onnx_frontend.py).

Motivation

In ONNX opset 17, DFT and STFT were introduced as the standard Fourier transform operators for audio and signal processing pipelines. While DFT support is being added, STFT is currently missing in the Relax ONNX frontend.

STFT is widely used in speech recognition, audio processing, and generative audio pipelines (such as Whisper, EnCodec, AudioLDM), which segment raw waveforms into overlapping windowed frames and compute discrete Fourier transforms. Importing ONNX models containing STFT fails with:

NotImplementedError: The following operators are not supported: ['STFT']

This operator was also highlighted in the ONNX operator coverage RFC tracking issue #20124 under Signal Processing operators.

Operator Specification (ONNX Opset 17+)

  • Inputs:
    1. signal: Input tensor of shape (batch, signal_length) or (batch, signal_length, 1) or (batch, signal_length, 2) (for complex audio signal).
    2. frame_step: Scalar integer representing the hop size between consecutive frames.
    3. window (optional): 1-D window tensor (e.g. Hann, Hamming) of length frame_length.
    4. frame_length (optional): Scalar integer representing the length of each frame (defaults to window length if window is provided).
  • Attributes:
    • onesided: int (default 1) - whether to return half spectrum for real signals.
  • Outputs:
    1. output: 4-D tensor of shape (batch, frames, dft_unique_bins, 2) representing real and imaginary frequency components.

Expected Behavior

The Relax ONNX frontend should implement STFT by framing the input signal (sliding window with frame_step and frame_length), applying the window function if provided, and computing discrete Fourier transforms across frames.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions