ONNX Extension

info

This feature is in Beta, contact us to apply for our early access testing program!

About ONNX

ONNX (Open Neural Network Exchange) is an open standard for representing machine learning models. It serves as a bridge between different deep learning frameworks, allowing models to be exchanged and used across platforms. ONNX is useful for various audio processing use-cases, including speech recognition, audio classification, music analysis, sound synthesis, and more. By converting models into the ONNX format, developers can leverage the interoperability it offers, enabling seamless integration and deployment of audio processing models across different frameworks, tools, and hardware platforms, thereby fostering collaboration and facilitating the adoption of deep learning models in the audio domain.

Example

ONNX.MLProcessor

Switchboard Editor Preview

Switchboard JSON

{
  "title": "ONNX MLProcessorNode example",
  "nodes": [
    {
      "id": "sineNode",
      "type": "Switchboard.SineGenerator"
    },
    {
      "id": "onnxNode",
      "type": "ONNX.MLProcessor",
      "assets": {
        "model": "https://switchboard-sdk-public.s3.amazonaws.com/assets/models/onnx/linear_layer_volume_adjustment_gain_0.5_weights_32x32.with_runtime_opt.ort"
      }
    }
  ]
}

Although in it's early stges of development and not publically available yet, here we show a demonstration of an ONNX.MLProcessor node which is using ONNX Runtime to run realtime audio inference on an audio signal. In this particular example, the model is simply trained to half the gain of it's input signal.

Running ONNX models in Switchboard

Switchboard SDK seamlessly integrates with your existing machine learning infrastructure, making it effortless to incorporate your models into a production environment. It provides a smooth transition from the development and research phase to real-world deployment. It enables real-time inference, allowing your machine learning models to process audio data instantaneously.

Switchboard SDK provides several features that make using machine learning models for audio processing easier. Some of these features are:

• Model Integration: seamless integration with ONNX, allowing developers to easily load and use pre-trained models for audio processing tasks.
• Preprocessing Functions: built-in preprocessing functions specifically designed for audio data. These functions include spectrogram computation, MFCC extraction, audio normalization, or resampling, which simplify the data preparation stage before feeding it into the models.
• Inference and Prediction: efficient and optimized inference capabilities, allowing developers to run audio data through the machine learning models and obtain predictions or feature representations in real-time.
• Visualization and Debugging Tools: visualization tools to facilitate the inspection of intermediate model outputs or audio features, aiding in model debugging and understanding the internal workings of the audio graph.
• Integration with Audio I/O: interfaces to handle audio input/output, allowing seamless integration with audio devices, file formats, or streaming services for real-time audio processing.

Node types

The ONNX Extension provides the following audio nodes for a Switchboard SDK audio graph:

Node	Description
ONNX.MLSource	A source node that runs the ONNX model and generates audio data with the specified post-processing. Ideal for use with generative audio use cases.
ONNX.MLSink	A sink node that runs the ONNX model and receives audio data with the specified pre-processing. Ideal to be used for classifier type models.
ONNX.MLProcessor	A processor node that runs the ONNX model and receives audio data with the specified pre-processing, then generates audio data with the specified post-processing. Ideal for applications in audio transformation, such as Noise Reduction, Source Separation, and Voice Conversion.

Download SDK Extension

You can download this SDK extension from our Downloads Page.

Visit the page to access the latest version and start integrating it into your project!