RTNeural Extension

info

This feature is in Beta, contact us to apply for our early access testing program! Please try our Switchboard Editor to quickly try and share Switchboard projects that use RTNeural.

About RTNeural

RTNeural is a light-weight ML inferencing library designed with the intention of being used in real-time systems, specifically real-time audio processing. You can find out more about the project on the RTNeural GitHub page.

Example

Below is at example where we define a Switchboard JSON graph to play a guitar sample with a WebAudio.AudioPlayer node, then pass the output into a RTNeural.MLProcessor node. In this particular example we're using an open-source guitar distortion model from the GuitarLSTM project, converted into RTNeural JSON modal format.

You can also use the Upload button in the UI to upload an RTNeural JSON model from your computer.

Switchboard Editor Preview

Switchboard JSON

{
  "title": "RTNeural distortion",
  "nodes": [
    {
      "id": "BufferSource1",
      "type": "WebAudio.AudioPlayer"
    },
    {
      "type": "RTNeural.MLProcessor",
      "assets": {
        "model": "https://switchboard-sdk-public.s3.amazonaws.com/assets/models/distortion.json"
      }
    }
  ],
  "tracks": [
    {
      "label": "EMH",
      "BufferSource1": "https://switchboard-sdk-public.s3.amazonaws.com/assets/audio/music/guitar-48k.wav"
    }
  ]
}

We've created an experimental RTNeural extension for Switchboard allowing you to insert real-time ML interfence powered signal processing into any Switchboard audio engine.

Below is at example where we define a Switchboard JSON graph to play a guitar sample with a WebAudio.AudioPlayer node, then pass the output into a RTNeural.MLProcessor node. In this particular example we're using an open-source guitar distortion model from the GuitarLSTM project, converted into RTNeural JSON modal format.

You can also use the Upload button in the UI to upload an RTNeural JSON model from your computer.

Node types

The Switchboard RTNeural Extension provides the following audio nodes for a Switchboard SDK audio graphs:

Switchboard JSON Node	Description
RTNeural.MLProcessor	A processor node that runs the RTNeural model and receives audio data with the specified pre-processing, then generates audio data with the specified post-processing. Ideal for applications in audio transformation, such as Noise Reduction, Source Separation, and Voice Conversion.

info

The RTNeural.MLProcessor node currently support mono audio only, down-mixing stereo to mono before its passed into the interfence engine and then up-mixing back to stereo as its passed back out of the RTNeural inferencing engine.

Download SDK Extension

You can download this SDK extension from our Downloads Page.

Visit the page to access the latest version and start integrating it into your project!