How To Provide Virtual Backgrounds In Your Meetings

A much loved feature in video meetings is to blur or replace your background. This feature Virtual Background is part of the Eyeson UI, the options are "blur" and "strong blur", backgrounds provided by Eyeson, or a custom image via API provided by you.

Here's how you can provide virtual background with our default UI and even in your own custom UI.

info

This is a client-side functionality

API with default UI aka Eyeson GUI

If you're using Eyeson's default UI, by default the virtual background is turned off, you need to enable this feature when you start the meeting (see API Reference). There are 3 new related options:

options[custom_fields][virtual_background] = true # default false
options[custom_fields][virtual_background_allow_guest] = true # default false
options[custom_fields][virtual_background_image] = "https://example.com/image.png" # adds a "Custom image"

Virtual background with your own UI

If you want to your own UI, you need to follow the next steps.

This feature is based on TensorFlow Lite, therefore make sure to include the needed tflite files in your project. They have to be located in a public folder with the following structure:

vendor/tflite/
|-- tflite.js
|-- tflite.wasm
|-- tflite-simd.js
|-- tflite-simd.wasm
|-- models/
|   |-- selfie_segmentation_landscape.tflite

The TFLite Scripts and TFLite Model are available on Jitsi Meet GitHub Repository and Google Mediapipe GitHub Repository.

info

Since version 1.8.9, the script path can be configured. The default value is "vendor/tflite/".

import eyeson from 'eyeson';
eyeson.config.vbgScriptPath = 'feature/virtual-background/';
// OR even absolute path
eyeson.config.vbgScriptPath = 'https://cdn.myawesomeapp.com/virtual-background/'; // take care about CORS

Make sure to check for support in the current browser.

import { FeatureDetector } from 'eyeson';
const virtualBackgroundSupport = FeatureDetector.canVirtualBackground();

Background types

Available types are off, blur:8, blur:16, and image:<url>.

Setting the type off will simply disable the virtual background.

The number after blur: is the blur radius in pixels. A higher number does increase the blur effect. Unfortunately, blur is not supported in every browser. (Hint: On caniuse.com you can find a full browser support list).

To check for blur support in the current browser, we provide a helper function.

import { FeatureDetector } from 'eyeson';
const virtualBackgroundBlurSupport = FeatureDetector.canvasBlurSupport();

The image type allows to set an absolute or relative URL to an image file.

• "image:https://images.another-domain.com/images/background.png"
• "image:images/vbg/image-1.png"

You can set any static image as background, best results with a ratio of 4:3, and will be scaled to cover the full background.

Set or change a background type

The following example shows how to set a background type. When it is set to off, the virtual background is disabled.

Keep in mind, that the DeviceManager will only preview and update the options. To actually use it in the meeting, read the next part "meeting usage".

import { DeviceManager } from 'eyeson';

const deviceManager = new DeviceManager();

document.querySelector('#dropdown-background-type').value = DeviceManager.getStoredVirtualBackgroundType();

document.querySelector('#dropdown-background-type').addEventListener('change', event => {
    const type = event.target.value;
    deviceManager.setVirtualBackgroundType(type);
});

Meeting usage

In order to actually use the virtual background in the meeting you have to set another parameter.

eyeson.start(<access_key>, { audio: true, video: true, virtualBackground: true });

let virtualBackgroundActive = true;

function toggleVirtualBackground() {
    virtualBackgroundActive = !virtualBackgroundActive;
    eyeson.send({ type: 'start_stream', audio: true, video: true, virtualBackground: virtualBackgroundActive });
}

Temporary pause and resume

In some cases it might be necessary to quickly disable the running virtual background. For example if you want to show a document. The algorithm would detect this as the person's background and so it becomes transparent.

let suspendVirtualBackground = false;

function suspendVirtualBackground() {
    suspendVirtualBackground = !suspendVirtualBackground;
    const state = suspendVirtualBackground ? 'suspend' : 'resume';
    eyeson.send({ type: 'suspend_virtual_background', state });
}