Example app to enhance video meetings with foreground layers

Some key advantages of Eyesons MCU+ are that all participants see the same and the video stream can contain additional visuals and data. That means, you can choose one of the provided layouts and show images in the background and foreground layers.

Figure 1. Animation that shows the stream components.

Most of the time, you can just use an image that you've already prepared as an overlay. But there are cases where you need to assemble an image from various parts and apply it on-the-fly.

caution

You can only use images smaller than or equal to 1280x720 in widescreen mode or 1280x960 in fullscreen mode.

info

Smaller images are positioned top left of the video stream as overlay.

Example web app

In this case we are going to write a small HTML file using some Javascript to show how to add custom visuals to your Eyeson video meetings.

This example includes

• designing an overlay and drawing it with canvas
• uploading a local image and placing it in a canvas
• converting the canvas to a blob so it can be shown in Eyeson via an AJAX request

Canvas to blob

The API endpoint for layer can take a binary image file as parameter. We're drawing the image on a canvas element, creating a blob from it and sending this to the Eyeson API. The access_key can be extracted from the meeting link.

drawCanvas();

canvas.toBlob(blob => {
    const formData = new FormData();
    formData.set('file', blob, 'overlay.png');
    formData.set('z-index', '1');
    fetch(`https://api.eyeson.team/rooms/${access_key}/layers`, {
        method: 'POST',
        body: formData
    });
}));

Upload a local image

Here's how a local image is being uploaded and made available for the canvas.

let image = null;
const changeImage = ({ target }) => {
    const [file] = target.files;
    const url = URL.createObjectURL(file);
    const img = new Image();
    img.onload = () => {
        image = img;
        URL.revokeObjectURL(url);
        drawCanvas();
    };
    img.src = url;
};

Placing the contents on the canvas

All the components are drawn in a specific order. The canvas has the same dimensions as the MCU video stream.

const drawCanvas = () => {
    canvas.width = 1280;
    canvas.height = 720;

    if (image) {
        const scale = imageSlider.valueAsNumber / 100;
        const width = image.width * scale;
        const height = image.height * scale;
        context.drawImage(image, imageX, imageY, width, height);
    }
    
    if (caption !== '') {
        context.font = '36px sans-serif';
        context.fillStyle = 'white';
        context.fillText(caption, captionX, captionY);
    }
}

note

When using layers you might want to disable the SFU mode in default room configuration, or simply set any layout to enable MCU mode.

Obviously, if you are going to implement this kind of feature on a bigger scale, you need to adjust the code and repositories accordingly.

Here are some more advanced examples: Local Video Overlay and Virtual Backgrounds.

Download code example

We've prepared a full working example that contains the code shown in this post.

Download eyeson-canvas-overlay-example.zip