Adaptive video layout with example app

Thanks to the implementation of Custom layout map API in Eyeson, you can now change the layout without having the need for pre-defined layouts provided by us. This has some major upsides regarding the control over the stream content.

In this tutorial we want to use the adaptive layout to arrange users around an image for a seamless presentation.

Example web app

You can instantly update the stream content during a live session using some HTML and Javascript.

This example includes

• a local image uploading script which draws it on a canvas
• layout calculation depending on the image size
• sending canvas and calculated layout to show in Eyeson via AJAX requests

Upload an image

Here's how a local image is being uploaded and made available for the canvas.

let image = null;
const onChooseFile = ({ target }) => {
    const [file] = target.files;
    const url = URL.createObjectURL(file);
    const img = new Image();
    img.onload = () => {
        image = img;
        URL.revokeObjectURL(url);
        drawCanvas();
    };
    img.src = url;
};

Figure 1. Simplified script for uploading an image

Draw the canvas content

Depending on the uploaded imageRatio, the position of the image and the position and size of the participants are defined and visualised in the canvas. Also an adjustable spacing parameter gap is added.

let layout_map = null;
const canvas = document.querySelector('#canvas-background');
const context = canvas.getContext('2d');

const drawCanvas = () => {
    const gap = document.querySelector('#slider-gap').valueAsNumber;
    context.clearRect(0, 0, canvas.width, canvas.height);
    const imageRatio = image.width / image.height;
    if (imageRatio < 0.6) {
        const imageHeight = canvas.height - 2 * gap;
        const imageWidth = Math.floor(imageHeight * imageRatio);
        const userWidth = Math.floor((canvas.width - 5 * gap - imageWidth) / 3);
        const userHeight = Math.floor((canvas.height - 4 * gap) / 3);
        layout_map = [
            [2 * gap + imageWidth, gap, userWidth, userHeight],
            [3 * gap + imageWidth + userWidth, gap, userWidth, userHeight],
            [4 * gap + imageWidth + 2 * userWidth, gap, userWidth, userHeight],
            [2 * gap + imageWidth, userHeight + 2 * gap, userWidth, userHeight],
            [3 * gap + imageWidth + userWidth, userHeight + 2 * gap, userWidth, userHeight],
            [4 * gap + imageWidth + 2 * userWidth, userHeight + 2 * gap, userWidth, userHeight],
            [2 * gap + imageWidth, 2 * userHeight + 3 * gap, userWidth, userHeight],
            [3 * gap + imageWidth + userWidth, 2 * userHeight + 3 * gap, userWidth, userHeight],
            [4 * gap + imageWidth + 2 * userWidth, 2 * userHeight + 3 * gap, userWidth, userHeight],
        ];
        context.drawImage(image, gap, gap, imageWidth, imageHeight);
        context.fillStyle = 'rgba(255,255,255,0.2)';
        layout_map.forEach(([x, y, width, height]) => {
            context.fillRect(x, y, width, height);
        });
    }
    else if (imageRatio < 1.2) {
        ...
    }
    else if (imageRatio <= 1.5) {
        ...
    }
    else {
        ...
    }
}

Figure 2. Simplified script for creating a layout map and drawing the canvas content

Send background and layout to Eyeson

The API endpoint for layer can take a binary image file as parameter. We're drawing the image on a canvas element, creating a blob from it and sending this to the Eyeson API. The access_key can be extracted from a meeting link.

The API endpoint for layout receives a layout map that resembles the canvas content.

const sendBackground = () => {
    canvas.toBlob(blob => {
        const formData = new FormData();
        formData.set('file', blob, 'background.png');
        formData.set('z-index', '-1');
        fetch(`https://api.eyeson.team/rooms/${access_key}/layers`, {
            method: 'POST',
            body: formData,
        });
    });
};

const sendLayout = () => {
    const formData = new FormData();
    formData.set('layout', 'auto');
    formData.set('name', 'adaptive-layout');
    formData.set('map', JSON.stringify(layout_map));
    for (let i = 0; i < layout_map.length; i++) {
        formData.append('users[]', '');
    }
    fetch(`https://api.eyeson.team/rooms/${access_key}/layout`, {
        method: 'POST',
        body: formData,
    });
};

drawCanvas();
sendBackground();
sendLayout();

Figure 3. Simplified script for instantly updating the stream content

Figure 4. Images with various ratios are uploaded to show the adaptive layout

note

If you are going to implement this kind of feature on a bigger scale, you need to adjust the code and repositories accordingly.

Read more about Custom layout map API and see other tutorials like Enhance video meetings with foreground layers or Local Video Overlay.

Download code example

We've prepared a full working example that contains the code shown in this post.

Download eyeson-adaptive-layout-example.zip