Using the Network Bandwidth Profile API

The Network Bandwidth Profile API makes it possible to specify how the downlink bandwidth of a Room Participant should be distributed among its subscribed tracks. Using this API developers can assign higher bandwidth to higher priority tracks, protect audio quality and keep the consumed network and battery resources under control.

Twilio Rooms use a Media Server that is based on the SFU (Selective Forwarding Unit) architecture, where Participants commonly publish up to two video tracks (e.g. webcam and/or screen share) but may subscribe to many. The number of subscribed tracks typically grows in proportion to N-1 where N is the total number of Participants in the room. When scalable video codecs (such as VP8 Simulcast) are used, each track can be forwarded using multiple qualities. Hence, the Media Server needs to decide which quality is assigned to each track. In other words, it needs to determine how the available downlink bandwidth is allocated among those tracks. This requires addressing the following problems:

• To guarantee that the network does not get overloaded, which may happen if all tracks are subscribed with their maximum quality.
• To guarantee that videos rendered using larger UI areas are allocated more bandwidth than thumbnail videos.
• To avoid mobile devices consuming more bandwidth or battery than what is required for each specific use-case.
• To limit the number of displayed video tracks to what is appropriate for every use-case.
• To preserve audio quality in case of severe congestion.

Expand image

The Network Bandwidth Profile API has been created to address these issues. The following sections explain how to use it.

By default Bandwidth Profiling is disabled. This means that if you are not interested in Network Bandwidth Profiles or if you don't use the Network Bandwidth Profile API you should not experience any changes in your application behavior. For activating Bandwidth Profiling in a Participant, you just need to specify a bandwidthProfile at connect time. Once Network Bandwidth Profiles is enabled for a Participant, it cannot be disabled.

A Network Bandwidth Profile specifies how a Participant's downlink bandwidth is consumed. Notice that it is per-Participant meaning that different Participants may have different Network Bandwidth Profiles. This also means that Bandwidth Profiling can be activated on some Participants and be inactive on others in the same Room. Developers can specify Network Bandwidth Profiles as a connect option.

JavaScript SDK (v2.0.0+)

1
//Specifying a Network Bandwidth Profile at connect time
2
const { connect } = require('twilio-video');
3

4
const room = await connect(token,{
5
  name: "my-new-room",
6
  bandwidthProfile: {
7
    video: {
8
      mode: 'collaboration',
9
      dominantSpeakerPriority: 'high'
10
    }
11
  }
12
});

Android SDK (v5.8.0+)

1
Room room = Video.connect(context, new ConnectOptions.Builder(accessToken)
2
                .bandwidthProfile(new BandwidthProfileOptions(new VideoBandwidthProfileOptions.Builder()
3
                         // Minimum subscribe priority of Dominant Speaker's RemoteVideoTracks
4
                        .dominantSpeakerPriority(TrackPriority.HIGH)
5
                        // Subscription mode: COLLABORATION, GRID, PRESENTATION
6
                        .mode(BandwidthProfileMode.COLLABORATION)
7
                        .build()))
8
                .build(),
9
        roomListener);

iOS SDK (v3.4.0+)

1
let videoOptions = VideoBandwidthProfileOptions { builder in
2
    // Minimum subscribe priority of Dominant Speaker's RemoteVideoTracks
3
    builder.dominantSpeakerPriority = .high
4

5
    // Maximum bandwidth (Kbps) to be allocated to subscribed RemoteVideoTracks
6
    builder.maxSubscriptionBitrate = 6000
7

8
    // Subscription mode: collaboration, grid, presentation
9
    builder.mode = .presentation
10

11
    // Track Switch Off mode: .detected, .predicted, .disabled
12
    builder.trackSwitchOffMode = .predicted
13
}
14
let bandwidthProfileOptions = BandwidthProfileOptions(videoOptions: videoOptions)
15

16
// Use bandwidth profile to build the connectOptions
17
let connectOptions = ConnectOptions(token: accessToken) { builder in
18
    builder.bandwidthProfileOptions = bandwidthProfileOptions
19
}
20

21
// Connect to a Room with connectOptions
22
room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)

The following table summarizes the meaning of the different bandwidthProfile parameters:

The main objective of the Network Bandwidth Profile API is to split the available downlink bandwidth among a Participant's subscribed video tracks. This is performed by an algorithm that determines how much bandwidth is allocated to each track as a function of the track priority and of the specified Network Bandwidth Profile. The mode parameter controls the behavior of the algorithm. Currently, mode can take the following values: grid (default), collaboration and presentation.

Grid mode should be used in use-cases where all subscribed video tracks are equally important. When this mode is used, the available bandwidth is split uniformly among all the tracks independently on their priorities. This makes grid mode useful for UIs displaying all video tracks in a matrix, so that none of them is enhanced over the rest.

Grid mode should also be used when scalable video codecs are not possible. In other words: if most participants in a room do not use VP8 Simulcast then, most probably, grid mode will generate higher quality of experience than any other mode.

When a participant is configured to use grid mode, the available downlink bandwidth is split uniformly among all the received tracks, which are treated as equals independently of their priority. Hence, when this mode is used, developers will typically use just the same priority and for all the video tracks or will set the same renderDimensions for all the used priorities.

Expand image

The following rules of thumb may help you understand how grid mode works:

• If bandwidth is large enough, all video tracks will be assigned their maximum bandwidth. This maximum depends on the renderDimensions and more specifically, is proportional to the display area (i.e. width x height).
• If bandwidth decreases, the available amount will be split equally among all the tracks. Hence, all tracks drop their bandwidth by the same fraction at the same time.
• If bandwidth decreases further video tracks will start switching-off in lower to higher priority order. Notice that the track with highest priority will never be switched off.

Collaboration mode is for applications where some video tracks are more important than others but we still want to keep the rest of the video tracks visible. For example, in videoconferencing meetings having the typical UI layout where the dominant speaker is rendered in the central area and the rest of participants are rendered as thumbnails in a bottom row.

When a participant is configured to use collaboration mode, the available bandwidth is split in proportion to the render area (i.e. renderDimensions) assigned to each track. Hence, more relevant tracks are going to be allocated higher bandwidth. Due to this, collaboration mode requires Simulcast to be used by most video publishers (or at least by higher relevance publishers). In other words, unless there is a good reason for it, developers should avoid using collaboration mode together with plain VP8 or H.264 codecs.

Expand image

The following information may help you understand how collaboration mode works:

• If bandwidth is large enough, all video tracks will be assigned their maximum bandwidth. This maximum depends on the renderDimensions and more specifically, is proportional to the display area (i.e. width x height).
• If bandwidth decreases, the available amount will be split among tracks in proportion to the renderDimensions so that tracks having double area will be assigned double bandwidth.
• If bandwidth decreases even further, the allocated amount will go down in that proportion while trying to keep all tracks visible. This means that, if necessary, higher priority tracks will decrease their quality to avoid lower priority switch-offs. If bandwidth continues decreasing, at some point all tracks will get to their minimum and switch-offs will start in lower to higher priority order. Notice that the track with highest priority will never be switched off.

Presentation mode is for use-cases where some video tracks are critical for the end-user experience and must be preserved at any cost. For example, in webinars where a speaker shares some kind of presentation to an audience of viewers. In this case, viewers webcams should rather be switched-off if the webinar screen share quality is at risk.

When a participant is configured to use presentation mode, higher priority tracks will be assigned all the bandwidth they require so that only when all the tracks of a given priority are at their maximum the lower priority ones will be allocated. As happens with collaboration, this mode requires most participants to publish tracks (and specifically the highest priority track) with a scalable video codec such as VP8 Simulcast.

Remark also that, if bandwidth is large enough, both collaboration and presentation behave equivalently and assign the maximum bandwidth to each track in proportion to its renderDimensions. However, when it's not possible to allocate all that bandwidth both modes diverge. In that case, presentation has the objective of keeping the highest priority video track quality, while collaboration is designed for keeping higher priority tracks with higher quality only if the continuity of the rest of tracks is not compromised.

Expand image

The following may help you understand how presentation mode works:

• If bandwidth is large enough, all video tracks will be assigned their maximum bandwidth. This maximum depends on the renderDimensions and more specifically, is proportional to the display area (i.e. width x height).
• If bandwidth decreases, higher priority video tracks will remain at their maximum while lower priority tracks will decrease, even if that decrease causes them to switch-off.
• If bandwidth decreases even further, lower priorities will be switched-off before decreasing higher priority tracks bandwidth. Only when all lower priority tracks are switched-off the higher priority tracks may decrease their allocation.

When a Participant's downlink bandwidth is insufficient, resulting congestion and packet loss may cause significant degradation to both audio and video quality. To avoid congestion, the Network Bandwidth Profile API algorithms monitor and estimate the available downlink bandwidth and decrease video tracks quality accordingly. However, any real-time video track has a minimum limit under which bandwidth cannot be further reduced. In the Twilio Video engine this limit is about 30Kbps for VP8 and H.264 and at approximately 60Kbps for VP8 Simulcast. Hence, if bandwidth keeps degrading, at some point all video tracks will reach that limit. Beyond that point, congestion will be severe enough that the application becomes unusable. To deal with this, we have introduced the trackSwitchOffMode parameter. trackSwitchOffMode allows developers to control wether less relevant video tracks should be switched off in case of congestion so that higher priority ones are protected.

Using trackSwitchOffMode: predicted

Twilio algorithms constantly estimate the available bandwidth. When predicted is used, lower priority video tracks will be switched-off as soon as such estimation indicates there may be congestion. Hence, predicted will act before congestion happens. Thanks to it, when predicted is used, a downlink bandwidth drop will typically not affect subscribed audio quality or higher priority video quality. However, the drawback is that the predicted bandwidth estimation algorithm is aggressive towards protecting higher priority tracks. This causes track switch-offs at the earliest signs of congestion. In summary: predicted offers maximum protection of audio and higher priority video tracks at the cost of having more lower priority video track switch-offs. By default trackSwitchOffMode is predicted.

Using trackSwitchOffMode: detected

When detected is used, track switch-offs will only happen after congestion actually happens. Thanks to this, detected will avoid any unnecessary switch-offs. However, as congestion is detected through packet loss, a downlink bandwidth drop when using this mode will typically cause choppy audio and frozen video during a few seconds. We only recommend to use detected if video continuity is very important for your use-case.

Using trackSwitchOffMode: disabled

When disabled is used, congestion will not cause any track switch-offs. Hence, in the case of severe bandwidth drop, congestion will grow and, with it, relevant packet loss will appear causing audio to be unintelligible and video to remain frozen. In general, our recommendation is not to use this mode unless you have a very good reason to do it.

Expand image

What track switching-off means

When a video track is switched-off for a Participant, Twilio's Media Server removes all the track downlink traffic. This means that, for that specific Participant, the track will consume zero bandwidth. For managing track switch-offs in your application, you may find useful the following information:

• Track switch-offs are always subscriber side. In other words, only the remote tracks received at a Participant can be switched-off.
• Track switch-offs happen "per-Participant". This means that a track may be off for a given Participant (e.g. one having poor bandwidth) but on for others.
• Switched-off tracks will appear as frozen video in the UI (i.e. will show the last received frame). Hence, a switched-off track will not be rendered as a black video.

How Track switch-offs work

The algorithms that control track switch-off enforce the following rules:

• Audio tracks are never switched-off.
• The Network Bandwidth Profile mode influences the switch-off behavior. Check the mode documentation below for further details.
• Tracks with higher priority will always be switched off last. Hence, for example, if a track with priority high is switched off, then it is guaranteed that all tracks with priority standard or low have been switched off previously.
• Among tracks with the same priority, the track being associated to the participant with higher speaking activity will be switched off last. Hence, for example, if two standard tracks are candidates to be switched off, the one associated to the participant who speaks less will be preferred.

After a track has been switched-off, it may eventually be switched on again. Track switch-on happens based on the following principles:

• As with the switch-off process, track switch-ons happen "per-Participant".
• To avoid oscillations, the off-to-on switching process has hysteresis. This means that it's not enough for the available bandwidth to recover, but that recovery must sustain during a reasonable time period (typically around 20 seconds) to have the track back to on.
• When multiple tracks are off, the track with highest priority will be switched on first.

Each time a track is switched off/on for a Participant the Twilio Media Server sends a notification. Developers can subscribe to those events that are published at the RemoteVideoTrack object, as the following code snippets illustrate:

JavaScript SDK (v2.0.0+)

1
remoteTrackPublication.on('subscribed', remoteTrack => {
2
  remoteTrack.on('switchedOff', () => {
3
    //You may update your UI accordingly
4
    // You can also determine whether a particular RemoteTrack is switched off.
5
    assert.equal(remoteTrack.isSwitchedOff, true);
6
    console.log(`The RemoteTrack ${remoteTrack.name} was switched off`);
7
  });
8

9
  remoteTrack.on('switchedOn', () => {
10
    //You may update your UI accordingly
11
    // You can also determine whether a particular RemoteTrack is switched off.
12
    assert.equal(remoteTrack.isSwitchedOff, false);
13
    console.log(`The RemoteTrack ${remoteTrack.name} was switched on`);
14
  });
15
});

Android SDK (v5.8.0+)

1
RemoteParticipant.Listener remoteParticipantListener = new RemoteParticipant.Listener() {
2
    @Override
3
    public void onVideoTrackSwitchedOff(
4
        @NonNull RemoteParticipant remoteParticipant,
5
        @NonNull RemoteVideoTrack remoteVideoTrack) {}
6

7
    @Override
8
    public void onVideoTrackSwitchedOn(
9
        @NonNull RemoteParticipant remoteParticipant,
10
        @NonNull RemoteVideoTrack remoteVideoTrack) {}
11
}

iOS SDK (v3.4.0+)

1
extension ViewController : RemoteParticipantDelegate {
2
    func remoteParticipantSwitchedOnVideoTrack(participant: RemoteParticipant,
3
                                               track: RemoteVideoTrack) {}
4

5
    func remoteParticipantSwitchedOffVideoTrack(participant: RemoteParticipant,
6
                                                track: RemoteVideoTrack) {}
7
}
8

Sometimes developers want to limit the total downlink bandwidth consumed by their applications. There may be many reasons for this:

• To minimize the battery consumption on mobile devices.
• To save costs.
• To reserve higher qualities to premium users.
• etc.

Expand image

When maxSubscriptionBitrate is set in a Participant, that Participant video downlink will never consume more than the specified value expressed in bps. However note that it may consume less as far as the actual available network bandwidth is below.

By default, the maximum downlink bandwidth is capped to 4Mbps in Rooms. Hence, values of maxSubscriptionBitrate over those limits will have no effect. Default value is:

• maxSubscriptionBitrate: 0 (meaning "no limit"), in desktop browsers. Hence, desktop browsers default to the above mentioned caps.
• maxSubscriptionBitrate: 2,400,000 in mobile SDKs.

It is important to remark that the use of maxSubscriptionBitrate for setting a bandwidth constraint may generate switch-off in your video tracks. For example, if you work in presentation mode with a high priority video track with default renderDimensions and set maxSubscriptionBitrate to say 500,000, then you will be probably experiencing permanent track switch-offs for all the tracks except the main one.

The Dominant Speaker refers to the Participant having highest audio activity at a given time. You can activate dominant speaker detection following Twilio's official documentation. This feature is useful when you want to enhance the dominant speaker in your UI: for example, by rendering her video in the central area and with larger size. To do this appropriately, the Network Bandwidth Profile API should be used to set a higher priority to the dominant speaker video tracks. However, as the dominant speaker changes dynamically, it may be hard to figure out what are the tracks that should be prioritized at any time. To solve this problem the Network Bandwidth Profile API exposes the dominantSpeakerPriority parameter. Using it, developers can set the minimum priority that should be automatically assigned to the dominant speaker video tracks.

To understand how dominantSpeakerPriority works imagine a Room with dominant speaker detection activated where a Network Bandwidth Profile is defined with dominantSpeakerPriority: 'high'. Imagine that in that Room all video tracks are set to their default priority (i.e. standard). In that case, when Participant Alice becomes dominant speaker, automatically all Alice's video tracks will become high priority. If later Bob becomes dominant speaker, automatically Alice's video tracks will go back to standard, as she isn't dominant any longer, and Bob's video tracks will be upgraded to high.

Expand image

As the rest of parameters in the Network Bandwidth Profile API, dominantSpeakerPriority works per-Participant. This means that the priority upgrades only happen on the subscriptions of the specific participant where that Network Bandwidth Profile has been defined. This also means that the dominant speaker is relative to those subscriptions. Let's illustrate that with an example. Imagine the following Room:

• Participant Alice subscribes to Bob's and Carl's tracks
• Participant Bob subscribes to all the Room tracks.
• Participant Carl subscribes only to Bob's and to Dave's tracks.
• Participant Dave subscribes only to Alice's tracks.

Imagine also that, at a given time, the speaking activity of the Participants goes in this order:

• Alice (with highest speaking activity), Dave, Bob and Carl (with lowest speaking activity).

As the dominant speaker is the Participant with highest speaking activity among all subscriptions, then the following will hold for that specific instant:

• For Alice: the dominant speaker is Bob.
• For Bob: the dominant speaker is Alice.
• For Carl: the dominant speaker is Dave.
• For Dave: the dominant speaker is Alice.

In addition, please notice the following:

• This property indicates the minimum priority of the dominant speaker video tracks. In other words, dominantSpeakerPriority can only upgrade the priority of the dominant speaker but never downgrade it. For example, if a developer sets dominantSpeakerPriority: 'standard' and the dominant speaker has a video track published with priority high, that video track will stay as high.
• By default Network Bandwidth Profiles have dominantSpeakerPriority: 'standard'.
• If dominant speaker detection has not been activated using the Dominant Speaker Detection API, then setting dominantSpeakerPriority will have no effect.

There are many types of multi-party applications where the video of all participants is not rendered on screen at the same time. If a participant's video is not on screen then there is an opportunity to save CPU and bandwidth by switching off the incoming video track. When the participant needs to be displayed on screen again, the video track can be easily switched on again. Depending on developer preference this video track switch on or off can be performed automatically by the SDK or via explicit manual control.

Using clientTrackSwitchOffControl: 'auto'

With this setting the client SDK will automatically switch on or off a RemoteVideoTrack depending on the visibility of the view/document, video attachment or application lifecycle. This automatic switch on/off of the RemoteVideoTrack is managed by the SDK. This is the default setting.

Using clientTrackSwitchOffControl: 'manual'

Use this setting if the client application requires the ability to explicitly switch on or off a RemoteVideoTrack based on a desired behavior. In general switching off RemoteVideoTracks that are not currently being rendered will result in the best video quality for the tracks that are being rendered.

JavaScript SDK (v2.14.0+)

1
const room = await connect(token, {
2
  bandwidthProfile: {
3
    video: {
4
      clientTrackSwitchOffControl: 'manual'
5
    }
6
  }
7
});

1
// Explicitly switch off a track
2
remoteTrack.switchOff();

Android SDK (v6.4.0+)

1
BandwidthProfileOptions bandwidthProfileOptions = new BandwidthProfileOptions(
2
        new VideoBandwidthProfileOptions.
3
                Builder().
4
                // Use "manual" for explicit track switch on/off
5
                .clientTrackSwitchOffControl(ClientTrackSwitchOffControl.MANUAL)
6
                .build());
7
ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)
8
        .bandwidthProfile(bandwidthProfileOptions)
9
        .build();
10

11
Video.connect(context, connectOptions, roomListener);

1
// the application can now request that specific RemoteVideoTracks be switched off or on, e.g.
2
RemoteVideoTrack.switchOn();

iOS SDK (v4.5.0+)

1
let connectOptions = ConnectOptions(token: accessToken) { (builder) in
2
  builder.bandwidthProfileOptions = BandwidthProfileOptions(
3
    videoOptions: VideoBandwidthProfileOptions { builder in
4
      builder.clientTrackSwitchOffControl = .manual
5
    }
6
  )
7
}
8

9
let room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)

1
// The application can now request that specific RemoteVideoTracks be switched off or on, e.g.
2
remoteVideoTrack.switchOff()

In general it is best for the Twilio Media Server to deliver the video resolution to the client that closely matches the current render dimensions for the corresponding video element or view in the UI. This ensures the most efficient use of both CPU resources on the client and the downstream bandwidth. This in turn delivers the best end user experience. Depending on developer preference this alignment of video track resolution with actual render dimensions can be performed automatically by the SDK or via explicit manual control.

Using contentPreferencesMode: 'auto'

With this setting the client SDK will automatically detect and communicate the render dimensions of each video element to the Twilio Media Server. As the UI changes and the render dimensions of each video element or view are changed, the SDK automatically detects this and the resolution of the RemoteVideoTracks are dynamically adjusted. This is the default setting.

Using contentPreferencesMode: 'manual'

Use this setting if the client application wants to explicitly communicate the preferred render dimensions for each RemoteVideoTrack and adjust them dynamically.

JavaScript SDK (v2.14.0+)

1
const room = await connect(token, {
2
  bandwidthProfile: {
3
    video: {
4
      contentPreferencesMode: 'manual'
5
    }
6
  }
7
});

1
// Indicate that the remoteTrack should be delivered at a maxiumum of 320x240 resolution
2
remoteTrack.setContentPreferences({
3
    renderDimensions: { width: 320, height: 240 }
4
});

Android SDK (v6.4.0+)

1
BandwidthProfileOptions bandwidthProfileOptions = new BandwidthProfileOptions(
2
        new VideoBandwidthProfileOptions.
3
                Builder().
4
                // Use "manual" for explicit content preferences (renderDimensions)
5
                .videoContentPreferencesMode(VideoContentPreferencesMode.MANUAL)
6
                .build());
7
ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)
8
        .bandwidthProfile(bandwidthProfileOptions)
9
        .build();
10

11
Video.connect(context, connectOptions, roomListener);

1
// Indicate that the remoteTrack should be delivered at a maxiumum of 320x240 resolution
2
remoteVideoTrack.setContentPreferences(new VideoContentPreferences(new VideoDimensions(320, 240)));

iOS SDK (v4.5.0+)

1
let connectOptions = ConnectOptions(token: accessToken) { (builder) in
2
  builder.bandwidthProfileOptions = BandwidthProfileOptions(
3
    videoOptions: VideoBandwidthProfileOptions { builder in
4
      builder.contentPreferencesMode = .manual
5
    }
6
  )
7
}
8

9
let room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)

1
// Request that the remoteTrack be delivered at 320x240 resolution
2
remoteVideoTrack.setContentPreferences(VideoContentPreferences { builder in
3
  builder.renderDimensions = VideoDimensions(width: 320, height: 240)
4
})
5

Deprecated - use clientTrackSwitchOffControl instead

Note: setting both maxTracks and clientTrackSwitchOffControl is not allowed and will raise an exception.

When setting maxTracks to N, Twilio guarantees that at any given time no more than N video tracks will be on. Hence, Twilio keeps only the N most relevant tracks for you based on the following:

• Higher priority tracks are preferred.
• Within a priority, tracks having higher speaking activity are preferred. This is sometimes called an N-Loudest policy. Note that N-Loudest (i.e the N participants with higher speaking activity at a given time) is not the same than the other policy called Last-N (i.e. the last N dominant speakers). Twilio Network Bandwidth Profiles do not support Last-N for maxTracks.

Note that all the lower priority video tracks beyond the specified maxTracks will be switched off. Hence, given that priorities may change dynamically, your application will receive track switch-off/on notifications to indicate which tracks are visible at any given time.

Note also that maxTracks related switch-offs are not due to congestion and are not affected by the trackSwitchOffMode parameter. Therefore, even if you set trackSwitchOffMode: disabled, you may still have track switch-offs if a Participant subscribes to more than maxTracks video tracks.

Just for illustration, if you have a videoconferencing application with:

• A screen share track with priority high.
• 40 webcam tracks with priority standard.
• You now set maxTracks: 5.

Then, the algorithm will only allow the following 5 tracks to be on:

• The screen share track.
• The 4 webcam tracks corresponding to the 4-loudest participants (i.e. the 4 participants with the highest speaking activity at that time).

Remark also that maxTracks: 0 should be read as "unlimited" meaning that Twilio will try to send all the subscribed tracks to the Participant. Note also that, by default, maxTracks: 0.

While maxTracks is "unlimited" by default, the recommended maximum value is 16.

This recommendation is highly dependent on the bitrate associated with each subscribed video track. For optimal results with two or more visible tracks you should ensure that you specify a low track priority for nearly all video tracks and that your renderDimensions for the low priority tracks are no more than 60p to 180p per track.

Deprecated - use contentPreferencesMode instead

Note: Setting both renderDimensions and contentPreferencesMode in the bandwidth profile is not allowed and will raise an exception.

Video bandwidth depends on resolution. For example, if a track is to be rendered in FullHD (1920x1080) it may make sense to reserve 4Mbps for it. However, for a thumbnail with CIF (352x240) size that amount would not be justified.

Using renderDimensions developers can specify the display size, in pixels, they plan to use to render video tracks on a Participant's UI. Thanks to this, Twilio can determine what's the maximum bandwidth that should be allocated to each video track. This means that, even if the available bandwidth is in excess, our algorithms will never allocate more than what is needed to render the track with he appropriate specified size. This has multiple positive side effects:

• Network resources are preserved and used only when it makes sense.
• Bandwidth is only allocated to tracks that really needed.
• Battery life is enhanced.

In the Network Bandwidth Profile API, renderDimensions are specified per priority:

high: Specifies the render width and height (in pixels) of high priority tracks. Defaults to HD resolution:

1
high: {
2
  width: 1280,
3
  height: 720
4
}

standard: Specifies the render width and height (in pixels) of standard priority tracks. Defaults to VGA resolution:

1
standard: {
2
  width: 640,
3
  height: 480
4
}

low: Specifies the render width and heigh (in pixels) of low priority tracks. Defaults to QCIF resolution:

1
low: {
2
  width: 176,
3
  heigh: 144
4
}

It is important to remark that, from the practical perspective, our algorithms use renderDimensions to compute the maximum video Track bandwidth. This has multiple implications you should be aware of:

• The renderDimensions must be understood as a hint to the desired actual rendering dimensions of video Tracks. That means that renderDimensions don't need to be exact. In other words, you don't need to invest much of your developing time figuring out how to update your renderDimensions every time the display size changes. You must concentrate instead on providing approximate hints that keep the actual proportions of your UI. That means that if a video track A is to be rendered with double size than video track B, then A's priority should have double renderDimensions than B's.
• The renderDimensions don't need to match with the actual video track dimensions. The video track dimensions will depend on aspects such as your capture constraints and the status of the network. Remember that renderDimensions are just a hint that you provide to help the bandwidth allocation algorithms and not a specification on actual video width and height.
• The Network Bandwidth Profile API implicitly assumes that all video tracks within a given priority are to be rendered using the same size. Experience shows that in most common use-cases this assumption is true. However, if you have a use-case where you really need different dimensions our recommendation is to set renderDimensions so that the resulting area (width x height) is the average of all the render areas of the tracks on that priority.

As stated above, collaboration and presentation modes are only recommended when scalable video codecs such as VP8 Simulcast are used. Using them together with plain VP8 and H.264 will sometimes generate undesired behaviors caused by the interdependencies that appear among the different track subscribers. This may lead to problems like the following:

• The subscribed bitrate of a non-scalable track may be significantly under the available bandwidth of that subscriber, even if the publisher has a high speed network. For example, you may have a publisher and a subscriber connected to a GB cable network but have them communicating low quality video at 50Kbps due to a third Participant connecting from a deficient network.
• When Participants publish multiple non-scalable video tracks, the bandwidth allocation for all of them will drop to the one of the lowest priority track. This may generate priority inversions in the allocation. For example, if a Participant is publishing a VP8 webcam with priority low and a VP8 screen-share with priority high, from the perspective of the bandwidth allocation, the screen-share will behave as having priority low. Notice that switch-offs will still preserve priority other though.