This documentation is for reference only. We are no longer onboarding new customers to Programmable Video. Existing customers can continue to use the product until December 5, 2026.
We recommend migrating your application to the API provided by our preferred video partner, Zoom. We've prepared this migration guide to assist you in minimizing any service disruption.
The Network Bandwidth Profile API makes it possible to specify how the downlink bandwidth of a Group Room Participant should be distributed among its subscribed tracks. Using this API developers can assign higher bandwidth to higher priority tracks, protect audio quality and keep the consumed network and battery resources under control.
The Network Bandwidth Profile API is only available for Group Rooms.
Twilio Group Rooms use a Media Server that is based on the SFU (Selective Forwarding Unit) architecture, where Participants commonly publish up to two video tracks (e.g. webcam and/or screen share) but may subscribe to many. The number of subscribed tracks typically grows in proportion to N-1 where N is the total number of Participants in the room. When scalable video codecs (such as VP8 Simulcast) are used, each track can be forwarded using multiple qualities. Hence, the Media Server needs to decide which quality is assigned to each track. In other words, it needs to determine how the available downlink bandwidth is allocated among those tracks. This requires addressing the following problems:
The Network Bandwidth Profile API has been created to address these issues. The following sections explain how to use it.
The Network Bandwidth Profile API is only available in Group Rooms (including Small Group Rooms). The following table illustrates current support:
Twilio Video SDK | Network Bandwidth Profile API Support (Group Rooms only) |
---|---|
JavaScript | 2.0.0+ |
Android | 5.8.0+ |
iOS | 3.4.0+ |
By default Bandwidth Profiling is disabled. This means that if you are not interested in Network Bandwidth Profiles or if you don't use the Network Bandwidth Profile API you should not experience any changes in your application behavior. For activating Bandwidth Profiling in a Participant, you just need to specify a bandwidthProfile
at connect time. Once Network Bandwidth Profiles is enabled for a Participant, it cannot be disabled.
A Network Bandwidth Profile specifies how a Participant's downlink bandwidth is consumed. Notice that it is per-Participant meaning that different Participants may have different Network Bandwidth Profiles. This also means that Bandwidth Profiling can be activated on some Participants and be inactive on others in the same Room. Developers can specify Network Bandwidth Profiles as a connect option.
JavaScript SDK (v2.0.0+)
1//Specifying a Network Bandwidth Profile at connect time2const { connect } = require('twilio-video');34const room = await connect(token,{5name: "my-new-room",6bandwidthProfile: {7video: {8mode: 'collaboration',9dominantSpeakerPriority: 'high'10}11}12});
Android SDK (v5.8.0+)
1Room room = Video.connect(context, new ConnectOptions.Builder(accessToken)2.bandwidthProfile(new BandwidthProfileOptions(new VideoBandwidthProfileOptions.Builder()3// Minimum subscribe priority of Dominant Speaker's RemoteVideoTracks4.dominantSpeakerPriority(TrackPriority.HIGH)5// Subscription mode: COLLABORATION, GRID, PRESENTATION6.mode(BandwidthProfileMode.COLLABORATION)7.build()))8.build(),9roomListener);
iOS SDK (v3.4.0+)
1let videoOptions = VideoBandwidthProfileOptions { builder in2// Minimum subscribe priority of Dominant Speaker's RemoteVideoTracks3builder.dominantSpeakerPriority = .high45// Maximum bandwidth (Kbps) to be allocated to subscribed RemoteVideoTracks6builder.maxSubscriptionBitrate = 600078// Subscription mode: collaboration, grid, presentation9builder.mode = .presentation1011// Track Switch Off mode: .detected, .predicted, .disabled12builder.trackSwitchOffMode = .predicted13}14let bandwidthProfileOptions = BandwidthProfileOptions(videoOptions: videoOptions)1516// Use bandwidth profile to build the connectOptions17let connectOptions = ConnectOptions(token: accessToken) { builder in18builder.bandwidthProfileOptions = bandwidthProfileOptions19}2021// Connect to a Room with connectOptions22room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
The following table summarizes the meaning of the different bandwidthProfile
parameters:
The main objective of the Network Bandwidth Profile API is to split the available downlink bandwidth among a Participant's subscribed video tracks. This is performed by an algorithm that determines how much bandwidth is allocated to each track as a function of the track priority and of the specified Network Bandwidth Profile. The mode
parameter controls the behavior of the algorithm. Currently, mode
can take the following values: grid
(default), collaboration
and presentation
.
Grid mode should be used in use-cases where all subscribed video tracks are equally important. When this mode is used, the available bandwidth is split uniformly among all the tracks independently on their priorities. This makes grid
mode useful for UIs displaying all video tracks in a matrix, so that none of them is enhanced over the rest.
Grid mode should also be used when scalable video codecs are not possible. In other words: if most participants in a room do not use VP8 Simulcast then, most probably, grid
mode will generate higher quality of experience than any other mode.
When a participant is configured to use grid
mode, the available downlink bandwidth is split uniformly among all the received tracks, which are treated as equals independently of their priority. Hence, when this mode is used, developers will typically use just the same priority and for all the video tracks or will set the same renderDimensions
for all the used priorities.
The following rules of thumb may help you understand how grid
mode works:
renderDimensions
and more specifically, is proportional to the display area (i.e.
width
x
height
).
Collaboration mode is for applications where some video tracks are more important than others but we still want to keep the rest of the video tracks visible. For example, in videoconferencing meetings having the typical UI layout where the dominant speaker is rendered in the central area and the rest of participants are rendered as thumbnails in a bottom row.
When a participant is configured to use collaboration mode, the available bandwidth is split in proportion to the render area (i.e. renderDimensions
) assigned to each track. Hence, more relevant tracks are going to be allocated higher bandwidth. Due to this, collaboration
mode requires Simulcast to be used by most video publishers (or at least by higher relevance publishers). In other words, unless there is a good reason for it, developers should avoid using collaboration
mode together with plain VP8 or H.264 codecs.
The following information may help you understand how collaboration
mode works:
renderDimensions
and more specifically, is proportional to the display area (i.e.
width
x
height
).
renderDimensions
so that tracks having double area will be assigned double bandwidth.
Presentation mode is for use-cases where some video tracks are critical for the end-user experience and must be preserved at any cost. For example, in webinars where a speaker shares some kind of presentation to an audience of viewers. In this case, viewers webcams should rather be switched-off if the webinar screen share quality is at risk.
When a participant is configured to use presentation mode, higher priority tracks will be assigned all the bandwidth they require so that only when all the tracks of a given priority are at their maximum the lower priority ones will be allocated. As happens with collaboration
, this mode requires most participants to publish tracks (and specifically the highest priority track) with a scalable video codec such as VP8 Simulcast.
Remark also that, if bandwidth is large enough, both collaboration
and presentation
behave equivalently and assign the maximum bandwidth to each track in proportion to its renderDimensions
. However, when it's not possible to allocate all that bandwidth both modes diverge. In that case, presentation
has the objective of keeping the highest priority video track quality, while collaboration
is designed for keeping higher priority tracks with higher quality only if the continuity of the rest of tracks is not compromised.
The following may help you understand how presentation
mode works:
renderDimensions
and more specifically, is proportional to the display area (i.e.
width
x
height
).
When a Participant's downlink bandwidth is insufficient, resulting congestion and packet loss may cause significant degradation to both audio and video quality. To avoid congestion, the Network Bandwidth Profile API algorithms monitor and estimate the available downlink bandwidth and decrease video tracks quality accordingly. However, any real-time video track has a minimum limit under which bandwidth cannot be further reduced. In the Twilio Video engine this limit is about 30Kbps for VP8 and H.264 and at approximately 60Kbps for VP8 Simulcast. Hence, if bandwidth keeps degrading, at some point all video tracks will reach that limit. Beyond that point, congestion will be severe enough that the application becomes unusable. To deal with this, we have introduced the trackSwitchOffMode
parameter. trackSwitchOffMode
allows developers to control wether less relevant video tracks should be switched off in case of congestion so that higher priority ones are protected.
Using trackSwitchOffMode: predicted
Twilio algorithms constantly estimate the available bandwidth. When predicted
is used, lower priority video tracks will be switched-off as soon as such estimation indicates there may be congestion. Hence, predicted
will act before congestion happens. Thanks to it, when predicted
is used, a downlink bandwidth drop will typically not affect subscribed audio quality or higher priority video quality. However, the drawback is that the predicted
bandwidth estimation algorithm is aggressive towards protecting higher priority tracks. This causes track switch-offs at the earliest signs of congestion. In summary: predicted
offers maximum protection of audio and higher priority video tracks at the cost of having more lower priority video track switch-offs. By default trackSwitchOffMode
is predicted
.
Using trackSwitchOffMode: detected
When detected
is used, track switch-offs will only happen after congestion actually happens. Thanks to this, detected
will avoid any unnecessary switch-offs. However, as congestion is detected through packet loss, a downlink bandwidth drop when using this mode will typically cause choppy audio and frozen video during a few seconds. We only recommend to use detected
if video continuity is very important for your use-case.
Using trackSwitchOffMode: disabled
When disabled
is used, congestion will not cause any track switch-offs. Hence, in the case of severe bandwidth drop, congestion will grow and, with it, relevant packet loss will appear causing audio to be unintelligible and video to remain frozen. In general, our recommendation is not to use this mode unless you have a very good reason to do it.
What track switching-off means
When a video track is switched-off for a Participant, Twilio's Media Server removes all the track downlink traffic. This means that, for that specific Participant, the track will consume zero bandwidth. For managing track switch-offs in your application, you may find useful the following information:
How Track switch-offs work
The algorithms that control track switch-off enforce the following rules:
mode
documentation below for further details.
high
is switched off, then it is guaranteed that all tracks with priority
standard
or
low
have been switched off previously.
standard
tracks are candidates to be switched off, the one associated to the participant who speaks less will be preferred.
After a track has been switched-off, it may eventually be switched on again. Track switch-on happens based on the following principles:
Each time a track is switched off/on for a Participant the Twilio Media Server sends a notification. Developers can subscribe to those events that are published at the RemoteVideoTrack
object, as the following code snippets illustrate:
JavaScript SDK (v2.0.0+)
1remoteTrackPublication.on('subscribed', remoteTrack => {2remoteTrack.on('switchedOff', () => {3//You may update your UI accordingly4// You can also determine whether a particular RemoteTrack is switched off.5assert.equal(remoteTrack.isSwitchedOff, true);6console.log(`The RemoteTrack ${remoteTrack.name} was switched off`);7});89remoteTrack.on('switchedOn', () => {10//You may update your UI accordingly11// You can also determine whether a particular RemoteTrack is switched off.12assert.equal(remoteTrack.isSwitchedOff, false);13console.log(`The RemoteTrack ${remoteTrack.name} was switched on`);14});15});
Android SDK (v5.8.0+)
1RemoteParticipant.Listener remoteParticipantListener = new RemoteParticipant.Listener() {2@Override3public void onVideoTrackSwitchedOff(4@NonNull RemoteParticipant remoteParticipant,5@NonNull RemoteVideoTrack remoteVideoTrack) {}67@Override8public void onVideoTrackSwitchedOn(9@NonNull RemoteParticipant remoteParticipant,10@NonNull RemoteVideoTrack remoteVideoTrack) {}11}
iOS SDK (v3.4.0+)
1extension ViewController : RemoteParticipantDelegate {2func remoteParticipantSwitchedOnVideoTrack(participant: RemoteParticipant,3track: RemoteVideoTrack) {}45func remoteParticipantSwitchedOffVideoTrack(participant: RemoteParticipant,6track: RemoteVideoTrack) {}7}8
Sometimes developers want to limit the total downlink bandwidth consumed by their applications. There may be many reasons for this:
When maxSubscriptionBitrate
is set in a Participant, that Participant video downlink will never consume more than the specified value expressed in bps. However note that it may consume less as far as the actual available network bandwidth is below.
By default, the maximum downlink bandwidth is capped to 4Mbps in Group Rooms. Hence, values of maxSubscriptionBitrate
over those limits will have no effect. Default value is:
maxSubscriptionBitrate: 0
(meaning "no limit"), in desktop browsers. Hence, desktop browsers default to the above mentioned caps.
maxSubscriptionBitrate: 2,400,000
in mobile SDKs.
It is important to remark that the use of maxSubscriptionBitrate
for setting a bandwidth constraint may generate switch-off in your video tracks. For example, if you work in presentation
mode with a high
priority video track with default renderDimensions
and set maxSubscriptionBitrate
to say 500,000, then you will be probably experiencing permanent track switch-offs for all the tracks except the main one.
The Dominant Speaker refers to the Participant having highest audio activity at a given time. You can activate dominant speaker detection following Twilio's official documentation. This feature is useful when you want to enhance the dominant speaker in your UI: for example, by rendering her video in the central area and with larger size. To do this appropriately, the Network Bandwidth Profile API should be used to set a higher priority to the dominant speaker video tracks. However, as the dominant speaker changes dynamically, it may be hard to figure out what are the tracks that should be prioritized at any time. To solve this problem the Network Bandwidth Profile API exposes the dominantSpeakerPriority
parameter. Using it, developers can set the minimum priority that should be automatically assigned to the dominant speaker video tracks.
To understand how dominantSpeakerPriority
works imagine a Group Room with dominant speaker detection activated where a Network Bandwidth Profile is defined with dominantSpeakerPriority: 'high'
. Imagine that in that Room all video tracks are set to their default priority (i.e. standard
). In that case, when Participant Alice becomes dominant speaker, automatically all Alice's video tracks will become high
priority. If later Bob becomes dominant speaker, automatically Alice's video tracks will go back to standard
, as she isn't dominant any longer, and Bob's video tracks will be upgraded to high
.
As the rest of parameters in the Network Bandwidth Profile API, dominantSpeakerPriority
works per-Participant. This means that the priority upgrades only happen on the subscriptions of the specific participant where that Network Bandwidth Profile has been defined. This also means that the dominant speaker is relative to those subscriptions. Let's illustrate that with an example. Imagine the following Room:
Imagine also that, at a given time, the speaking activity of the Participants goes in this order:
As the dominant speaker is the Participant with highest speaking activity among all subscriptions, then the following will hold for that specific instant:
In addition, please notice the following:
dominantSpeakerPriority
can only upgrade the priority of the dominant speaker but never downgrade it. For example, if a developer sets
dominantSpeakerPriority: 'standard'
and the dominant speaker has a video track published with priority
high
, that video track will stay as
high
.
dominantSpeakerPriority: 'standard'
.
dominantSpeakerPriority
will have no effect
.
There are many types of multi-party applications where the video of all participants is not rendered on screen at the same time. If a participant's video is not on screen then there is an opportunity to save CPU and bandwidth by switching off the incoming video track. When the participant needs to be displayed on screen again, the video track can be easily switched on again. Depending on developer preference this video track switch on or off can be performed automatically by the SDK or via explicit manual control.
Using clientTrackSwitchOffControl: 'auto'
With this setting the client SDK will automatically switch on or off a RemoteVideoTrack depending on the visibility of the view/document, video attachment or application lifecycle. This automatic switch on/off of the RemoteVideoTrack is managed by the SDK. This is the default setting.
Using clientTrackSwitchOffControl: 'manual'
Use this setting if the client application requires the ability to explicitly switch on or off a RemoteVideoTrack based on a desired behavior. In general switching off RemoteVideoTracks that are not currently being rendered will result in the best video quality for the tracks that are being rendered.
JavaScript SDK (v2.14.0+)
1const room = await connect(token, {2bandwidthProfile: {3video: {4clientTrackSwitchOffControl: 'manual'5}6}7});
1// Explicitly switch off a track2remoteTrack.switchOff();
Android SDK (v6.4.0+)
1BandwidthProfileOptions bandwidthProfileOptions = new BandwidthProfileOptions(2new VideoBandwidthProfileOptions.3Builder().4// Use "manual" for explicit track switch on/off5.clientTrackSwitchOffControl(ClientTrackSwitchOffControl.MANUAL)6.build());7ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)8.bandwidthProfile(bandwidthProfileOptions)9.build();1011Video.connect(context, connectOptions, roomListener);
1// the application can now request that specific RemoteVideoTracks be switched off or on, e.g.2RemoteVideoTrack.switchOn();
iOS SDK (v4.5.0+)
1let connectOptions = ConnectOptions(token: accessToken) { (builder) in2builder.bandwidthProfileOptions = BandwidthProfileOptions(3videoOptions: VideoBandwidthProfileOptions { builder in4builder.clientTrackSwitchOffControl = .manual5}6)7}89let room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
1// The application can now request that specific RemoteVideoTracks be switched off or on, e.g.2remoteVideoTrack.switchOff()
In general it is best for the Twilio Media Server to deliver the video resolution to the client that closely matches the current render dimensions for the corresponding video element or view in the UI. This ensures the most efficient use of both CPU resources on the client and the downstream bandwidth. This in turn delivers the best end user experience. Depending on developer preference this alignment of video track resolution with actual render dimensions can be performed automatically by the SDK or via explicit manual control.
Using contentPreferencesMode: 'auto'
With this setting the client SDK will automatically detect and communicate the render dimensions of each video element to the Twilio Media Server. As the UI changes and the render dimensions of each video element or view are changed, the SDK automatically detects this and the resolution of the RemoteVideoTracks are dynamically adjusted. This is the default setting.
Using contentPreferencesMode: 'manual'
Use this setting if the client application wants to explicitly communicate the preferred render dimensions for each RemoteVideoTrack and adjust them dynamically.
JavaScript SDK (v2.14.0+)
1const room = await connect(token, {2bandwidthProfile: {3video: {4contentPreferencesMode: 'manual'5}6}7});
1// Indicate that the remoteTrack should be delivered at a maxiumum of 320x240 resolution2remoteTrack.setContentPreferences({3renderDimensions: { width: 320, height: 240 }4});
Android SDK (v6.4.0+)
1BandwidthProfileOptions bandwidthProfileOptions = new BandwidthProfileOptions(2new VideoBandwidthProfileOptions.3Builder().4// Use "manual" for explicit content preferences (renderDimensions)5.videoContentPreferencesMode(VideoContentPreferencesMode.MANUAL)6.build());7ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)8.bandwidthProfile(bandwidthProfileOptions)9.build();1011Video.connect(context, connectOptions, roomListener);
1// Indicate that the remoteTrack should be delivered at a maxiumum of 320x240 resolution2remoteVideoTrack.setContentPreferences(new VideoContentPreferences(new VideoDimensions(320, 240)));
iOS SDK (v4.5.0+)
1let connectOptions = ConnectOptions(token: accessToken) { (builder) in2builder.bandwidthProfileOptions = BandwidthProfileOptions(3videoOptions: VideoBandwidthProfileOptions { builder in4builder.contentPreferencesMode = .manual5}6)7}89let room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
1// Request that the remoteTrack be delivered at 320x240 resolution2remoteVideoTrack.setContentPreferences(VideoContentPreferences { builder in3builder.renderDimensions = VideoDimensions(width: 320, height: 240)4})5
Deprecated - use clientTrackSwitchOffControl
instead
Note: setting both maxTracks
and clientTrackSwitchOffControl
is not allowed and will raise an exception.
When setting maxTracks
to N, Twilio guarantees that at any given time no more than N video tracks will be on. Hence, Twilio keeps only the N most relevant tracks for you based on the following:
maxTracks
.
Note that all the lower priority video tracks beyond the specified maxTracks
will be switched off. Hence, given that priorities may change dynamically, your application will receive track switch-off/on notifications to indicate which tracks are visible at any given time.
Note also that maxTracks
related switch-offs are not due to congestion and are not affected by the trackSwitchOffMode
parameter. Therefore, even if you set trackSwitchOffMode: disabled
, you may still have track switch-offs if a Participant subscribes to more than maxTracks
video tracks.
Just for illustration, if you have a videoconferencing application with:
high
.
standard
.
maxTracks: 5
.
Then, the algorithm will only allow the following 5 tracks to be on:
Remark also that maxTracks: 0
should be read as "unlimited" meaning that Twilio will try to send all the subscribed tracks to the Participant. Note also that, by default, maxTracks: 0
.
While maxTracks
is "unlimited" by default the following are the recommended maximum values based on Room type:
Small Group Room | (Regular) Group Room | |
---|---|---|
MaxTracks | 8 | 16 |
These recommendations are highly dependent on the bitrate associated with each subscribed video track. For optimal results with 2 or more visible tracks you should ensure that you specify a low track priority for nearly all video tracks and that your renderDimensions for the low priority tracks are no more than 60p to 180p per track.
Deprecated - use contentPreferencesMode
instead
Note: Setting both renderDimensions
and contentPreferencesMode
in the bandwidth profile is not allowed and will raise an exception.
Video bandwidth depends on resolution. For example, if a track is to be rendered in FullHD (1920x1080) it may make sense to reserve 4Mbps for it. However, for a thumbnail with CIF (352x240) size that amount would not be justified.
Using renderDimensions
developers can specify the display size, in pixels, they plan to use to render video tracks on a Participant's UI. Thanks to this, Twilio can determine what's the maximum bandwidth that should be allocated to each video track. This means that, even if the available bandwidth is in excess, our algorithms will never allocate more than what is needed to render the track with he appropriate specified size. This has multiple positive side effects:
In the Network Bandwidth Profile API, renderDimensions
are specified per priority:
high
: Specifies the render width
and height
(in pixels) of high priority tracks. Defaults to HD resolution:
1high: {2width: 1280,3height: 7204}
standard
: Specifies the render width
and height
(in pixels) of standard priority tracks. Defaults to VGA resolution:
1standard: {2width: 640,3height: 4804}
low
: Specifies the render width
and heigh
(in pixels) of low priority tracks. Defaults to QCIF resolution:
1low: {2width: 176,3heigh: 1444}
It is important to remark that, from the practical perspective, our algorithms use renderDimensions
to compute the maximum video Track bandwidth. This has multiple implications you should be aware of:
renderDimensions
must be understood as a hint to the desired actual rendering dimensions of video Tracks. That means that
renderDimensions
don't need to be exact. In other words, you don't need to invest much of your developing time figuring out how to update your
renderDimensions
every time the display size changes. You must concentrate instead on providing approximate hints that keep the actual proportions of your UI. That means that if a video track A is to be rendered with double size than video track B, then A's priority should have double
renderDimensions
than B's.
renderDimensions
don't need to match with the actual video track dimensions. The video track dimensions will depend on aspects such as your capture constraints and the status of the network. Remember that
renderDimensions
are just a hint that you provide to help the bandwidth allocation algorithms and not a specification on actual video width and height.
renderDimensions
so that the resulting area (
width
x
height
) is the average of all the render areas of the tracks on that priority.
As stated above, collaboration
and presentation
modes are only recommended when scalable video codecs such as VP8 Simulcast are used. Using them together with plain VP8 and H.264 will sometimes generate undesired behaviors caused by the interdependencies that appear among the different track subscribers. This may lead to problems like the following:
low
and a VP8 screen-share with priority
high
, from the perspective of the bandwidth allocation, the screen-share will behave as having priority
low
. Notice that switch-offs will still preserve priority other though.