Using the Track Priority API

Overview

Not all video tracks are equal. For example, in a webinar application the screen share track is more important than the webcams, while in a video collaboration service the dominant speaker has more relevance than the rest of the participants.

To account for this, Twilio has created the Track Priority API, which makes it possible to set the priority of Tracks. The priority of a Track can be set to any of the following values:

high
standard
low

Track priorities can be understood as the relative relevance of Tracks. This means that, from the Twilio perspective, any Track marked as high will be more important than any standard Track, which in turn will be more relevant that any low priority Track.

Currently, only the Network Bandwidth Profile API consumes Track priorities to determine which Tracks are more relevant from the bandwidth allocation perspective. In the future Twilio may use Track priorities for other further purposes.

SDK Compatibility

Twilio Video SDK	Track Priority API
JavaScript	2.0.0+
Android	5.8.0+
iOS	3.4.0+

Publisher Track Priorities

The publisher Track priority is the priority given to a track by its publisher. By default, the publisher Track priority is standard. However, developers can specify a different priority at publish time. Once set, the publisher Track priority can be read both at the local and remote Track publications. Developers can also update the publisher Track priority using the setPriority primitive. In that case, all remote subscribers will be notified through an event. The following code snippets illustrate how this works:

JavaScript SDK (Required v2.0.0+)

1//You can set the publisher track priority at publish time
2const localTrackPublication = await room.localParticipant.publishTrack(localTrack, {
3  priority: 'high' //choose among 'high', 'standard' or 'low'
4});
5//The publisher Track priority can also be read.
6assert.equal(localTrackPublication.priority, 'high');
7
8//The publisher Track priority is propagated to the RemoteTrackPublication.
9remoteParticipant.on('trackPublished', remoteTrackPublication => {
10  console.log(`RemoteTrackPublication with priority ${remoteTrackPublication.publishPriority}`);
11});
12
13//You can also update the track priority of a published track
14localTrackPublication.setPriority('low')
15
16//That change will fire an event in all subscribers indicating the updated priority
17remoteTrackPublication.on('publishPriorityChanged', priority => {
18  console.log('The publisher has changed the priority this Track to "${priority}"');
19  assert.equal(remoteTrackPublication.publishPriority, priority);
20});

Android SDK (Required v5.8.0+)

1// You can set the publisher track priority at publish time
2LocalTrackPublicationOptions localTrackPublicationOptions = new LocalTrackPublicationOptions(TrackPriority.HIGH);
3localParticipant.publishTrack(localVideoTrack, localTrackPublicationOptions);
4
5 // The publish priority is also represented in remote audio, video, and data track publications.
6remoteVideoTrackPublication.getPublishPriority();
7
8// You can also update the track priority of a published track
9localVideoTrackPublication.setPriority(TrackPriority.STANDARD);
10
11// That change will result in a RemoteParticipant.Listener callback
12@Override
13public void onVideoTrackPublishPriorityChanged(
14        @NonNull final RemoteParticipant remoteParticipant,
15        @NonNull final RemoteVideoTrackPublication remoteVideoTrackPublication,
16        @NonNull final TrackPriority trackPriority) {
17    Log.d(TAG, "The publisher has changed the priority of this Track to " + trackPriority)
18}

iOS SDK (Required v3.4.0+)

1// You can set the publisher track priority at publish time
2let localVideoTrackPublicationOptions = LocalTrackPublicationOptions(priority: .high)
3localParticipant.publishVideoTrack(localVideoTrack, publicationOptions: localVideoTrackPublicationOptions);
4
5// The publish priority is also represented in remote audio, video, and data track publications.
6let publishPriority = remoteVideoTrackPublication.publishPriority
7
8// You can also update the track priority of a published track
9localVideoTrackPublication.priority = .high
10
11// That change will result in a RemoteParticipantDelegate callback
12extension ViewController : RemoteParticipantDelegate {
13    func remoteParticipantDidChangeAudioTrackPublishPriority(participant: RemoteParticipant,
14                                                             priority: Track.Priority,
15                                                             publication: RemoteAudioTrackPublication) {}
16}

Subscriber Track Priorities

As stated above, the publisher Track priority propagates to all remote subscribers. However, sometimes, there are use-cases where subscribers need to set their very specific priorities. To deal with this, we have introduced subscriber Track priorities: a mechanism that allows a subscriber to override the publisher Track priority. The following code snippets illustrate how this works:

JavaScript SDK (Required v2.0.0+)

1//You can set the publisher track priority at the publisher
2const localTrackPublication = await room.localParticipant.publishTrack(localTrack, {
3  priority: 'low' //choose among 'high', 'standard' or 'low'
4});
5
6//That priority is propagated to all subscribers
7assert.equal(remoteTrackPublication.priority, 'low');
8
9//Subscribers can now override the publisher side priority on the remote Track
10remoteTrack.setPriority('high')
11
12//This subscriber Track priority only affects that specific subscriber
13assert.equal(remoteTrack.priority,'high')
14//The overridden priority is still visible
15assert.equal(remoteTrackPublication.priority, 'low');
16
17//We can revert back to the publisher priority
18remoteTrack.setPriority(null)
19assert.equal(remoteTrack.priority, null)
20assert.equal(remoteTrackPublication.priority, 'low');

Android SDK (Required v5.8.0+)

1// You can set the publisher track priority at publish time
2LocalTrackPublicationOptions localTrackPublicationOptions = new LocalTrackPublicationOptions(TrackPriority.LOW);
3localParticipant.publishTrack(localVideoTrack, localTrackPublicationOptions);
4
5 // That priority is propagated to all subscribers
6remoteVideoTrackPublication.getPublishPriority();
7
8// Subscribers can now override the publisher side priority on the remote Track
9remoteVideoTrack.setPriority(TrackPriority.HIGH);
10
11// We can revert back to the publisher priority
12remoteVideoTrack.setPriority(null);

iOS SDK (Required v3.4.0)

1// You can set the publisher track priority at publish time
2let localVideoTrackPublicationOptions = LocalTrackPublicationOptions(priority: .low)
3localParticipant.publishVideoTrack(localVideoTrack, publicationOptions: localVideoTrackPublicationOptions);
4
5// That priority is propagated to all subscribers
6let publishPriority = remoteVideoTrackPublication.publishPriority
7
8// Subscribers can now override the publisher side priority on the remote Track
9remoteVideoTrack.priority = .high
10
11// We can revert back to the publisher priority
12remoteVideoTrack.priority = nil

Using Publisher and Subscriber Track Priorities

Developers can use publisher and subscriber Track priorities for setting the Track relevance. The difference among both are subtle but relevant:

Publisher Track priorities are designed to set the relevance of a track "for all the Room subscribers". For example, in a collaboration application a screenshare Track may be set to publisher Track priority high while all the webcams may be set to low. That would make the screen share to be (by default) much more relevant than the webcams for all the Participants.
Subscriber Track priorities are designed to modify the relevance of a track "for a specific subscriber". In the above mentioned application, a specific subscriber may be more interested on seeing the reaction of another participant to the presentation rather than on the screen share. In that case, developers can override the publisher Track priority for that specific subscriber. For example, when the end-user clicks on the webcam of her interest, the screen share can be set a subscriber Track priority low and the webcam a subscriber Track priority high. This would allow that end-user to enhance on her UI that specific webcam.

Hence, publisher Track priorities are useful for specifying the desired default relevance of tracks for an application, while subscriber Track priorities are a mechanism developers can use to customize priorities to the needs of specific end-users.