Understanding Video Rooms APIs

The Rooms REST API is based on the following REST resources and dependencies:

Expand image

A Room represents a virtual space where end-users communicate. Technically, a Room is a computing resource that provides Real-time Communications (RTC) services to client applications through a set of APIs. More specifically, a Room provides:

• A session: so that end-users can connect/disconnect to/from Rooms. When an end-user connects, we say it is a Room Participant.
• An RTC Service: so that Participants can exchange audio, video, and data tracks using WebRTC.

In Twilio, a Room is identified by a SID: a string that uniquely identifies that Room inside Twilio. A Room SID always start with the characters RM and looks like this: RMdba2b037253f3a9a81a591681e88ce96.

Twilio Rooms REST API allows developers to:

• Create new Rooms (sending a POST request to the /Rooms REST resource)
• List all the Rooms in an Account (sending a GET request to the /Rooms REST resource)
• Manage a given Room (sending a POST request to modify or a GET request for information to the /Rooms/{RoomSid} resource)

You can review the Understanding Video Rooms Guide and the Rooms REST API Reference Documentation for more information about Rooms.

A Participant represents a client (i.e. an end-user) that is connected to a Room and can use the Room's communication capabilities. In a given Room there can be zero (i.e. an empty Room) or more Participants. A Participant can be connected only to one Room.

In Twilio, a Participant is uniquely identified by a SID with a PA prefix like the following: PA734b33d8bde2a3b3c7720964c87961c9.

Twilio Rooms REST API allows developers to:

• List all the Participants in a Room (sending a GET request to the /Participants subresource)
• Manage a given Participant (sending a POST request to modify or a GET request for information to the /Participants/{ParticipantSid} subresource)

A Track is a stream of bytes that contain the data generated by a multimedia source such as a microphone or a camera. Twilio Rooms are based on a publish/subscribe model. This means that a Participant can publish media tracks to the Room. The rest of the Room's Participants can then subscribe to these tracks and start receiving the media information.

In Twilio, a Track is uniquely identified by a SID with the MT prefix like the following: MT6e87a66bd033cdced4efe1267c595a16.

For a given Participant, the Twilio Rooms REST API allows developers to:

• List all the Tracks published by the Participant (sending a GET request to the /PublishedTracks subresource of the Participant). In these Tracks, the media bytes are sent by the Participant.
• List all the Tracks subscribed by the Participant (sending a GET request to the /SubscribedTracks subresource of the Participant). In these Tracks, the media bytes are received by the Participant.

A SubscribeRule is a specification of the Tracks a given Participant should subscribe to. For a given Participant, the Twilio Rooms REST API allows developers to:

• Modify the Tracks a Participant is subscribed to (sending a POST request to the /SubscribeRules subresource of the Participant)
• List the SubscribeRules of a Participant (sending a GET request to the /SubscribeRules subresource of the Participant)

Developers can use the Rooms REST API directly firing the HTTP request to the appropriate endpoints. As an alternative, Twilio has created server-side helper libraries that expose all our REST APIs capabilities in a consistent and seamless way.

Status Callbacks allow you to receive events related to the REST resources managed by Twilio. In order to use Status Callbacks you need to configure your Application Server to be able to receive HTTP requests issued by Twilio. Status callbacks events will notify your application when relevant events occur to your Rooms, Participants and Tracks indicating the appropriate SIDs to identify the affected resources. Check the Programmable Video Status Callbacks Guide for further information.

Video Rooms SDKs are available on three different platforms: JavaScript, Android, and iOS. These SDKs share similar type hierarchies based on the following classes: Room, Participant, TrackPublication, and Track. The following UML diagram shows their relationship.

Expand image

This type represent Rooms from the perspective of the client. When a client connects to a Twilio Room it obtains a Room object instance. The Room class has primitives and events that developers can use to:

• Get the Room name and SID
• Get notified when participants connect or disconnect
• Disconnect from the Room.

This snippet shows how to connect to a Room using the JavaScript SDK:

1
const {connect} = require('twilio-video');
2
connect('$TOKEN', { name: 'my-room' }).then(room => {
3
  console.log('I connected to Room ${room.name} that has SID ${room.sid}')
4
}, error =>{
5
  console.error('Unable to connect to Room: ${error.message}')
6
})

This type represents a Participant from the perspective of the client. A Room can have zero or more Participants.

A TrackPublication represents the communication of a media Track from a Participant to the Room. A Participant can have zero or more TrackPublications.

This type represents a Track from the perspective of the client.

Our client SDK APIs are designed in "first person". This means that developers should think about them from the perspective of the client where the code is running. From that perspective, there are two types of Participants as illustrated in the following diagram:

Expand image

The LocalParticipant is the Participant on "this" client (i.e. on the client the code is running). The LocalParticipant can access the client hardware resources and manages the status and preferences of the client with respect to the Room. Developers can obtain a reference to the LocalParticipant from the Room instance, as the following code illustrates:

1
localParticipant = room.localParticipant
2
console.log('The LocalParticipant identity is ${localparticipant.identity}')
3
console.log('The LocalParticipant SID is ${localParticipant.sid}')

RemoteParticipants represent all other Participants in the Room that are not on this specific client. RemoteParticipant objects can be seen as stubs that represent remote client status and capabilities on "this" client. RemoteParticipants can be obtained from the Room, as the following code illustrates:

1
//List RemoteParticipants connected to the Room
2
room.participants.forEach(remoteParticipant => {
3
  console.log('RemoteParticipant ${remoteParticipant.identity} is connected')
4
  console.log('RemoteParticipant SID is ${remoteParticipant.sid}')
5
});
6

7
//Get notified when a RemoteParticipant connects to the Room
8
room.on('participantConnected', remoteParticipant => {
9
  console.log('RemoteParticipant ${remoteParticipant.identity} just connected')
10
  console.log('RemoteParticipant SID is ${remoteParticipant.sid}')
11
});

Our SDKs support three Track subtypes:

• AudioTrack: It represents a stream of bytes being generated by an audio source such as a microphone.
• VideoTrack: It represents a stream of bytes being generated by a video source such as a webcam or a screen.
• DataTrack: It represents a stream of bytes (or characters) that are generated by the application. In other words, developers can use a DataTrack to communicate arbitrary data among participants with very low latency. Notice that all Tracks being published by a Participant share the same underlying transport. Hence, the latency properties of the DataTrack bytes published by a Participant should be the same than the latency properties of the audio and video information published by the Participant.

Expand image

Following the above mentioned "first person" API mode, each Track subtype has two versions:

LocalAudioTrack, LocalVideoTrack and LocalDataTrack are the local Track subtypes. Be aware that:

• Instances of these types can only be created at the local SDK.
• Local Tracks do not have a SID because they are not known by Twilio (i.e. they are local to the SDK they have been created in). Tracks are only assigned a SID when they are published.
• Local Tracks have an ID that identifies them locally (i.e. the ID of a published Local Track does not match with the ID of the subscribed Remote Track).
• Local Tracks can be assigned a name at creation time.
• LocalVideoTrack can be attached to the local UI to create a camera preview.

1
const { createLocalVideoTrack, createLocalAudioTrack } = require('twilio-video');
2
createLocalAudioTrack({name:'john-audio-track').then(localAudioTrack => {
3
  console.log('Created LocalAudioTrack with id ${localAudioTrack.id}')
4
})
5
createLocalVideoTrack({name:'alices-webcam'}).then(localVideoTrack => {
6
  console.log('Created LocalVideoTrack with id ${localVideoTrack.id}')
7
  const localMediaContainer = document.getElementById('local-media');
8
localMediaContainer.appendChild(localVideoTrack.attach());
9
})

RemoteAudioTrack, RemoteVideoTrack and RemoteDataTrack are the remote Track subtypes.

• Instances of these types can only exists at the RemoteParticipants that are subscribed to them.
• A Local Track published at a given Participant will be seen as a Remote Track by the rest of subscribed Participants.
• Remote Tracks always have a SID that matches with the SID assigned to the corresponding published Track
• Remote Tracks have a name that matches with the name assigned to the corresponding published Track.
• Remote Tracks are held by the owning RemoteParticipant

1
remoteParticipant.on('trackSubscribed', remoteTrack => {
2
  console.log('Received a ${remoteTrack.kind} track with SID ${remoteTrack.sid}')
3
  document.getElementById('remote-media-div').appendChild(track.attach());
4
});

As with Track, TrackPublication also has three media-dependent subtypes:

• AudioTrackPublication: it represents the publication of an AudioTrack.
• VideoTrackPublication: it represents the publication of a VideoTrack.
• DataTrackPublication: it represents the publication of DataTrack.

Expand image

There are also local and remote subtypes for them:

LocalAudioTrackPublication, LocalVideoTrackPublication and LocalDataTrackPublication represent local Tracks that have been published to the Room by the LocalParticipant. When a Track is published to the Room it gets a SID assigned. That SID can be recovered at the Local TrackPublication object, as the following snippet shows:

1
localParticipant.publishTrack(localTrack).then(localTrackPublication => {
2
  console.log('Track ${localTrack.name} was published with SID ${localTrackPublication.tracksid}'
3
})

The LocalParticipant keeps track of all its publications:

1
localParticipant.tracks.forEach(localTrackPublication => {
2
  console.log('LocalTrackPublication with kind=${localTrackPublication.kind}')
3
})

RemoteAudioTrackPublication, RemoteVideoTrackPublication and RemoteDataTrackPublication are stubs that represent the local TrackPublications of other participants. In other words, a Local TrackPublication on a given Participant will be seen as a Remote TrackPublication on the rest of Participants.

1
remoteParticipant.on('trackPublished', remoteTrackPublication => {
2
  console.log('A track with sid ${remoteTrackPublication.tracksid} was published to the Room')
3
})

To publish a Track developers need to use the publish primitive of the LocalParticipant. When doing so, a new local TrackPublication is created for the specified local Track. Local TrackPublications are owned by the LocalParticipant. Remark also that a Local TrackPublication always have associated one LocalTrack (i.e. the LocalTrack being published) as shown on the following UML diagram:

Expand image

Every Local Track Publication at a LocalParticipant will be seen as a Remote Track Publication at the rest of RemoteParticipants. As a consequence, remote TrackPublication subtypes map to Local TrackPublications.

Note that a Local TrackPublication is always associated to a LocalTrack (i.e. the Track being published). However, Remote TrackPublications behave differently:

• A Remote TrackPublication has zero Remote Tracks when "this" client is not subscribed to the such published track.
• A Remote TrackPublication has one Remote Track when "this" client is subscribed to such published Track.

Expand image

Let's use an example to illustrate our API Local and Remote Perspectives. Imagine you create a Room using our Room REST API. Also imagine your code creates and distributes the appropriate Access Tokens among participants. In this situation, suppose Alice (client A) and Bob (client B) connect doing the following

• Alice joins and gets assigned PA_A as her participant SID.
• Alice publishes a LocalAudioTrack, a LocalVideoTrack and a LocalDataTrack that are assigned SIDs MT_A_A, MT_A_V, and MT_A_D respectively.
• Alice subscribes to all Tracks.
• Bob joins and gets assigned PA_B as his participant SID.
• Bob publishes a LocalAudioTrack, a LocalVideoTrack and a LocalDataTrack that are assigned SIDs MT_B_A, MT_B_V and MT_B_D respectively.
• Bob subscribes to all Alice's Audio and Data Tracks, but not to her video Tracks.

In this case, the situation will look like the following

Expand image

From Alice's perspective (as code executes on Alice's SDK):

• The LocalParticipant SID is PA_A.

• The LocalParticipant has three LocalTrackPublications:

  • A LocalAudioTrackPublication with tracksid=MT_A_A. The associated track is a LocalAudioTrack that captures Alice's microphone.
  • A LocalVideoTrackPublication with tracksid=MT_A_V. The associated track is a LocalVideoTrackPublication that captures Alice's webcam.
  • A LocalDataTrackPublication with tracksid=MT_A_D. The associated track is a LocalDataTrack where Alice can write her data blobs.

• There is one RemoteParticipant with SID PA_B.

• There are three RemoteTrackPublications:

  • A RemoteAudioTrackPublication with tracksid=MT_B_A. As Alice is subscribed to this publication, it has a RemoteAudioTrack instance associated that Alice can use to reproduce Bob's audio.
  • A RemoteVideoTrackPublication with tracksid=MT_B_V. As Alice is subscribed to this publication, it has a RemoteVideoTrack instance associated that Alice can use to display Bob's video.
  • A RemoteDataTrackPublication with tracksid=MT_B_D. As Alice is subscribed to this publication, it has a RemoteDataTrack instance associated that Alice can use to read Bob's data blobs.

From Bob's perspective (as code executes on Bob's SDK):

• The LocalParticipant SID is PA_B.

• The LocalParticipant has three LocalTrackPublications:

  • A LocalAudioTrackPublication with tracksid=MT_B_A. The associated track is a LocalAudioTrack that captures Bob's microphone.
  • A LocalVideoTrackPublication with tracksid=MT_B_V. The associated track is a LocalVideoTrack that captures Bob's webcam.
  • A LocalDataTrakPublication with tracksid=MT_B_D. The associated track is a LocalDataTrack where Bob can write his data blobs.

• There is one RemoteParticipant with SID PA_A.

• There are three RemoteTrackPublications:

  • A RemoteAudioTrackPublication with tracksid=MT_A_A. As Bob is subscribed to this publication, it has a RemoteTrack instance associated that Alice can use to reproduce Alice's audio.
  • A RemoteVideoTrackPublication with tracksid=MT_A_V. As Bob is not subscribed to this publication, it does not have any associated RemoteVideoTrack instance and cannot display Alice's video.. If Bob got subscribed to this publication, the publication would get that association and would fire a trackSubscribed event to notify it.
  • A RemoteDataTrackPublication with trackSid=MT_A_D. As Bob is subscribed to this publication, it has a RemoteDataTrack instance associated where Bob can read Alice's data blobs.