This guide provides you with an overview of how to build your video application with the Twilio Programmable Video Android SDK.
If you haven't already done so, take a look at the open source video collaboration app and quickstart apps. Then come back to this guide for more detail on how to add video to your own app.
Let's start with an overview of the Programmable Video API:
Room
represents a real-time audio, data, video, and/or screen-share session, and is the basic building block for a Programmable Video application.Participants
represent client applications that are connected to a Room and sharing audio, data, and/or video media with one another.Tracks
represent the individual audio, data, and video media streams that are shared within a Room.LocalTracks
represent the audio, data, and video captured from the local client's media sources (for example, microphone and camera).RemoteTracks
represent the audio, data, and video tracks from other participants connected to the Room.The following code samples illustrate common tasks that you as a developer may wish to perform related to a Room and its Participants.
To start using the Android Programmable Video SDK in your apps, you need to perform a few basic tasks first.
The Android Video SDK is distributed through Maven Central.
To install the Android Video SDK, ensure the following configuration is in your build.gradle file:
Add the following lines to your build.gradle file.
1allprojects {2repositories {3mavenCentral()4}5}67// The Video library resides on Maven Central8implementation 'com.twilio:video-android:7.5.1'910android {11compileOptions {12sourceCompatibility 1.813targetCompatibility 1.814}15}
Add the following lines to your proguard-project.txt file.
1-keep class tvi.webrtc.** { *; }2-keep class com.twilio.video.** { *; }3-keepattributes InnerClasses
The Android SDK supports Android API level 21 and higher. It is built for armeabi-v7a, arm64-v8a, x86, and x86_64 architectures.
API Keys represent credentials to access the Twilio API. You use them to:
Follow the instructions in the API Keys Overview doc to create a new API Key for your project.
To execute the code samples below, you'll need to generate an Access Token. An Access Token is a short-lived credential used to authenticate your client-side application to Twilio.
You can generate an Access Token using either the Twilio CLI or a Twilio Helper Library.
For application testing purposes, the Twilio CLI provides a quick way to generate Access Tokens that you can then copy/paste into your application. In a production application, you should use the Twilio Helper Libraries because your back-end server will need to generate an Access Token for every user in your application.
To use the CLI, you will need to install the Twilio CLI and log in to your Twilio account from the command line; see the CLI Quickstart for instructions. Then, you can install the Token CLI plugin with the following command:
twilio plugins:install @twilio-labs/plugin-token
To generate an Access Token, run the following command. --identity
is a required argument and should be a string that represents the user identity for this Access Token.
twilio token:video --identity=<identity>
You can find examples of how to generate an Access Token for a participant using Twilio Helper Libraries in the User Identity and Access Token guide.
Call Video.connect()
to connect to a Room from your Android application. Once connected, you can send and receive audio and video streams with other Participants who are connected to the Room.
1private Room.Listener roomListener() {2return new Room.Listener() {3@Override4public void onConnected(Room room) {5Log.d(TAG,"Connected to " + room.getName());6}7}8}910public void connectToRoom(String roomName) {11ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)12.roomName(roomName)13.audioTracks(localAudioTracks)14.videoTracks(localVideoTracks)15.dataTracks(localDataTracks)16.build();17room = Video.connect(context, connectOptions, roomListener);18}
You must pass the Access Token when connecting to a Room. You may also optionally pass the following:
The name of the Room specifies which Room you wish to join. If you have enabled client-side Room creation for your Account and a Room by that name doesn't already exist, it will be created upon connection. If a Room by that name is already active, you'll be connected to the Room and receive notifications from any other Participants also connected to the same Room. Room names must be unique within an Account.
If you have enabled client-side Room creation, or ad hoc Room creation, any new Room you create via the Android SDK will follow the default Room settings that you've specified in your Account. These settings include options like a StatusCallback
URL where you can receive Room creation and other webhook events, the maximum number of Participants, automatic recording, and more. You can view and update your default Room settings in the Twilio Console.
You can also create a Room using the Rooms REST API. Look at the REST API Rooms resource docs for more details.
If you'd like to join a Room you know already exists, you handle that the same way as creating a room: pass the Room name to the connect
method.
Once in a Room, you'll receive a participantConnected
event for each Participant that successfully joins. Querying the participants
getter will return any existing Participants who have already joined the Room.
1private Room.Listener roomListener() {2return new Room.Listener() {3@Override4public void onConnected(Room room) {5Log.d(TAG,"Connected to " + room.getName());6}7}8}910public void connectToRoom(String roomName) {11ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)12.roomName(roomName)13.audioTracks(localAudioTracks)14.videoTracks(localVideoTracks)15.dataTracks(localDataTracks)16.build();17room = Video.connect(context, connectOptions, roomListener);18}
You can capture local media from your device's microphone, camera or screen-share on different platforms in the following ways:
In an Android application, begin capturing audio data by creating a LocalAudioTrack
, and begin capturing video by adding a LocalVideoTrack
with an associated VideoCapturer
. The Android Video SDK provides customizable video capturers for both camera and screen capture.
1// Create an audio track2boolean enable = true;3LocalAudioTrack localAudioTrack = LocalAudioTrack.create(context, enable);45// A video track requires an implementation of a VideoCapturer. Here's how to use the front camera with a Camera2Capturer.6Camera2Enumerator camera2Enumerator = new Camera2Enumerator(context);7String frontCameraId = null;8for (String cameraId : camera2Enumerator.getDeviceNames()) {9if (camera2Enumerator.isFrontFacing(cameraId)) {10frontCameraId = cameraId;11break;12}13}14if(frontCameraId != null) {15// Create the CameraCapturer with the front camera16CameraCapturer cameraCapturer = new Camera2Capturer(context, frontCameraId);1718// Create a video track19LocalVideoTrack localVideoTrack = LocalVideoTrack.create(context, enable, cameraCapturer);2021// Rendering a local video track requires an implementation of VideoSink22// Let's assume we have added a VideoView in our view hierarchy23VideoView videoView = (VideoView) findViewById(R.id.video_view);2425// Render a local video track to preview your camera26localVideoTrack.addSink(videoView);2728// Release the audio track to free native memory resources29localAudioTrack.release();3031// Release the video track to free native memory resources32localVideoTrack.release();33}
For some use cases you may wish to connect as a publish-only Participant that is not subscribed to any Tracks. You can disable automatic subscription behavior via ConnectOptions
.
1public void connectToRoom(String roomName) {2ConnectOptions connectOptions = new ConnectOptions.Builder(accessToken)3.roomName("my-room")4.enableAutomaticSubscription(false)5.build();6room = Video.connect(context, connectOptions, roomListener);7}
When you join a Room, Participants may already be present. You can check for existing Participants in the connected
event callback by using the participants
getter.
1// Connect to room2Room room = Video.connect(context, connectOptions, new Room.Listener() {3@Override4public void onConnected(Room room) {}56@Override7public void onConnectFailure(Room room, TwilioException e) {}89@Override10public void onDisconnected(Room room, TwilioException e) {}1112@Override13public void onRecordingStarted(Room room) {}1415@Override16public void onRecordingStopped(Room room) {}1718@Override19public void onParticipantConnected(Room room, RemoteParticipant participant) {20Log.i("Room.Listener", participant.getIdentity() + " has joined the room.");21}2223@Override24public void onParticipantDisconnected(Room room, RemoteParticipant participant) {25Log.i("Room.Listener", participant.getIdentity() + " has left the room.");26}27);2829// ... Assume we have received the connected callback3031// After receiving the connected callback the LocalParticipant becomes available32LocalParticipant localParticipant = room.getLocalParticipant();33Log.i("LocalParticipant ", localParticipant.getIdentity());3435// Get the first participant from the room36RemoteParticipant participant = room.getRemoteParticipants().get(0);37Log.i("HandleParticipants", participant.getIdentity() + " is in the room.");38
When Participants connect to or disconnect from a Room that you're connected to, you'll be notified via an event listener: Similar to Room Events, Twilio will fire Participant events if the StatusCallback webhook URL is set when the Room is created. These events help your application keep track of the participants who join or leave a Room.
1private Room.Listener roomListener() {2return new Room.Listener() {34@Override5public void onParticipantConnected(Room room, RemoteParticipant participant) {6Log.v(TAG, "Participant connected: " + participant.getIdentity());7}89@Override10public void onParticipantDisconnected(Room room, RemoteParticipant participant) {11Log.v(TAG, "Participant disconnected: " + participant.getIdentity());12}13};14}
To see the Video Tracks being sent by remote Participants, we need to render them to the screen:
1// First, we set a Media Listener when a Participant first connects:2private Room.Listener roomListener() {3return new Room.Listener() {4@Override5public void onParticipantConnected(Room room, RemoteParticipant participant) {6participant.setListener(remoteParticipantListener());7}8};9}1011/* In the Participant listener, we can respond when the Participant adds a Video12Track by rendering it on screen: */13private RemoteParticipant.Listener remoteParticipantListener() {14return new RemoteParticipant.Listener() {15@Override16public void onVideoTrackSubscribed(RemoteParticipant participant,17RemoteVideoTrackPublication remoteVideoTrackPublication,18RemoteVideoTrack remoteVideoTrack) {19primaryVideoView.setMirror(false);20remoteVideoTrack.addSink(primaryVideoView);21}22};23}
Sometimes you need to make sure you're looking fantastic before entering a Room. We get it. Each SDK provides a means to render a local camera preview outside the context of an active Room:
1/* The CameraCapturer is a default video capturer provided by Twilio which can2capture video from the front or rear-facing device camera */3private CameraCapturer cameraCapturer;45/* A VideoView receives frames from a local or remote video track and renders them6to an associated view. */7private VideoView primaryVideoView;89// Start the camera preview10LocalVideoTrack localVideoTrack = LocalVideoTrack.create(context, true, cameraCapturer);11primaryVideoView.setMirror(true);12localVideoTrack.addSink(primaryVideoView);1314// Release the local video track to free native memory resources once you are done15localVideoTrack.release();
Note: See this snippet to see how to initialize a CameraCapturer.
You can disconnect from a Room you're currently participating in. Other Participants will receive a participantDisconnected
event.
1// To disconnect from a Room, we call:2room.disconnect();34// This results in a call to Room.Listener#onDisconnected5private Room.Listener roomListener() {6return new Room.Listener() {7@Override8public void onDisconnected(Room room, TwilioException e) {9Log.d(TAG,"Disconnected from " + room.getName());10}11};12}
The Video SDK will raise notifications when a Room is reconnecting due to a network disruption. A Room reconnection is triggered due to a signaling or media reconnection event.
1private Room.Listener roomListener() {2return new Room.Listener() {34/*5* Exception will be either TwilioException.SIGNALING_CONNECTION_DISCONNECTED_EXCEPTION or6* TwilioException.MEDIA_CONNECTION_ERROR_EXCEPTION7*/8@Override9public void onReconnecting(Room room, TwilioException exception) {10Log.v(TAG, "Reconnecting to room: " + room.getName() + ", exception = " + exception.getMessage());11}1213@Override14public void onReconnected(Room room) {15Log.v(TAG, "Reconnected to room " + room.getName());16}17};18}
The Programmable Video REST API
allows you to control your video applications from your back-end server via HTTP requests. To learn more check out the Programmable Video REST API docs.
If you are experiencing echo on Android, attempt the following changes prior to making a support ticket. Please reference the following snippets and all of the recommended API calls. You can also find this information in the troubleshooting audio section of the Android Quickstart README.
Use software echo cancellation
WebRtcAudioUtils.setWebRtcBasedAcousticEchoCanceler(true);
Use hardware noise suppression
WebRtcAudioUtils.setWebRtcBasedNoiseSuppressor(false);
Use hardware automatic gain control
WebRtcAudioUtils.setWebRtcBasedAutomaticGainControl(false);
We love feedback and questions especially those with helpful debugging information so we can diagnose and respond quickly. When submitting issues or support tickets, it would be great if you add the following:
After gathering the above information, you can get help in a few ways:
To enable debug level logging, add the following code in your application:
1/*2* Set the log level of the Video Android SDK3*/4Video.setLogLevel(LogLevel.DEBUG);56/*7* If your application is experiencing an issue related to a specific8* module, you can set the log level of each of the following modules.9*/10Video.setModuleLogLevel(LogModule.CORE, LogLevel.DEBUG);11Video.setModuleLogLevel(LogModule.PLATFORM, LogLevel.DEBUG);12Video.setModuleLogLevel(LogModule.SIGNALING, LogLevel.DEBUG);13Video.setModuleLogLevel(LogModule.WEBRTC, LogLevel.DEBUG);