This is the second part of the tutorial for getting started with Twilio Video using Python/Flask and JavaScript. In this part, you'll create a JavaScript frontend that can connect to the backend Flask web server you created in Part One. If you haven't already created a backend server, refer back to the server code and setup in Part One, and then come back to complete this section.
You can also view the completed code from both parts of the tutorial in GitHub.
Before beginning this tutorial, you should ensure you are using one of the supported browsers for the Twilio Video JavaScript SDK.
In the project's root directory (video_tutorial
or whatever you named the project's main directory), create a templates
directory. This is where you'll store your HTML files. To make the new directory, run the following command in your terminal:
mkdir templates
In this application, you'll only need one HTML file. Create a new file called index.html
in the templates
directory. This file will contain the HTML structure for the video application, which should include:
<div>
where Participants' videos will be displayed after they've joined.main.js
file, which is where you'll write the code for controlling the video application.Add the following code to templates/index.html
:
1<!DOCTYPE html>2<html>3<head>4<meta charset="utf-8" />5<title>Twilio Video Demo</title>6{/* Twilio Video CDN */}7<script src="https://sdk.twilio.com/js/video/releases/2.15.2/twilio-video.min.js"></script>8</head>9<body>10<form id="room-name-form">11Enter a Room Name to join: <input name="room_name" id="room-name-input" />12<button type="submit">Join Room</button>13</form>14<div id="video-container"></div>15<script src="static/main.js"></script>16</body>17</html>
Note that this index.html
file links to version 2.15.2 of the Twilio Video JavaScript SDK. You can find the CDN link for the most recent version of the Twilio Video JavaScript SDK here.
Next, you'll want to have your Flask server render this index.html
template when someone goes to your web app, instead of showing the string "In progress!" as it does right now. In your app.py
file, change the line that returns "In progress!" to return render_template("index.html")
.
1@app.route("/")2def serve_homepage():3return render_template("index.html")
If your server is not running, make sure you are in your virtual environment by running source venv/bin/activate
(MacOS/Unix) or venv\Scripts\activate.bat
(Windows). Then, start up the server by running the following command:
python app.py
After the server starts, navigate to localhost:5000
in your browser. You should see the form with an input box and a submit button.
In index.html
, you linked to a JavaScript file that doesn't exist yet. In this step, you'll create it and start adding code.
In a second terminal window, navigate to your main project directory, and create a new directory called static
in the root directory of the project:
mkdir static
Then, create a file called main.js
within the static
directory, and open that file in a text editor.
First, you'll create variables for the form
, video container div
, and input
HTML elements, to identify them later. Copy and paste the following code into the static/main.js
file:
1const form = document.getElementById("room-name-form");2const roomNameInput = document.getElementById("room-name-input");3const container = document.getElementById("video-container");
Next, you'll create a function called startRoom
that you'll call when the form is submitted. Copy and paste the following code into the static/main.js
file, under the variable declarations:
1const startRoom = async (event) => {2// prevent a page reload when a user submits the form3event.preventDefault();4// hide the join form5form.style.visibility = "hidden";6// retrieve the room name7const roomName = roomNameInput.value;89// fetch an Access Token from the join-room route10const response = await fetch("/join-room", {11method: "POST",12headers: {13Accept: "application/json",14"Content-Type": "application/json",15},16body: JSON.stringify({ room_name: roomName }),17});18const { token } = await response.json();19console.log(token);20};
The startRoom
function hides the Join Room form. Then it submits a request to the /join-room
route with the value the user entered in room-name-input
. The /join-room
route either finds an existing Video room with that name or creates one, and gives back an Access Token that this participant can use to join the room.
For now, the code just retrieves the Access Token and prints the token to the console with console.log
. In the next step, you'll use the token to join the video room.
At the bottom of the main.js
file, below the startRoom
function, add an event listener on the form so that when it's submitted, startRoom
runs:
form.addEventListener("submit", startRoom);
You can now navigate to localhost:5000
in your browser, enter a room name in the form, click "Join Room", and see the Access Token in the browser's console. (You'll need to open the Developer Tools for your browser to see the console.)
Here's the full code for static/main.js
so far:
1const form = document.getElementById("room-name-form");2const roomNameInput = document.getElementById("room-name-input");3const container = document.getElementById("video-container");45const startRoom = async (event) => {6// prevent a page reload when a user submits the form7event.preventDefault();8// hide the join form9form.style.visibility = "hidden";10// retrieve the room name11const roomName = roomNameInput.value;1213// fetch an Access Token from the join-room route14const response = await fetch("/join-room", {15method: "POST",16headers: {17Accept: "application/json",18"Content-Type": "application/json",19},20body: JSON.stringify({ room_name: roomName }),21});22const { token } = await response.json();23console.log(token);24};2526form.addEventListener("submit", startRoom);
This tutorial is using WebRTC Go rooms, which are free rooms that allow a maximum of two participants in the room at one time. If you request Access Tokens for more than the maximum number of participants in one room, you'll receive an error that there are too many participants in the room. This tutorial doesn't cover error handling in these cases — for now, if you run into this error, you can try joining a new room with a different name. You'll see the error appear in your browser's console if it happens.
Now that your application can retrieve an Access Token, you can use that token to join the video room in the browser. The Twilio Video JavaScript SDK has a connect
method that you can call to connect to a Twilio Video room.
Copy and paste the following code underneath the startRoom
function in static/main.js
.
1const joinVideoRoom = async (roomName, token) => {2// join the video room with the Access Token and the given room name3const room = await Twilio.Video.connect(token, {4room: roomName,5});6return room;7};
This creates a function called joinVideoRoom
, which passes the Access Token and an object of ConnectOptions
to the Twilio video connect
method. The only ConnectOption
you pass in is the name of the room that you're connecting to.
The connect
method returns a Promise that will eventually either resolve to a Room object or be rejected with an error.
Then, call the joinVideoRoom
function after you've retrieved the Access Token in the startRoom
function. For now, you can console.log
the video room object. In the next step, you'll start adding code to display participants' video streams.
Update the startRoom
function with the following lines of code. Note that you're removing the console.log(token)
line and replacing it with a call to the joinVideoRoom
function:
1// fetch an Access Token from the join-room route2const response = await fetch("/join-room", {3method: "POST",4headers: {5Accept: "application/json",6"Content-Type": "application/json",7},8body: JSON.stringify({ room_name: roomName }),9});10const { token } = await response.json();1112// join the video room with the token13const room = await joinVideoRoom(roomName, token);14console.log(room);15};
Now if you navigate to localhost:5000
in your browser, enter a room name, and click "Join Room", you'll connect to a Twilio Video room. If you open your browser's console, you should see a log line with the room object. You might also be prompted to grant browser access to your camera and microphone devices, because once you connect to a video room, the room will start receiving your audio and video data.
Here's the full main.js
with these additions up to now.
1const form = document.getElementById("room-name-form");2const roomNameInput = document.getElementById("room-name-input");3const container = document.getElementById("video-container");45const startRoom = async (event) => {6// prevent a page reload when a user submits the form7event.preventDefault();8// hide the join form9form.style.visibility = "hidden";10// retrieve the room name11const roomName = roomNameInput.value;1213// fetch an Access Token from the join-room route14const response = await fetch("/join-room", {15method: "POST",16headers: {17Accept: "application/json",18"Content-Type": "application/json",19},20body: JSON.stringify({ room_name: roomName }),21});22const { token } = await response.json();2324// join the video room with the token25const room = await joinVideoRoom(roomName, token);26console.log(room);27};2829const joinVideoRoom = async (roomName, token) => {30// join the video room with the Access Token and the given room name31const room = await Twilio.Video.connect(token, {32room: roomName,33});34return room;35};3637form.addEventListener("submit", startRoom);
Once a web client joins a video room with an Access Token, it becomes a Participant in the room. The Twilio Video JavaScript SDK distinguishes between local and remote participants; a web client's local participant is the one who joined the video room from that browser, and all other participants are considered remote participants.
After you've successfully connected to a video room, you'll want to display each participant's video and audio on the page. The room object you got back from the connect
method has an attribute called localParticipant
, which represents the local participant, and an attribute called participants
, which is a list of the remote participants that are connected to the room.
The next step in this code is to display the video and audio data for the participants. This will involve:
The logic for adding the video and audio data on the page will come later; right now, you can create an empty function called handleConnectedParticipant
under the startRoom
function. You'll fill out the body of the function in the next section.
const handleConnectedParticipant = (participant) => {};
Next, you'll add calls to the handleConnectedParticipant
function. In main.js
, add the following lines to the startRoom
function, right after the call the joinVideoRoom
function. Note that you're removing the console.log(room)
line and replacing it with a call to handleConnectedParticipant
on the local participant.
1// join the video room with the token2const room = await joinVideoRoom(roomName, token);34// render the local and remote participants' video and audio tracks5handleConnectedParticipant(room.localParticipant);6room.participants.forEach(handleConnectedParticipant);7room.on("participantConnected", handleConnectedParticipant);8};
In these new lines of code, you call handleConnectedParticipant
on the local participant and any remote participants in the room. (In this application, you're using WebRTC Go Rooms, which have a maximum of two participants, so there will only be one remote participant.)
The room will send a participantConnected event whenever a new participant connects to the room. The code also listens for this event and calls the handleConnectedParticipant
function whenever that event triggers.
Here's the full main.js
code up to this point:
1const form = document.getElementById("room-name-form");2const roomNameInput = document.getElementById("room-name-input");3const container = document.getElementById("video-container");45const startRoom = async (event) => {6// prevent a page reload when a user submits the form7event.preventDefault();8// hide the join form9form.style.visibility = "hidden";10// retrieve the room name11const roomName = roomNameInput.value;1213// fetch an Access Token from the join-room route14const response = await fetch("/join-room", {15method: "POST",16headers: {17Accept: "application/json",18"Content-Type": "application/json",19},20body: JSON.stringify({ room_name: roomName }),21});22const { token } = await response.json();2324// join the video room with the token25const room = await joinVideoRoom(roomName, token);2627// render the local and remote participants' video and audio tracks28handleConnectedParticipant(room.localParticipant);29room.participants.forEach(handleConnectedParticipant);30room.on("participantConnected", handleConnectedParticipant);31};3233const handleConnectedParticipant = (participant) => {};3435const joinVideoRoom = async (roomName, token) => {36// join the video room with the Access Token and the given room name37const room = await Twilio.Video.connect(token, {38room: roomName,39});40return room;41};4243form.addEventListener("submit", startRoom);
In this section, you'll complete the logic for showing each participant's audio and video. Before doing this, you should understand the concept of participant tracks
.
All participants have tracks. There are three types of tracks:
By default, when a participant connects to a video room, Twilio will request their local audio and video data. You can use ConnectOptions
when you join a video room with the connect
method to control whether or not a local participant sends their local video and audio data. For example, if you wanted to create an audio-only chat room, you could create a room and set video: false
in ConnectOptions
, and then the room would not get participants' video data:
1const room = await Twilio.Video.connect(token, {2room: roomName,3video: false,4});
In this application, you'll receive both the audio and video data from participants.
Video room tracks follow a publish/subscribe pattern. A participant publishes their video, audio, and/or data tracks, and all other participants can subscribe to those published tracks. A participant cannot subscribe to an unpublished track.
When a participant publishes a track, it means they are making the data from that track available to other participants. By default, participants will publish their video and audio tracks when they join a room. A local participant can choose to unpublish or disable certain tracks as well. You can take advantage of this to build something like mute functionality; you could write a function that allows a local participant to stop publishing their audio track. This would make the audio track no longer available to other participants in the room, while still keeping their video track available.
You can update the default track subscription setting with the Track Subscriptions API, which allows you to specify which tracks to subscribe to automatically. For example, if you were building a video presentation application, you might want participants to only subscribe to one participant's audio track (the presenter), and not subscribe to anyone else's (the audience).
In this tutorial, you'll stick with the default track publication setting, with every participant publishing their audio and video tracks, and other participants automatically subscribing to those tracks.
Given these details, you can now complete the handleConnectedParticipant
function in main.js
. You'll separate the logic into a few different pieces.
Remove the empty handleConnnectedParticipant
function and replace it with the code below.
1const handleConnectedParticipant = (participant) => {2// create a div for this participant's tracks3const participantDiv = document.createElement("div");4participantDiv.setAttribute("id", participant.identity);5container.appendChild(participantDiv);67// iterate through the participant's published tracks and8// call `handleTrackPublication` on them9participant.tracks.forEach((trackPublication) => {10handleTrackPublication(trackPublication, participant);11});1213// listen for any new track publications14participant.on("trackPublished", handleTrackPublication);15};1617const handleTrackPublication = () => {}
In the body of handleConnectedParticipant
, you first create a new <div>
element for a participant, where you'll put all of this participant's tracks. (This will be helpful when a participant disconnects from the video app — you can remove all of their tracks from the page by removing this <div>
.)
Then, you loop through all of the participant's published tracks by looping over the participant's tracks
attribute, which contains the participant's TrackPublication
objects. A TrackPublication
object represents a track that has been published.
TrackPublication
objects can be either LocalTrackPublication
or RemoteTrackPublication
objects. In this application's case, you don't need to distinguish between local or remote objects, because they're treated the same. You might want to distinguish between local and remote tracks if you wanted to do something like display the local participant's video differently than other participants' videos.
You pass those TrackPublication
objects to a new function called handleTrackPublication
, which will eventually add the track onto the page. handleTrackPublication
is an empty function underneath the handleConnectedParticipant
function now, but you'll complete it in the next step.
You also listen for any new track publications from a participant. Whenever there's a trackPublished event, the handleTrackPublication
function runs.
This section is complex, as you're dealing with several different objects: Participants
, TrackPublications
, and Tracks
.
It can be helpful to console.log
these objects as you're starting to understand what they do. Feel free to add console.log
statements throughout the code and take time to look at what the objects include. As an example, you might add console.log
lines to the handleConnectedParticipant
function and inspect the participant
and trackPublication
objects:
1// iterate through the participant's published tracks and2// call `handleTrackPublication` on them3console.log("participant: ", participant);4participant.tracks.forEach((trackPublication) => {5console.log("trackPublication: ", trackPublication);6handleTrackPublication(trackPublication, participant);7});
In the next function, handleTrackPublication
, you'll be working with individual trackPublication
s from a participant.
Remove the empty handleTrackPublication
function, and replace it with the following code:
1const handleTrackPublication = (trackPublication, participant) => {2function displayTrack(track) {3// append this track to the participant's div and render it on the page4const participantDiv = document.getElementById(participant.identity);5// track.attach creates an HTMLVideoElement or HTMLAudioElement6// (depending on the type of track) and adds the video or audio stream7participantDiv.append(track.attach());8}9};
First, you create an internal function, displayTrack
, to handle rendering the data on the page. It will receive a track and find the <div>
that you created earlier for containing all of a participant's tracks. Then, you'll call track.attach()
and append that to the participant's <div>
.
track.attach()
creates either an HTMLVideoElement
, if the track is a VideoTrack
, or an HTMLAudioElement
for an AudioTrack
. It then adds the video or audio data stream to that HTML element. This is how video will ultimately show up on the page, and how audio will be played.
You'll only call displayTrack
on tracks that you are subscribed to. Update the handleTrackPublication
function with the following code, underneath the displayTrack
function:
1const handleTrackPublication = (trackPublication, participant) => {2function displayTrack(track) {3// append this track to the participant's div and render it on the page4const participantDiv = document.getElementById(participant.identity);5// track.attach creates an HTMLVideoElement or HTMLAudioElement6// (depending on the type of track) and adds the video or audio stream7participantDiv.append(track.attach());8}910// check if the trackPublication contains a `track` attribute. If it does,11// we are subscribed to this track. If not, we are not subscribed.12if (trackPublication.track) {13displayTrack(trackPublication.track);14}1516// listen for any new subscriptions to this track publication17trackPublication.on("subscribed", displayTrack);18};
You can identify whether or not you are subscribed to a published track by seeing if the trackPublication
has a track
attribute. The track
is the stream of audio, video, or data that you'll add to the page. If the trackPublication
does have a track
, this participant is subscribed and you can display the data. Otherwise, this local participant has not been subscribed to the published track.
This code also adds an event listener so that any time you subscribe to a new track, displayTrack
runs.
Here's the full code for the main.js
file:
1const form = document.getElementById("room-name-form");2const roomNameInput = document.getElementById("room-name-input");3const container = document.getElementById("video-container");45const startRoom = async (event) => {6// prevent a page reload when a user submits the form7event.preventDefault();8// hide the join form9form.style.visibility = "hidden";10// retrieve the room name11const roomName = roomNameInput.value;1213// fetch an Access Token from the join-room route14const response = await fetch("/join-room", {15method: "POST",16headers: {17Accept: "application/json",18"Content-Type": "application/json",19},20body: JSON.stringify({ room_name: roomName }),21});22const { token } = await response.json();2324// join the video room with the token25const room = await joinVideoRoom(roomName, token);2627// render the local and remote participants' video and audio tracks28handleConnectedParticipant(room.localParticipant);29room.participants.forEach(handleConnectedParticipant);30room.on("participantConnected", handleConnectedParticipant);31};3233const handleConnectedParticipant = (participant) => {34// create a div for this participant's tracks35const participantDiv = document.createElement("div");36participantDiv.setAttribute("id", participant.identity);37container.appendChild(participantDiv);3839// iterate through the participant's published tracks and40// call `handleTrackPublication` on them41participant.tracks.forEach((trackPublication) => {42handleTrackPublication(trackPublication, participant);43});4445// listen for any new track publications46participant.on("trackPublished", handleTrackPublication);47};4849const handleTrackPublication = (trackPublication, participant) => {50function displayTrack(track) {51// append this track to the participant's div and render it on the page52const participantDiv = document.getElementById(participant.identity);53// track.attach creates an HTMLVideoElement or HTMLAudioElement54// (depending on the type of track) and adds the video or audio stream55participantDiv.append(track.attach());56}5758// check if the trackPublication contains a `track` attribute. If it does,59// we are subscribed to this track. If not, we are not subscribed.60if (trackPublication.track) {61displayTrack(trackPublication.track);62}6364// listen for any new subscriptions to this track publication65trackPublication.on("subscribed", displayTrack);66};6768const joinVideoRoom = async (roomName, token) => {69// join the video room with the Access Token and the given room name70const room = await Twilio.Video.connect(token, {71room: roomName,72});73return room;74};7576form.addEventListener("submit", startRoom);
You should now be able to go to localhost:5000
, join a video room, and see your video. Then, from a separate tab, you can join the same room and see both participants' video and hear audio.
You now have a working Video application! The last piece is to clean up after a participant closes their browser or disconnects. In those cases, you should disconnect them from the room and stop displaying their video.
Copy and paste the following handleDisconnectedParticipant
function in static/main.js
under the handleTrackPublication
function.
1const handleDisconnectedParticipant = (participant) => {2// stop listening for this participant3participant.removeAllListeners();4// remove this participant's div from the page5const participantDiv = document.getElementById(participant.identity);6participantDiv.remove();7};
In this function, when a participant disconnects, you remove all the listeners you had on the participant, and remove the <div>
containing their video and audio tracks.
Then, add an event listener in the startRoom
function, underneath the calls to handleConnectedParticipant
. When a participantDisconnected
event happens, call the handleDisconnectedParticipant
function that you wrote.
1// render the local and remote participants' video and audio tracks2handleConnectedParticipant(room.localParticipant);3room.participants.forEach(handleConnectedParticipant);4room.on("participantConnected", handleConnectedParticipant);56// handle cleanup when a participant disconnects7room.on("participantDisconnected", handleDisconnectedParticipant);8};
You'll also want to detect when someone closes the browser or navigates to a new page, so that you can disconnect them from the room. This way, other participants can be alerted that the participant left the room. Underneath the event listener for participantDisconnected
, add two more event listeners: one for pagehide
, and one for beforeunload
. When either of these events occur, the local participant should disconnect from the room.
1// handle cleanup when a participant disconnects2room.on("participantDisconnected", handleDisconnectedParticipant);3window.addEventListener("pagehide", () => room.disconnect());4window.addEventListener("beforeunload", () => room.disconnect());5};
Here's the final main.js
with all of this functionality:
1const form = document.getElementById("room-name-form");2const roomNameInput = document.getElementById("room-name-input");3const container = document.getElementById("video-container");45const startRoom = async (event) => {6// prevent a page reload when a user submits the form7event.preventDefault();8// hide the join form9form.style.visibility = "hidden";10// retrieve the room name11const roomName = roomNameInput.value;1213// fetch an Access Token from the join-room route14const response = await fetch("/join-room", {15method: "POST",16headers: {17Accept: "application/json",18"Content-Type": "application/json",19},20body: JSON.stringify({ room_name: roomName }),21});22const { token } = await response.json();2324// join the video room with the token25const room = await joinVideoRoom(roomName, token);2627// render the local and remote participants' video and audio tracks28handleConnectedParticipant(room.localParticipant);29room.participants.forEach(handleConnectedParticipant);30room.on("participantConnected", handleConnectedParticipant);3132// handle cleanup when a participant disconnects33room.on("participantDisconnected", handleDisconnectedParticipant);34window.addEventListener("pagehide", () => room.disconnect());35window.addEventListener("beforeunload", () => room.disconnect());36};3738const handleConnectedParticipant = (participant) => {39// create a div for this participant's tracks40const participantDiv = document.createElement("div");41participantDiv.setAttribute("id", participant.identity);42container.appendChild(participantDiv);4344// iterate through the participant's published tracks and45// call `handleTrackPublication` on them46participant.tracks.forEach((trackPublication) => {47handleTrackPublication(trackPublication, participant);48});4950// listen for any new track publications51participant.on("trackPublished", handleTrackPublication);52};5354const handleTrackPublication = (trackPublication, participant) => {55function displayTrack(track) {56// append this track to the participant's div and render it on the page57const participantDiv = document.getElementById(participant.identity);58// track.attach creates an HTMLVideoElement or HTMLAudioElement59// (depending on the type of track) and adds the video or audio stream60participantDiv.append(track.attach());61}6263// check if the trackPublication contains a `track` attribute. If it does,64// we are subscribed to this track. If not, we are not subscribed.65if (trackPublication.track) {66displayTrack(trackPublication.track);67}6869// listen for any new subscriptions to this track publication70trackPublication.on("subscribed", displayTrack);71};7273const handleDisconnectedParticipant = (participant) => {74// stop listening for this participant75participant.removeAllListeners();76// remove this participant's div from the page77const participantDiv = document.getElementById(participant.identity);78participantDiv.remove();79};8081const joinVideoRoom = async (roomName, token) => {82// join the video room with the Access Token and the given room name83const room = await Twilio.Video.connect(token, {84room: roomName,85});86return room;87};8889form.addEventListener("submit", startRoom);
You're done! You now have a video chat application that you can use for connecting two people with audio and video.
Right now, the application is only running locally. To make it accessible to others without deploying the application, you can use ngrok. Download ngrok and follow the setup instructions (you'll unzip the ngrok download file and can then start it up). Once it's downloaded, start ngrok on port 5000.
./ngrok http 5000
You should see output that looks like this:
1ngrok by @inconshreveable (Ctrl+C to quit)23Session Status online4Update update available (version 2.3.40, Ctrl-U to update)5Version 2.3.356Region United States (us)7Web Interface http://127.0.0.1:40408Forwarding http://04ae1f9f6d2b.ngrok.io -> http://localhost:50009Forwarding https://04ae1f9f6d2b.ngrok.io -> http://localhost:50001011Connections ttl opn rt1 rt5 p50 p90120 0 0.00 0.00 0.00 0.00
Copy the "Forwarding" address and send that to a friend so they can join your video chat!
There's so much more that you can add on to your application now that you've built the foundation. For example, you can:
For more inspiration, check out the Twilio Video React quick deploy app, which demonstrates a wide range of video functionality, or try out more Video tutorials on the Twilio Blog.