In this guide, we'll show you how to configure Audio and Video input and output devices from your Twilio Video application. Taking advantage of the ability to control input and output devices lets you build a better end user experience.
The CameraSource
class captures from an AVCaptureDevice
, and provides frames to a LocalVideoTrack
.
Add selected video source to local track
1// The default initializer will succeed if the front camera is available.23if let camera = CameraSource(delegate: self),4let videoTrack = LocalVideoTrack(source: camera) {56// VideoView is a VideoRenderer and can be added to any VideoTrack.7let renderer = VideoView(frame: view.bounds)89// Add renderer to the video track10videoTrack.addRenderer(renderer)1112self.localVideoTrack = videoTrack13self.camera = camera14self.view.addSubview(renderer)15} else {16print("Couldn't create CameraSource or LocalVideoTrack")17}181920// Select between the front and back camera.21func flipCamera() {22var newDevice: AVCaptureDevice?2324if let camera = self.camera, let captureDevice = camera.device {25if captureDevice.position == .front {26newDevice = CameraSource.captureDevice(position: .back)27} else {28newDevice = CameraSource.captureDevice(position: .front)29}3031if let newDevice = newDevice {32camera.selectCaptureDevice(newDevice) { (captureDevice, videoFormat, error) in33if let error = error {34print("Error selecting capture device.\ncode = \((error as NSError).code) error = \(error.localizedDescription)")35}36}37}38}39}
Optionally set VideoEncodingMode in ConnectOptions
VideoEncodingMode represents the modes of behavior of the videoEncodingMode
property in ConnectOptions
. The videoEncodingMode
API is mutually exclusive with existing codec management APIs EncodingParameters.maxVideoBitrate
and preferredVideoCodecs
. The default value is unset.
auto
- In this mode, the SDK selects the video codecs and manages encodings automatically.
1let connectOptions = ConnectOptions(token: accessToken) { (builder) in2builder.roomName = "my-room"3builder.videoEncodingMode = .auto4}5var room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
Typically, the audio input and output route is chosen by the end user in Control Center. By default TwilioVideo will manage the application's AVAudioSession
and configure it for video conferencing use cases. If you wish to modify audio behavior, including session configuration, you can create your own DefaultAudioDevice
and provide it as an option before connecting to a Room.
1// Override the device before creating any Rooms or Tracks.2self.audioDevice = DefaultAudioDevice()3TwilioVideoSDK.audioDevice = self.audioDevice45let localAudioTrack = LocalAudioTrack()6let connectOptions = ConnectOptions(token: accessToken) { (builder) in7builder.roomName = "my-room"89if let audioTrack = localAudioTrack {10builder.audioTracks = [ audioTrack ]11}12}13var room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
To configure the AVAudioSession
, DefaultAudioDevice
executes the kDefaultAVAudioSessionConfigurationBlock
by default. You can alter the AVAudioSession
configuration for your application by providing a block
to the DefaultAudioDevice
. For example, the input and output audio routes can be overwritten by altering the AVAudioSession
configuration in the block that you provide to DefaultAudioDevice
.
The following example demonstrates how to configure AVAudioSession
for voice chat scenarios. This will prefer using the device's receiver and bottom microphone like a voice call in the Phone app.
1// Change the audio route after connecting to a Room.2self.audioDevice.block = {3do {4DefaultAudioDevice.DefaultAVAudioSessionConfigurationBlock()56let audioSession = AVAudioSession.sharedInstance()7try audioSession.setMode(.voiceChat)8} catch let error as NSError {9print("Fail: \(error.localizedDescription)")10}11}1213self.audioDevice.block();
You must use your own AVAudioEngineDevice
audio device that uses the kAudioUnitSubType_RemoteIO
componentSubType
of the AudioComponentDescription
in order to change microphone orientations.
The next Objective-C example demonstrates how to select the back microphone. This might be useful if your subject is behind the phone, and being captured with the back camera.
12// Change this to `AVAudioSessionOrientationFront` if you wish to use the front microphone.3NSString *microphone = AVAudioSessionOrientationBack;45typeof(self) __weak weakSelf = self;6self.audioDevice.block = ^ {7kTVIDefaultAVAudioSessionConfigurationBlock();8[weakSelf setMicrophoneInUse:microphone];9};10self.audioDevice.block();1112- (void)setMicrophoneInUse:(NSString *)nextDataSource {13NSError *theError = nil;14BOOL result = YES;1516AVAudioSession *session = [AVAudioSession sharedInstance];1718result = [session setActive:YES error:&theError];1920// Get the set of available inputs. If there are no audio accessories attached, there will be21// only one available input -- the built in microphone.22NSArray *inputs = [session currentRoute].inputs;2324// Locate the Port corresponding to the built-in microphone.25AVAudioSessionPortDescription *builtInMicPort = nil;26for (AVAudioSessionPortDescription *port in inputs) {27if ([port.portType isEqualToString:AVAudioSessionPortBuiltInMic]) {28builtInMicPort = port;29break;30}31}3233if ([builtInMicPort.preferredDataSource.orientation isEqualToString:nextDataSource]) {34return;35}3637// loop over the built-in mic's data sources and attempt to locate the specified microphone38AVAudioSessionDataSourceDescription *theDataSource = nil;39for (AVAudioSessionDataSourceDescription *source in builtInMicPort.dataSources) {40if ([source.orientation isEqual:nextDataSource]) {41theDataSource = source;42break;43}44} // end data source iteration4546if (theDataSource) {47theError = nil;48if ([theDataSource.orientation isEqualToString:AVAudioSessionOrientationBack]) {49result = [theDataSource setPreferredPolarPattern:AVAudioSessionPolarPatternSubcardioid error:&theError];50if (!result) {51NSLog (@"Failed to set AVAudioSessionPolarPatternSubcardioid failed");52}53} else if ([theDataSource.orientation isEqualToString:AVAudioSessionOrientationFront]) {54result = [theDataSource setPreferredPolarPattern:AVAudioSessionPolarPatternCardioid error:&theError];55if (!result) {56NSLog (@"Failed to set AVAudioSessionPolarPatternCardioid failed");57}58}5960// Set a preference for the front data source.61theError = nil;62result = [builtInMicPort setPreferredDataSource:theDataSource error:&theError];63if (!result) {64// an error occurred. Handle it!65NSLog(@"setPreferredDataSource failed");66}67}6869// Make sure the built-in mic is selected for input. This will be a no-op if the built-in mic is70// already the current input Port.71theError = nil;72result = [session setPreferredInput:builtInMicPort error:&theError];73if (!result) {74// an error occurred. Handle it!75NSLog(@"setPreferredInput failed");76}77}