Changelog: Twilio Video iOS 3.x

Enhancements

• isRecording property in TVIRoom now accurately reflects the current recording state of the TVIRoom. In the previous versions of the SDK, isRecording could return false positives. In TVIRoomDelegate the roomDidStartRecording and roomDidStopRecording callbacks will now be invoked when recording for at least a single track in the TVIRoom has begun and ended respectively.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a race condition that could cause a crash or unnecessary web socket signaling reconnection.
• Fixed a race condition that could cause a crash when closing the signaling web socket.
• Fixed a bug where private IP addresses could be sent to insights when TVIConnectOptions.networkPrivacyPolicy was set to TVILocalNetworkPrivacyPolicyBlockLocal.
• Two small memory leaks were fixed, one when iced response is received in icing state, and one for signaling data tracks. [ISDK-3046]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed an SCTP related WebRTC security vulnerability detailed here.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

This release improves support for local network privacy on iOS 14.0 and iPadOS 14.0.

By default, the SDK no longer uses your local network on iOS 14 and above. Twilio Video balances privacy and performance, blocking communication routes that use your local network inappropriately in Peer-to-Peer Rooms while still ensuring a high rate of direct connections on iOS 14 devices.

If you use Peer-to-Peer Rooms then we recommend that you update your applications to this release in order to prepare for iOS 14. If you need features like multi-party video, network quality or recordings, then use Group Rooms. These Rooms use media servers provided by Twilio that are available on the public internet.

By default, Twilio Video does not access the local network on iOS 14 and above. If you want to allow Twilio Video to use the local network in a Peer-to-Peer Room then you can override the TVILocalNetworkPrivacyPolicy.

1
let connectOptions = ConnectOptions( token: accessToken, builder {
2
    // A permissions request may be triggered on iOS 14.
3
    builder.networkPrivacyPolicy = .allowAll
4
})

If you select TVILocalNetworkPolicyAllowAll then you must add a privacy usage description with the NSLocalNetworkUsageDescription key in your application's .plist file.

You can find more information about network privacy in our Getting Started Guide.

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a memory corruption crash that occurs when recovering from a media server failure in Group Rooms.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

API Changes

• TVIError is now defined using the NS_ERROR_ENUM macro which makes consuming TVIError objects more idiomatic in Swift, such as using in a switch/case statement.

1
func roomDidDisconnect(room: Room, error: Error?) {
2
    if let error = error as? TwilioVideoSDK.Error {
3
        switch error.code {
4
            case .roomRoomCompletedError:
5
                print ("The room completed gracefully")
6
            case .participantDuplicateIdentityError:
7
                print("Disconnected becasue another participant with the same identity joined.")
8
            default:
9
                print("Something else happened.")
10
        }
11
    }
12
}

Bug Fixes

• The camera capture background task logic has been refactored and iOS will no longer report a background task risk of termination warning. [ISDK-2878]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• Reduced connection times by acquiring IceServers during the initial handshake with Twilio's signaling server rather than sending a request to a different endpoint
• IceServers are now localized to your Participant's region rather than using Global Low Latency routing
• The SDK no longer depends on ecs.us1.twilio.com:443

API Changes

• The following TVIIceOptions properties are deprecated and setting them has no effect:
  • TVIIceOptions.abortOnIceServersTimeout
  • TVIIceOptions.iceServersTimeout

Bug Fixes

• Fixed a bug where IceServers might not be fetched on a dual-stack device where the IPv6 network interface is not reachable. [CSDK-3295]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed the issue where unpublishing and then republishing a TVILocalVideoTrack using VP8 simulcast does not complete. [CSDK-3433]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

You can now configure how your available downlink bandwidth will be distributed among your subscribed RemoteVideoTracks by using a new optional ConnectOptions parameter bandwidthProfile. For more details and best practices, visit the Network Bandwidth Profile API guide. Here is a brief example:

1
let videoOptions = VideoBandwidthProfileOptions { builder in
2
    // Minimum subscribe priority of Dominant Speaker's RemoteVideoTracks
3
    builder.dominantSpeakerPriority = .high
4

5
    // Maximum bandwidth (Kbps) to be allocated to subscribed RemoteVideoTracks
6
    builder.maxSubscriptionBitrate = 6000
7

8
    // Max number of visible RemoteVideoTracks. Other RemoteVideoTracks will be switched off
9
    builder.maxTracks = 5
10

11
    // Subscription mode: collaboration, grid, presentation
12
    builder.mode = .presentation
13

14
    // Configure remote track's render dimensions per track priority
15
    let renderDimensions = VideoRenderDimensions()
16

17
    // Desired render dimensions of RemoteVideoTracks with priority low.
18
    renderDimensions.low = VideoDimensions(width: 352, height: 288)
19

20
    // Desired render dimensions of RemoteVideoTracks with priority standard.
21
    renderDimensions.standard = VideoDimensions(width: 640, height: 480)
22

23
    // Desired render dimensions of RemoteVideoTracks with priority high.
24
    renderDimensions.high = VideoDimensions(width: 1280, height: 720)
25

26
    builder.renderDimensions = renderDimensions
27

28
    // Track Switch Off mode: .detected, .predicted, .disabled
29
    builder.trackSwitchOffMode = .predicted
30
}
31
let bandwidthProfileOptions = BandwidthProfileOptions(videoOptions: videoOptions)
32

33
// Use bandwidth profile to build the connectOptions
34
let connectOptions = ConnectOptions(token: accessToken) { builder in
35
    builder.bandwidthProfileOptions = bandwidthProfileOptions
36
}
37

38
// Connect to a Room with connectOptions
39
room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)

As part of the Bandwidth Profile API, you can now specify the mode to control remote video track switch off behavior. The new Track.SwitchOffMode enum can be specified in the VideoBandwidthProfileOptions class and can be set to one of the following:

• Track.SwitchOffMode.predicted - In this mode, RemoteVideoTracks are pro-actively switched off when network congestion is predicted by the bandwidth estimation mechanism. This mode is used by default if not specified.
• Track.SwitchOffMode.detected - In this mode, RemoteVideoTracks are switched off only when network congestion is detected.
• Track.SwitchOffMode.disabled - In this mode, RemoteVideoTracks will not be switched off. Instead tracks will be adjusted to lower quality.

Track switch off events are provided via two additional callbacks to RemoteParticipantDelegate.

1
extension ViewController : RemoteParticipantDelegate {
2
    func remoteParticipantSwitchedOnVideoTrack(participant: RemoteParticipant,
3
                                               track: RemoteVideoTrack) {}
4

5
    func remoteParticipantSwitchedOffVideoTrack(participant: RemoteParticipant,
6
                                                track: RemoteVideoTrack) {}
7
}

While publishing a local track, you can now optionally specify it's publish priority.

1
/*
2
 * Publish a local track with specified priorities.
3
 *
4
 * In addition, local track publications now contain a priority property which reflect the priority of
5
 * the published track.
6
 */
7
let localVideoTrackPublicationOptions = LocalTrackPublicationOptions(priority: .high)
8
localParticipant.publishVideoTrack(localVideoTrack, publicationOptions: localVideoTrackPublicationOptions);
9

10
let localAudioPublicationOptions = LocalTrackPublicationOptions(priority: .high)
11
localParticipant.publishAudioTrack(localAudioTrack, publicationOptions: localAudioPublicationOptions);
12

13
let localDataPublicationOptions = LocalTrackPublicationOptions(priority: .low)
14
localParticipant.publishDataTrack(localDataTrack, publicationOptions: localDataPublicationOptions);
15

16
/*
17
 * The publish priority is also represented in remote audio, video, and data track publications.
18
 */
19
let publishPriority = remoteVideoTrackPublication.publishPriority

Also, the publisher's TrackPriority for the corresponding local Track can be updated after a track has been published.

localVideoTrackPublication.priority = .high

When a TVITrackPriority is updated after the track has been published, remote participants will be alerted to the change via the [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forAudioTrack], [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forVideoTrack] and [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forDataTrack] delegate methods.

1
extension ViewController : RemoteParticipantDelegate {
2
    func remoteParticipantDidChangeAudioTrackPublishPriority(participant: RemoteParticipant,
3
                                                             priority: Track.Priority,
4
                                                             publication: RemoteAudioTrackPublication) { }
5

6
    func remoteParticipantDidChangeVideoTrackPublishPriority(participant: RemoteParticipant,
7
                                                             priority: Track.Priority,
8
                                                             publication: RemoteVideoTrackPublication) { }
9

10
    func remoteParticipantDidChangeDataTrackPublishPriority(participant: RemoteParticipant,
11
                                                            priority: Track.Priority,
12
                                                            publication: RemoteDataTrackPublication) { }
13
}

Additionally, you can set a subscriber side priority for a TVIRemoteVideoTrack.

1
/*
2
 * Set a remote video track priority to HIGH. Note that this feature is only available
3
 * for RemoteVideoTrack.
4
 */
5
remoteVideoTrack.priority = .high
6

7
/*
8
 * Get the current remote video track priority.
9
 */
10
let remoteVideoTrackPriority = remoteVideoTrack.priority

This signals to the media server the relative importance of the track with respect to other tracks that may be shared to the Room. The media server takes this into account while allocating a subscribing RemoteParticipant's bandwidth to the corresponding remote track. If you do not specify a priority, the RemoteVideoTrack.priority property will return nil, and the media server will use the RemoteVideoTrackPublication.publishPriority assigned by the publisher instead.

New APIs

• Added TVITrackPriority for track priorities of Room.
• Added a new readonly property publishPriority of type TVITrackPriority in TVIRemoteAudioTrackPublication, TVIRemoteVideoTrackPublication, and TVIRemoteDataTrackPublication. These readonly properties fetch the priorities at which the remote Audio, Video, and Data Tracks were published.
• Added a new class TVILocalTrackPublicationOptions to specify track publication options when publishing tracks via TVILocalParticipant.
• Added additional publish track methods that allow developers to specify TVILocalTrackPublicationOptions -
  • [TVILocalParticipant publishAudioTrack:publicationOptions:]
  • [TVILocalParticipant publishVideoTrack:publicationOptions:]
  • [TVILocalParticipant publishDataTrack:publicationOptions:]
• Added a new property priority of type TVITrackPriority in TVILocalAudioTrackPublication, TVILocalVideoTrackPublication and TVILocalDataTrackPublication classes.
• Added the [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forAudioTrack], [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forVideoTrack] and [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forDataTrack] delegate methods.

Bug Fixes

• Fixed a crash when the SDK was processing a local SDP that had incorrect number of SSRCs when applying VP8 simulcast.

Known Issues

• Unpublishing and then republishing a TVILocalVideoTrack using VP8 simulcast does not complete. As a workaround, disable the Track instead. [CSDK-3433]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• Previously, twilio-video would fail connection attempts if the signaling server was busy with too many requests. Now, it will try again after a server specified backoff period either until it is successfully connected or the server asks it to stop trying. In this case, the -[TVIRoomDelegate room\:didFailToConnectWithError:] method is called with TVIErrorSignalingServerBusyError.

1
func roomDidFailToConnect(room: Room, error: Error) {
2
    let theError = error as NSError
3
    if theError.domain == TwilioVideoSDK.ErrorDomain,
4
        theError.code == TwilioVideoSDK.Error.signalingServerBusyError {
5
        print("Twilio's signaling server cannot accept connection requests at this time.")
6
    }
7
}

API Changes

• Added error code definitions:
  • TVIErrorSignalingServerBusyError
  • TVIErrorRoomAccountLimitExceededError
  • TVIErrorParticipantAccountLimitExceededError
  • TVIErrorTrackDataTrackMessageTooLargeError
  • TVIErrorTrackDataTrackSendBufferFullError
  • TVIErrorMediaDtlsTransportFailedError
  • TVIErrorMediaIceRestartNotAllowedError

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancement

• TVICameraSource now stops the underlying AVCaptureSession during deallocation if the developer had not stopped it using one of the stopCapture methods. It is still suggested that the developer calls one of the stopCapture methods at a time that is appropriate for their application and not relying on the SDK to do it. #445
• Publish stats reports every 10 seconds and active ICE candidate pair messages every 20 seconds. Previously both were reported every 4 seconds.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a bug when establishing a signaling WebSocket that might cause only one address family (IPv4 or IPv6) to be tried.
• Fixed a network handoff bug where the media was not switching over to low cost network.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a potential crash when destroying a TVILocalVideoTrack while the Track's TVICameraSource (or custom TVIVideoSource) is still delivering frames. [ISDK-2792]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed the issue where the NS_SWIFT_NAME mapping for CameraSourceOptionsBuilder.Block would fail. #72

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed parsing issue when a Remote Participant's Network Quality Level is zero. [CSDK-3325]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

API Changes

• Implemented Network Quality for Remote Participants in Group Rooms:
  • To enable the Network Quality functionality, set the TVIConnectOptions.networkQualityEnabled property to YES when connecting to a Group Room. By default this enables network quality level changes to be reported for the Local Participant. To also receive network quality level changes for the Remote Participants, a configured TVINetworkQualityConfiguration object needs to be supplied to the TVIConnectOptions.networkQualityConfiguration property.
  • The networkQualityLevel property has been promoted from the TVILocalParticipant class to the TVIParticipant base class.
  • To determine the current network quality level for both Local and Remote Participants, query the networkQualityLevel property. Note, this will return TVINetworkQualityLevelUnknown if:
    • The TVIConnectOptions.networkQualityEnabled property was set to NO OR
    • Using a Peer-to-Peer room OR
    • The network quality level has not yet been computed
  • Implementing the [TVILocalParticipantDelegate localParticipant:networkQualityLevelDidChange:] method on your TVILocalParticipantDelegate will allow you to receive callbacks when the network quality level changes for the Local Participant.
  • Implementing the [TVIRemoteParticipantDelegate remoteParticipant:networkQualityLevelDidChange:] method on your TVIRemoteParticipantDelegate will allow you to receive callbacks when the network quality level changes for the Remote Participant.

1
@IBAction func connectToRoom(sender: AnyObject) {
2
    let connectOptions = ConnectOptions(token: accessToken) { (builder) in
3
        builder.isNetworkQualityEnabled = true
4
        builder.networkQualityConfiguration = NetworkQualityConfiguration(localVerbosity: .minimal,
5
                                                                          remoteVerbosity: .minimal)
6
        builder.roomName = "my-room"
7
    }
8

9
    room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
10
}
11

12
...
13

14
// MARK: LocalParticipantDelegate
15
func localParticipantNetworkQualityLevelDidChange(participant: LocalParticipant, networkQualityLevel: NetworkQualityLevel) {
16
    print("Local Participant Network Quality Level Changed: \(networkQualityLevel)")
17
}
18

19
// MARK: RemoteParticipantDelegate
20
func remoteParticipantNetworkQualityLevelDidChange(participant: RemoteParticipant, networkQualityLevel: NetworkQualityLevel) {
21
    print("Remote Participant (\(participant.identity)) Network Quality Level Changed: \(networkQualityLevel)")
22
}

Enhancements

• Reduced bandwidth usage for Insights reporting by up to 75%.
• Data track per-message size limit has been increased to 64 KB. [CSDK-3197]

Bug Fixes

• Fixed intermittent crash when sending data on a data track while in the process of closing the track due to remote participant disconnect. [CSDK-3245]
• TVIIceCandidatePairStats.availableOutgoingBitrate is populated with the available send bandwidth when publishing Tracks.
• TVIIceCandidatePairStats.availableIncomingBitrate is populated with the available receive bandwidth when subscribing to Tracks in a Group Room.
• Fixed a crash which could occur when destroying a TVIRoom while the TCP handshake for the signaling connection is in progress.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• Reduced memory allocations when cropping is applied with -[TVIVideoSource requestOutputFormat:] by caching the cropping attributes.
• Added TVICameraSourceOptions.rotationTags, which defaults to TVICameraSourceOptionsRotationTagsKeep. You might want to set this option to TVICameraSourceOptionsRotationTagsRemove in order to:
  • Save power and battery life when you know that the video will be published in a Group Room
  • Receive images that are properly oriented for Computer Vision analysis
  • Use AVAssetWriter for recordings without worrying about transformation matrices
• Performance Improvements:
  • With TVICameraSourceOptionsRotationTagsRemove, we see significant improvement with the H.264 video codec on modern iPhones.
  • With TVICameraSourceOptionsRotationTagsRemove, we see small improvement with the VP8 video codec (non-simulcast) on modern iPhones.

1
* Tested on iPhone 11
2
* Group Room
3
* Capture resolution, fps, encoding params: 640x480x30 @ 1200 kbps
4
* Video call duration: 5 mins / run
5
* Measured using Instruments
6
** Load scheduled mostly on high efficiency cores, with performance cores disabled.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a possible deadlock that could occur when destroying a TVIRoom without first calling -[TVIRoom disconnect].
• Fixed a crash which could occur when destroying a TVIRoom with dominant speaker enabled.

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

3.0 is now generally available. Thank you for all the feedback from our beta adopters.

This release includes:

• Global low latency (GLL) signaling with IPv4 and IPv6 support.
• Improved Swift APIs for a more idiomatic Swift development experience.
• Support for UIScene and multi-window apps on iOS 13 and iPadOS 13.

Bug Fixes

• Fixed a regression in 3.0.0-beta5 and 3.0.0-beta6 where the Participant's TLS handshake might be delayed by up to 250 milliseconds. [CSDK-3117]

Known Issues

• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

This release completes the API changes needed for Twilio Video to be fully compatible with the iOS 13.0 and iPadOS 13.0 SDKs. This release also removes support for iOS 10. Twilio Video 2.x will continue to support iOS 9.0 and later.

API Changes

• The minimum supported iOS version has been raised to 11.0.
• TVICameraSource can monitor UIWindowScene for orientation and state changes in iOS 13.0+.
• Removed manual orientation control APIs from TVICameraSource and introduced the TVICameraSourceOrientationTracker protocol along with the TVIUserInterfaceTracker class.
• The following example demonstrates how to track a UIWindowScene:

1
// Track the orientation of the key window's scene.
2
let options = CameraSourceOptions { (builder) in
3
    if let keyScene = UIApplication.shared.keyWindow?.windowScene {
4
        builder.orientationTracker = UserInterfaceTracker(scene: keyScene)
5
    }
6
}
7
let camera  = CameraSource(options: options, delegate: self)

1
// Forward UIWindowScene events
2
func windowScene(_ windowScene: UIWindowScene,
3
                 didUpdate previousCoordinateSpace: UICoordinateSpace,
4
                 interfaceOrientation previousInterfaceOrientation: UIInterfaceOrientation,
5
                 traitCollection previousTraitCollection: UITraitCollection) {
6
    UserInterfaceTracker.sceneInterfaceOrientationDidChange(windowScene)
7
}

Known Issues

• The Participant's TLS handshake might be delayed by up to 250 milliseconds. [CSDK-3117]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

API Changes

• Added TVILocalParticipant.signalingRegion. You can use this property to determine where your Participant connected to a Room, especially when using the default value of "gll" for TVIConnectOptions.region.
• Added TVIRoom.mediaRegion. You can use this property to determine where media is being processed in a Group Room.
• Updated the TVIConnectOptionsBuilder.region property to no longer throw an exception when an empty string is provided and instead default to gll.
• When a Participant connects to a Room, the WebSocket handshake is required to complete in 15 seconds or less, otherwise TVIErrorSignalingConnectionError is raised.
• TVIEncodingParameters now expresses maximum bitrates in Kilobits per second (Kbps) instead of bits per second (bps).

1
// Before
2
let params = EncodingParameters(audioBitrate: UInt(64000),
3
                                videoBitrate: UInt(800000))

1
// After
2
let params = EncodingParameters(audioBitrate: UInt(64),
3
                                videoBitrate: UInt(800))

Enhancements

• The iOS SDK is built with Xcode 11.0
• TVIVideoView registers for UIScene lifecycle notifications when it is contained within a UIWindowScene on iOS 13.0+. Rendering is permitted in the following states:
  • UISceneActivationStateForegroundActive at 60 Hz
  • UISceneActivationStateForegroundInactive at 30 Hz
• TVIVideoView suspends its internal CADisplayLink timer when it is removed from a UIWindow.
• Increased the reliability of TVILocalDataTrack by monitoring its send buffer. Sending too many messages, or sending single messages larger than 16 KB no longer causes the underlying channel(s) to close. These requests are ignored instead. [CSDK-3037]

Bug Fixes

• Fixed a bug where TVICameraSource might produce distorted frames after being interrupted with AVCaptureSessionInterruptionReasonVideoDeviceInUseByAnotherClient on iOS 13.0. #53
• Making the SDK more aggressive at establishing a connection by trying to connect multiple sockets to different endpoints in parallel. [CSDK-2942]
• TVILocalDataTrack should no longer end up in an inconsistent state after attempting to send a message larger than 16 KB. [CSDK-3081]

Known Issues

• The Participant's TLS handshake might be delayed by up to 250 milliseconds. [CSDK-3117]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

API Changes

• Introduced TVIErrorSignalingDnsResolutionError, which is now raised instead of TVIErrorSignalingConnectionError in the following scenarios:
  • The device has misconfigured DNS Server(s) on its active network interface.
  • The region provided in TVIConnectOptions was invalid.
  • The device lost its Internet connection before the query could complete.

Enhancements

• Reduced connection times by removing a round trip when:
  • Reconnecting after a signaling connection failure
  • Connecting with TVIConnectOptions.iceOptions, and overridden Servers
  • Connecting with default ICE servers (us1 only)
  • [CSDK-2902]

Bug Fixes

• Fixed a bug where Participants remain connected to a Room even after terminal media failures. [CSDK-3023]
• Fixed a crash when Network Quality or Dominant Speaker is enabled, connectivity to the Room is lost, and the Room is deleted from within TVIRoomDelegate.roomDidDisconnect(). [CSDK-2950]
• Fixed a bug where resources might be leaked when destroying a Room. [CSDK-2822]
• Fixed a bug where network monitoring would continue on closed connections in a Peer-to-Peer Room. [CSDK-2947]
• Fixed a crash on the iOS 13 simulator when rendering received H.264 video, or any other video in the TVIPixelFormatYUV420BiPlanarFullRange or TVIPixelFormatYUV420BiPlanarVideoRange formats. [ISDK-2631]

Known Issues

• In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• The Participant's signaling connection now conforms to Twilio's TLS & Cipher Suite Security Policy. Support for TLS versions older than 1.2 has been removed. [CSDK-3020]
• Adjusted the buffer sizes used for signaling messages to reduce network fragmentation. [CSDK-2920]
• Setting TVILogModuleSignaling enables logging of low level connection events at TVILogLevelTrace. [ISDK-2446]

Bug Fixes

• WebSocket errors are handled immediately, rather than waiting for a timeout to occur. [CSDK-3025]
• Handle rare exceptions when constructing a WebSocket. [CSDK-2755]

Known Issues

• Future 3.0.0-beta releases will reduce the number of round-trips required to connect to a Room. [CSDK-2902]
• In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

API Changes

• Renamed + [TwilioVideoSDK version] to + [TwilioVideoSDK sdkVersion] to alleviate the clash with NSObject's version method.
• Removed the type argument label for the Swift definition of [TVICameraSource captureDeviceForPosition:type]. In Swift, the function is now declared as class func captureDevice(position: AVCaptureDevice.Position, deviceType: AVCaptureDevice.DeviceType) -> AVCaptureDevice?.
• Updated the Swift audio sample rate constant names to remove the redundant Audio. For instance AudioFormat.AudioSampleRate8000 is now AudioFormat.SampleRate8000.
• Updated the Swift audio format channels constant names to remove the redundant Audio. For instance AudioFormat.AudioChannelsMono is now AudioFormat.ChannelsMono.
• Refactored the TVILocalParticipantDelegate and TVIRemoteParticipantDelegate methods to follow the did naming convention for consistency with the other delegate methods. See the Migrating from 2.x to 3.x guide for more information.

Bug Fixes

• The SDK will consistently recover the Participant's connection after multiple failed network handovers, or multiple failures on a single network. [CSDK-2957, CSDK-3007]
• Failures in the Participant's connection are always detected within 15 seconds. [CSDK-2927]
• Improved DNS retry logic and error handling. [CSDK-2801]
• Disabled server side deflate compression due to occasional errors when reading messages. [CSDK-3004]
• Security improvements. [CSDK-2986, CSDK-3005]

Known Issues

• Future 3.0.0-beta releases will reduce the number of round-trips required to connect to a Room. [CSDK-2902]
• Setting TVILogModuleSignaling does not produce any logging. [ISDK-2446]
• In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
• If a WebSocket error occurs, it might not be handled for up to 15 seconds. [CSDK-3025]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

The 3.0 release removes support for iOS 9.0, and raises the minimum supported iOS version to 10.0. Twilio Video 2.x releases will continue to support iOS 9.0 and later.

Enhancements

• The SDK uses a new WebSocket based signaling transport, and communicates with globally available signaling Servers over IPv4 and IPv6 networks.
• Added the TVIConnectOptions.region property. By default, the Client will connect to the nearest signaling Server determined by latency based routing. Setting a value other than "gll" bypasses routing and guarantees that signaling traffic will be terminated in the region that you prefer.

1
let connectOptions = ConnectOptions(token: accessToken) { (builder) in
2
    builder.region = "us1"
3
    builder.roomName = "my-room"
4
}
5
self.room = TwilioVideoSDK.connect(with: connectOptions, delegate: self)

• Participants are considered to be reconnecting within 15 seconds, and are disconnected from a Room after 45 seconds of lost connectivity. #99

API Changes

• Twilio Video has been updated for better Swift interoperability. See the Migrating from 2.x to 3.x guide for more information.
  • The TwilioVideo Obj-C class has been renamed to TwilioVideoSDK to reduce possible namespace issues in Swift due to the Module containing a Type with the same name.
  • The TVI prefix has been removed from all Twilio Video types
  • Many of the delegate function declarations have been revamped for better clarity of their intent
  • Some of the class method declarations have been revamped for better clarity of their intent
• The TVIAudioSessionActivated() and TVIAudioSessionDeactivated() methods have been removed as the Video SDK no longer needs to terminate the signaling connection when an app is backgrounded.
• Removed the deprecated TVIVideoCapturer APIs:
  • TVIVideoCapturer
  • TVICameraCapturer
  • TVIScreenCapturer
  • TVIVideoConstraints
  • TVILocalVideoTrack.capturer
  • TVILocalVideoTrack.constraints
  • TVILocalVideoTrack.trackWithCapturer:
  • TVILocalVideoTrack.trackWithCapturer:enabled:constraints:name:
• Removed TVIAudioOptions that are no longer supported by WebRTC:
  • TVIAudioOptions.levelControl
  • TVIAudioOptions.levelControlInitialPeakLevelDBFS
  • TVIAudioOptionsBuilder.levelControl
  • TVIAudioOptionsBuilder.levelControlInitialPeakLevelDBFS
• Removed the TVIVideoRenderer.optionalPixelFormats property due to lack of use. A TVIVideoRenderer is expected to support any valid TVIPixelFormat or drop incoming frames if it can't process them.

Bug Fixes

• Participants can send and receive messages that are larger than 16 KB.
• Renamed kDefaultAVAudioSessionConfigurationBlock to kTVIDefaultAVAudioSessionConfigurationBlock so that TwilioVoice.framework and TwilioVideo.framework may be used in the same application.
• Fixed a crash that could occur when using a TVIVideoSource, and the source's buffer pool becomes empty.

Known Issues

• Future 3.0.0-beta releases will reduce the number of round-trips required to connect to a Room. [CSDK-2902]
• If heartbeats are not received, the signaling connection may take up to 45 seconds to be disconnected instead of 15 seconds. [CSDK-2927]
• The SDK may crash if Network Quality or Dominant Speaker is enabled, and connectivity to the Room is lost, and the Room is deleted from within [TVORoomDelegate didDisconnectWithError:]. [CSDK-2950]
• Setting TVILogModuleSignaling does not produce any logging. [ISDK-2446]
• In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
• The WebSocket might close with a partial_deflate_block (code 5) error. [CSDK-2949]
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact