Changelog: Twilio Video iOS 4.x

API Changes

• New error code TVIErrorParticipantSessionLengthExceededError (53216) is added to support configurable participant session limit.

Bug Fixes

• Fixed a bug where media could incorrectly be detected as not flowing after the network connectivity changes. [VIDEO-7685]

Known Issues

• Audio playback fails when running a simulator on a Mac Mini. #182
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Enhancements

• Added support for Audio Only rooms.
• Improved reconnection time after network connectivity disruptions.
• This release is built with Xcode 12.5.1

API Changes

• Two new error codes TVIErrorRoomAudioOnlyFlagNotSupportedError (53124) and TVIErrorRoomTrackKindNotSupportedError (53125) are added to support Audio Only rooms.

Known Issues

• Audio playback fails when running a simulator on a Mac Mini. #182
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Bug Fixes

• Fixed an interoperability bug between JavaScript, iOS and Android SDKs which could cause subscription events not to fire in a Peer-to-Peer or Go Room. [VIDEO-7334] #211

Known Issues

• Audio playback fails when running a simulator on a Mac Mini. #182
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

This release adds support for Apple Silicon arm64 Macs, updates WebRTC and modernizes our use of several WebRTC APIs. Thank you for your feedback on 4.6.0-beta1 and for your interest in Apple Silicon development.

Features

• TwilioVideo.xcframework now supports the simulator on Apple Silicon arm64 Macs. #144
  • This results in a size increase for the SDK during development.
  • App store download size is not affected.

Enhancements

• This release is based on WebRTC M88.
• The SDK uses Unified Plan SDP semantics instead of Plan-B. This brings several important benefits:
  • Improved interoperability with Firefox, Safari and Chrome in Peer-to-Peer and Go Rooms
  • Track level operations like publishing and unpublishing are more reliable
• Improved performance of Room.getStats() by adopting the W3C standardized RTC stats API
• Unused codecs for a track are removed from the local SDP offer in order to reduce the SDP size, once codecs for the track have been negotiated. [VIDEO-3532]

API Changes

• The ordering of RemoteParticipantDelegate.onAudioTrackSubscribed() and RemoteParticipantDelegate.onVideoTrackSubscribed() is not strictly guaranteed. In this release the ordering of these callbacks may be different when compared to previous releases.
• The value of LocalVideoTrackStats.framesEncoded now reflects the total number of frames encoded for a simulcast track.

Bug Fixes

• Fixed performance problems with Room.getStats() where block callbacks could sometimes be significantly delayed. [CSDK-3475]
• Fixed an interoperability issue with twilio-video.js when publishing media in Peer-to-Peer or WebRTC Go Rooms. #931

Known Issues

• Audio playback fails when running a simulator on a Mac Mini. #182
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

This release contains a significant update to the Bandwidth Profile API. It allows for more efficient use of bandwidth and CPU in multi-party applications. In addition it provides developers with more dynamic control over which video tracks are delivered to the client and the preferred video resolution of the tracks. These capabilities are provided via the Client Track Switch Off Control and Video Content Preferences settings.

Existing Bandwidth Profile settings will continue to function as before, however we recommend developers update their Bandwidth Profile settings to make use of these new capabilities at their earliest convenience.

• This feature allows subscribers to control whether the media for a RemoteVideoTrack is received or not. Client Track Switch Off Control has two modes of operation:
  • auto (default): The SDK determines whether tracks should be switched off based on renderer attachment, view visibility, and application lifecycle.
  • manual: The application requests that individual tracks be switched off or on using the RemoteVideoTrack.switchOff() / switchOn() methods.
• Note: If your application previously set the maxTracks property to limit the number of tracks visible, you should migrate to using clientTrackSwitchOffControl to take advantage of this feature.

• This feature allows subscribers to specify preferences about the media that they receive on a RemoteVideoTrack. Video content preferences has two modes of operation:
  • auto (default): The SDK specifies content preferences based on video view size. A RemoteVideoTrack rendered by a VideoView with larger dimensions will get a higher quality video compared to a RemoteVideoTrack rendered by a VideoView with smaller dimensions.
  • manual: The application specifies the content preferences for individual tracks using RemoteVideoTrack.setContentPreferences().
• Note: If your application previously set the renderDimensions property, you should migrate to using contentPreferencesMode to take advantage of this feature.

Both of these features are available in Group Rooms and are enabled by default if your application specifies BandwidthProfileOptions during connect.

1
let connectOptions = ConnectOptions(token: accessToken) { (builder) in
2
  builder.bandwidthProfileOptions = BandwidthProfileOptions(
3
    videoOptions: VideoBandwidthProfileOptions { builder in
4
      // Use "auto" default. Be sure to remove "maxTracks" and "renderDimensions".
5
    }
6
  )
7
}
8

9
let room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)

If you don't want the SDK to automatically switch on/off RemoteVideoTracks then specify ClientTrackSwitchOffControl.manual and VideoContentPreferencesMode.manual instead.

1
let connectOptions = ConnectOptions(token: accessToken) { (builder) in
2
  builder.bandwidthProfileOptions = BandwidthProfileOptions(
3
    videoOptions: VideoBandwidthProfileOptions { builder in
4
      builder.clientTrackSwitchOffControl = .manual
5
      builder.contentPreferencesMode = .manual
6
    }
7
  )
8
}
9

10
let room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)

Participants that connect with ClientTrackSwitchOffControl.manual can request which RemoteVideoTracks are switched on or off.

1
func didSubscribeToVideoTrack(videoTrack: RemoteVideoTrack,
2
                             publication: RemoteVideoTrackPublication,
3
                             participant: RemoteParticipant) {
4
  if participant.identity != "Bob" {
5
    videoTrack.switchOff()
6
  }
7
}

Subscribers that connect with VideoContentPreferencesMode.manual can request which resolution they prefer to receive RemoteVideoTracks in.

1
func didSubscribeToVideoTrack(videoTrack: RemoteVideoTrack,
2
                             publication: RemoteVideoTrackPublication,
3
                             participant: RemoteParticipant) {
4
    videoTrack.setContentPreferences(VideoContentPreferences { builder in
5
        builder.renderDimensions = VideoDimensions(width: 320, height: 240)
6
    })
7
}

• The VideoBandwidthProfileOptions.maxTracks property is now deprecated and will raise a warning when set. Calling RemoteVideoTrack.switchOn() or RemoteVideoTrack.switchOff() after setting maxTracks is not allowed and will raise an exception.
• The VideoBandwidthProfileOptions.renderDimensions property is now deprecated and will raise a warning when set. Calling RemoteVideoTrack.setContentPreferences() after setting renderDimensions is not allowed and will raise an exception.

• Fixed a bug where correct recording state of the Room was not available right after the RoomDelegate.roomDidConnect(room:) delegate was invoked. [VIDEO-4616]
• Fixed a bug that could cause a crash when changing log level to Trace at runtime. [VIDEO-5893]

• Video Content Preferences might prefer larger video than needed when device orientations are mismatched. For example, if a participant in landscape mode publishes video, then the subscribing participant must also be in landscape mode in order for the correctly sized simulcast layers to be selected. The same is true for the portrait orientation.
• When the publisher is publishing video at 720p with VP8 simulcast enabled and the subscriber varies the hints from 180p, 360p, and 720p, sometimes the subscriber receives larger video than expected.
• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Enhancements

• 100 Participant Group Rooms Pilot Program: A Group Room created with max participants greater than 50 is structured to support a small number of presenters and a large number of viewers. It has the following behavioral differences compared to regular Group Rooms:
  • The RoomDelegate.participantDidConnect() delegate method is invoked when a RemoteParticipant connects to the Room and publishes at least one Track.
  • The RoomDelegate.participantDidDisconnect() delegate method is invoked when a RemoteParticipant disconnects from the Room or unpublishes all of its Tracks.
  • If a RemoteParticipant unpublishes all of its tracks (resulting in the RoomDelegate.participantDidDisconnect() delegate method being invoked) and later republishes a track, a new RemoteParticipant object will be provided in the subsequent RoomDelegate.participantDidConnect() delegate method invocation with the same Participant Sid as before.
  • The maximum number of published Tracks in a Room at the same time cannot exceed 16. Attempts to publish more Tracks will result in a publication failure with ParticipantMaxTracksExceededError unless one or more published Tracks is unpublished.
  • Contact your Twilio Account Executive to enroll in this pilot program.

Bug Fixes

• Fixed a crash when network manager was still invoking onNetworksChanged callback while network monitor was being destroyed.

Known Issues

• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

API Changes

• Added new error code, CameraSource.Error.cameraPermissionDenied

Bug Fixes

• CameraSouce.startCapture() now properly raises the CameraSource.Error.cameraPermissionDenied error when attempting to start capturing the camera when a user has declined the camera permissions.

Known Issues

• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Enhancements

• The minimum iOS version supported at build time has been lowered from 11.0 to 9.0. Though the Twilio Video APIs are only available on iOS 11.0 and above, you can still build iOS 9.x and 10.x apps with the Video SDK.

API Changes

• Calling VideoTrack.addRenderer() multiple times with the same VideoRenderer now throws a helpful NSInvalidArgumentException instead of firing NSAssert.

Known Issues

• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• The minimum supported iOS version has been lowered from 12.0 to 11.0. On iOS 11.x simulators, video rendering with VideoView is not supported.

Bug Fixes

• Fixed a bug where the VideoView was not applying the native scale factor to the drawable size. This was a regression introduced in version 4.0.1.#159

Known Issues

• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a bug where the video renderer was setting the drawable size incorrectly. This was causing pixelated video when rendered on an external display using the Screen Mirroring.#123
• Fixed a bug where publishing multiple Tracks with the same name may result in a crash if network quality is enabled.

Distribution Changes

• Removed the Poco section from Acknowledgements.md since Poco is no longer a dependency

Known Issues

• VideoView may display pixelated video. #159
• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• The iOS SDK is based on WebRTC-83.
• The iOS SDK is built with Xcode 12.0.
• Twilio Video is now delivered as an .xcframework. The .xcframework package includes the .dSYM and .bcsymbolmap files to allow developers to symbolicate crash reports.
• Twilio Video is now distributed as a Swift Package.
• Enabled Metal rendering on the iOS simulator.
• You can now share video of your app's screen to a Room using AppScreenSource. The Video SDK uses ReplayKit internally for in-app screen capture. AppScreenSource conforms to VideoSource and uses RPScreenRecorder to capture video of your app's screen. Here is a brief example:

1
if let source = AppScreenSource(), let track = LocalVideoTrack(source: source) {
2
    room.localParticipant?.publishVideoTrack(track)
3
    source.startCapture()
4
}

• Discontinuous transmission (DTX) is enabled by default for the Opus codec. Disabling DTX will result in higher bitrate for silent audio while using the Opus codec. The TVIOpusCodec class now has a new initializer [TVIOpusCodec initWithDtxEnabled:] and a property dtxEnabled.
• Added the TVIParticipantState enumeration.
• Added the TVIParticipant.state property.
• Added new delegate methods [TVIRoomDelegate room:participantIsReconnecting:] and [TVIRoomDelegte room:participantDidReconnect:] to TVIRoomDelegate. These callbacks will be raised when a TVIRemoteParticipant is attempting to reconnect to a room due to a signaling network interruption. NOTE: It can take up to 15 seconds for our signaling backend to detect that a RemoteParticipant's connection has been disrupted due to a network degradation or handoff.
• isRecording property in TVIRoom now accurately reflects the current recording state of the TVIRoom. In the previous versions of the SDK, isRecording could return false positives. In TVIRoomDelegate the roomDidStartRecording and roomDidStopRecording callbacks will now be invoked when recording for at least a single track in the TVIRoom has begun and ended respectively.

API Updates

• Added AppScreenSource to control video capture of your app's screen.
• Removed the TVIRemoteParticipant.connected property in favor of TVIParticipant.state.

Removed the following deprecated APIs

• Removed the deprecated OpenGL Video Renderer APIs:
  • VideoView.RenderingType
  • [VideoView initWithFrame:delegate:renderingType:renderingType]
• Removed the deprecated abortOnIceServersTimeout and iceServersTimeout properties from IceOptions and IceOptionsBuilder.

Distribution Changes

• Twilio Video iOS SDK is no longer being shipped as a static library.
• Twilio Video no longer depends on the Poco C++ Libraries.

Known Issues

• The TwilioVideo.xcframework does not currently support the simulator on Apple Silicon arm64 Macs.
• Carthage is not currently a supported distribution mechanism for Twilio Video. Carthage does not currently work with .xcframeworks as documented here. Once Carthage supports binary .xcframeworks, Carthage distribution will be re-added.
• Unpublishing and republishing a TVILocalAudioTrack or TVILocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
• Publishing multiple tracks with the same name may result in a crash if network quality is enabled. To avoid this, use unique names for each track in the TVIRoom.

Size Impact