Changelog: Twilio Video iOS Latest

• Programmable Video iOS SDK 5.8.3 [XCFramework] (checksum: 748818a981846612cdef64ac1fe3b8593e75f002116a16421eb5d589540fcfa8).

Bug Fixes

• This release uses a WebRTC patch that addresses a potential ICE gathering or network handover issue on iOS 18.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work arounds.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

• Programmable Video iOS SDK 5.8.2 [XCFramework] (checksum: 2f4466fd2bcbfc497fbe928099fb81d9d521996d2a008f7736cd64dcb857ecc2).

Enhancements

• The Privacy Manifest is updated since the SDK does not collect API usage or call quality data for tracking purposes according to Apple's documentation.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workarounds.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

• Programmable Video iOS SDK 5.8.1 [XCFramework] (checksum: 39ea1c10856e7b3019f0311b34d97a149074c8db79fd6e59618399d592919245).

Enhancements

• The SDK now includes the privacy manifest in the XCFramework. The SDK collects analytic data such as WebRTC stats when the insightsEnabled property is enabled in TVIConnectOptions. The required reason APIs included in the privacy manifest are for internal usage by WebRTC and are not used directly by the Video SDK.
• The SDK XCFramework is now signed with Twilio's distribution signature.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workarounds.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Report

Enhancements

• This version contains 'VideoTrackStoringSampleBufferVideoView' class which simplifies the implementation of Picture in Picture.
• This release is based on Chromium WebRTC 112.
• iSAC Codec is no longer supported.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workarounds.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Report

This release is now available through CocoaPods again as we have resolved the issue.

Bug Fixes

• Updated documentation for didFailToPublishVideoTrack callback.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

This release adds the possibility to enable camera multitasking for a video session.

Enhancements

• Camera multitasking allows users to continue broadcasting their camera even if the video application is put to background.

Limitations

This feature is supported on a limited set of devices:

• iPad Pro (4th generation)
• iPad Pro (5th generation)
• iPad Air (5th generation)

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

This release is not available through CocoaPods because CocoaPods has incorrectly removed the ownership of this project from Twilio. We will publish the pod for this release as soon as the ownership issue is resolved.

Enhancements

• This release now uses WebRTC-m105

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• The following thread performance checker warning has been resolved

"Thread Performance Checker: -[AVCaptureSession startRunning] should be called from background thread. Calling it on the main thread can lead to UI unresponsiveness"

Maintenance

• Bitcode is now disabled for all builds due to deprecation by Apple since the release of Xcode 14. See excerpt below from Xcode 14 Release Notes: Deprecations

"Starting with Xcode 14, bitcode is no longer required for watchOS and tvOS applications, and the App Store no longer accepts bitcode submissions from Xcode 14."

"Xcode no longer builds bitcode by default and generates a warning message if a project explicitly enables bitcode: "Building with bitcode is deprecated. Please update your project and/or target settings to disable bitcode." The capability to build with bitcode will be removed in a future Xcode release. IPAs that contain bitcode will have the bitcode stripped before being submitted to the App Store. Debug symbols can only be downloaded from App Store Connect / TestFlight for existing bitcode submissions and are no longer available for submissions made with Xcode 14. (86118779)"

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Features

• This release introduces support for a noise cancellation audio device, NoiseCancellationProcessor via a new TwilioAudioProcessors SDK

API Changes

• TVIAudioDeviceFormatChanged method is now marked as deprecated and will raise a warning to use TVIAudioDeviceReinitialize instead. The API will be removed in v6.0+.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Enhancements

• Picture in Picture mode is now supported with the SDK. A new renderer SampleBufferVideoView is added for picture in picture support. In order for picture in picture to work on iOS 15+, your app must have com.apple.developer.avfoundation.multitasking-camera-access entitlement.

The following code, from our Twilio Video iOS App, illustrates implementing picture in picture using SampleBufferVideoView:

1
class PictureInPictureViewController: UIViewController {
2

3
    @IBOutlet weak var videoView: VideoView!
4

5
    private var pipController: AVPictureInPictureController!
6
    private var pipVideoCallViewController: AVPictureInPictureVideoCallViewController!
7

8
    override func viewDidLoad() {
9
        let sampleBufferVideoCallView = SampleBufferVideoView()
10
        sampleBufferVideoCallView.contentMode = .scaleAspectFit
11

12
        pipVideoCallViewController = AVPictureInPictureVideoCallViewController()
13
        pipVideoCallViewController.preferredContentSize = CGSize(width: 1080, height: 1920)
14
        pipVideoCallViewController.view.addSubview(sampleBufferVideoCallView)
15

16
        sampleBufferVideoCallView.translatesAutoresizingMaskIntoConstraints = false
17
        let constraints = [
18
            sampleBufferVideoCallView.leadingAnchor.constraint(equalTo: pipVideoCallViewController.view.leadingAnchor),
19
            sampleBufferVideoCallView.trailingAnchor.constraint(equalTo: pipVideoCallViewController.view.trailingAnchor),
20
            sampleBufferVideoCallView.topAnchor.constraint(equalTo: pipVideoCallViewController.view.topAnchor),
21
            sampleBufferVideoCallView.bottomAnchor.constraint(equalTo: pipVideoCallViewController.view.bottomAnchor)
22
        ]
23
        NSLayoutConstraint.activate(constraints)
24

25

26
        sampleBufferVideoCallView.bounds = pipVideoCallViewController.view.frame
27

28
        let pipContentSource = AVPictureInPictureController.ContentSource(
29
            activeVideoCallSourceView: videoView,
30
            contentViewController: pipVideoCallViewController
31
        )
32

33
        pipController = AVPictureInPictureController(contentSource: pipContentSource)
34
        pipController.canStartPictureInPictureAutomaticallyFromInline = true
35
        pipController.delegate = self
36
    }
37
}
38

Please note that the render hints are not supported in picture in picture mode except for the track switch on and off hints when the SampleBufferVideoView renderer is added and removed from the RemoteVideoTrack respectively. As a result when the app is in the background and in picture in picture mode, the displayed RemoteVideoTrack will still be active (e.g Dominant Speaker track), consuming bandwidth, and it will have the default spatial dimension determined by the media server.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed a bug in Adaptive Simulcast where processing of capturer parameter change events was causing a crash and debug check failure. [VIDEO-11409]

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Max video bandwidth limits are now applied to each simulcast layer when simulcast is enabled, and the expected behavior for max video bitrates with simulcast is better documented.
• Fixed several bugs that could cause crashes and deadlocks when a Room is disconnected / destroyed.
• Fixed a use-after-free bug associated with tasks posted to TaskQueue that could lead to undefined behavior.
• Fixed a bug which could cause a crash in IceConnectionActivityMonitor when reconnecting to or disconnecting from a Room.
• Fixed a race condition where subscribed callbacks for Remote Participant could be raised for tracks that have been ended.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Bug Fixes

• Fixed symbol naming conflicts for side-by-side support of third-party WebRTC builds. #207

Known Issues

• On iOS 15.4+, lower frame rate may be observed when H.264 codecs are used. #238
• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Features

• This release introduces Adaptive Simulcast along with new Video Encoding modes for publishers. Adaptive Simulcast reduces publisher resource usage by only sending those encodings for a video track that are needed by subscribers or by the media server (when recording is enabled). This is an opt-in feature and can be enabled by setting the new Connect Option videoEncodingMode to auto, which allows the SDK to select video codecs, use adaptive simulcast and manage encodings automatically. This feature cannot be used alongside EncodingParameters.maxVideoBitrate.

API Changes

• Added the videoEncodingMode property to ConnectOptions.
• The videoEncodingMode API is mutually exclusive with existing codec management APIs EncodingParameters.maxVideoBitrate and preferredVideoCodecs.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]

Size Impact

Maintenance

• The minimum iOS version supported is 12.2.
• This release is built with Xcode 13.

Enhancements

• Microphone access will only be requested when recording audio and publishing an audio track.

Notes

• Starting with iOS 14.5, apps can overwrite the audio interruption caused by smart cover use on iPads. See #225 for more details.

Bug Fixes

• Fixed a bug which could occur when connected to a Peer-to-Peer or Go Room and publishing a second video track before another Participant joins, causing the connection to be terminated or the second video track not to be fully established.
• Fixed a bug which could cause the Room connection to be terminated by the server.
• Fixed a bug which could cause the media connection not to be re-established after network changes.
• Fixed a bug where audio fails to start in the iOS Simulator on M1 Mac Mini when no microphone is present.

Known Issues

• Audio playback fails in some cases when running a simulator on a Mac Mini. #182
• Unpublishing and republishing a LocalAudioTrack or LocalVideoTrack might not be seen by Participants. #34
• H.264 video might become corrupted after a network handoff. #147
• iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested workaround.
• Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]