TwiML™ Voice: <Transcription>

The table below lists <Transcription>'s supported attributes, which modify the <Transcription> behavior. All attributes are optional.

The user-specified name of this Real-Time Transcription. This name can be used to stop the Real-Time Transcription.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', name: 'Contact center transcription'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" name="Contact center transcription" />
5
    </Start>
6
</Response>

The statusCallbackUrl attribute is the relative or absolute URL of an endpoint. Twilio sends Real-Time Transcription status updates and the call's transcript data to this URL.

Twilio sends a POST request to this URL whenever one of the following occurs:

• A Real-Time Transcription session starts. This is called the transcription-started event.
• Utterances (partial or final) of transcribed audio is available. This is called the transcription-content event.
• A Real-Time Transcription session stops. This is called the transcription-stopped event. This event occurs when a Real-Time Transcription session is stopped via API or TwiML, or when the call ends.
• An error occurs. This is called the transcription-error event.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url"/> 
5
    </Start>      
6
</Response>

When a Real-Time Transcription is started and a session is created, Twilio sends an HTTP POST request to your statusCallbackUrl for the transcription-started event. This event provides initial details about the transcription session.

These HTTP requests contain the properties listed below.

Example of a transcription-started event payload:

1
{
2
  "TranscriptionSid": "GT8fbf72a043b98407a3ce68331cd0030a",
3
  "Timestamp": "2024-06-25T18:45:12.135751Z",
4
  "AccountSid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
5
  "ProviderConfiguration": "{\"profanityFilter\":\"true\",\"speechModel\":\"telephony\",\"enableAutomaticPunctuation\":\"true\",\"hints\":\"Alice Johnson, Bob Martin, ACME Corp, XYZ Enterprises, product demo, sales inquiry, customer feedback\"}",
6
  "Name": "Chris Transcription",
7
  "OutboundTrackLabel": "agent",
8
  "LanguageCode": "en-US",
9
  "PartialResults": "false",
10
  "InboundTrackLabel": "customer",
11
  "TranscriptionEvent": "transcription-started",
12
  "CallSid": "CAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
13
  "TranscriptionEngine": "google",
14
  "Track": "both_tracks",
15
  "SequenceId": "1"
16
}

When an individual utterance (partial or final) of audio is transcribed, Twilio sends an HTTP POST request to your statusCallbackUrl for the transcription-content event. This event provides TranscriptionData results for the transcribed audio.

These HTTP requests contain the properties listed below.

Example of a transcription-content event payload when partialResults is equal to false:

1
{
2
  "LanguageCode": "en-US",
3
  "TranscriptionSid": "GT8fbf72a043b98407a3ce68331cd0030a",
4
  "TranscriptionEvent": "transcription-content",
5
  "CallSid": "CAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
6
  "TranscriptionData": "{\"transcript\":\"Hello, this is Sam from Horizon Financial Services. Just letting you know this call may be recorded for quality purposes. How can I assist you today?\",\"confidence\":0.9956335}",
7
  "Timestamp": "2024-06-25T18:45:21.454203Z",
8
  "Final": "true",
9
  "AccountSid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
10
  "Track": "outbound_track",
11
  "SequenceId": "2"
12
}

Example of a transcription-content event payload when partialResults is equal to true:

1
{
2
  "LanguageCode": "en-US",
3
  "TranscriptionSid": "GT6ebb54a123f0c86b70605a4925836f69",
4
  "Stability": "0.9",
5
  "TranscriptionEvent": "transcription-content",
6
  "CallSid": "CAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
7
  "TranscriptionData": "{\"transcript\":\"Hello, this is Sam from Horizon Financial Services. Just letting you know this call may be recorded for\"}",
8
  "Timestamp": "2024-06-25T16:30:21.600697Z",
9
  "Final": "false",
10
  "AccountSid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
11
  "Track": "outbound_track",
12
  "SequenceId": "70"
13
}

When a Real-Time Transcription session is stopped or ends, Twilio sends an HTTP POST request to your statusCallbackUrl for the transcription-stopped event. This event provides final details about the transcription session.

These HTTP requests contain the properties listed below.

An example of the transcription-stopped event payload:

1
{
2
  "TranscriptionSid": "GT8fbf72a043b98407a3ce68331cd0030a",
3
  "TranscriptionEvent": "transcription-stopped",
4
  "CallSid": "CAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
5
  "Timestamp": "2024-06-25T18:45:23.839266Z",
6
  "AccountSid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
7
  "SequenceId": "3"
8
}

When an error occurs during a Real-Time Transcription session, Twilio sends an HTTP POST request to your statusCallbackUrl for the transcription-error event.

These HTTP requests contain the properties listed below.

Example of a transcription-error event payload:

1
{
2
  "TranscriptionSid": "GT20dfa03c8cf8aa8d0c4aeccde5558b66",
3
  "Timestamp": "2023-10-19T22:33:22.611Z",
4
  "AccountSid": "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
5
  "SequenceId": "3",
6
  "TranscriptionEvent": "transcription-error",
7
  "CallSid": "CAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
8
  "TranscriptionErrorCode": "32655",
9
  "TranscriptionError": "Provider Unavailable"
10
}

The languageCode attribute specifies the language in which the transcription should be performed. It accepts a BCP-47 standard language code, such as en-US for American English. This attribute is useful for ensuring that the transcription engine correctly understands and processes the spoken language.

The following TwiML example demonstrates how to specify the languageCode attribute for a transcription for Mexican Spanish. This ensures that the transcription is performed in the specified language, which is particularly useful for calls in languages other than English.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', languageCode: 'es-MX'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" languageCode="es-MX" /> 
5
    </Start>      
6
</Response>

The track attribute specifies which audio track should be transcribed. It can take one of the following values: inbound_track, outbound_track, or both_tracks. This attribute is useful for determining whether to transcribe the audio coming from the caller, the callee, or both.

The following TwiML example demonstrates how to specify the track attribute for a transcription.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', track: 'inbound_track'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" track="inbound_track" /> 
5
    </Start>      
6
</Response>

The inboundTrackLabel attribute allows you to associate an alphanumeric label with the inbound track being transcribed. This can be useful for identifying and differentiating the inbound audio stream in the transcription results. Using labels helps to clearly identify who is speaking, especially in multi-party conversations or call center scenarios.

Refer to the Track labels section below to understand the importance of using labels.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', inboundTrackLabel: 'agent', outboundTrackLabel: 'customer'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" inboundTrackLabel="agent" outboundTrackLabel="customer" /> 
5
    </Start>      
6
</Response>

In an inbound call scenario, the call is initiated by the customer and received by the agent or business person. Here, the inbound audio track (agent's speech) is labeled for clarity in the transcription results.

1
<Response>
2
  <Start>
3
    <Transcription track="inbound_track" inboundTrackLabel="agent" />
4
  </Start>
5
</Response>

In this example, the inbound audio track is labeled as "agent". This is useful for scenarios like customer support calls, where distinguishing the agent's responses from the customer's speech is crucial for understanding the interaction.

In an outbound call scenario, the call is initiated by the agent or business person and received by the customer. Here, the inbound audio track (customer's speech) is labeled for clarity in the transcription results.

1
<Response>
2
  <Start>
3
    <Transcription track="inbound_track" inboundTrackLabel="customer" />
4
  </Start>
5
</Response>

In this example, the inbound audio track is labeled as "customer". This is useful for scenarios like sales calls, where distinguishing the customer's speech in the transcription can help in analyzing customer feedback and engagement.

The outboundTrackLabel attribute allows you to associate an alphanumeric label with the outbound track being transcribed. This can be useful for identifying and differentiating the outbound audio stream in the transcription results. Using labels helps to clearly identify who is speaking, especially in multi-party conversations or call center scenarios.

Refer to the Track labels section below to understand the importance of using labels.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', inboundTrackLabel: 'agent', outboundTrackLabel: 'customer'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" inboundTrackLabel="agent" outboundTrackLabel="customer" /> 
5
    </Start>      
6
</Response>

In an inbound call scenario, the call is initiated by the customer and received by the agent or business person. Here, the outbound audio track (customer's speech) is labeled for clarity in the transcription results.

1
<Response>
2
  <Start>
3
    <Transcription track="outbound_track" outboundTrackLabel="customer" />
4
  </Start>
5
</Response>

In this example, the outbound audio track is labeled as "customer". This is useful for scenarios like customer support calls, where distinguishing the customer's speech from the agent's responses is crucial for understanding the interaction.

In an outbound call scenario, the call is initiated by the agent or business person and received by the customer. Here, the outbound audio track (agent's speech) is labeled for clarity in the transcription results.

1
<Response>
2
  <Start>
3
    <Transcription track="outbound_track" outboundTrackLabel="agent" />
4
  </Start>
5
</Response>

In this example, the outbound audio track is labeled as "agent". This is useful for scenarios like sales calls, where distinguishing the agent's speech in the transcription can help in analyzing the effectiveness of the sales pitch.

The transcriptionEngine attribute allows you to specify the name of the speech-to-text transcription provider to be used. This can be useful for leveraging specific features or optimizations provided by different transcription engines.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', transcriptionEngine: 'google'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" transcriptionEngine="google" /> 
5
    </Start>      
6
</Response>

The speechModel attribute allows you to specify which speech model to use for the transcription.

Maps to Transcription Model in Google terminology. Different speech models can optimize for different use cases, such as phone calls, video, or enhanced models for higher accuracy.

Refer to the Google documentation to understand each speech model's specific capabilities and configurations.

The telephony speech model is optimized for phone call audio and can provide better accuracy for this type of audio.

The long speech model is optimized for long-form audio, such as lectures or extended conversations, and can provide better accuracy for lengthy audio.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', speechModel: 'telephony', transcriptionEngine: 'google'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" speechModel="telephony" transcriptionEngine="google" /> 
5
    </Start>      
6
</Response>

Maps directly to the profanityFilter in Google's RecognitionFeatures object. The profanityFilter attribute allows you to enable or disable the filtering of profane words in the transcription. When enabled, the transcription engine will attempt to mask or omit any detected profanities in the transcription results.

The example below demonstrates how to enable the profanity filter for the transcription. This is useful for ensuring that any profane language is masked or omitted in the transcription output.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', profanityFilter: false, transcriptionEngine: 'google'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" profanityFilter="false" transcriptionEngine="google" /> 
5
    </Start>      
6
</Response>

Maps to StreamingRecognitionResult specifically when ("is_final"=false) in Google Terminology. The partialResults attribute allows you to enable or disable the delivery of interim transcription results. When enabled, the transcription engine will send partial (interim) results as the transcription progresses, providing more immediate feedback before the final result is available.

The example below demonstrates how to enable partial results for the transcription.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', partialResults: true, transcriptionEngine: 'google'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" partialResults="true" transcriptionEngine="google" /> 
5
    </Start>      
6
</Response>

The hints attribute contains a list of words or phrases that the transcription provider can expect to encounter during a Real-Time Transcription. Using the hints attribute can improve the transcription provider's recognition of words or phrases you expect from your callers.

You may provide up to 500 words or phrases in this list, separating each entry with a comma. Your hints may be up to 100 characters each, and you should separate each word in a phrase with a space, e.g.:

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', hints: 'Alice Johnson, Bob Martin, ACME Corp, XYZ Enterprises, product demo, sales inquiry, customer feedback'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" hints="Alice Johnson, Bob Martin, ACME Corp, XYZ Enterprises, product demo, sales inquiry, customer feedback" /> 
5
    </Start>      
6
</Response>

The hints attribute also supports Google's class token list to improve recognition. You can pass a class token directly in the hints attribute, as shown in the example below.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', hints: '$OOV_CLASS_ALPHANUMERIC_SEQUENCE'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" hints="$OOV_CLASS_ALPHANUMERIC_SEQUENCE" /> 
5
    </Start>      
6
</Response>

Maps to Automatic Punctuation in Google Terminology. The enableAutomaticPunctuation attribute allows you to enable or disable automatic punctuation in the transcription. When enabled, the transcription engine will automatically insert punctuation marks such as periods, commas, and question marks, improving the readability of the transcribed text.

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const start = response.start();
5
start.transcription({statusCallbackUrl: 'https://example.com/your-callback-url', enableAutomaticPunctuation: true, transcriptionEngine: 'google'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Start>
4
        <Transcription statusCallbackUrl="https://example.com/your-callback-url" enableAutomaticPunctuation="true" transcriptionEngine="google" /> 
5
    </Start>      
6
</Response>

Twilio's transcription service supports a variety of languages and models. The examples provided below are specific to Google Speech-to-Text. Depending on the language, certain attributes like speechModel, profanityFilter, and enableAutomaticPunctuation may have different levels of support. For the most up-to-date and comprehensive information, please refer to the Google Speech-to-Text Supported Languages documentation.

This example demonstrates how to configure transcription for Chinese (Simplified, China) using the Chirp Model with support for automatic punctuation.

1
<Response>
2
  <Start>
3
    <Transcription 
4
      transcriptionEngine="google" 
5
      languageCode="cmn-Hans-CN" 
6
      speechModel="chirp" 
7
      enableAutomaticPunctuation="true" />
8
  </Start>
9
</Response>

In this example, the profanityFilter attribute, hints attribute, and other advanced features are not supported for this configuration.

This example demonstrates how to configure transcription for Spanish (Spain) using the telephony model with full support for all attributes.

1
<Response>
2
  <Start>
3
    <Transcription 
4
      transcriptionEngine="google" 
5
      languageCode="es-ES" 
6
      speechModel="telephony" 
7
      profanityFilter="true" 
8
      enableAutomaticPunctuation="true" />
9
  </Start>
10
</Response>

In this example, the telephony model supports automatic punctuation and profanity filter, but not model adaptation (e.g., hints).

This example demonstrates how to configure transcription for Hindi (India) using the short model with support for specific attributes.

1
<Response>
2
  <Start>
3
    <Transcription 
4
      transcriptionEngine="google" 
5
      languageCode="hi-IN" 
6
      speechModel="short" 
7
      enableAutomaticPunctuation="true" 
8
      profanityFilter="true" 
9
      hints="संपर्क, सेवा, समर्थन, ग्राहक" 
10
      modelAdaptation="true" />
11
  </Start>
12
</Response>

In this example, the short model supports automatic punctuation, profanity filter, model adaptation, and hints.

This example demonstrates how to configure transcription for French (Canada) using the long model with support for specific attributes.

1
<Response>
2
  <Start>
3
    <Transcription 
4
      transcriptionEngine="google" 
5
      languageCode="fr-CA" 
6
      speechModel="long" 
7
      hints="service à la clientèle, rendez-vous, commande" />
8
  </Start>
9
</Response>

In this example, the long model supports model adaptation through hints, but does not support automatic punctuation, profanity filter, or spoken punctuation.

If specifying inboundTrackLabel or outboundTrackLabel, the call direction mapping table below can be used as a guide.

Note: A call that has an "outbound" direction is a call that is outbound from Twilio, i.e., from Twilio to a customer.

If you provided a name attribute when starting a Real-Time Transcription session, you can stop a Real-Time Transcription using TwiML or via API.

Given a Real-Time Transcription that was started with the following TwiML instructions:

1
<Response>
2
  <Start>
3
    <Transcription name="Contact center transcription" />
4
  </Start>
5
</Response>

You can stop the Real-Time Transcription with the following TwiML instructions:

1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2

3
const response = new VoiceResponse();
4
const stop = response.stop();
5
stop.transcription({name: 'Contact center transcription'});
6

7
console.log(response.toString());

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
    <Stop>
4
        <Transcription name="Contact center transcription" />
5
    </Stop>
6
</Response>

If a name was not provided, you can stop an in-progress Real-Time Transcription via API using the SID of the Transcription. See the RealtimeTranscription resource API reference page for more information.