Add Translation and Transcription to Voice Calls with SignalWire RELAY

Create a voicemail number that translates and transcribes messages and displays them on a webpage!

Moheeb Zara | Developer Advocate

Jun 20, 2022

SignalWire RELAY is an advanced interactive communications API that lets you build powerful applications using voice and SMS. It can run in the browser or server side. To dive deeper into RELAY, check our in-depth getting started guide.

In this example, available on GitHub, we use RELAY and the Google Cloud Speech and Translate APIs. The following instructions will help you get up and running using CodeSandBox, a browser based platform for running code.

There are two parts to this example. The first is a backend API that serves call records and also runs a consumer which is a class that encapsulates the main connection and call handling features of RELAY.

The second is a React frontend application that displays call records by calling the backend. It shows the number calls came from, a translation of the audio, and the ability to play the original call audio.


Prerequisites

To run this example you will need to first create a SignalWire account and then configure a SignalWire Space with an inbound phone number.

API Credentials

To run the example a SignalWire Project ID and API Token are required. Follow these instructions to create a new API Token.

Configure SignalWire Phone Number to a context

A RELAY consumer can register with an arbitrary context. This context can be used to direct phone numbers to the consumer application in certain situations. The instructions below show you how to configure a context on a SignalWire Phone Number in the event of incoming calls.

Navigate to the edit page for your newly created SignalWire Phone Number from the SignalWire Space dashboard.

Set the Relay context that your phone number forwards to as translator

Google Speech and Translate

When the RELAY consumer receives a call it will prompt the user to leave a message and press the # key. Their response is recorded by SignalWire and a URL where the wav file containing the audio is stored is fetched from the RELAY API. That audio file is downloaded and then passed to the Google Speech and Translation API.

To access the Google Speech to text and language translation API, you need to create an account and some credentials.

Follow these instructions https://cloud.google.com/speech-to-text/docs/libraries

Be sure to include permissions for the translation API as well.

Upon completion you will have downloaded a JSON file with your google key.

Save this file as google-key.json

We’ll use this file later.

Configure and Run the Backend

You can follow the instructions in the GitHub repository on how to run this locally or you can use a CodeSandBox we’ve created that is ready to run.

https://codesandbox.io/s/sw-relay-consumer-translate-transcribe-public-fec50i

Once the CodeSandBox is open, upload your google-key.json file to the root directory of the project.

Next, create a file .env with the following, replacing with values that correspond with your SignalWire account from the previous steps.

SIGNALWIRE_PROJECT_KEY=<Your-Signalwire-Project-Key>
SIGNALWIRE_TOKEN=<your-Token>
VERIFIED_NUMBER=<Your signalwire phone number including the region code>
GOOGLE_APPLICATION_CREDENTIALS=google-key.json

CodeSandbox will auto update and run on any file save operation.

To use in the next section, copy the URL of the running instance from the embedded browser. This URL won’t change and as long as you have the CodeSandBox window open the backend will continue to run.

You can now test that the consumer is working by calling your SignalWire Phone number!

To change the languages that are detected and translated, you can alter the Google Translate configuration in the helper.js file. For a list of supported language codes see the Google Documentation. This example will detect if the caller is speaking in Spanich, English, or Japanese and translate to English.

Running the Frontend

Open the CodeSandBox for the frontend app

https://codesandbox.io/s/sw-relay-transcribe-translate-frontend-public-1bj43d

The frontend needs to know the backend URL .Edit the .env file and add the URL copied from the previous section.

API_URL=<Your backend URL>

Once you save the file the sandbox will restart.

Navigating to the React App hosted in CodeSandbox will now show call records and translations, if there are any.


Congrats! You’ve deployed a custom voice call translation and transcription app! From here you can continue to play with and modify the application. Be sure to checkout our documentation to learn how much more you can do from this starting point!

Here at SignalWire, we love our community of developers! Find us on the SignalWire Twitter and be sure to join our SignalWire Community Slack. We’re always happy to help answer questions and we look forward to engaging with you!