Extend Twilio Autopilot by Adding FAQ Functionality

When working with Twilio Autopilot, dealing with FAQs is not as straightforward as with typical user Intents and Tasks.

In this turorial, I'll show you how to add FAQ functionality to a Twilio Autopilot powered bot using Twilio Functions and Elasticsearch, then show you a working result running in WhatsApp.

Let's get started.

Components and architecture of our FAQ integration

In our case, we use the following components to build an FAQ integration:

  • Autopilot Tasks – to direct to the FAQ module. For example, we can make an “unresolved” case, which means Autopilot could not match an existing Intent to the user input (and we then try to see if the query corresponds to something in the FAQ)
  • Twilio Function – to take the specific user input that was not matched to an intent and send it to the backend for analysis.
  • Elasticsearch – to link the backend of the previous step to our knowledge. Here we create the relevant shards that hold the FAQ base. 

Based on the above, the process will proceed as follows:

Autopilot FAQ process

Below you can find information on how to set up all the components.

Set up the Autopilot task

As mentioned above, in this example we direct to the FAQ when Autopilot can't match the user query to an existing Task.

If we don't find a match, we do the following:

  • Create a new task named unresolved
  • Set the new task as "Fallback" and save it
  • Program the new task

All you need in this case is to direct to the /faq function (shown below). You do not need to provide any training phrases for this task as all it does is route to the function that implements the FAQ search functionality.

Task for FAQ integration in a function

And here's how the redirection works:

Task to redirect to the FAQ route

Set up the Twilio Function

For these unresolved routes, we need something to field our requests. That's where Twilio Functions comes into play.

The main steps of our function are:

  • Get user input from the previous step
  • Prepare the payload including the user input. (In this example we use the "More Like This" query. You can find more info here)
  • Send the request to the Elastic shards 
  • Receive the result and show it or say it to the user. (In this example we take only the highest match result. We could easily retrieve more results depending on your use case.)
exports.handler = function(context, event, callback) {
    

    console.log("Current Input: " + event.CurrentInput);

    let userInput = event.CurrentInput;

    /*content, category as the field names that we used for this example*/
    let properties = {
        query: {
            more_like_this: {
                fields: [
                    "content",
                    "category"
                ],
                like: userInput,
                min_term_freq: 1,
                min_doc_freq: 1,
                max_query_terms: 20
            }
        }
    };

    console.log(properties);
    const request = require('request');
    request({
        url: '<FAQ_BASE_URL>/sample/document/_search',
        method: "GET",
        json: properties
    }, function(error, response, body) {

        console.log(error);
        console.log(JSON.stringify(response));
        console.log(body);

        let data = body;
        let answer;

        //We use only the top match
        answer = data.hits.hits[0]._source.content;

        let responseObject = {
            "actions": [{
                "say": {
                    "speech": answer
                }

            }]
        };
        callback(null, responseObject);

    });
};

And that's all you need on the routing side. Next we'll look at setting up the FAQ knowledgebase in elasticsearch.

Create the FAQ documentation

For this step, download elasticsearch from here and follow the guide in order to deploy the instance locally.

Upon deploying, test that everything is working by trying port 9200 on localhost:

FAQ elastic setup

For this test, I added some sample FAQs directly to elasticsearch shards for category and content. Here I used Postman to make direct API calls, using the relevant elasticsearch endpoints. (If you need help, you can find these here.)

Here's how it looks:

FAQ shards

Obviously, this workflow isn't manageable for a huge FAQ. However, it gets you started on an architecture to approach the integration, and shows the calls you'll need to make.

If you are building out a full set of FAQs, here is one approach I'd suggest:

  • Allow a user to upload/edit a CSV with the FAQs in a particular format (or through a simple UI)
  • When the user is satisfied, allow them to submit the document
  • Upon submission, your code would make the relevant API calls to store the documents in the shards

End result: elasticsearch driven FAQ integration in Twilio Autopilot

And how does it look? I'm glad you asked.

Below I've taken a screenshot of the bot flow working in WhatsApp. (You can imagine the bot working in a similar way over Voice, except by reading out the response)

FAQ result

Autopilot integrations your business needs

Your business probably maintains one – or a few – FAQs. In many cases, companies maintain hundreds of questions… attempting to build Tasks out of them is almost impossible and very inefficient.

Unlike straightforward flows, these questions vary significantly. Instead of creating a standalone intent for "Where are you located?", for example, you'd rather provide a direct response instead of following a flow.

Hopefully this integration tutorial helps you as you look for a better path for serving this type of content. I can't wait to see how you use it – and always feel free to reach out to an expert to talk it through.

Evangelos Resvanis is a Solutions Engineer at Twilio. He is currently working on his favorite topics, Autopilot and Twilio Functions, exploring and expanding the possibilities offered by these tools. He can be reached at eresvanis [at] twilio.com and on Twitter at @eresvanis.

Source: Twilio

Leave a Reply

Your email address will not be published.


*