Couple weeks ago I published an article here in the blog outlining the process of creating a new machine learning model that has the ability of predict diabetes for a given person based-upon a set of data provided. That one is the first article of a series of three, and is available in here. Go check it out!

In today’s post I’m going to discuss the process we went through to build out the diabetes predictor assistant, which is a Bot predictor we built on top of one of Microsoft’s Power Platform services, Power Virtual Agents. It also relies on Power Automate, or just Flow (which is another service under Power Platform’s portfolio) to perform certain API calls to our backend (built throughout the last post) on behalf of the Bot.

If you had the chance to take a look on the previous post, you remember the scenario. If you don’t, before moving forward with this reading, I strongly recommend you doing so, as we’re going to build on top of that. The Figure 1 depicts the proposed architecture we agreed upon with Americas University’s technical teams.

Figure 1. Architectural view of the scenario

We already have our “Diabetes Predictor API” published in production and sitting onto an instance of Azure Kubernetes Service (we call it inference cluster), so we’re ready to start designing conversations flows for our digital assistant (“Bot Predictor” in Figure 1) which will serve as primary interface between AU’s customers and the predictor API in the back.

Setting up Power Platform (PP)

As mentioned early on, in order to build out our digital assistant or Bot Predictor, we’re going to rely on Power Platform Virtual Agents. This way, the very first step we need to take is to make sure we have that service properly licensed and configured.

If this is new to you, here is a word or two about Power Platform.

Power Platform is a Microsoft’s set of cloud services that enables people on creating applications without the need of having previous knowledge about programming. Through a very simple, straightforward and intuitive interface (pretty similar to Power Point), users can create robust and scalable applications by grabbing and dropping built-in components (forms, cameras, so on so forth) and adding some logic to it by mixing Excel’s logic model (formules, macros, etc). Pretty neat stuff.

If you want to lean more about Power Platform services, I do recommend a free training under Microsoft’s learning portal that takes you through every detail of the service. It can be accessed from here.

Power Platform has its own licensing model (disconnected from any eventual Azure or Office 365 subscriptions you might have in your company), being possible to pick and choose from either per user, per app or pay-as-you-go models. It was designed this way to allow platform independency, meaning that you don’t have to have any previous commercial relationship with Microsoft to build out products with Power Platform. Obviously, if you’re already tied to a given Azure or O365 tenant, your Power Platform environment will be tied to those. Licensing plans can be easily understood by following this link.

A 30-day trial is currently available. If you want to follow along with de implementation here described and don’t have any existing Power Platform license tied to an existing Microsoft tenant, you can head out to this page and get it done for free.

Getting started with Power Virtual Agents

As briefly mentioned early on, Power Virtual Agents (or simple PVA) is one of the components of Power Platform. It is an out-of-the-box Bot builder platform, designed to highly simplify the process of building intelligent and scalable digital agents with all its caveats (underlying infrastructure, downstream APIs, and so on). It is available to customers through a web UI, which you can have a first look through Figure 2.

Figure 2. Power Virtual Agent’s UI

Obviously, it brings natively a strong integration with all Microsoft’s cloud services (including Azure and O365), which enables the creation of complex scenarios.

Once a Bot is ready to go to production and meet actual users, PVA makes available a very simple distribution tool that allow companies to put it out there through several channels, including Microsoft Teams, Facebook, Twillio, Telegram, Web, Cortana, Skype, and many more (Figure 3).

Figure 3. Managing conversation’s channels in PVA

It is also important to mention at this point that Power Virtual Agent service is logically based on three key elements: Topic, Entity and Action, which I briefly touch below.

  • Topic – A topic defines the path of a conversation, and the overall purpose of a topic is to answer a question. An example topic could be “store opening hours”, and the purpose of this would be to answer store opening hour questions. A bot will be associated with one or more topics.
  • Entity – An entity describes a piece of information that the bot receives. For example, a bot could ask “what location are you interested in?”. In this case, city would be the name of the entity.
  • Action – A bot can call actions based on the user response. Typically, this would be a Power Automate Flow that could send emails, or to initiate the 1000s of tasks that are possible through Flow. This could include data activities, the escalation of the conversation to human agent, switching/restarting a topic, and more.

You could have as many Topics, Entities and Actions and needed but, for the sake of simplicity, I will centralize the entire flow of our Bot’s conversation within one single topic: “Greetings”. Important to mention, though that this is not recommended for a real Bot. Ideally, you should break down all different real conversation’s topics throughout different PVA’s Topics.

Creating a new Bot

Time to get our hands dirty. We’re going to start by creating a new Bot. To do so, after accessing https://web.powerva.microsoft.com/ and get yourself logged in, on the dark green menu at the very top, give a click on the icon of a Bot, as displayed by Figure 4.

Figure 4. Bot menu

That click will trigger up the appearance of a new window, which allows you to create a new Bot for a given environment (please, notice the dropdown right below) by clicking on the dark green button”+ New bot”. Figure 5 depicts that window.

Figure 5. Creating a new Bot

Clicking on that “+ New bot” button will bring a up a new modal window. This new one is going to ask you few additional information about the Bot itself. “Name”, “Language” and “Environment” (should be pre-populated). Once you fill out those information, just hit “Create” for Power Virtual Agent service provision a new Bot infrastructure.

Figure 6. Creating “Diabetes Predictor Bot”

It takes only a couple seconds for your new Bot to be ready. Once is finishes, you will be able to see a screen pretty similar to what is presented by Figure 2.

Take a time to familiarize yourself with it browsing through the different components presented in there. What does stand out in there and you should be mindful of:

  • Left hand side menu. There is where you’re going to be able to navigate through main aspects of your Bot, like: Topics, Entities, view Analytic data, Publish and additional management aspects of it.
  • “Test bot” window. This is where we can test our Bot as we go. It does simulate a final user experience with the Bot and allows to collect feedback from it.

Designing the Bot

Bot created. Now, time to design it and the very first thing we need to do to accomplish that is minimally think about the general flow we want our users to go through once they get into the real conversation. What AU’s team had in mind for Diabetes Predictor Bot is listed below.

  1. User wakes up our Bot with a greeting.
  2. Bot then asks for user’s name and persists it somewhere for later usage.
  3. Bot then asks questions related to diabetes aspects: Blood pressure, Serum, etc, and then, persists it to later usage.
  4. Once all the data is gathered, our Bot will then perform a HTTP call to the diabetes predictor API (the one we built in the first article of this series).
  5. Upon return with proper results, our Bot will then present the results to the final user.
  6. Then, our Bot asks the user if they would like to see recommendations based upon similar cases found in AU’s dataset history.
  7. If user confirms its interest, a new HTTP call is made by our Bot to a different API; This one is specialized on doing recommendations (we’ll discuss the implementation of this API in a future article).
  8. Upon return, our Bot then present the recommendations and finishes up the chat.

As you might have noticed, steps 4 and 7 perform HTTP calls. To do so, we rely on another component of Power Platform suite; Power Automate. Don’t worry. We will get into the details of those calls when the time comes.

Nothing really fancy but it does accomplish what AU’s technical team thinks is the ideal experience for final users consuming information from our Bot.

Step 1: User wakes up our Bot with a greeting

I mentioned that early on, but it is important to reinforce it here. For the purpose of this demo and sake of simplicity, we’re going to concentrate all the steps our Bot bot need to undertake at one single topic “Greeting”, which is built-in in every Power Virtual Agent Bot. However, for productive scenarios, we don’t recommend doing so as it can compromise the overall performance and experience of building out a new Bot. Cool?!

With that said, let’s get our hands dirty and navigate through the “Topics” option, on the left-hand-side menu. Once in there, you should be seeing a screen that lists all the built-in Topics offered by the tool. You could create a new one if you would like, but at this point, we’re going to leverage the “Greeting” one (Figure 7).

Figure 7. Browsing “Topics” and “Greetings”

Clicking on “Greetings” will take you to an intermediate screen that explains the purpose of that topic and gives you the ability to access “topic’s authoring canvas”, the place where we can visually design each interaction within a given topic. Figure 8 showcases it.

Figure 8. Accessing authoring canvas

Once in the Canvas, first thing we’re going to do is to add a group of messages welcoming the user. Those messages are going to be displayed to the final user in response to some greeting message first typed in.

The very first message I want to display is the one below.

Hi! I'm a virtual doctor. I'm here to help you find out how good (or bad) you're doing with your Diabetes treatment.

For that, I will just replace the original message being shown in the default node card by the one I just described. Final result is presented by Figure 9.

Figure 9. Welcome message 1

Step 2: Bot then asks for user’s name and persists it somewhere for later usage

Next, we want to know user’s name so we can intimate the conversation. Because the name should be reusable throughout our Bot, we need to persist that information. For that, we’re going to leverage the concept of global variables, made available for Power Virtual Agents service.

We then add a new node card of type “Question” by giving a click at the “+” icon, available right below the previous welcome message node card, as depicted by Figure 10.

Figure 10. Adding a new node card

In the new question node card (which has the ability to serve as input), we want to display users a question and save their response (name) into a global variable that we’re going to call “personName”. What designates the bot’s global scope for the variable is the marker “bot.” in front of variable’s name. The final result of that card can be seen through Figure 11.

Figure 11. Asking user’s name

Next, we just nominally thank the user for providing the name by referencing the variable we just set up, “bot.personName” in a message node card (Figure 12).

Figure 12. Referencing personName variable

As a quick recap, at this point, we have a interesting sequence of activities set up to happen when the user greets up the Bot: We send a welcome message; Then, we ask for user’s name and store it in a variable; Finally, we thank the user with a dynamic message referencing the newly created variable. Figure 13 depicts the sequence of actions.

Figure 13. Welcome actions sequence

Step 3: Bot then asks questions related to diabetes aspects: Age, blood pressure, Serum, etc, and then, persists it to later usage

Next, our Bot needs to collect critical data related to the current status of diabetes treatment our users are supposedly going through. Information like age, blood pressure, serum, body mass, and so forth have to be gathered before hand, as the AI service predictor we’re going to call later on, depends on it.

For the record, we’re going to rely on the same temporary storage structures (variables) to hold users’ answers to those questions. Figures 14 to 24 present the configuration of each question and its variables definition.

Figure 14. Age
Figure 15. Gender
Figure 16. Body Mass Index (BMI)
Figure 17. Blood Pressure
Figure 18. Serum 1 (S1)
Figure 19. Serum 2 (S2)
Figure 20. Serum 3 (S3)
Figure 21. Serum 4 (S4)
Figure 22. Serum 5 (S5)
Figure 23. Serum 6 (S6)
Figure 24. Progression measurement

Lots of questions, huh? As I said before, they’re all needed and our ML model engine depends on those to properly predict diabetes level moving forward.

Theoretically, we have all the information we need and are ready to call out our predictor service back in Azure Kubernetes Service. However, I want to make sure the data is accurate, so I ask user for a confirmation upon summarization of its answers in a question node card (Figure 25).

Figure 25. Summarizing answers and taking a path

Notice that the answer to that question (is the collected data accurate or not?) is a boolean, which means it will come out either true or false. When it happens, Power Virtual Agent service automatically infers that we might need to take one of two paths depending on user’s response: one if that comes back “true”, and another if that returns “false”. That’s the reason by which you’re seeing two different paths at that point.

In our case, if the user confirms the information provided is accurate, then we go left and call the predictor service API. In other hand, if they disagree with that statement, we go right and restart the flow by calling a specialized topic called “Start over”, which has the ability to restart the conversation (Figure 26).

Figure 26. If response comes false, conversation restarts

Step 4: Once all the data is gathered, our Bot will then perform a HTTP call to the diabetes predictor API

We know by a fact that, if user’s response comes out “false”, conversation’s flow restarts. However, what happens if user confirms everything is accurate? Well, we teased that already. It is going to rely on a special node card type known as “Action”.

We need this action node card in play due the fact we need perform an external HTTP call to our Diabetes Predictor Service API, currently sitting in AKS.

Here is what is going to happen:

  1. Upon creation of a new action node card (Figure 27), PVA is going to give you an option to create a new Power Automate Flow (Figure 28).
Figure 27. Calling a new action
Figure 28. Creating a new flow

This is going to tie together this instance of PVA’s Bot with the new instance of Power Automate Flow so that, every time a new call happens from the Bot, it knows exactly how to pass parameters along to right flow and receive information back.

That action (create a new flow) will also take you to another browser tab with Power Automate canvas screen ready for edition. Figure 29 shows what the Power Automate canvas screen will look like.

Figure 29. Power Automate canvas

From there, we’re ready to start building out our HTTP call flow.

First, we need to define all the parameters (adding new inputs under the Power Virtual Agents task) this flow is going to be receiving from our PVA bot. Those parameters are stored as variables within our Bot. To keep things simple and easy to related, flow’s parameters will have the same name as Bot variables.

Figure 30 shows the final configuration of my Power Virtual Agents task, which now holds all the parameters needed to perform the HTTP call.

Figure 30. Power Virtual Agents task

If you remember from the first article, the way our diabetes predictor model prints out the result of its processing, is by defining a final scored label number, which represents the likelihood of diabetes be happening to a given patient in 12 months from now. If that number is over 200, then the patient will be diabetic. If that trash-hold doesn’t hit, then they should be fine.

That’s the reason by which we need to initialize a new variable in our flow. That variable will hold the number we just talked about (scored label), which will return as response to our HTTP call to the prediction service. Figure 31 shows that configuration.

Figure 31. Initializing scoredLabel variable

Next, we perform the HTTP call to our predictor API and for that, we use a task called “HTTP” within our flow.

The predictor API expects some specific information coming in, both in the Header and Body. In the Header we authenticate ourselves and define the content-type, while in the body, we pass the actual payload formatted properly. Figure 32 presents that configuration.

Figure 32. Configuring the HTTP call to diabetes predictor API

Next, we need to handle the response. Once it comes in JSON format and considering we are interested on a small portion of that return, it will be better for us to transform that response in an actual object, so that we can easily interact/manipulate its properties and values.

That’s the reason by which the next step in our flow is “Parse JSON”. All we have to do here is to define where in the HTTP response the JSON we are seeking for is sitting at (in this case, Body) and then set up the object’s schema it will be converted to. With that, the task will know how to properly handle the incoming JSON. Figure 33 depicts the configuration I have in place.

Figure 33. Transforming HTTP response in JSON object

Next, we need to iterate within object’s properties. If you noticed, there is an array in the HTTP response called “WebServiceOutput0”. This array has the ability to host multiple Scored Labels, depending on how the output of the model was configured. In our case, by design, we will always have one output. However, the service groups the scored label information within an array, reason by which we need to iterate over.

To do so, I will leverage an “Apply to each” type task to iterate over array’s items and combine it with the “Append to string variable” task within each iteration, so that we can set up the returning variable (scoredLabel we initialized at the beginning of the flow). This iteration will run once. Figure 34 presents that configuration.

Figure 34. Iterating over array “WebServiceOutput0” to capture scoredLabel

Finally, all we have to do now is return the final result of our flow back to our Bot. As the result is been held by variable “scoredLabel”, we can simple return it from “Return value(s) to Power Virtual Agents” task, as presented by Figure 35.

Figure 35. Returning scored label to called Bot

Back to PVA’s canvas, the final step to complete the HTTP call is to bound back every Bot’s variable with the parameters being expected by the Power Automate flow we just finished. We do this by editing the action node card previously created for the external call. Figure 36 shows how your card’s configuration should look like.

As you can see, we have three sections in this card. The first one shows each Bot’s variable being bounded to a flow’s parameters. The second section shows which flow we are calling to. The third section shows the value we’re getting back from flow. Pretty neat, huh?!

Figure 36. Action call final set up

Step 5: Upon return with proper results, our Bot will then present the results to the final user

All good thus far, so at this point, we should be ready to display the prediction made by our predictor API to the final user via Bot.

We know by a fact that, if the “scoredLabel” returned is greater than 200, it is a clear indication that the patient asking for the prediction will be diabetic again in 12 months, which is not true if that number doesn’t hit that limit.

So, what we’re going to do is to put together a condition that evaluates “scoredLabel” on the exact same terms I mentioned earlier. If it is greater than 200, show a warning message. If it is not, show a congratulations message, as depicted by Figure 37.

Figure 37. Evaluating HTTP call’s return and advising user

Let’s give it a try? Sure. For this, we’re going to use our “Test bot” window, which has been following our Topic’s canvas along the way. To start the action, just send a “hi” or whatever greeting that comes to your mind. Figure 38 shows what I had after entering all the values.

Figure 38. Testing the conversation flow and bring external results from API call

Step 6: Then, our Bot asks if they would like to see recommendation based on similar cases

If the user is currently under 200 level, then our Bot congratulate him/her and finish the conversation. Now, what happens if they’re not doing a good work and their levels are running higher?

Well, then we try to help a bit more by proposing some recommendations based on similar cases we found on another dataset. To do that recommendation, we rely on a different AI model, designed specifically to do recommendations based on certain aspects being measured.

This is going to be the subject of our next article in the series. We’re going to spin up the Recommendations API and have our Bot set up to communicate with it.

Stay tunned!


0 Comments

Leave a Reply

Avatar placeholder

Your email address will not be published.