This is the third post of a series of four covering the process I went through with Americas University’s technical team to create a digital diabetes predictor. The first one introduces the scenario, solution’s rational and ends up with the building of the AI model that will serve as prediction engine in Azure ML. The second guides through the process of creating a digital assistant (Bot) that uses the AI model previously created to predict user’s diabetes rate in 12 months. If you’re arriving just now to the series, I strongly recommend the read of the other two to get the context and foundation of the solution.

Today’s article covers another aspect of the solution: potential additional medicine and/or physical activities that are recommended depending on the level of diabetes, age, weight and so on.

Revisiting the overall solution

If you’ve been following the series, you might (or might not, that’s why I revisit it here) remember the overall architecture (Figure 1) that was put together to address the need of predicting diabetes.

Figure 1. Diabetes Predictor architecture

Considering the user interacts with the prediction engine from a Bot, it is important to revisit its navigation flow (it was explained in details on the second article of the series but worth taking a look again here) so that we know how it fits on the architecture above.

  1. User wakes up our Bot with a greeting.
  2. Bot then asks for user’s name and persists it somewhere for later usage.
  3. Bot then asks questions related to diabetes aspects: Blood pressure, Serum, etc, and then, persists it to later usage.
  4. Once all the data is gathered, our Bot will then perform a HTTP call to the diabetes predictor API (the one we built in the first article of this series).
  5. Upon return with proper results, our Bot will then present the results to the final user.
  6. Then, our Bot asks the user if they would like to see recommendations based upon similar cases found in AU’s dataset history.
  7. If user confirms its interest, a new HTTP call is made by our Bot to a different API; This one is specialized on doing recommendations .
  8. Upon return, our Bot then present the recommendations and finishes up the chat.

If you remember the second article of the series, we were left off on the step where the user gets back the result of the prediction based on the data provided (diabetes rate in 12 months), which maps out to step 5 in above’s list. Moving forward, we’re going tackle steps 6 – 8, recommending additional care to be taken if prediction result came above 200, and obviously, if the user consented on receiving that.

Picking from were we were left off

Figure 2 below depicts how the prediction result is presented to the user once they provide the set of information required.

Figure 2. Predicting diabetes rate in 12 months and informing it to the user

That is useful and for most users, it should be enough. However, AU’s team came up with an additional challenge. Considering they have an additional dataset that correlates medication and types of exercises recommended for certain diabetes types and users’ profiles, AU’s health department wanted to try to create an “additional care recommendation” feature and make it available through digital assistant.

Good news is, there is a “Recommendations API” made available by Azure team in Github that is meant to help with scenarios like the one just described. It means, once provided a dataset with historical facts data, recommendations engine is set to be able to provide what action/activities are recommended for a given requester (in our case a diabetic patient) based upon current parameters sent (in our case, it maps out to user’s diabetic profile).

Then, what we’re going to do from here is 1) spin up a new piece of infrastructure to host our new API; 2) train recommendation’s model with AU’s dataset; 3) adjust the digital assistant to interact with the API from our digital assistant; 4) format results to show to final user.

A word about Recommendation API

I’ve been referring to the recommendation engine as an API because, ultimately, this is how data coming out of it will made available for consumers and because it is coherent with the scenario we’re working on. However, infrastructure-wise, the ideal reference to it should be “Recommendation Template” as it packs together a set of resources that need to be in place for the engine to properly work. Figure 3 breaks down the underlying resources needed for the engine to work.

Figure 3. Recommendation components

As you can see, there are three main components in Azure that need to be deployed to bring the recommendation engine to life: A Web App (part of the App Services), a Storage Account and a Web Job (which is a built-in feature in App Services).

While Web App will be responsible for receiving any incoming requests and handling them to proper internal methods for recommendation generation, storage service will be in charge of storing datasets and models themselves, plus decoupling “training requests” through storage queue. Finally, Web Jobs come in to play when a new “model training” request arrives into the API. It does run an internal process that leverages updated data, model and retrain the engine.

You can see detailed information about how the Recommendation API works in this Github repository.

Deploying Recommendation API

Deploying Recommendation API in Azure is pretty simple and straightforward. That’s because an Azure Resource Manager (ARM) Template was put together by Azure’s engineering team to simplify the process of getting all the pieces together. If you’re interested on knowing the details of how it gets deployed by analyzing the script, take a look here.


Click Deploy to Azure to deploy the above resources in Azure. Clicking on that link, will redirect you to Azure portal, so you must have an active Azure subscription to create a custom deployment. The default parameters should suffice. If you wish to change them, you may go ahead and select a different value from the dropdown for the Account Type, Hosting Plan SKU, and the App Insights Location. Figure 4 shows up components deployed.

Figure 4. Recommendation API components deployed

The recommendation solution has been deployed. Now, we can use the service to perform a first train on the recommendation models and start getting recommendations.

Before we do that though, there are couple of aspects we need to clarify before calling out any training endpoints. They are:

In the Azure portal, go to the newly deployed resource group. From there, on the left-hand menu, click on “Deployments”. Then, after selecting the deployment, give a click on “Outputs” (also in the left). The blade presented (Figure 5) should be filled with important information about how to connect to API’s endpoints, access keys, storage connection string, and more.

Figure 5. Recommendation solution outputs
  1. The first piece of information being shown refers to the base URI for API calling. Because this is being deployed in a Web App in Azure, your URL should look like “https://{something}“.
  2. Refers to a Web UI that comes along and serves different purposes in the context of the API. You can use it to train new model, see prediction scores, and more. Keep in mind you’re going to need to provide the admin key for this UI to work.
  3. Brings the base URI that points out to Swagger (now Open API) specification for the recommendation API.
  4. Gives you the admin primary key which was designed to give system’s administrator access to everything that happens internally to the API itself. This one can also be used for all API operations, including model creation and deletion. The adminKey is the key that can be used for all API operations and gives full error stack on any internal errors.
  5. Gives you the recommender primary key. This one can only be used to get recommendations. This is the key you would use on the client or website requesting recommendations.
  6. Gives you the connection string to the storage account deployed with the recommendation solution.

Dataset and historical data

Last but not least, we need a historical dataset that correlates additional diabetes treatments to each patient profile so that we can train our model to generate future recommendations.

The way API’s model was designed to work requires two different datasets to begin with. One conceptually defined as “Catalog”, as it aggregates all the elements and its properties that recommendation will be projected against, and a second one defined as “Usage Events”, which aggregates events (in our case, diabetes treatments) that happened on top of the items cataloged.

Figure 6 depicts the “Catalog” dataset I’m submitting to the engine. Figure 7 depicts the “Usage Events” one.

Figure 6. Catalog dataset
Figure 7. Usage Events dataset

For the model to be properly trained, we need to put those two datasets (in txt format) in the right place. If you remember the beginning of this article, I mentioned that datasets are hosted in the storage account that is deployed together with the service.

In preparation for our first training, what we need to do as next step is to create a new container in the storage account. That container will hold both datasets. To do so, just get back to Azure portal, select the storage account that was deployed with the recommendation solution and on left-side menu, select “Containers”. Once in there, just click “+ Container” and type in “datasets”. As public access level, select “Private”. Figure 8 shows how the containers list blade should look like at the end of this process.

Figure 8. Creating a new container

Now, get into the newly created container (by clicking on the name of it) and just upload the two dataset files (available here). At the end, container’s content should look like Figure 9.

Figure 9. Datasets successfully uploaded into datasets container

Done. Now we have everything we need to perform our first training against the recommendation model. We do this by calling a service endpoint made available specifically for it. That HTTP call has the respect the following restrictions:

  • It has to have an administrator key header (x-api-key) set up.
  • It has to call the endpoint “https://{something}“.
  • It has to pass the following body payload to the service.

Body’s payload sets up couple key parameters for the engine.

  • “description” allows us to set up a high-level descriptive name for the training process.
  • “blobContainerName” allows us to point out the container that holds the datasets so that, model knows where to look when performing the training.
  • “usageRelativePath” allows us to set up the “Usage Events” dataset file.
  • “catalogFileRelativePath” allows us to set up the “Catalog” dataset file.
  • “evaluationUsageRelativePath” defines the file it should use for evaluation (compare training data with a different set of real data). For the purpose of this demo we’re using the same file, but real world scenarios would require a different sampling.
  • “supportThreshold” defines how conservative the model is. Number of co-occurrences of items to be considered for modeling.
  • “cooccurrenceUnit” indicates how to group usage events before counting co-occurrences. A ‘User’ cooccurrence unit will consider all treatments by the same user as occurring together in the same session.
  • “similarityFunction” defines the similarity function to be used by the model. Lift favors serendipity, Co-occurrence favors predictability, and Jaccard is a nice compromise between the two.
  • “enableColdItemPlacement” indicates if recommendations should also push cold items via feature similarity.
  • “enableColdToColdRecommendations” determines whether the similarity between pairs of cold items (catalog items without usage) should be computed. If set to false, only similarity between cold and warm items will be computed, using catalog item features. Note that this configuration is only relevant when enableColdItemPlacement is set to true.
  • “enableUserAffinity” allows us to define for personalized recommendations, it defines whether the event type and the time of the event should be considered as input into the scoring.
  • “allowSeedItemsInRecommendations” allow seed items (items in the input or in the user history) to be returned as recommendation results.
  • “enableBackfilling” allow to backfill with popular items when the system does not find sufficient recommendations.
  • “decayPeriodInDays” decay period in days. The strength of the signal for events that are that many days old will be half that of the most recent events.

Figure 10 shows what I got back when calling the training model endpoint with all the requirements in place.

Figure 10. New model training completed successfully

Success! 201 status returned and body’s response give us more details about the event “creation”. Now, we can finally try asking for a recommendation to the API. For that call to work, we need to set it up according the schema below.

GET https://{something}{modelId}/recommend?itemId={item to test}

x-api-key: your_recommendation_api_key

Content-Type: application/json

The return I get is presented below. It picks the first six results with its scores (the confidence the model projects for the recommendation).

     "recommendedItemId": "5",
     "score": 0.45787626504898071
     "recommendedItemId": "12",
     "score": 0.12505614757537842
     "recommendedItemId": "8",
     "score": 0.049780447036027908

Great! Recommendation API seems to be working properly so now we can safely move to the next step, which is all about integrating our digital assistant with this API.

Integrating digital assistant with Recommendation API

We came to a point where the user just got back it diabetes rate in 12-months. What’s next? In that case, we first assess if the user is currently under any medication to diabetes. If user responds “no”, then we just message the final user with a final message. If a “yes” comes back, we inquire if user would like to see additional recommendations to their case, as you can see through Figure 11.

Figure 11. Assessing about additional recommendation

In the event of the user agreeing on seeing additional recommendation, we make sure the medication is among the ones cataloged in the dataset we used to train our recommendation model. That’s it. We then call out a flow that reaches out the recommendation API, gets the result back return the final user to our digital assistant. Figure 12 depicts that process.

Figure 12. Validation medication and calling the API

The overall flow is pretty similar to what we did to call out the predictor API on the second article of the series.

First, we receive the information about the medication user is currently taking. Then, we initialize an empty variable (which will bear the recommendation result later on). Because we will need to iterate through the recommendation’s array, we initialize an “index” variable. Please, see Figure 13.

Figure 13. Initializing global values

Then, we call the API, as you can see through Figure 14.

Figure 14. Calling the API

Upon return, we then parse the return to an array object so that we can easily iterate through array’s items. For the conversion to happen, we need to provide the payload schema (please see Figure 15).

Figure 15. Parsing JSON result

Next, we then iterate among array’s items. For the purpose of this solution, we’re interested only on the three first recommendations (reason by which we top the iteration on 2). Here we use the variable “index”, mentioned earlier.

Figure 16. Iterating through recommendation’s items

Then, we just return the variable “recommendation” (which was set up within the “Condition 2” above) back to Power Virtual Agent. Please, see Figure 17.

Figure 17. Returning recommendation back to PVA

Once the data arrives back in Bot’s conversation’s flow, we just plot it out to the user. Depending on the profile informed, the recommendation will either be additional medication or exercises. Figure 18 depicts the final communication process.

Figure 18. Recommending additional medication/exercises

Great! Now, we ready to try this final process. I went through the whole conversation flow and, at the end, I validate the recommendation action, as you can see through the Figure 19.

Figure 19. Recommendation of additional care in action

Done! That wraps up the article. Hopefully, it will help you on getting started with both AI, Power Virtual Agents and more. Feel free to add comments and ask questions.

The upcoming article in the series (it will be the last one) will guide us through the process of publishing the digital assistant and making it available throughout different channels.

See you there!


Leave a Reply

Avatar placeholder

Your email address will not be published.