{"id":14960,"date":"2023-07-13T09:33:33","date_gmt":"2023-07-13T08:33:33","guid":{"rendered":"https:\/\/sovanta.com\/?p=14960"},"modified":"2023-07-13T09:34:09","modified_gmt":"2023-07-13T08:34:09","slug":"how-chatgpt-makes-your-experience-management-survey-more-intelligent","status":"publish","type":"post","link":"https:\/\/sovanta.com\/en\/how-chatgpt-makes-your-experience-management-survey-more-intelligent\/","title":{"rendered":"How ChatGPT makes your Experience Management survey more intelligent"},"content":{"rendered":"\n

In less than a year, ChatGPT is all of a sudden everywhere. New business models are rapidly proliferating, but you don\u2019t need to search far at all to see how Large Language Models can support existing business solutions and processes. In this blog post, Larissa Haas, Senior Data Scientist at sovanta, shows how integrating ChatGPT into Qualtrics can automate tedious follow-up surveys.<\/strong><\/p>\n\n\n\n

Use Case: Employee Technology Experience Surveys<\/h2>\n\n\n\n

Companies are increasingly interested in the extent to which employees utilize their software tools and whether such tools meet employee expectations. These assessments seek to capture what\u2019s referred to as Employee Technology Experience<\/a> (ETX). They correlate analytical data, such as login frequency or other usage patterns, with surveys that collect employee perceptions about using the technology.<\/p>\n\n\n\n\n

\n
\n
\n
\n
\n

Employees are leaving unusable technology behind<\/h3>\n

What digital offerings do employees need to complete their daily tasks more effectively? Do employees even use the tools provided, or do they need to find a workaround due to a \u2026<\/p>\n

\n read more<\/a>\n <\/div>\n <\/div>\n <\/div>\n
\n \"\" <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n\n\n\n

Systems like Qualtrics XM greatly simplify the work of obtaining and processing the necessary experiential and operational data. Until recently, however, tasks like fully analyzing open text survey responses or picking a direction for additional analysis have resisted automation. ChatGPT and other Large Language Models can take these tasks over, saving time, money, and valuable human analytical capacity. How? Let\u2019s take a look, sketching out a use-case where ChatGPT takes answers from a completed Qualtrics survey and generates follow-on questions for the next one.<\/p>\n\n\n\n

ChatGPT automatically asks your follow-up question<\/h2>\n\n\n\n

Imagine a typical ETX project. As a transformation effort runs its course, the project team wants to monitor if their technological transformation makes good progress. At a critical touchpoint a survey window opens and closes to collect feedback and when this happened, the project team finds patterns within responses using various digital tools. Then they formulate additional questions for another user questionnaire, everything with the goal in mind to get the right answers to shepherd the project along.<\/p>\n\n\n\n

By integrating ChatGPT into Qualtrics, this entire process gets a lot easier. How? We replace the manual analysis and formulation of follow-up questions with an intermediate step hosted on Cloud Foundry which sends carefully designed prompts  to ChatGPT (or any other custom Large Language Model). ChatGPT generates brand new questions, which we use to automate the creation of a survey via the Qualtrics Survey API.<\/p>\n\n\n\n

\"\"
Architecture: ChatGPT automates the creation of a survey on SAP BTP<\/figcaption><\/figure>\n\n\n\n

Advantages of ChatGPT over other text analysis tools<\/h2>\n\n\n\n

Before we go any further, a clarification. Don\u2019t platforms like Qualtrics already include machine learning approaches that summarize text, cluster text into topics, or extract keywords? They do, but the advantage of ChatGPT is its greatly reduced need for structure. The previous generations of text analysis tools do not allow us to extract insights on topics that we have previously defined, since to do this requires that we make assumptions about what we will find in the data. <\/p>\n\n\n\n

For a clustering algorithm, we need to know the number of clusters beforehand. For a classification algorithm, we need to define the topics we want to classify. For a text summary, we might lose data if the words have a high ambiguity. ChatGPT understands the \u201cmeaning\u201d behind the words thanks to its very large language model. It understands that we want the \u201cmost important\u201d topics, not the top three, and it knows that it should formulate questions out of them. This step of text generation makes it very easy for us to use within this use case, especially in comparison to existing alternatives.<\/p>\n\n\n\n

Let\u2019s review the technical setup<\/h2>\n\n\n\n

How did we realize this? The following paragraphs show you what we did technically to perform this use case, and as you will see, it was not as hard as it sounds.<\/p>\n\n\n\n

Qualtrics Survey Setup and Intercept<\/strong><\/p>\n\n\n\n

The first step to start this journey is to have a Qualtrics survey at hand. In my case I implemented a classical ETX Survey with four short questions:<\/p>\n\n\n\n

  1. How much do you like using [TOOL<\/em> NAME<\/em>] for your daily work? 5-point-scale<\/li>
  2. How do you rate [TOOL<\/em> NAME<\/em>] in terms of\u2026
    1. \u2026 how effective you feel while working with it. (0-100)<\/li><\/ol>
      1. \u2026 how much you enjoy working with it. (0-100)<\/li><\/ol>
        1. \u2026 how much time it saves you. (0-100)<\/li><\/ol><\/li>
        2. How frequent do you use [TOOL<\/em> NAME<\/em>]?<\/li>
        3. What do you else think about [TOOL<\/em> NAME<\/em>]?<\/li><\/ol>\n\n\n\n
          \"\"
          Demo application with survey: closed question<\/figcaption><\/figure>\n\n\n\n

          This last question was the most important in our approach, though the others are also handy to filter and analyze the results later. In the free text question, we often find the most valuable insights.<\/strong> It\u2019s a tall order to receive open-ended feedback, so if we do receive answers, they should be taken seriously and we should derive actions from those key insights.<\/p>\n\n\n\n

          \"\"
          Demo application with survey: open question<\/figcaption><\/figure>\n\n\n\n

          The survey was then added as a website intercept directly within the homepage of our tool. The advantage of this approach is that employees can provide their feedback directly from within the tool they are working. With Qualtrics you can configure different versions, for example showing the survey when certain triggers are met, or only showing it to a specific fraction of users.<\/p>\n\n\n\n

          Controller on SAP BTP Cloud Foundry<\/strong><\/p>\n\n\n\n

          Now that you have started collecting results, you can think about your middleware. In our case we used a Python FastAPI service deployed on the SAP BTP Cloud Foundry to handle the next steps. This Python service does the following:<\/p>\n\n\n\n