{"id":39600,"date":"2025-02-17T20:25:54","date_gmt":"2025-02-17T19:25:54","guid":{"rendered":"https:\/\/sovanta.com\/?p=39600"},"modified":"2025-02-27T20:26:18","modified_gmt":"2025-02-27T19:26:18","slug":"magical-generative-ai-buzzwords-and-where-to-use-them","status":"publish","type":"post","link":"https:\/\/sovanta.com\/en\/magical-generative-ai-buzzwords-and-where-to-use-them\/","title":{"rendered":"Magical Generative AI buzzwords and where to use them"},"content":{"rendered":"\n

\u201cOur RAG application integrates seamlessly with AI agents and utilizes Knowledge Graph technology to autonomously fulfill user requests.\u201d – If that sentence sounds confusing, you\u2019re not alone. Many customer inquiries mix these buzzwords, leaving especially non-technical audiences overwhelmed by the vast array of available technologies. That\u2019s why Niklas Fr\u00fchauf, Data Scientist at sovanta, is here to cut through the noise. He\u2019ll clarify these concepts, bring structure to the buzzword jungle, and, most importantly, highlight the real business value and future use cases they enable.<\/strong><\/p>\n\n\n\n

LLMs and their limitations<\/h2>\n\n\n\n

Let\u2019s start simple: Large Language Models (LLMs)<\/strong> such as ChatGPT, Gemini and recently DeepSeek are the main building block for most of today\u2019s Generative AI approaches. At their core, they are trained on terabytes of public text data, rendering them capable of \u201cgenerating\u201d text based on arbitrary input \u201cprompts\u201d. Due to the nature of their training process, they are unable to recall information that was made public after their training was completed, and obviously also unable to recall information that is not public, such as company-internal guidelines and documents. Instead, they will in the best case tell you that they cannot answer the question, and in the worst case confidently \u201challucinate\u201d an incorrect answer<\/strong>. Asking ChatGPT about an existing contract with sovanta thus is most likely not going to give you any meaningful information. Sure, you could fine-tune these LLMs<\/strong> on company-specific data, but many studies<\/a> show that unless you have loads of high-quality documents available, this will degrade their overall performance. Add to that the rather high time (and cloud costs!) required to do so, and it quickly becomes infeasible, especially for smaller- or medium-sized companies.<\/p>\n\n\n\n

Shift to Retrieval-Augmented Generation (RAG)<\/h2>\n\n\n\n

As a result, companies can shift to an approach called \u201cRetrieval-Augmented Generation\u201d (RAG) <\/strong>or \u201cdocument grounding\u201d. Basically, your internal documents are first indexed into a vector database, using a specific \u201cembedding model\u201d which turns each section of the document into a list of numbers that represent the semantic content of this \u201cchunk\u201d. Whenever a user wants to ask a question that may be related to these documents, the question is also embedded (using the exact same model as during indexing), and the chunks with the highest semantic similarity are considered further<\/strong> (\u201cretrieval\u201d). Next, these chunks (sometimes also called \u201ccontext information\u201d) are passed to the LLM together with the user question, enabling answers based on internal data<\/strong> as well as allowing the user to check the contents of files that were found during the retrieval process, providing much-needed transparency<\/strong>. In our example, indexing your contracts with RAG would then allow a user to ask questions such as \u201cWhat\u2019s the title and duration of the contract we have with sovanta?\u201d without the need for LLM fine-tuning.<\/p>\n\n\n\n

What\u2019s more, it is technically feasible to self-host all components (Ollama for Open-Source LLMs, Transformers + ONNX for embeddings), thus reducing the dependency on 3rd party vendors. Our GenAI DocumentChat<\/a> follows this technological approach, too, relying on SAP-hosted Open Source Models via their Business Technology Platform (BTP), and leveraging the SAP HANA Vector Engine for quick document retrieval.<\/p>\n\n\n\n\n

\n
\n
\n
\n
\n

GenAI DocumentChat<\/h3>\n

Our GenAI DocumentChat lets users \u201cchat\u201d with your internal documentation, legal texts, compliance guides, and other complex documents.<\/p>\n

\n More<\/a>\n <\/div>\n <\/div>\n <\/div>\n
\n \"\" <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n\n\n\n

Next: Knowledge Graphs<\/h2>\n\n\n\n

However, this classical RAG approach has drawbacks: What if the answer to a user question requires more than let\u2019s say the top 10 most similar chunks, instead running aggregations<\/strong> on top of them? Imagine having loaded all your PDF-based contracts with hundreds of thousands of sections into your RAG system, just for a user to ask a question such as \u201cHow many contracts do we have with companies in Berlin?\u201d. This can be addressed using what’s often referred to \u201cKnowledge Graphs\u201d<\/strong> or \u201cTriplet Stores\u201d, an approach founded in 90s research on \u201cSemantic Web\u201d. In addition to (or instead of) using classic RAG with embeddings, all documents are turned into a graph representation, where the nodes usually represent concepts\/entities, and the edges represent relationships. <\/p>\n\n\n\n

In our above example, you could have a node for each contract partner (\u201csovanta\u201d), connected to each legal entity or business partner (e.g. \u201csovanta Germany\u201d), in turn linked to the city (\u201cHeidelberg\u201d) and a node for each contract (\u201cGenAI Discovery Workshop\u201d). The neat part: You don\u2019t have to manually generate this graph; instead you can ask the LLM to extract those nodes and relationships from each document while they are being indexed. Using specific Graph Databases (Neo4j, the upcoming HANA Knowledge Graph Engine<\/a> and others) together with a specific query language (such as SPARQL or Cypher) allows you to quickly insert and search for complex graphs. Tasked with a user request such as the one above, the LLM is used to turn it into it\u2019s corresponding query representation<\/strong>:<\/p>\n\n\n\n

SELECT COUNT(?contract) WHERE { ?company a :Company ; :hasLocation ?location . ?location :locatedIn \"Berlin\"^^xsd:string . ?company :hasContract ?contract . }<\/code><\/p>\n\n\n\n

This query can be executed against the graph database<\/strong>, returning the correct number to the LLM to create a final answer for the user. As you can see, Knowledge Graphs are mainly useful when you are working with a set of similar documents that have a somewhat clear domain data model, but which are not yet available in tabular\/structured form in any of your company systems.<\/p>\n\n\n\n

Function Calling and Tools<\/h2>\n\n\n\n

But what if you already have your data in structured format somewhere? Or you have an external system that has a clear-defined REST API to access? In these cases, it may make sense to switch to the method of \u201cfunction calling\u201d <\/strong>(or user-provided \u201ctools\u201d<\/strong>). Ideally, you could also hook up not one but multiple functions to the system, allowing the user to ask questions such as \u201cWhat was the last purchase order we signed with sovanta?\u201d but also \u201cWho\u2019s the client partner for sovanta, and what\u2019s his phone number?\u201d in the same context. The approach is similar to the one using Knowledge Graphs – First, the LLM is presented with the user question and available tools\/functions, then selects a function to execute together with the function inputs, runs the function and returns the result back for a final answer<\/strong>. <\/p>\n\n\n\n

Technically speaking, a Knowledge Graph lookup is also simply a function! In our example, we may have two functions: Firstly, the \u201clookup_purchase_orders\u201d one (expecting e.g. a filter argument) and the \u201clookup_business_partners\u201d (the same). The initial LLM call would then decide to execute \u201clookup_purchase_orders\u201d with \u201cfilter=vendor LIKE %sovanta%\u201d order by createdAt DESC LIMIT 1\u201d, which in turn calls the responsible system, producing the correct result and retuning that for a final user answer. This approach also allows the LLM to trigger e.g. business processes and workflows, open tickets, and many more options, making it a prime approach for supporting end to end user requests with a clearly defined scope.<\/p>\n\n\n\n

Spotlight on “(AI) Agents”<\/h2>\n\n\n\n

Sadly, requests from user are not always straightforward, and sometimes require cross-referencing results from various different tools. Additionally, they may refer to specific edge cases that cannot be solved by a simple function call. What would you do if a user uploads a vendor offer and asks the system to create the corresponding purchase requisition? Obviously, the exact approach would depend on your corporate guidelines and the system that is used for managing the purchase requisitions. You may have different fields that are required for service or for product requisitions.<\/p>\n\n\n\n

This is where the approach of AI Agents<\/strong> shines – instead of having a very narrow system that can only fulfil straightforward requests, you simply rely on LLMs to autonomously plan and iteratively execute various tools<\/strong> to ultimately fulfil a user request. The output of the plan step for our example could look something like this \u201cFirst, I need to check the PDF to extract line item information. Then, I need to check the company guidelines to verify if additional information is needed. Next, I need to find the system that stores the purchase requisitions and fetch the documentation of its external REST API to find the correct format. Once this is done, I need to re-check required fields and ask the user for clarification and approval. Only then I can send the final HTTP request.\u201d Keeping track of internal state, the Agent(s) iterate(s) until they have fulfilled their task, showcasing an ability to react to errors and incorporating new knowledge as it is uncovered<\/strong>.<\/p>\n\n\n\n\n

\n
\n
\n
\n
\n

AI Agent Starter<\/h3>\n

AI agents are intelligent software solutions powered by artificial intelligence, designed to automate complex tasks, make decisions, and independently optimize processes.<\/p>\n

\n more<\/a>\n <\/div>\n <\/div>\n <\/div>\n
\n \"Von <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n\n\n\n

Summing it up: Generative AI buzzwords<\/h2>\n\n\n\n

That was a lot of information at once. Let\u2019s quickly recap on a high level:<\/p>\n\n\n\n