Ontologies revisited: from the setbacks of the Semantic Web to a map of the business for AI agents

What is an ontology: concepts, properties, relationships and rules

This is Part 1 (principles) of the series "Ontologies: Hype and Reality." Ontologies come up often again in discussions of AI agents. Yet what an ontology actually is, and why it was once forgotten and then came back, is rarely explained well. This part sets out the principles.

W3C's OWL 2 overview document source defines an ontology as "a formalized vocabulary of terms, often covering a specific domain and shared by a community of users." The meaning of a term is determined by its relationships to other terms in the ontology. An OWL 2 ontology consists of classes (concepts), properties, individuals (actual objects) and data values. Put simply, there are concepts (classes) such as "customer," "contract" and "driver," and each concept has properties such as "sign-up date" or "license type." Relationships such as "a driver operates a vehicle" link the concepts. On top of this come rules of the form "if these conditions hold, this conclusion follows."

An example in a Thoughtworks article source shows why rules matter. A language model can read documents and infer that "a driver operates a vehicle." But it has no way of knowing this company's rule that drivers without a particular customs clearance cannot be assigned to international routes. Such rules are scattered across people's heads, policy documents and departmental practices; they are not anywhere a machine can reason over them. An ontology is the work of writing down exactly this "meaning and rules of the business" in a form machines can read.

How it differs from schemas, taxonomies, semantic layers and knowledge graphs

Ontologies are often confused with several tools that look similar. The Thoughtworks article source places schemas, controlled vocabularies (taxonomies), semantic layers, ontologies and knowledge graphs on a single continuum. One end is "how is the data stored," the middle is "what does that data mean," and the other end is "what can a machine infer and act on." Each step up increases expressiveness. In exchange, cost and management burden grow, and more people are needed to own and look after it.

Briefly, each works like this. A database schema defines how data is held in tables, columns and keys. A taxonomy fixes the terms to be used and groups them into a hierarchy. A semantic layer unifies the definitions of metrics and terms such as "revenue" or "active customer" across the whole company. According to the same article, the semantic layer may be the most widely used layer in practice. But because it only standardizes definitions and does not contain the relationships and logic an agent can follow, it sits lower on the expressiveness axis.

The most confusing pair is the ontology and the knowledge graph. The same article source draws a clear distinction: an ontology is the blueprint (classes, relationships, rules), and a knowledge graph is that blueprint populated with real data. The knowledge graph is also what an agent actually queries. The LLM–knowledge graph roadmap paper paper summarizes that knowledge graphs store facts as triples of the form (head entity, relation, tail entity). The table below compares what each of the five tools contains.

Why didn't the Semantic Web spread widely in companies?

Ontologies first came to prominence on the stage of the Semantic Web. The goal was to make web content easier for machines to process source. The standards did survive. RDF writes all information as triples made of three elements: subject, predicate and object. As the same entity appears in multiple triples, the information links up into a graph, which is queried with a query language called SPARQL source. The RDF primer source gives examples such as attaching machine-readable information to web pages using the schema.org vocabulary, or linking data to Wikidata. The first version of OWL appeared in 2004, and OWL 2 became a W3C Recommendation in 2012 source.

Inside companies, however, the story is different. First, there is a lot to learn. The OWL 2 document source alone lists several syntaxes, such as Turtle, OWL/XML and Manchester, in addition to RDF/XML, the mandatory syntax for exchange. Its semantics are defined on the basis of a description logic called SROIQ (a fragment of first-order logic). Reasoners are powerful tools that check whether classes are consistent with each other and whether one class is subsumed by another. But using them requires dedicated tools, query languages, and people with knowledge of logic. The Thoughtworks article source calls this the "translation gap." Ontology engineers know formal systems but not the business, while domain experts know the business but not formal modeling. Transferring knowledge back and forth between them through interviews is slow and lossy.

The second trap is the single enterprise-wide model. The same article source points out that because an ontology describes reality, there is a temptation to model everything before releasing anything. Such projects tend to fizzle out after about a year. The article describes the enterprise master model as simply a neat name for the trap of trying to "boil the ocean."

The third, and most underestimated, problem is maintenance. A hand-edited model cannot keep up with the pace at which regulations change, organizations are restructured and products evolve. Building happens once, but keeping it aligned with reality never ends. Yet under the traditional approach, the owner disappears when the project ends. The article source says that without an owner and regular reviews, a model goes stale within two quarters, and calls this "a governance problem dressed up as a technology problem." As an alternative, it recommends treating the ontology as a product rather than a project. The traditional approach takes 12–24 months to deliver value, while the product approach plans 4–8 weeks per iteration. In short, the standards were not lacking. Ontologies struggled to take root in practice because there was a lot to learn, the scope overflowed, and there was no organization to look after them.

Why they are rising again: LLMs and agents need a map of the business

What changed the mood was LLMs (large language models). The LLM–knowledge graph roadmap paper lists the weaknesses of LLMs as follows: their knowledge is implicit in their parameters, they hallucinate, they make decisions based on probabilities, they are black boxes, and they lack domain-specific or new knowledge. Knowledge graphs, by contrast, are structured, accurate, interpretable and well suited to holding domain knowledge. However, they are hard to build and always incomplete. The roadmap lays out three ways to combine the two: using knowledge graphs to enhance LLMs, using LLMs to help with knowledge graph tasks (including construction), and having the two support each other.

These weaknesses become more visible now that agents touch business systems and act on behalf of companies. The Thoughtworks article source says that agents can read documents but do not have the company's operational logic, so they eventually hit a wall. OG-RAG paper, an ontology-grounded RAG (retrieval-augmented generation) method, closes this gap at the retrieval stage. Standard RAG uses domain-agnostic embeddings to find similar text chunks. OG-RAG first organizes documents into a hypergraph according to a domain ontology. A hyperedge is a connection that groups several related facts at once. It then selects only the minimal set of groups needed to answer the question and passes them to the LLM. In evaluations in agriculture and news, this method increased recall of accurate facts by 55% and answer correctness by 40% across four LLMs. In a user study, the time to link answers to their supporting evidence fell by 30%, and on reasoning tasks that apply predefined rules, accuracy was 27% higher.

On the side of constraining agent behavior, there is an enterprise agent study paper. It binds agents with a three-layer ontology covering roles, domains and interactions. The authors point to an "asymmetry" in current enterprise systems: they use ontologies to constrain only the input side (context assembly, tool discovery, governance criteria) and do not check outputs. They therefore propose a framework that also verifies answers, reasoning and compliance on the output side. In experiments with 1,800 runs across five industries and three LLMs, agents combined with ontologies were significantly ahead in metric accuracy and role consistency. The effect was largest in domains the LLM knew poorly from its training data. In the Vietnamese localization domain, the improvement was twice that in the English domain.

An ontology is not, however, a guardrail in itself. The Thoughtworks article source draws the line: an ontology tells an agent "what is allowed," but that is knowledge, not enforcement. Rules take effect when a deterministic executor actually checks them and the result is followed, and when it can be shown that the check took place. The article also says that many operational rules belong in a policy engine rather than in formal axioms. Moreover, retrieval quality comes not from the blueprint but from the graph populated with real data. So how well entity resolution (finding and merging records of the same entity) and relation extraction are done sets the upper bound on quality.

Will LLMs reduce the cost of building them?

In the Semantic Web era, the biggest cost was people extracting concepts and writing down relationships one by one. Now there is a clear trend toward handing this drafting work to LLMs. OntoEKG paper is a pipeline that splits the work into two stages. It first extracts key classes and properties from a company's unstructured documents, then arranges them into a logical hierarchy and exports them as standard RDF. On an evaluation set built from documents in data, finance and logistics, it achieved a fuzzy-matching F1 of 0.724 in the data domain. At the same time, the authors note that limitations emerged in deciding how far the scope should extend and in hierarchical reasoning.

So the realistic picture is "the LLM writes, people select." The Thoughtworks article source recommends using LLM pipelines to extract draft taxonomies from existing schemas, API definitions and wikis. Experts' work then shifts from writing definitions to reviewing and refining drafts. But the same article states firmly that this only speeds up building and does nothing for maintenance. In other words, the third reason for failure discussed above remains even with LLMs.

Plant Memory paper, which deals with equipment maintenance records, is an example that puts human review at the center of the design. The system first extracts facts from SAP maintenance work orders using rules, and uses the LLM only as an optional "second reader." When the LLM reads something differently from the rule-based extraction, it does not overwrite the result; it only raises that fact's review priority. Every fact carries a confidence score, the source text it was based on, and a verification status. Technicians check facts in a review queue, and when new terms appear, they approve or reject proposed ontology changes. The verified graph is served to people as a web app and to AI agents as an MCP (Model Context Protocol) server. However, the evaluation data consisted of 400 synthetic work orders, so the results should not be read as performance in a real plant.

Where things are heading: answers that show their grounds, and research that questions the checkers

Recent research can be read in two directions. One uses ontologies as grounding to show, alongside an answer, "how certain it is." OA-RAG paper is an agent that mixes formal reasoning and LLM reasoning, using both the ontology's logical structure and human-written annotations. Its distinguishing feature is that for each answer it reports the ontology observations used as evidence and the level of support (entailed, supported, related, unrelated). On a benchmark built from three large ontologies and on six medical question-answering tasks, it had the best overall performance among the methods compared. Stronger support levels were generally associated with higher accuracy. However, these are benchmark results and do not indicate performance in actual clinical care. In fields like medicine, where outcomes directly affect people, final judgments must be made by humans.

The other direction questions the ontology itself. It is increasingly common to wrap agents that answer over knowledge graphs in ontology-based checkers. A checker study paper directly tested the underlying assumption that "a formal schema is a reliable judge." The researchers compared three kinds of ontology: curated ontologies refined by people, ontologies generated by LLMs, and schemas extracted from data. Each was used as prompt context, as a checker, or as both. Then, on public graphs and on a private graph with nearly 1 billion triples, they counted what the checkers caught and what they flagged incorrectly. The results are fairly sobering. Some curated schemas, used as published, contradicted their own data. On UniProt, the curated ontology flagged correct answers as wrong more often than the other two approaches. On real agent answers, the precision of those flags was only marginally better than flagging at random.

Putting the two trends together, the message is clear. Ontologies are useful for grounding agents' answers and actions. But their effectiveness depends on whether the ontology is maintained to match current data. Having been expensively curated does not by itself make an ontology a good judge.

A first test of hype versus reality: is this a problem that needs an ontology?

The Thoughtworks article source says the first decision should be a "diagnosis," not an "ambition." The question to ask is not "Do we want an ontology?" but "What is the minimum semantic structure that solves the problem at hand?" The same article finds that most organizations try to climb to a higher layer than they actually need. A well-managed semantic layer and disciplined metadata alone deliver much of the value available in analytics and search. The places where an ontology is worth its cost are narrower: questions that require multi-hop reasoning, automated checking of constraints, and integration across multiple domains that flat models cannot represent.

The questions for this decision can be summarized as five. First, can the business write down specific questions the current system cannot answer (competency questions)? The same article recommends using this list of questions as the gate for funding and tying each question to a decision or a monetary amount. If the business cannot come up with questions, the issue is not a missing semantic structure but unclear value. Second, does getting the answer require following relationships across several concepts, or is it enough to align the definition of "revenue"? If the latter, a semantic layer or a well-organized table is enough. Third, does a machine need to check rules that must not be violated? If so, the executor or policy engine that actually performs those checks must be designed along with it. Fourth, does the LLM know the domain poorly? As the results of the enterprise agent study paper suggest, the more a domain involves company-specific terminology or localization, the larger the ontology's effect may be. Fifth, is there an owner who has defined who decides when terms change and how often to sync with the source systems source?

Common mistakes are also worth noting: trying to model the whole company at once, believing that maintenance is solved because an LLM produced the draft, assuming that adding an ontology creates a guardrail, and trusting the check results of a curated ontology as they are. As the checker study paper warned, the last mistake can backfire by making agents "correct" answers that were right.

Further reading, and on to Part 2

If you are new to the topic, start with the Thoughtworks article source. It covers the continuum from schemas to knowledge graphs, why ontology programs fail, and how to treat an ontology as a product, all in one piece. If you want to understand the standards, read the triple examples in the RDF primer source and then the structural overview in the OWL 2 overview source. For research, we suggest this order. First, get the big picture from the LLM and knowledge graph roadmap paper. Next, look at OG-RAG paper and the enterprise agent study paper to see how retrieval and action constraints are designed in practice. Finally, read the checker study paper for balance.

The conclusion of this part is as follows. An ontology is still useful as a 'framework', but its value comes from operations: filling it with real data and keeping it aligned over time. That leaves the next question. What changes if an ontology is not left as a static map of knowledge but becomes an operational layer that executes real business actions on top of data, such as approvals, assignments and order changes? Would the problems of maintenance and accountability then be solved, or would they become heavier? Part 2 uses these questions to look at how Palantir turned the ontology into 'an operational layer that puts actions on top of data'.