Marketing Renaissance

Tag: Michael Polanyi

  • Why Handovers Don’t Work.

    What happens when your star performer decides to quit?

    If you’re like most managers you congratulate them, ask if you can get them to stay, and then get straight to planning their handover.

    Most companies I’ve worked at have elaborate systems for this exact situation. Most of them work very well — until they don’t. In my experience, no matter how elaborate, even the best ones work only 50% of the time. Pretty soon you run into the biggest weakness of any handover process: edge cases.

    An edge case is a problem that carries meaningful impact but happens infrequently. Easy to miss, but impossible to skip when it’s staring right at you.

    Now before you judge the documentation process or suggest using AI, let me tell you this: expert knowledge is hard to capture even with the best tools and processes.

    Here’s why.

    A philosopher named Hubert Dreyfus1 spent years studying how humans develop skill. What he found was unexpected. Experts don’t just accumulate knowledge, they reorganise it into heuristics, recognisable patterns, and perception.

    Simply put, expertise is as much learning through lived experience as it is acquiring deeper knowledge.

    Here’s how we know this.

    In the 1940s, a Dutch chess player and psychologist named Adriaan de Groot wanted to understand what separated great chess players from good ones. The obvious assumption was that grandmasters were simply smarter.

    And their intelligence gave them the edge to think further ahead, consider more moves, and process more combinations. That assumption, ironically, is still how many organisations treat expertise today: as innate ability rather than accumulated experience.

    So de Groot ran an experiment. He showed chess positions to players of different skill levels and asked them to think aloud as they analysed the board. What he found changed the was unexpected.

    The grandmasters weren’t thinking more. In many cases they were thinking less. They considered fewer moves than intermediate players but the moves they considered were almost always the right ones.

    De Groot couldn’t fully explain the mechanism. That came later, when Herbert Simon and William Chase picked up the work in the 1970s. They discovered that grandmasters had memorised somewhere between 50,000 and 100,000 meaningful board patterns or chunks, as they called them, accumulated over years of play. When a grandmaster looks at a board they aren’t seeing 32 pieces. They’re recognising configurations, the way you recognise a face without consciously processing each individual feature.

    Dreyfus took this further and made it philosophical. While a novice follows rules. For the grandmaster, decades of pattern recognition have dissolved into instinct.

    That’s the first clue to our handover problem.

    When you ask as expert to explain their thinking, what you get is a reconstruction built after the fact. The reasoning is plausible, often technically accurate, but it isn’t the actual cognitive process. It’s a story told to explain something that happened faster than language.

    This is what makes handovers structurally impossible to fix after the fact.

    Most handover documents contain reasonable questions about the job, the projects, the clients, the processes. Experts can answer them accurately. But the knowledge that handles edge cases isn’t organised as answers to questions, it’s organised as responses to situations.

    Like the grandmaster reading the position of pieces on a board, it only becomes accessible when the right situation activates it.

    Michael Polanyi captured this with a phrase that has stayed with me:

    ‘We know more than we can tell.’

    But even that undersells it. It’s not just that expert knowledge is hard to articulate.

    It’s that the very process of becoming expert reorganises knowledge into forms that are faster, more contextually sensitive, and more integrated than language allows.

    The next clue comes from Herbert Simon, the same Simon who helped explain the grandmaster’s chunking.

    Studying how people make decisions under complexity, in organisations, in economics, in everyday life, he found that nobody optimises. Not really…

    He wanted to understand how any mind, human or machine, navigates a world too complex to fully process. His answer was bounded rationality: the idea that minds don’t optimise, they satisfice.

    We search until we find something good enough, using the categories and heuristics available to us, and we stop. This is the only viable strategy for a finite mind operating in an infinitely complex world.

    The implication for expertise is precise: when you sit down to document your knowledge, you would find it much easier to write down what’s prompted and what’s top of mind. Easy peasy.

    But you won’t be able to consciously thing of every edge case unless you’re actively prompted. They knowledge exists in your experience but it’s never stored as something a finite mind could retrieve on demand.

    So what do you do?

    The answer isn’t a better offboarding process. It’s a different relationship with expertise altogether.

    One that doesn’t wait for the resignation letter. Expert knowledge needs to be treated as something you harvest continuously, while the expert is still performing, while the knowledge is still alive and activated in real situations.

    The most underused tool for this is reflection and managers can facilitate it in regular 1-2-1s.

    When your star performer has a great quarter, don’t just celebrate and move on. Help them contextualise it by asking them to walk you through exactly the specifics: What did they choose to do? What did they choose not to do, and why? Where did they make a judgment call that isn’t in any playbook?

    Be mindful that is not an interrogation. it is a way to contextualise knowledge that’s as valuable for the expert as it is for the organisation. Repeated over months and years, it builds something no exit interview ever could.

    One of my favourite experts on the subject is Peter Senge. He spent years studying why some organisations learn and others don’t. His conclusion was that the unit of capability in any organisation isn’t the individual — it’s the team.

    Top performers matter, but an organisation that depends on them is fragile by design. What makes organisations genuinely capable is the degree to which knowledge circulates, gets tested, gets refined.

    The handover problem is really a symptom of this: organisations that treat expertise as individually owned discover what they’ve lost only when the individual leaves.

    Most organisations have many people, each carrying expertise that is partially tacit, partially compiled, partially invisible even to themselves. The moment you try to build systems that run on shared expertise, this stops being an offboarding problem and becomes a competitive advantage.

    1Hubert Dreyfus: https://en.wikipedia.org/wiki/Hubert_Dreyfus

  • The Tacit Knowledge Problem

    In the 1970s, a team of researchers set out to build a computer system that could teach surgery.

    The idea was straightforward. Find the best surgeons in the world. Record everything they did and turn it into a training program that could transmit world-class surgical skill to the next generation of doctors.

    They found the surgeons. They recorded everything. And then they ran into a problem that nobody had anticipated.

    The best surgeons couldn’t explain what made them good.

    When they sat down and tried to articulate what they were doing and why, the accounts they gave were incomplete. They described the mechanics. But they couldn’t describe was the accumulated judgment of ten thousand procedures compressed into instinct that had become invisible even to themselves.

    The researchers had set out to capture expertise. What they discovered instead was that expertise, at its highest levels, resists capture.

    That discovery has a name. It is one of the most important and most underappreciated ideas in the history of human knowledge. And understanding it is the only way to understand why building systems that perform at expert level is so much harder than it looks.

    The Philosopher Who Saw It First

    Michael Polanyi was not the kind of person you would expect to reshape the field of artificial intelligence. He was a Hungarian-born chemist who had fled Nazi Germany in the 1930s, eventually landing at the University of Manchester where he spent the second half of his career not doing chemistry but thinking about what scientific knowledge actually is, how it develops, and how it moves from one generation of scientists to the next.

    By the 1950s Polanyi had become increasingly troubled by a dominant assumption in Western philosophy of knowledge, the idea that genuine knowledge is knowledge that can be made fully explicit.

    That if you truly understand something you should be able to state it clearly, defend it logically, and transmit it to anyone willing to pay attention. Knowledge, in this view, is essentially propositional. It lives in sentences. It can be written down.

    Polanyi thought this was fundamentally wrong.

    And he spent the better part of two decades building the argument against it.

    His most concentrated statement of that argument came in 1966 in a slim book called The Tacit Dimension. The book opens with a single sentence that contains the entire problem: “We can know more than we can tell.”

    It sounds simple. It is not.

    What Polanyi Actually Meant

    Polanyi’s argument begins with perception, the most basic act of knowing something.

    Consider how you recognise a face. You can look at a photograph of someone you know and identify them instantly, across years, across changes in weight and hair and age.

    You are doing something genuinely sophisticated – processing a complex pattern and matching it against memory with remarkable reliability. But you wouldn’t be able to explain ( which features you used, how you weighted them, what the decision rule was) exactly how you did it to someone. Not because the knowledge isn’t there.

    Because the knowledge doesn’t exist in a form that can be stated.

    Polanyi called this tacit knowledge or knowledge that we hold and use reliably but cannot fully articulate. He distinguished it from explicit knowledge i.e. knowledge that can be stated, written down, and transmitted through language and instruction.

    The distinction sounds straightforward but its implications are radical. Because Polanyi’s claim was not just that some knowledge happens to be tacit. His claim was that tacit knowledge is foundational.

    He believed that all explicit knowledge rests on a substrate of tacit knowledge that can never be fully surfaced. You cannot make everything explicit because the act of making something explicit always relies on tacit capacities that are doing the work underneath.

    He illustrated this with what he called the subsidiary-focal distinction. When you use a hammer, your attention is focused on the nail. The hammer itself (its weight, its balance, the feel of it in your hand) is present to you, but subsidiarily – you are not focusing on it. You are focusing through it. If you shift your attention to the hammer itself, you lose your grip on the task. The tacit knowledge that makes you competent with the tool only functions when it stays tacit.

    This is why expertise is so hard to teach and so hard to transfer.

    The expert is not withholding anything. They are focusing through their knowledge, not on it. Asking them to articulate it is like asking them to stare at the hammer instead of the nail. The act of articulation disrupts the very thing you are trying to capture.

    The Iceberg

    The most useful way to think about expertise is as an iceberg.

    Above the surface sits explicit knowledge, everything that can be stated, taught, written down, encoded in manuals and training programs and textbooks. This is the knowledge that moves easily. You can put it in a document and send it across the world. It survives the death of the person who held it. It can be transmitted to ten people as easily as to one.

    Below the surface sits tacit knowledge. It’s vastly larger, and entirely invisible from above. This is the knowledge that makes the difference between someone who knows the rules and someone who can actually perform.

    It includes:

    Perceptual knowledge: the ability to notice what matters. The experienced radiologist who sees something in a scan that a resident misses. The fund manager who reads a room full of executives and knows within minutes whether the business is actually healthy. They are perceiving things that are genuinely there, but their perception has been trained by years of experience into a sensitivity that cannot be directly transmitted.

    Procedural knowledge: knowing how, as distinct from knowing that. You can read every book ever written about riding a bicycle without being able to ride one. The knowledge of how to ride lives in the body, in the calibration of balance and response that only practice builds. Professional skills work the same way. The senior copywriter who reads a brief and knows immediately what angle to take is not applying a rule. They are drawing on something built from thousands of briefs processed over years.

    Contextual judgment: knowing when the rules apply and when they don’t. This is perhaps the most valuable and most elusive dimension of expertise. Textbooks describe how things work in general. Experts know how they work in this situation, with these constraints, given what happened last time. That situational sensitivity is almost impossible to encode because it is not a rule at all.

    The knowledge of what to ignore is perhaps the least discussed but most practically important. Experts are not just better at processing relevant information. They are better at filtering out irrelevant information. They have learned, through experience, what doesn’t matter. That negative knowledge is as hard to transfer as the positive kind.

    When Tacit Knowledge Is Lost

    The organisational implications of tacit knowledge loss are severe and largely invisible until it is too late.

    When an expert leaves an organisation what walks out the door is not just the explicit knowledge they held. That part, if the organisation was reasonably diligent, has probably been documented somewhere. What walks out the door is everything below the surface. The perceptual sensitivity built over decades. The contextual judgment that knew when the documented process didn’t apply. The feel for what mattered and what didn’t.

    This loss is structurally invisible because explicit knowledge is easy to see and tacit knowledge is not.

    Organisations inventory what they can measure. They document processes, capture decisions, build knowledge bases. And then they are surprised when the person who wrote the process document leaves and everything quietly starts going wrong gradually, in the accumulation of small decisions that the documentation doesn’t cover and the new person doesn’t know how to make.

    NASA experienced this in one of its most documented forms. After the Apollo program ended in the early 1970s, the organisation went through waves of restructuring and downsizing.

    When NASA began planning a return to the moon decades later, it discovered that significant tacit knowledge about how to build certain components had simply ceased to exist within the organisation. The documentation was there but embodied, practiced, judgment-laden knowledge was not.

    This pattern repeats across industries and new graduates, however well trained, cannot replicate what the experienced staff did without being able to say why.

    Why This Problem Is Acute Now

    For most of the history of organisations, tacit knowledge loss was a serious but manageable problem. It was addressed, imperfectly, through apprenticeship and practice rather than instruction.

    The medieval guild system was essentially a tacit knowledge transfer mechanism. So is the residency system in medicine or the partnership track in professional services. You spend years watching someone who knows what they’re doing, and eventually some of what they know moves into you.

    Apprenticeship is slow and expensive. But it works, because tacit knowledge can be transferred through observation, practice under guidance, and through the accumulated experience of being in the room when an expert makes a hundred decisions and slowly developing a feel for why.

    The agentic era has broken this in a specific and important way.

    The promise of AI agents is that you can encode expert-level performance into a system and deploy it faster, cheaper, and more consistently than any human expert. The appeal is obvious. The problem is that the entire premise depends on being able to get the expertise into the system in the first place.

    The real question is transferring expertise that is mostly tacit.

    This means that most agent implementations are not actually encoding expertise. They are encoding the explicit layer – the documented processes, the stated rules, the guidebook version of how things work.

    Most organisations have deployed that explicit layer at scale and called it an expert system.

    What they have actually built is a very fast, very consistent, very scalable average.

    It performs well on the cases that the explicit rules cover. It fails, sometimes catastrophically, on the cases that require the judgment, the contextual sensitivity, the feel for when the rules don’t apply that lives below the surface of what any expert can easily say.

    The gap between a competent agent and an expert-level agent is almost entirely a tacit knowledge gap.

    It is not a technology or a model problem. It is the same problem the surgical researchers hit in the 1970s, the same problem Feigenbaum hit sitting with chemists in the 1960s, the same problem Polanyi was describing in 1966.

    We can know more than we can tell. And until you have a method for surfacing what can’t easily be told, you are building on the visible part of the iceberg and wondering why the system keeps running into things it didn’t see coming.

    What This Means in Practice

    Polanyi’s insight makes this problem legible. And this is where every solution begins.

    If tacit knowledge cannot be extracted through direct questioning, it can be approached through other means. Through observation rather than interview. Through cases rather than principles. Through contrast rather than description. Through the careful, patient work of watching experts perform and finding ways to surface the knowledge they are focusing through rather than on.

    That work has a name and a methodology.

    It is the discipline of elicitation and it is where the practical response to the tacit knowledge problem lives.

    But before elicitation can work, you have to understand what you are trying to elicit. You have to know that the knowledge you need is not sitting on the surface waiting to be asked for.

    You have to know that the iceberg is mostly underwater, and that the part you can see is not representative of the part you can’t.

    That understanding is what Polanyi gave us. And it is why, sixty years after he wrote it, his single sentence still contains everything you need to know about why this problem is hard.

    We can know more than we can tell.

  • Knowledge Engineering

    Picture a group of scientists in the summer of 1956. Not just any scientists, these are the sharpest minds of their generation, the kind of people who genuinely believe they can do what nobody has done before. They’ve gathered at Dartmouth College in New Hampshire with a bold idea and a proposal to match.

    The proposal was written by a mathematician named John McCarthy who was the kind of person who believed that if you couldn’t solve a problem it was because you hadn’t thought about it carefully enough.

    The proposal said they could figure out how to make machines think. Not someday. This summer.

    Now, nobody in that room actually believed it would take just two months. But they did believe it was possible.

    Human reasoning, they argued, was essentially a formal system. A set of rules. And if you could identify the rules, you could replicate the reasoning in a machine.

    The summer came and went. They didn’t crack it. Most participants drifted in and out — only McCarthy, Minsky, and a mathematician named Ray Solomonoff stayed for the full eight weeks.

    They argued about approaches, worked mostly on their own ideas, and eventually packed up and went home.

    But something did happen that summer that none of them had planned for. For the first time, all the people working on this problem had been in the same room.

    McCarthy had insisted on calling the field artificial intelligence. Claude Shannon thought the name was too dramatic. He lost that argument. The name stuck.

    They went back to their universities. They kept working. But now they were working on the same named thing, and they knew who else was in the game.

    The summer project failed. The idea it launched and the community that formed around it turned out to be worth more than any result they could have produced that summer.

    The Man Who Asked a Different Question

    Fast forward about a decade. The AI field is now a real institution. DARPA is writing serious checks. The Stanford AI Lab, MIT, Carnegie Mellon are the places to be if you’re brilliant and ambitious and want to work on the hardest problems in science.

    Graduate students are being recruited from the best programs in the country. The culture is intense, competitive, and deeply optimistic. Nobody has built the thinking machine yet, but the prevailing mood is that it’s only a matter of time.

    Into this world arrives Edward Feigenbaum.

    Feigenbaum had studied under Herbert Simon at Carnegie Mellon. Simon is one of the founding figures of AI, a man who believed human decision-making could be modeled as a computational process and had the Nobel Prize to show for a career built on that belief. Feigenbaum absorbed that conviction but added a practical instinct that would eventually set him apart from his peers.

    He thought the field was aiming at the wrong target.

    Building general intelligence (a machine that could think about anything) was too hard and too vague.

    The better move was to go narrow and go deep. Pick a specific domain, find the world’s best experts in it, and build a system that could perform at their level. Prove it works. Then do it again in another domain.

    It sounds obvious now but at the time it was a minority view in a field riding high on the dream of general intelligence. Feigenbaum didn’t care. He was more interested in what actually worked than in what was theoretically elegant.

    He got his first real chance to test the idea when a Nobel Prize-winning geneticist named Joshua Lederberg knocked on his door.

    The Problem in the Lab

    Lederberg had a practical problem. His lab was generating mass spectrometry data faster than his chemists could interpret it.

    A mass spectrometer fires a beam of electrons at a molecule, breaks it apart, and produces a kind of fingerprint that an expert chemist can read to figure out what the original molecule was. It’s painstaking work that requires deep expertise. Lederberg wanted to know if a machine could do it.

    Feigenbaum said yes. And together they began building DENDRAL: the first serious attempt to encode expert-level knowledge into a computer system.

    Chemistry has known rules. Molecules behave according to principles that can be written down. Feigenbaum and his team encoded those principles and the system started producing reasonable results.

    The early work was promising.

    DENDRAL could handle straightforward cases when the molecules behaved exactly the way the textbooks said they would. But chemistry in practice is messier than chemistry in textbooks.

    When they pushed the system into more complex cases it started producing answers that were technically consistent with the rules but wrong. And no amount of refining the existing rules fixed it.

    Something was missing. The knowledge in the published literature wasn’t the whole picture. Expert chemists were doing something beyond what had ever been written down. The only way forward was to go and ask them.

    The Wall

    The chemists could solve the problems. They were reliable, accurate, and fast. You could test them and they’d get it right. But when Feigenbaum and his team asked them to explain exactly how they couldn’t really give a clear answer.

    The accounts they gave were good. They walked through cases, described what they were looking for, traced their reasoning step by step.

    But they were incomplete in ways nobody in the room could see at the time. They skipped steps they didn’t realise they were skipping. They applied judgment they couldn’t fully articulate. They knew things they could not say.

    Feigenbaum had stumbled onto something that would reshape his entire understanding of the problem. The barrier to building an expert system wasn’t computational. The inference engine, the search algorithms, the formal logic wasn’t the hard part. The hard part was getting the knowledge in.

    He called it the knowledge acquisition bottleneck. And naming this challenge reframed the entire question.

    It was no longer just ‘can machines reason?’ It was something deeper and harder:

    • What is knowledge?
    • How do humans actually hold it, and
    • How on earth do you move it from a human mind into a machine?

    That question would take decades to answer. It still hasn’t been fully answered. But asking it clearly was the beginning of a new discipline.

    The Doctor in the Room

    A few years later, a medical student named Edward Shortliffe sat down to do something similar in medicine.

    MYCIN, the system Shortliffe built under Feigenbaum’s supervision in the early 1970s, was designed to diagnose bacterial blood infections and recommend antibiotic treatments.

    The stakes were as high as they get. The wrong antibiotic at the wrong dose can kill a patient. So MYCIN needed to reason at the level of a specialist in infectious disease.

    Shortliffe spent years in sessions with those specialists. They were cooperative, intelligent, and genuinely trying to help. They would walk him through cases, explain their reasoning, describe what they were looking for. And the accounts they gave were good enough to build a working system.

    But Shortliffe noticed the same gaps.

    It wasn’t just that the doctors struggled to explain their reasoning. It was that large parts of their reasoning were no longer available to them as conscious thought.

    Years of seeing thousands of patients had compressed their knowledge into something faster and more automatic than deliberate thinking. They didn’t work through a diagnosis, so much so that they recognised it. The way an experienced driver doesn’t think about changing gears. The way a native speaker doesn’t think about grammar.

    This is what made the problem structural rather than just difficult. You couldn’t solve it by asking better questions or running longer sessions.

    The knowledge had been absorbed so completely into expert intuition that it no longer existed as something that could be directly retrieved and handed over.

    MYCIN eventually encoded the knowledge of those specialists into roughly 600 production rules — IF this condition, THEN this action, with an attached certainty factor. When it was tested against specialist physicians in controlled evaluations, it performed at or above their level.

    A machine, reasoning from encoded rules, was diagnosing infections as well as doctors.

    It was never deployed clinically. Not because it failed, but because nobody could agree on who was responsible if it got something wrong. The liability questions proved harder than the technical ones. But the proof of concept was undeniable. The question now was how to do it reliably, at scale, across domains.

    What Is Knowledge Engineering?

    Knowledge Engineering is the practice of getting expertise out of human minds and into a form that machines can use.

    The knowledge engineer job is to surface what doctors, chemists, financial analysts, and other experts know, both what they’re consciously and unconsciously aware of into a structure a system can reason with.

    Knowledge Engineering works moves through four stages.

    Stage 1: The first is knowledge acquisition: the elicitation sessions, the interviews, the case walkthroughs. This is the hardest stage and the most underestimated.

    Naïve approaches fail consistently. If you just ask experts what they know, you get the idealized version. So knowledge engineers use methods designed to surface what experts do rather than what they say they do.

    One approach is to watch them work. Sit alongside an expert as they solve a real problem and ask them to think out loud as they go. What comes out is messier and more revealing than any interview.

    Another is to work through specific past situations where the expert made a consequential decision. Walking through a real case unlocks detail that abstract questioning never reaches. The expert remembers what they noticed, what they ignored, what made this situation different from the last one.

    A third is contrastive questioning: instead of asking an expert to describe what they do, ask them to explain the difference between two situations. Why did you treat these two cases differently? What did you see in one that you didn’t see in the other? Comparison forces precision in a way that open description rarely does.

    None of these methods extract knowledge perfectly. But they get closer to what experts actually do than asking them to explain themselves ever could.

    The second is knowledge representation: translating what you’ve acquired into a formal structure. Production rules. Semantic networks, decision trees, ontologies, are some examples.

    The third is knowledge validation: testing whether the encoded knowledge actually performs correctly across cases but the easy ones and the edge cases. A system that works on textbook cases and fails on real ones is not an expert system. It is a simulation of one.

    The fourth is knowledge maintenance: managing the knowledge base as a living thing over time. Domains evolve. Expertise updates.

    The Philosopher Nobody Expected

    The person who gave all of this its deepest intellectual foundation wasn’t a computer scientist. He was a Hungarian-British chemist turned philosopher named Michael Polanyi who had been thinking about the nature of expertise since the 1950s.

    In a short book called The Tacit Dimension, published in 1966, Polanyi wrote a sentence that the AI community would eventually find its way to: “We can know more than we can tell.”

    Polanyi’s argument was that expertise is not a large inventory of facts and rules that an expert has accumulated and could, in principle, recite. It is something more integrated than that.

    Years of experience get processed into pattern recognition, intuition, and judgment that operates faster than conscious reflection and is only partially accessible to it. The expert chemist, from the earlier example, when reading a spectrogram is not running through a checklist, they are doing something that feels perceptual and immediate and that immediacy is precisely what makes it so hard to transfer.

    Polanyi never wrote about AI. He was thinking about scientific knowledge and how it gets transmitted between generations of researchers. But when the knowledge engineering community discovered his work, it landed like a precise diagnosis of exactly what they were dealing with.

    That insight didn’t make the problem easier. But it made it legible. And legibility is where every solution begins.

    The Boom, the Collapse, and What Survived

    By the mid-1980s, knowledge engineering had become an industry. XCON was saving Digital Equipment Corporation an estimated $40 million a year. Hundreds of companies were building expert systems. The commercial application of AI, after decades of promise, seemed to have genuinely arrived.

    Then the systems started breaking.

    Early expert systems were brittle. They performed well within the boundaries of their encoded knowledge and failed unpredictably outside them. Maintaining large rule bases as domains evolved was expensive and error-prone. The knowledge that was accurate in 1982 was wrong or incomplete by 1988.

    XCON’s the expert system Digital Equipment Corporation built to configure its VAX computers, eventually grew to over 10,000 rules. Keeping it current required a dedicated team of knowledge engineers working continuously. That ongoing cost was one of the reasons large expert systems eventually became unsustainable.

    The AI Winter arrived. Funding contracted. The expert systems industry collapsed.

    The field responded by turning toward machine learning. Using statistical approaches that could learn patterns from data rather than requiring knowledge to be extracted and encoded by hand. It was an appealing solution to the bottleneck problem. Instead of laboriously eliciting knowledge from experts, you trained systems on large datasets and let them figure it out.

    What was gained in scalability was lost in other ways.

    The key difference: Statistical systems learn patterns, not principles. They can’t explain their reasoning. They generalise within the distribution of their training data and fail outside it. They don’t encode the kind of structured, causal understanding that experts use when they encounter something genuinely new.

    For a while Knowledge engineering fell out of fashion but the questions it was built to answer remained alive.

    Overtime, it evolved into ontology engineering, into organisational learning theory, into research on how experts actually make decisions under uncertainty.

    Why You’re Reading This Now

    Large language models have changed what’s possible. They have absorbed an extraordinary volume of human knowledge and can perform across a remarkable range of domains with genuine fluency. The optimism in the AI field today echoes the optimism of that summer at Dartmouth.

    But the knowledge acquisition bottleneck has not been solved by scale. It has been displaced.

    What LLMs cannot do, without deliberate intervention, is perform at expert level in specific organisational contexts. They have breadth without depth. They produce plausible output, which is not the same thing as correct output. They have patterns without the underlying causal understanding that experts use to reason about situations they’ve never seen before.

    Anyone building a system that needs to perform at expert level is facing the same problem Feigenbaum faced sitting with those chemists in the 1960s.

    How do you move what an expert actually knows into a system that can use it?

    That problem doesn’t yield to better prompts or bigger models. It yields to method. The method has a name. It is sixty years old and more relevant now than it has ever been.

    That’s what knowledge engineering is. And that’s why it’s where this story starts.