AI tools are now the horsepower of computer programming. They are generally great for writing glue-code and integration tasks, probably less than ideal for complex problems on complicated programming settings. What could be the reasons that prevent generators on full-scale adoption?

I am implementing a declarative programming language that facilitates the synergy between automatons and humans on software development by forcing AI tools to generate intuitive code and to allow human operators to understand what is in there. It is an attempt to lower the barriers by simplifying the programming experience.

For a rationale, I will touch here various aspects that includes cognitive aspects of problem-solving applied to programming and the role of AI tools such as LRMs on software development. I will provide arguments for stating that the accidental complexity of the programming system may worsening the performance of AI generators as well as it affects human developers.

Maybe you want to read “Pull Down Complexity with Kubrick” in PDF?

Started as a data programming language for pairing database queries with programming controls, Kubrick may evolve in an advance integrated programming environment with the footprint of a Jupiter notebook and the immediacy of a spreadsheet. It lets agents to focus on what they want to achieve making easy things easily without expressivity losses.

To mitigate the lack of solution productivity¹ in generators - the way that an optimizer generates deriving combinations of solutions from a set of given axioms - I extended the language for helping agents to cover combinatorial problems. Think of answer set programming (ASP) but with an eye on integration and usability.

This is an ongoing project that was the topic of my Master’s dissertation “Programming Language and System for Enhancing AI-Assisted Software Development” I defended last December 2025, and it summarized several insights I gathered during my experience in the field. I just recently started to open source it and I will gradually share it on GitHub².

Cognitive Aspects of Programming

AI tools are everywhere. In every domain it is possible to find features that can benefit of AI capabilities. If we exclude certain high-risk sectors - nuclear plants, aviation control - AI is already assisting human labour, and the pace of adoption will certainly accelerate.

Language models were conceived for NLP tasks and trained with all sort of available text, so it is quite natural to think about their application in programming code generation. After all, programming is text-based for being written and read like any other text. Is it then reasonable that LRMs exhibits appreciable programming skills as they do on generating phrases?

With AI Tools it is not so bizarre to ship live entire applications in hours rather than weeks, but their effectiveness may fall short on accomplishing what prompters want them to do. Complaints regarding GenAI performances are summarized in:

• Problems with multi-step reasoning.
• Struggles with mutable states and side effects.
• Shallow code understanding.
• Fail to meet requirements.
• Generate complicated code.
• Despite same prompt, they generate different code.
• Despite different prompt, they generate the same code.

Building a program essentially means to face it from two distinct sides: the problem domain and the solution domain. Understanding the problem to solve is the most important task, and if it is more or less difficult to grasp, it is related to its essential complexity. On the other hand, the complexity of the solution domain is referred to as accidental complexity, and it is introduced by the ecosystem necessary to implement the solution.

“If I had an hour to solve a problem, I’d spend 55 minutes thinking about the problem and 5 minutes thinking about the solution” - A. Einstein

The programming system consists of several substrates that include libraries, tools, and external systems, and when it increases the programming effort increases exponentially. While the essential complexity isn’t negotiable, the accidental one must be kept at the lower level possible.

Many difficulties automatons and developers shows on generating code might be due to a common root: the accidental complexity carried by the programming system. The intuition behind this project is to address this issue from its foundations.

While increasing the complexity of a system absorbs more of the engineer’s working memory, it also increases the perplexity³ an LRMs on solving the same task. Perplexity measures how unexpected a token is to the LRMs, and that means that the accidental complexity negatively impacts the generation of code, with more chances of introducing bugs not only by developers but also by automatons. Humans and the artificial agents show a significant positive correlation when faced with similar alienating settings.

How to reduce complexity

Do you want to reduce accidental complexity and favor agents on programming? Be inspired by what teachers do for preventing⁴ plagiarism through AI, and apply the opposite strategies for making programming easier:

Uniform experience

From the point of programming experience, what is daunting³ for agents is the heterogeneity of the ecosystem. Different paradigms for configuration, data access, service orchestration, remote procedure calls, testing and deployment. Each of those aspects requires specific skills and knowledge that increases accidental complexity. A proper programming environment should nullify those frictions by providing a consistent an uniform way to interact with it.

Open authorship

Software development often involves a hard separation between programmers and users. I believe that accidental complexity prevents final consumers to be empowered to change the software applications. Are there already some examples that allow that?

I think everybody has at least once used a spreadsheet application. One thing that’s very clear to users is that the spreadsheet does not have really a separate environment for programming and for use. A spreadsheet can be modified at any time by modifying the data or the formulas it contains.

Notebooks like Jupyter or Google Colab allow users to naturally split problems into smaller pieces, solve them individually with an immediate feedback. Those approaches lower the accidental complexity and Kubrick aims to combine both.

Self-sustainability

Self-sustainability refers to the extent to which a system’s behavior can be changed without having to step outside to a lower implementation level⁵

The most predictor of a low-code platform’s success is the ability to change the system’s behaviour from the deepest layers. This would be achieved by introducing macros, programming fragments intended to generate programs inside the programming system itself. Why not being inspired by traditional languages like Lisp for that? It has stood the test of time with great honor, thanks also to its homoiconic nature that enables to write macros easily.

Logic + functional

If we consider AI as an assistant, the programmer double-checks that what has been generated satisfy explicit and implicit requirements. Software code is more read than written and the immense capacity of the generators to flush out large quantity of code can easily saturate human scrutiny, urging the necessary of a clear, concise and easy language for encoding programs. We need to let agents to express what they want to achieve more than how to achieve it and remove the need of boilerplate code.
Immutability, control over side-effects, unification, pattern-matching can definitely help on dragging down complexity.

Relation algebra

I think one of the main complexity drivers is the impedance mismatch between query and programming languages. Those idiosyncratisms are usually mediated by ORMs frameworks but their slippery slope⁶ can trigger more problems than they solve. Relation algebra is the basic foundation of database theory and when combined with programming constrols, it provides a smooth experience for data-programming.

Combinatorics

When programmers encounter new code, they actively simulate the program’s behaviour in their head and create mental models of its structure and logic. This is why programming is more than a logically demanding task⁷ with significant implications on how people interact with code. This is confirmed by the use debugging tools, syntax highlighting, code formatting, visualization applets; are all there for supporting the brain’s reliance on logical simulations.

The attitude on playing code behaviours confirms that intelligence can’t be diverted from the ability to search through a potential infinite combination of concepts and rules. This is in summary the idea of productivity which refers to the compositional generation of optimal propositions from a valid set of grounded statements.

Do automatons excel on that? The building blocks of LRMs lack of intrinsic search, though efforts have been applied⁸ on forcing recursive reiterations on their conclusions for minimizing hallucinations. How would be possible to inject intentional search where it is lacking? This is why combinatorial programming can boost applications’ intelligence by commoditizing optimized solution search.

Make easy things easily

…and complex things doable.

Not a single artefacts comes out of the vacuum and Kubrick is not an exception. I’ve been inspired by a multitude of established ideas on re-arranging them and creating new ones. From Prolog I took the variables’ unification, the automatic pattern matching on method’s activation and the success/failure assertions for validating method invocations, typical of logic programming. From Julia I’ve borrowed the fundamental recursive data structures for combining choices, named tuples and sequences. From ASP I’ve taken the stable model semantics for reasoning on alternative solutions and choices, the alternative values.

A Combinatorial use case

Central to la language is the recursive data type that include sequences, named tuples and choices:

grocery orange type->fruit price->1.2,
  rating->5 origin->italy;spain 

Choices are alternative values (or expressions) to say that the fruit can either be 1st, 2nd, bio or local quality. Named tuples are key-value pairs and sequences are ordered collections, and any element can be a nested data structure.

Let’s assume we need to create purchase lists with only one item per type and grouped by origin:

cart Item -|1^Origin,1#Type| 
  grocery Item type->Type origin->Origin

You need to filter only purchase below 30€:

\ cart Item, grocery Item price->Price, 
    Tot->(group sum Price),
    < Tot 30

and get the best combination of items that maximizes product’s ratings and minimizes the total cost. View it as a sort of multi-objective Pareto optimization:

\> cart Item, grocery Item rating->Rating,
     group sum Rating
\< cart Item, grocery Item price->Price, 
     group sum Price

User experience

The prototype’s interface resemble a notebook environment with cells for writing code, for AI prompting and for visualizing results. While the program generation could be delegated to external LRMs, the program execution is performed locally in the browser. This approach combines the divide/conquer of notebooks, the immediacy of spreadsheet applications and the assisted program generation.

Process flow

When the user submit the prompt, the web application (Capriccio) delegate a specialized module (Avro) that augments the user prompt with Kubrick language reference documentation and forward the enriched request to the AI service. The generated code is then executed by the web application that display the results in the notebook cell.

Compile-time Workflows

The glamoured “agentic AI” trend put LRMs in the main stage of complex programming exploiting their ability to elaborate decisions based on intricate set of input data. In the reasoning and acting (ReAct) paradigm the LLM is called during the transition in a broader state-machine that describes the the entire process. Basically, the AI tool is adopted not for planning the entire workflow where specialized modules can be engaged for specific tasks, but for deciding which action to take at each step of the process. It is a quite limiting approach that just raises the complexity of the overall solution.

Why not let the AI tools to generate the entire workflow at once? Attempts⁹ in this direction have demonstrated that it is possible to substantially improve the effectiveness of AI generators. This achievement reinforces the thesis that a highly expressive and declarative language like the one presented in this project can improve the impact of agentic AI.

Next Steps: MCP integration

We can inform the generator about available functions and their interfaces (following Kubrick’s language) and run the GenAI as an orchestrator and model the entire process in compile-time.
Available functions can be those imported from libraries but also those exposed through model context protocols (MCP), a revisited protocol for service discovery.
In this way, the single components can be invoked by the symbolic runtime that execute the generated code with more fine-grained control over its execution.

Let’s Meet in Programming Experience Workshop 2026

This year the <Programming> 2026 will take place in Munich, Germany on Mar 16-20, 2026. I will present this work in the Programming Experience Workshop 2026¹⁰. If you are around, feel free to reach me for a chat!

Giancarlo Frison

References

The analogy can be applied to software applications. Comparing the average lifespan of a building with that of a software application, the impact of adaptability is more evident. Applications are disposable and prone to continuous updates, and cases of legacy systems remaining untouched after decades are as rare as anecdotal. Software is built to let users achieve specific goals. As building inhabitants change their uses and, consequently, their purposes and mediators, so too are programs clearly subject to evolution.

Mediators: point of interaction between agents (user, developers and AI generators) and the system. They are affordances such as APIs, design patterns, frameworks, but also GUIs, command lines, etc.

While users interact with the application via its available mediators, programmers leverage a different kind of them offered by the programming system to crystallize the output of their work into a runnable application.

Programming system: Integrated and complete set of tools sufficient for creating, modifying and performing programs. It encompasses programming languages, libraries, frameworks, development environments, and other tools that facilitate the software development process.

The more adaptive a programming system is, the easier it is to apply changes - and at a higher rate - than with less adaptive ones. A flexible programming system is better for the output. If the application is appreciated by customers, change requests inevitably arise, and an important aspect of evaluating whether to build a new application from scratch or update the current one is to consider the potential adaptability to changes. Certain mediators are involved in changing existing software, especially those more inherently tied to the application’s components. We can refer to them as substrates.

Substrates: various underlying mechanisms or interfaces that allow to interact with and modify the software​.

Substrates are the layers that encapsulate different modules and environments. They could be related to data management, dominated by the database interface, for example. Mediators may be also volatile archetypes, like prescribed design patterns for software development. When a generic design is found for a recurrent problem pattern, the solution is actually an abstract template for lowering the accidental complexity of the application. That archetype could be seen as a substrate applied to the application for a common problem.

Accidental vs Essential complexity: if you have ever faced a programming challenge like the ones you may be submitted during interviews, you may have experienced the frustration of not being able to solve it. This is the feeling the essential complexity manifests itself. On the other hand, when the programmer should update an intricated legacy system without documentation, the desperation can easily take over. The sorrow is now related to the accidental complexity, which is the kind of trouble introduced into the ecosystem. The essential is not negotiable, but the accidental must be kept at the lowest level.

Let final users be programmers

Mediators vary depending on the role of the agent. Those accessible to consumers may differ from those engaged by producers, as they are indeed very different. Programmers leverage mediators from the programming system, while end users interact with the application through exposed substrates, like GUIs. This separation is usually irreconcilable. Users may not expect to modify the program they are using. In fact, they have almost no chance to change it by themselves. More adaptability also means open authorship, a principle that empowers different roles to modify the software from lower-level substrates than those reserved for using it.

Real cases of open authorship

Are there any live examples of open authorship out there? If you are reading this article, you are probably doing so through a browser. You can open the menu, access “developer tools,” and inspect the page. If you have a bit of HTML knowledge, you can modify the page directly. Pay attention to the page itself, because changes are instantly reflected in the rendered view. The HTML substrate is accessible and modifiable, and the feedback is immediate. A browser lets the consumer be a producer to some extent at the same time. Spreadsheets are another example of open authorship. The grid is visually accessible, and the engine works at the cell level. Users can modify the data and the formulas, and results are promptly reported. Moreover, the formula substrate is hierarchical, as formulas can be encoded at different levels of capability, from simple domain-specific languages to more complex languages like VBScript or Python.

What’s programming?

Empowering users to modify software without the help of professional developers is a way to make it more resilient to the changes that occur over time and to extend the application’s longevity. As mentioned before, this can’t be achieved by eliminating the essential complexity of the problem. The user - whether programmer or empowered user - necessarily needs to understand the problem first and then implement a solution. As Einstein said, “95% of the time is spent understanding the problem; the little rest is on finding the solution”; the essential complexity is the critical part that programming must address first.

For human agents, programming is a logical task, not a language-based one. The brain is not just parsing text when interpreting a program. Rather, it is actively simulating the program’s behavior, following its logical sense and interpreting the effects. It’s quite a different cognitive process than following a text or a flow of thoughts. This may be why a fully committed programmer may exhibit contrasting language attitudes - as vague stereotypes circulating around nerds may confirm.

Evidence of the logical nature of programming can be seen in the tools usually offered by programming systems. Debuggers, code highlighting, code formatting, and other visualization tools are aimed at not exceeding the memory span and to focus user’s attention where it really matters. Those tools are designed for lowering the cognitive load of the mental module charged with the programming task.

What about AI agents? Is their path toward code generation similar to human programming?

Large language/reasoning models (LLRMs) show excellent results in many different tasks, demonstrating a general intelligence, but with uneven performance. To put it simply, LLRMs are essentially next-token predictors, trained to provide a probability distribution of what comes next in a given sequence of symbols. I’m afraid, it is evident that the answer to the previous question is negative. LLRMs are not even close, in abstract terms, to the way humans program. This is not, per se, an insurmountable problem; it just helps to understand why some problems may occur when using those agents to generate code. Recurrent complaints with LLRMs are summarized as follows:

1. Struggles with mutable states and side effects. Imperative programming languages are more prone to induce errors in AI agents due to the increasing burden of tracking the state of a program while it operates.
2. Shallow code understanding. LLRMs focus on the syntax and semantics of parts of code (like variable names) and less on the overall logic of the program. For example, obfuscated code causes AI agents to fail even on simple problems.
3. Problems with multi-step reasoning. This is a generalization of the first point.
4. Generates complicated code. LLRMs tend to produce convoluted code, increasing rather than lowering complexity. I guess this problem is due to the already high accidental complexity of the ecosystem, forcing automatons to fill in the boilerplate with even more glue code.
5. Fails to meet requirements. Generators do not implement the set of prescriptions in their entirety and refuse to implement them despite repeated requests.
6. Despite different prompts, it generates the same code.
7. Despite the same prompt, it generates different code.

All-in-one Development Environment

Some difficulties are remarkably similar to those encountered by human developers. Points (1), (3), and (4) are related to the inherited ecosystem and its idiosyncrasies. When it comes to the irrelevant and harmful composition of poorly designed substrates, the burden of keeping them aligned rises exponentially. Misaligned substrates are the source of impedance mismatch that necessitate further sub-problem elaboration specific to the amount of glue code involved. This affects not just humans but automatons as well.

Impedance mismatch: it refers to the difficulties when different system with incompatible data models need to interface with each other. For example, the object-oriented data model in Java clashes with relational model and with the interface (SQL) used in databases.

Unified Mode of Programming

To mitigate the problem, the goal is to build a platform using a minimum number of technologies, glued together in a programming system that allows the definition of a multitude of paradigms by adopting only a single way of expression: a core language sufficient to formalize data, computation, and external service access. This universal language should coherently allow many problem representations, minimizing the accidental complexity that derives from using non-unified integrated development systems.

I began this post by stressing the importance of adaptability for software success, and I can’t end it without mentioning an aspect of the low-code programming paradigm. In particular, if it is important to lower the cognitive barriers for expanding the pool of agents who can be actively involved in changing the tools they are using or creating, then those languages should be elastic enough to allow new versions of themselves specific to a domain (domain-specific language or DSL) while keeping unnecessary code away. A common feature of successful low-code languages is the capability to be changed by agents solely by using the language itself. This is the principle of self-sustainability.

Self-sustainability refers to the extent to which a system’s behavior can be changed from within itself, without having to step outside to a lower implementation level. A programming system that embraces self-sustainability allows its inner workings to be accessible from the user level, usually through macros, which are snippets of code that generate other code.

There is no conclusion to this post, this is just the part of the journey towards understanding what programming is and how to match humans and AI in a common adaptive ecosystem. Giancarlo Frison

• Basman, A., Tchernavskij, P., Bates, S., & Beaudouin-Lafon, M. (2018, April). An anatomy of interaction: co-occurrences and entanglements. In Programming’18 Companion - Conference Companion of the 2nd International Conference on Art, Science, and Engineering of Programming, Nice, France (pp. 188-196). ACM. https://doi.org/10.1145/3191697.3214328
• Clark, C., & Basman, A. (2017). Tracing a Paradigm for Externalization: Avatars and the GPII Nexus. In Companion to the First International Conference on the Art, Science and Engineering of Programming (Programming ’17) (Article 31, 5 pages). ACM. https://doi.org/10.1145/3079368.3079410
• Jakubovic, J., Edwards, J., & Petricek, T. (2023). Technical Dimensions of Programming Systems. The Art, Science, and Engineering of Programming, 7(3), 13. https://doi.org/10.22152/programming-journal.org/2023/7/13
• Petricek, T. (2022, April 28). No-code, no thought? Substrates for simple programming for all.

Hybrid AI brings together two very different approaches: symbolic AI, which works like traditional programming with rules and logic, and connectionist AI, which relies on neural networks, where large language models (LLMs) being the most advanced example. In this dissertation, I review some literature that tries to combine these methods to leverage their strengths for generating programs. I focus on the most interesting and useful papers, setting aside those that were less clear or relevant. While this overview may simplify some aspects or overlook certain details, my aim is to clarify the key ideas and highlight what has been achieved so far in making these two approaches work together.

Program synthesis (PS) is the automatic process of generating programs that accomplish specified objectives. A program consists of a set of instructions written in a formal language a symbolic engine can interpret and execute. Because of that, programs must be exactly right to achieve the intended outcome otherwise any deviation may lead to incorrect results or even the impossibility to run the program. The central challenge is the transformation of input - which often looks very different to the final program - into working code. The input may take different forms, as pointed below.

You can also Download the PDF version of this dissertation

Input as formal specification

The requirements might be written in a formal language, like test cases in the same language of the generated code or a even more abstract logical form. This is typical of the SyGus ¹ challenge, where specification are runnable code snippets.

An example of PS specification states the constraints of the new function max2, its signature and the tests for correctness:

;; The background theory is linear integer arithmetic
(set-logic LIA)
;; Name and signature of the function to be synthesized
(synth-fun max2 ((x Int) (y Int)) Int    
    ;; Declare the non-terminals that would be used in the grammar
    ((I Int) (B Bool))
    ;; Define the grammar for allowed implementations of max2
    ((I Int (x y 0 1
             (+ I I) (- I I)
             (ite B I I)))
     (B Bool ((and B B) (or B B) (not B)
              (= I I) (<= I I) (>= I I))))
)
(declare-var x Int)
(declare-var y Int)
;; Define the semantic constraints on the function
(constraint (>= (max2 x y) x))
(constraint (>= (max2 x y) y))
(constraint (or (= x (max2 x y)) (= y (max2 x y))))
(check-synth)

Programming by example

Rather than providing a strict set of requirements the program should adhere to, the generator is tuned on few pairs of input$\rightarrow$output as training samples and then the learned pattern is applied to a unpaired input.

One key detail:  the output in these examples isn’t the actual code itself, it’s rather the result of running the target program, and PS succeeds if the generated program produces the same output of the expected one. This type is named programming by example (PBE) and it is the approach adopted for example by ARC-AGI, an initiative for testing human-like intelligence in software agents².

Input in plain English

These inputs may take the form of high-level specifications written in natural language, such a brief description of what the program should do. An example is the ConCode³ dataset, where pairs of description and code allow the training of PS’s generators from English statements. Using natural language for defining programs has been indeed one of the hardest type of input to work with. The problem is that human language is ambiguous and fuzzy, while code needs to be exact, almost mathematical in its structure, as clearly stated by Dijkstra:

using natural language to specify a program is too imprecise for programs of any complexity⁴

The lack of rich and complex language models has been a barrier for this type of synthesis, but with LLMs it is possible to capture nuances that were previously out of reach.

Domain specific language

The generated program might be encoded in some general-purpose language that is designed - as the name suggests - to cover a very wide range of tasks. However, PS is usually tailored for narrowed domains where the language expressiveness is not the biggest advantage. This why a domain specific language (DSL) is usually more appropriate.

A DSL brings some other advantages to the process. First of all, a DSL reduces the range of possible solutions a program generator has to search through. DSLs are restricted in the scope and purposes and consequently the search space of possible programs is smaller. In favor of DSL there is also another point: DSLs tend to be more readable for humans. They do not prescribe unnecessary boilerplate code. Rather they keep only the meaningful parts that actually affect what the program will do.

Symbolic vs Statistical methods

Before going deep down to the methods used for PS, I would distinguish three approaches for PS: fully symbolic, fully neural, and hybrid (symbolic + statistical).

Fully symbolic

The exclusive-symbolic systems relies only procedural algorithms for searching solutions that do not comprise any statistical method. Basically, they do not take into account any learned pattern for speed-up the synthesis. Those methods usually fetch the entire search space and when the problem can’t be narrowed in a small number of possibilities, the task might turns out to be very slow or even unfeasible.

Fully neural

On the other end, statistical methods usually count on generalizations offered by neural networks, and thanks to their universal function approximation they tentatively map the input to the final program. Those methods have some drawbacks related to the massive amount of data they requires for obtaining convincing results⁵.

By their intrinsic nature, purely statistical methods lack of precision that usually is demanded for PS. This is a major issue since the output must run on a formal interpreter. To compensate, post-hoc filtering on multiple program generation is necessary. It consists on generating multiple candidate programs and filter them afterward. Usually it degrades performance with consequent defeat of real-time systems.

Hybrid: combine the best

An attentive observer may notice that the two paradigms above actually complement each other: while neural networks excel at approximations and fuzzy selections, symbolic methods perform best when nesting and composing primitive operations. So why not combine them by exploiting their strengths and mitigating their weaknesses? Hybrid methods attempt to do just that, but since they are fundamentally different, making them work together is not an easy task.

The integration of the two different ways is usually called neuro-symbolic AI (NeSy) and the boundary where one system ends and the other begins varies significantly on the method adopted by the authors. Generally, an appropriate metaphor for NeSy could be the one that emphasize the duality reason/intuition or the more glamoured dichotomy of Thinking Fast and Slow⁶. It basically describes how well the human (slow) reasoning integrates with the (fast) intuitive mental module. Similarly, in PS the symbiosis occurs for facilitating the search of the target program: the symbolic clockwork scans the possibilities while the neural module suggests the intuitions to search more efficiently.

Enumerative algorithms

What mainly distinguish symbolic from statistical methods is the presence of enumerative algorithms that represent one of the workhorse of the entire program synthesis. Each time the PS generates a candidate program, it is going to be validated in order to ensure correctness with the DSL syntax and the specifications⁷.

If the verification fails and the candidate does not satisfy the specifications, the system has just found a counterexample – an input on which the program produces an incorrect output that could be added in the history of failed trials for improving search on the next iterations. This is basically the process named counter-example guided inductive synthesis (CEGIS) and it is applied not only on PBE but also on specifications settings.

As previously mentioned, the fast and intuitive module is deployed to guide the search of candidates, but learning to search works best when it exploits existing search algorithms already proven useful for
the problem domain of interest - for example, AlphaGo exploits Monte Carlo Tree Search, DeepCoder uses also Satisfiability Modulo Theories (SMT) solvers. Which kind of searches are usually employed? A distinction should be remarked among different approaches I’ve seen in the surveyed methods.

Top-down enumeration

The top-down paradigm shows how an high-ranking problem could be decomposed in smaller parts that are nested or connected together. In programming, top-down algorithms are the deep first search, it’s companion breath first search or the more generic dynamic programming. Those algorithms are recursive since they unfold into branches that have a similar structure, but they suffer of a big problem.

Many derived branches ultimately prove irrelevant to the optimal solution. If we can infer which sub-problems are worth of scrutiny, we can significantly reduce computation time. This is why we need more sophisticated way than the ones mentioned above. The A* search combines backtracking search with heuristics that helps to find the right path⁸. Where do heuristics come from? Of course, from a probabilistic learned model. This is the intuition at work! With top-down enumeration, the model could be called just before the search and just once, since its output includes the elements to drive the search without further inquiries.

Because search branches operate independently - a property called Markovian - top-down methods work well for supervised learning. Consider a target output: a program represented as an abstract syntax tree (AST) is essentially a hierarchy of operations that unfold from top to bottom. If we can encode the AST in a neural network, it should somehow reflect inevitably a tree structure we can use to discriminate relevant branches to examine.

Bottom-up enumeration

In contrast, bottom-up searches do not offer the same Markovian characteristic. Bottom-up search builds programs from smaller components, and the correctness or usefulness of a program’s chunk might only become apparent when it’s combined with other segments later in the process.

What might be more appealing in bottom-up settings is that the process of building programs is closely related to the intuition that a human programmer has when he writes small functions first and then combine them to get the desired solution. This differs from the top-down approach, where you start with the big picture and break it down. Instead, bottom-up focuses on solving smaller, manageable problems first and then assembling them into more complex solutions⁹.

Because the search starts at the bottom of the AST tree and moves its way up, every sub-program it generates is already executable. At any stage of the search, any sub-problem has always a concrete value attached to it, which helps to assess how well it combines with other code segments¹⁰.

So, how does the model contribute to the search? Let’s talk about some probability-based methods that help those enumerations.

Probabilistic guided search

Most of practitioners agree that heuristics are useful for finding the right solution, and these shortcuts are based also on observed patterns. In particular, it is evident that not all programs are equally likely - some patterns appear more often than others. There definitely is a bias in the way programs are written. For instance, it is self-evident not a single program use both filter(> x 0) and filter(<= x 0) - that would be simply absurd.

Another suggestion for inferring which operations may be involved comes from an analysis of the training data labels (on PBE settings). Is there some kind of alignment on the output’s elements, for example a sorting? If so, then most likely there is a sort command in the target program¹¹.

Probabilistic grammar tree

Earlier, I exposed some reasons for using DSLs as target programs. One advantage is that DSLs come with a well-defined context-free grammar¹² (CFG) which is a formal way to define how languages are structured. A CFG is defined by a set of:

• terminal symbols - those atomic elements that can’t be derived by other elements.
• variables - to be computed by the symbolic engine and rule of productions (non-terminal elements).
• rules of production - the core of the CFG because it is where the generation can unfold the expressiveness of the language and cover myriads of use cases.

The rules determine the language’s expressiveness, allowing it to generate countless use cases. Think of it when forming sentences in English: not every word sequence is valid, and a CFG strictly defines which combinations are allowed. This rigidity contrasts with neural models, which learn from examples and work non-deterministically rather than following hard rules.

Probabilistic context-free grammar (pCFG) is an extension of CFG where each rule is associated with a probability that reflects the chance of choosing that particular rule when expanding the program in a sequences of terminals. Those probabilities (or weights) are statistically derived from a series of programs linked to a specific task¹³.

Attribute vector

Another way to learn heuristics from training data is the approach used in DeepCoder¹⁴ where the system uses a vector to represents the probability of programming features present in the target program. The encoded features depends of course on the DSL adopted for the task. For giving some examples, it might be present features such as:

• unary math operations - like abs, sqrt.
• binary arithmetic - like +, -.
• sequence helper methods - like head, tail, take, drop.
• set functionalities - like intersection, union, diff.
• filters - like >, <=, ==.
• tailored compositions of functions - like mergesort.

The (probabilistic) attribute vector is then used to elicit discriminatory decisions when the enumerator searches for viable solutions.

Argument selector

The two type of statistical models above fit greatly with top-down search since they capture the overall structure of the target program. In contrast, systems such as CrossBeam¹⁵ take a bottom-up approach. Instead of reasoning from the top level, the model decides how to combine previously explored operations to build new program components.

As we have seen, not only the programs are kept in history but also their execution result. In this way, in the following iterations those program/value pairs are then used to generate more complex candidates, until the desired solution is found. In this way, promising combinations are prioritized rather then exhaustively enumerate all possibilities.

Branch selector

In the neural guided deductive search (NGDS)¹⁶ the model is intended to rank potential extensions of a partial program. It takes the input-output examples and the current candidate program, then assigns weights to different branches the search algorithm could explore.

At beginning, the candidate program is of course empty, and from there starts the divide and conquer top-down approach by building the solution step-by-step from the root of the AST. When the enumerator reaches a branch, the model suggests the most promising path to follow - guiding the search toward a valid solution.

Auxiliary construction

What if the neural model doesn’t just help to speed-up the search by giving proper heuristics, but rather it generates new supporting information, helping the symbolic engine to discover a better path to the correct program? This is what has been proposed in Alpha Geometry¹⁷ with the purpose of creating proofs to verify geometrical theories¹⁸.

LLMs in the loop

Unlike specialized statistical models, LLMs can bring broader cognitive abilities to PS. Though they were designed to generate the next word in a sequence, they have shown surprising reasoning capabilities, the ones that can be valuable on generating code.

Whether LLMs genuinely reason or they just emulates it by retrieving training data is still controversial¹⁹. But for practical purposes, I’m more interested on observing what they can actually do, and integrating them into PS poses new challenges - as any new emerging technology.

Lack of consistency

A persistent challenge is the lack of consistency and generalization in reasoning behavior. The inconsistency arises when LLMs provide different answers to semantically equivalent input. For instance, simply altering the prompt’s tone - or even adding threats to the request - can significantly steer the LLM’s output. This suggests that LLMs probably don’t have a stable, logical framework guiding their answers.

Crafting effective prompts remains an art (maybe yet one of the fewer exclusive to humans?) and this unpredictability adds another layer of uncertainty to PS. Indeed, this kind of problem is well known to practitioners, and when a problem is familiar, there are usually ways to circumvent it, as those surveyed here.

When LLMs generate programs - and usually they return many wrong solutions - the correct solution is most often in the proximity of the wrong ones, and that, by searching in the neighborhood of the invalid proposals, we may be able to guide the search to find a solution faster by compiling a pCFG as described in a previous section.

Context is the king

LLMs being based on the transformer architecture possess long-term memory that enables to track distant elements in the request. This capability joint with vast ground knowledge that extends the mere narrow scope of the PS’s request, is indeed helpful on determining proper variations on the generated program.

As memory capabilities are present in more simpler neural architecture, in LLMs they surely reach undisputed highs. This aspect affects also the latent-space that governs the network capabilities. LLM’s embeddings incorporate much more information than in older neural architectures. RAG (retrieval augmented generation) is a method to encode vectors for enabling similarity search on very complex structures. Though similarity is indeed used for fuzzy searches, its importance on PS remains marginal due to the more tight requirements for generating programs. Program structures are closer to graph representations that hardly can be generated only by similarity means.

The leverage of high contextualization is well exploited in top-down search methods where an hierarchical view of the final goal gives an important help on splitting high-level problems into tiny ones. On the other hand, this reliance on context can become a drawback when using those powerful models for bottom-up searches. Starting from the big picture and trying to work backward, from basic functions to complex solutions, is much harder.

Hybrid methods

Alpha Geometry

As just previously mentioned, the intuitive hint comes from a neural model based on the transformer architecture. It is initially pre-trained with a large amount of synthetic-generated cases of pure geometrical deductions to create a grounding latent space for geometrical knowledge. Then the model is fine-tuned on a generating auxiliary statements to be added to the initial geometrical proposition²⁰.

DeepCoder

Or how to learn to write programs with a NeSy approach. It is based on PBE and it employs a neural network (no LLM) to encode an attribute vector that list the features the final program should include. The search will start by including the most promising features and it will test-and-prune inapplicable programs till it finds a solution that satisfy the examples provided²¹.

iLLM-synth

This method involves prompting the LLM to provide helper functions or syntactic suggestions based on the partially constructed program and any counterexamples encountered (CEGIS) and the provided problem specification (SyGus). The LLM’s feedback can even augment the grammar and update rule weights dynamically as the search progresses. A pCFG is created from LLM’s suggestions and the Symbolic searcher apply an A* algorithm based on the heuristics calculated in the pCFG. The search stops when a solution is found or a max cost is reached. As cost it is intended as a max depth in the search tree is reached, or the time budget is spent or when the diminishing return of searching does not worth further explorations²².

Neural guided deductive search

It is a PBE oriented method where a DSL provides the vocabulary of the operations that apply in different parts of the input. The method works by filtering the most likely useful operations (via statistical learning) and then chaining them together recursively, following a functional programming style²³.

Take for example the following problem: “Firstname Lastname” $\rightarrow$ “FL”:

• The model’s inference breaks it down into smaller problems by splitting them with the firstWord and lastWord functions.
• Those two branches are then computed separately by recursively solving “Firstname” and “Lastname”.
• The model finds the operating sequentially firstChar to the previous operations might lead to interesting results, that the model again will attempt to combine them together.

If this sounds over-simplified, well, it is. Real-world cases involve more complexity, but this gives the basic idea²⁴.

Crossbeam

Differently from previous approaches, the enumerative search follow the bottom-up prescriptions. Starting from the raw input, it applies most likely operations and their result will be used in the following iterations. The search is guided by a neural model trained on program examples. While in NGDS the model gives indications on the AST tree, in Crossbeam only the operations’ output is considered for ranking nested operations. The model learns how to combine evaluated sub-programs in a bottom-up manner²⁵.

HySynth

Differently from other methods, LLMs are prompted first to guess what will be the program for a PBE problem. Candidate programs are used to compile a pCFG which will then guide the following bottom-up search.

The search method falls under dynamic programming, meaning it builds the program from its smaller parts step by step, assigning a computational cost at each new chunk of code. This cost reflects the resources needed to execute that part, as well as the search process itself and it is used for filtering out unsuitable candidates. The DP-based search stores intermediate results to avoid redundant calculations, improving efficiency by skipping explored sub-programs²⁶.

Conclusion

In this brief overview I explore how different methods for PS can be combined in flexible way. Various approaches have been tested to find simple and effective ways to generate programs. Pure statistical methods, especially when they are based on LLMs, might provide simple methods that do not requires combinations of techniques but rather they rely on the universality of their broad context they can leverage for producing code.

Usually, fully neural approaches are effective when problems are simple or LLMs can compensate the lack of explicit knowledge in the input, or they can simply exploit massive amount of publicly available programming code. When none of the above conditions applies, LLMs are beneficial when paired with enumerative processes by guiding the search towards more plausible candidates. The intersection between symbolic and neural systems lies where probabilities may have a role on compiling heuristics. In particular, the flat semantic of a DSL can be more valuable for search when enriched with pCFG. author: Giancarlo Frison

For more logic based argumentation, here a deep dive into the world of defesabile logic.

Adopting neural processes in argumentative tasks is not a fresh idea. Usually there have been initiatives for argumentation mining - extraction of latent parts (claim, warrant, baking, etc.) from raw text - a kind of process similar to named entity recognition for example. Another task where deep learning have found applicability was argument generation which is reversal in the respect of mining. The generation phase transform a given argument’s structure into coherent and fluent paraphrases.

LLMs are quite effective in those tasks without particularly laborious setup. I’m referring to the capability of solving those tasks with minimal (few-shot learning) or even without (zero-shot learning) any sample on how to accomplish the required work.

Though LLMs have outstandingly raised the bar of machine intelligence - whatever meaning could be pointed to the terminology - they are affected by the collateral effects of their own designing.

The fundamental LLMs’ feature is about predicting what comes next the last token (word or any symbol), the so-called auto-regressive property of generative models that consists of recursively iterate over what has been already generated. This enable the generation of fluent text and the answering to an incredible variety of questions. The next-token prediction lays on the scale a minimalist process (thanks to the transformer architecture) to an enormous amount of data. But auto-regression leads also to hallucinations - generated chunks not aligned with the reality.

Those fictitious answers are a collateral effect of the LLM design. Basically, a LLM always hallucinates¹. Sometimes the generated output is sound and valid, sometimes it is just a well-assembled bunch of fantasies. One way to mitigate generation of statements not adherent to our world is the adoption of the chain of thought (CoT), a technique that is also adopted for argumentation, as we see next.

argumentation step by step

CoT is a technique used to encourage an LLM generating higher quality answers. As previously mentioned, an LLM generates a series of words (tokens) according to what submitted (or self-produced) and the corpus on which the model has been trained. If the request implies a concatenation of reasoning steps not explicitly expressed, the generative part will jump to the conclusions with the obvious risk to fail on sequencing a correct chain of reasoning, causing diversions and utterly wrong answers.

This approach is remarkably similar to what we do every time we mentally assess a complex argument with several check-points. This problem is mitigated by instructing on explicitly tracking the logical path from premises to conclusion. The goal of CoT is to divide a problem in small chunks and facilitate a reasoning in smaller steps, where the effort to connect premises and conclusions is much lower than the original (bigger) argument.

multiagent automated debate

The CoT is even more effective when it is augmented not by simply instructing the LLM to take care of the intermediate steps, but by arguing on each intermediate conclusion with a multiagent debate². What could be the opponent of an LLM agent? Of course, another LLM agent. This approach is inspired by the concept of “The Society of Mind”³ and the core idea is to have multiple LLMs independently generating arguments. The indipendent agents will critique and refine their responses based on the arguments and reasoning presented by the others, for converging eventually on a single, more accurate conclusion.

Peformance gap in multi-debating agents (Du, Yilun, et at. 2023)

The multiagent debate is mainly focused on unfolding a particular reasoning path while the focus of the task is not on searching alternative ways of thinking of assessing conclusions. Which is the point of the next paragraph.

self-consistency and contrastive argumentation

Consistency is a crucial aspect in argumentation and it is scrupulously controlled in logic frameworks like the ones previously depicted. The probabilistic nature of LLMs and their intrinsic hallucinating mechanisms may often clash with this important value, and without specific mechanisms to promote self-consistency, LLMs may produce conflicting outputs.

A variation of CoT that aims to boost (according to the authors) the performance of LLMs is a strategy that leverages the idea of generating multiple reasoning paths⁴ and then a self-consistency check could identify contradictory claims, discarding them and selecting only conflict-free candidates, or it will attempt to solve them by inspecting additional steps.

Self-consistency may be reinforced by a technique that help the LLM to discriminate good reasoning path from the bad ones. The contrastive argumentation works by providing sample of claims with good and bad conclusions with related explanations such as: if all dogs are mammals and all mammals have fur, does it follows that all dogs have fur? i.e.: $?has(dog,fur)$
\(\begin{eqnarray} has(x,fur) &\Leftarrow& is(x,mammal)\\ is(x,mammal) &\leftarrow& is(x,dog) \end{eqnarray}\)

• Correct Answer Explanation: Yes, since all dogs are mammals and all mammals have fur, it logically follows that all dogs must have fur.
• Incorrect Answer Explanation: No, some dogs might be hairless. (The incorrect explanation ignores the stated premises and introduces an exception not supported by the given information).

automated rhetoric

So far, we have seen the building blocks of the logic behind argumentation, which is addressed by the initiatives in automated argumentation as discussed in previous sections. The advent of LLMs offers new opportunities on addressing communicative aspects such as rhetoric, and enhance argumentation within human language.

Rhetoric is the study of effective speaking and writing for what it relates to persuasion. It requires understanding the distinction between the content and the form of communication. Rhetoricians emphasize their interdependence, highlighting the connection between language and meaning, argument and ornament, thought and its expression. This means linguistic forms are fundamental, not just to persuasion, but to thought itself.

content: topics of invention

The topics of invention (TI) are categories of thought that can be used to generate arguments. They are like lenses through which you can view the claim, the baking and even the evidence. They are used in combination to create a strong and persuasive argument. For example, you might start by defining the problem, then explore its causes and effects. You could then propose a solution, compare and contrast it to other solutions, make analogies with other arguments and cite expert testimony to support your claims. Finally, you could address potential objections and refute them using counterarguments⁵.

• Cause and Effect: This involves exploring the causes of a problem or the effects of a proposed solution.
• Comparison and Contrast: This involves comparing and contrasting different ideas, arguments, or proposals.
• Definition: This involves defining key terms and concepts in the argument.
• Testimony: This involves citing the opinions of experts or authorities to support your argument.
• Analogy: This involves drawing comparisons between different things to illustrate a point.
• Generalization: This involves making broad statements based on specific examples or evidence.
• Division and Partition: This involves breaking down a complex issue into smaller, more manageable parts.
• Concession and Refutation: This involves acknowledging the opposing side’s arguments and then refuting them.

form: figures of speech

It is subset of rhetorical devices that specifically use language in creative ways to convey meaning and evoke emotions. Rhetorical devices are techniques used to enhance communication and persuasion, and figures of speech are specific tools within this broader category.

• Simile: A comparison using “like” or “as.” Example: “Her eyes are like stars.”
• Metaphor: A direct comparison without using “like” or “as.” Example: “He is a lion in battle.”
• Personification: Giving human qualities to non-human things. Example: “The wind whispered through the trees.”
• Hyperbole: Exaggeration for emphasis. Example: “I’m so hungry, I could eat a horse.”
• Understatement: Deliberately saying less than what is meant. Example: “It’s a bit chilly outside” (when it’s freezing).
• Oxymoron: Combining contradictory terms. Example: “Bittersweet,” “jumbo shrimp”
• Pun: A play on words, often humorous. Example: “I’m reading a book about anti-gravity. It’s impossible to put down.”

how much are LLMs persuasive?

As one could expect from a training with a big slice of the entire human-written corpus, LLM can adopt the most effective and persuasive rhetoric strategies. Have ever opened a random prompt for a practical task? Usually they begin with “You are a X and you are demanded to…“ where X could be a professional figure or any kind of specifically behavioral-driven agent that the LLM should impersonate, in order to run the task with a very specific taste. Not only the adaptability to different prompts, but also the control over persuasive strategies is totally mastered. For example, LLMs push forward on moral values - especially on negative moral bias. This could be due maybe to the fact that we as humans are more sensitive to negative outcomes rather the positive ones, and LLM may emphasize more on bad effects of a particular action or policy, appealing to the audience’s desire to avoid causing harm.

LLM seems to use more complex grammatical and lexical sentences, probably because human readers may interpret the need for greater cognitive effort as an indicator of the argument’s significance or substance, despite the common notion that that simpler, easier-to-process arguments are inherently more persuasive⁶.

Conclusion

This is a really brief introduction on how argumentation is an interdisciplinary subject that is tackled from many different angles and perspectives. It triggers interests of a very diverse set of domains such as philosophy, logic, symbolic and neural processing, but also cognitive sciences and social matters. Each field of research move forward its own contribution that cover some aspects of the vast panorama that argumentation involves. Though, it is important to summarize them and synthesize some converging points where different fields can contribute to each other. For example the integration of logic soundness and validity into rhetorical devices - for enabling a more precise dialectic process - is an open issue that roots into symbolic-neural dualism that might look very promising from the point of view of automated argumentation, but that it is still far from very effective applicability.

In a famous old television series, a patient enters a clinic for having an argument but the specialist clear the demand by simply denying whatever patient says. For that, the patient complains that a mere denial of a proposition is not an argument. “An argument - my fellow doctor - is a connected series of propositions that are intended to support or refute some other kind of proposition!”. The client shouts.
Which is not so different from:

“Argumentation is a verbal, social, and rational activity aimed at convincing a reasonable critic of the acceptability of a standpoint by putting forward a constellation of propositions justifying or refuting the proposition expressed in the standpoint¹.”

Argumentation is inherently linguistic - in spoken or in written form - and it is a social activity as an interaction within two or more participants with different opinions. Without contested idea, argumentation would not take place. Having an “argument” implies responding to one’s another claim for supporting, attacking or defending positions. Argumentation is mainly a rational activity as an exchange of reasonable arguments, though rhetoric tactics can significantly influence the persuasive power of an arguer.
The nuanced spectrum of rhetoric can hardly be caught by reasoning and logical processes, but they could be resorted by other means. I’m referring in particular to large language models that they can tailor an argument for a particular audience.

Want to read more about it? Take a look into LLMs rhetorical skills

The latter point is crucial for depicting argumentation. Argumentation investigates the communicative aspects of reasoning and can be described as an interplay between logic and rhetoric. While rhetoric is a communication exercise to make an argument more or less appealing to the recipient, logic is a philosophical seek that concern that soundness of a proposition rather than championing persuasion as primary goal.

The first introduction to a systematic theory of argumentation comes from Dung and his abstract argumentation framework², where arguments are nodes in graph. An argumentation system is defined as $S=\langle A,R\rangle$ where $A$ is the argument and $R$ is the attack relation with other arguments. In the example below, $c$ is defeated both by $d$ and $b$. On the other hand, $b$ is defeated by $a$ which neutralizes the attack of $b$ towards $c$. According to the framework, an argument is undefeated if it has no attacks towards it, or if attackers are already all defeated. As the most casual reader might notice, the computational load required to check whether a statement is defeated or not depends on the size of graph, but “Hey man, this is why the computers stand for!”. It’s then beneficial to use automated means to reach a consensus.

structured argumentation

In abstract argumentation every argument is regarded as atomic. Dung’s framework does not dissect arguments into their constituent parts, and that could be seen as an overly simplification. This is why we can turn into structured argumentation. In this way, the different parts that compose an argument are declared explicitly.

An argument is then said to be structured in the sense that normally the premises and claim of the argument are made explicit, and the relationship between the premises and claim is formally defined³.

What are then the constituent parts of an an argument? Toulmin⁴ has traced a list that could be grossly summarized with:

• Claim. It is an explicit performative at the core of the argument. It’s the conclusion that needs to be proved (claimed) and stated to other participants.
• Data. These are the evidences available that trigger a validation or a refusal of a claim based on some warrants.
• Reason. It could be an explanation with the intent to tell why a claim is true. It could be a justification with the purpose to make a conclusion believable.
• Warrant. It is an implicit assumption that a reason is justifiably true. In the argument “doing regularly sport helps to loose weight” implies that loosing weight (of course, for those overweight) is a good thing. It is the final consequence that a supporting reason or evidence make the claim truly believable.
• Backing. There is plenty of scientific information that confirm low-weighted people can benefit of better health condition much longer than heavy-weight ones.
• Qualifier. Help to bound the claim statement in a scope on which it’s validity can be more successfully validated. Such as “most of the people will benefit”, does not say all in order to not be easily confuted by statistical outcomes.

The approach adopt by Toulmin depart a bit from the usual mathematical form of logic. According to him, the geometrical approach to argumentation is flawed to accommodate the complexity of real-world cases - especially those involving empirical evidence - and fails to capture the context-dependencies of an argument. The list we see above reflects more a jurisprudential⁴ rather than methematical approach. Real-world argumentation resemble legal cases more closely than math proofs. Most of arguments do not follow scrupulously the analytical syllogism “Socrates is human, all humans are mortal, therefore Socrates is mortal”. Claims are rather supported by evidence, the warrant’s backing, and the possibility of rebuttals.

explanations

Explanations aim to transfer the understanding of how the warrant status of a particular argument was obtained from a given argumentation framework. Walton⁵ has studied arguments and explanations from a philosophical point of view. He considers that the purpose of an argument is to get the hearer to come to accept something that is doubtful or unsettled, whereas the purpose of an explanation is to get the hearer to understand something that he already accepts as a fact.

Explaining consists in exposing something in such a way that it is understandable for the receiver of the explanation – so that he/she improves his/her knowledge about the object of the explanation – and satisfactory in that it meets the receiver’s expectations⁶.

practical applications

Argumentation is an art. As an art, argument has techniques and general principles, therefore it is a learned craft. Although there are suggested guidelines and argumentative tools, there is no science of argument. So why automatize it?
As we will see shortly, certain mechanics mostly related to logical reasoning could be intuitively automatized for handling complex issues with multiple perspectives and contrasting viewpoints. I’m referring to applications in the realm of legal systems, take for example how to check the consistency of the immense plethora of laws in a state legal system; or how to automatize intelligent agents to project provisioning orders in a supply chain management.

Courious on how sound arguments might help on explaining given recommendations? Here an implementation where products are suggested based on consumer preferences and grounded knowledge (knowledge graphs) to explain why a suggestion is better than another.

These are some example on how argumentation is permeating through not only our daily life but also the way we conduct business.

Elements of logic

To begin our endeavour in the intricacies of argumentation, we should deep into the mechanics and well-definable aspects of argumentation. We then start with logic reasoning. In here, I arrange the commonalities and peculiarities of 3 approaches to automatic argumentation: ABA⁷, ASPIC+⁸ and DeLP⁹.

negation

One of the most nuanced concept in logic programming is the negation. As we state in a casual conversation such as “the accused is not guilty” ($\neg guilty$) is the form of negation closer to the strict negation one might find in classical logic. The “classical” adjective stands for the kind of logic elaborated by ancient Greeks, with Aristotle as the greatest contributor. According to the excluded middle hypothesis, the accused is either guilty or he’s not. There are no shades of guilt - which are instead embraced by fuzzy logic - only black or white, if you let me indulge in the color metaphor.
There also different concepts related to negation, in particular adopted in practical logical programming languages, which is the negation as failure. According to that interpretation, the negation as failure is a broader concept the reasoning principle of assuming the negation of a statement is true if the system cannot prove the statement itself. What does it means concretely?
When in a supermarket I can’t find biscuits, I can assume that there is no biscuits in that supermarket. I may find them certainly elsewhere, but not there and not in that moment. The negation as failure consider the world-closed assumption: what can’t be found in the knowledge base (supermarket), simply doesn’t exist. This is particularly common in logic languages as DeLP, but might take slightly different names, such as default negation.

defeasible logic

Logic is the basic language of reasoning. Classical logic embrace the law of excluded middle, where what is already known to be true (or false) cannot change due to new incoming information (monotonicity). While those conditions simplify computation and the validation of logic statements, they are limiting the scope on which classical logical might apply:

logical argumentation has the ultimate goal to refuse or support a proposition, therefore the change of truth of any statement is an inherent and desirable property.

Classical logic can’t find a whole place in here. It must be supersede by an elastic framework that contemplate dynamic changes of truth. This where defeasible logic (DL) come to place. In classical style, once a statement is established, it is no longer possible to alter it without falling in conflicts and inconsistencies. Defeasible logic instead, by adhering to the non-monotonic paradigm, embrace beliefs change. Welcoming fresh information, DL may alter previously established conclusions. A way to put it into a practical example could be the statement: “the accused is not guilty if not proved otherwise”. By default, the accused is not guilty unless it is proved the contrary:
\(\begin{eqnarray}\neg guilty(x) &\leftarrow accused(x), not\; guilty(x) \end{eqnarray}\)

With that we make use of default negation which is itself non-monotonic - the $not$ fails when the predicate $guilty(x)$ is activate. Another approach implemented in ASPIC+ and DeLP is the introduction of defeasible rules, according to which a rule normally apply unless a more specific rule overrides the default one: \(\begin{eqnarray} \label{accused} \neg guilty(x) &\Leftarrow& accused(x)\\ \label{guilty} guilty(x) &\leftarrow& evidence(accused(x)) \end{eqnarray}\)The rule ($\ref{accused}$) affirms that by default the accused is not guilty, unless it is proved the contrary. The following strict rule ($\ref{guilty}$) expresses that if there are evidences against the accused, he is guilty - no matter what could be subsequently determined.

With defeasible rules, there is a distinct separation between the default behavior usually assumed, and the exceptions that might occur. Some examples:

• A bird usually flies ($\ref{defeasible}$), but a penguin does not ($\ref{strict}$).
• In ($\ref{dead}$) a combination of negation as failure and defeasible proposition translated as: “if we’re not sure someone is dead, we can assume he’s alive” - which is why death certificates exist.

The default rule has a specific notation ($\Leftarrow$) that distinguish its fallibility, while the other ($\leftarrow$) denotes a strict rule which has a higher ranking than the fallible one, and supersede all other lower ranking rules. Both rules can coexist in the same program without raising any conflict, but what happens when two or more conflicting defeasible rules apply? DeLP uses a dialectical process to decide which information prevails.

dialectical process

In the sample below, if both $mosquito$ and $dengue$ applies, What is expected the system to assert? \(\begin{eqnarray} \label{dengue} dangerous &\Leftarrow& mosquito, dengue\\ \label{mosquito} \neg dangerous &\Leftarrow& mosquito \end{eqnarray}\)If we apply the specificity tactic, the rule ($\ref{dengue}$) should have higher priority because it is more specific. i.e. it entails more literals than ($\ref{mosquito}$).
What if instead, the specificity tactic can’t be applied, because the rules’ specificity is the same?

Some notion of priority must be introduced in conflicting defeasible rules. It could be implemented by defining a series of inequalities representing the relative ranking order:\(\begin{eqnarray} r1&:&walk &\Leftarrow& sunny, free\_time\\ r2&:&\neg walk &\Leftarrow& sunny, allergy\_season\\ \label{pref} r1 &>& r2 \end{eqnarray}\)

In this example, if $sunny$, \(free\_time\) and \(allergy\_season\) apply, it’s definitely better to go for a $walk$ rather than stay home.

An interesting representational device - the presumption - is the defeasible rule with an empty antecedent. As the case of strict/defeasible rules, the presumption introduces fallibility even for facts. Strict facts can’t change, they are infallible like axioms in mathematics. Everything else, such as:
\(\begin{eqnarray}delivered(letter) \Leftarrow \{\}\end{eqnarray}\)
is fallible and can be overturned by a contrasting deduction or a fact \(\begin{eqnarray}\neg delivered(letter) \Leftarrow wrong\_address(letter)\end{eqnarray}\)

compelling arguments

A common definition of argument in the initiatives for automating it, is defined as:

an argument is the minimal set of defeasible rules that allow for the defeasible derivation of a conclusion, ensuring no contradictory literals are derived.

Taking it as a general line of principle of an argumentation system, what should we expect on querying $Q$ in a system? DeLP offers the most granular depiction of how an answer could be. For that, there are 4 possible answer to $Q$:

• $YES$. If $Q$ has no counterargument standing against it, or they are all ultimately defeated. $Q$ is warranted.
• $NO$. If $\neg Q$ is warranted. if the query is “Is the sky blue?” and there exists a warranted argument for “the sky is not blue”.
• $UNDECIDED$. if $Q$ nor $\neg Q$ are warranted. This is a stalemate in the dialectical process. It could arise if there are conflicting arguments of equal strength, leading to a situation where neither side prevails.
• $UNKNOWN$. if $Q$ is not in the knowledge base $K$. In here, the system lacks the necessary information to even begin constructing arguments for or against $Q$. This is the response it would get if $K$ contains information about birds and their flying abilities, and $Q$ is about the price of tea, as this information is simply out of scope.
  This nuanced approach to determining the result comes from the system’s handling of potentially incomplete and inconsistent information.

Though the logical argumentation initiatives acknowledge the value of structured and schematic segmentation of arguments, they don’t attempt to capture the Toulmin framework at that granularity level. The simpler structure of an argument is condensed as the tuple $\langle A,L \rangle$ where $A$ is the minimum set of rules (strict and defeasible) and $L$ is the claim. An argument is just a part of the whole system; in DeLP a defeasible logic program is described by $\langle \pi, \Delta \rangle$ where $\pi$ stands for strict rules and facts, on the other hand $\Delta$ indicates defeasible facts and rules. We can say $A$ is an argument for $L$ when:

1. Exists a defeasible derivation for $L$ from $\pi$.
2. No contradictory literals can be defeasibly derived from $\pi$.
3. If a rule in $A$ contains a negation ‘$not\; F$’, then $F$ can’t be in the defeasible derivation of $L$. This condition prevents circular reasoning and ensures that the argument is logically sound.
4. It is also promoted the principle of minimality, for which $A$ should only contain the essential set of literals for claiming $L$.

The second point could be explained in the following example. Consider a program $P$ formed by:
\(\pi = \left\{ \begin{array}{l} night \\ at\_market \\ \neg at\_home \leftarrow at\_market \end{array} \right\} \qquad \Delta = \{at\_home \Leftarrow night\}\)

The same literal \(at\_home\) is strictly negated but also reinstated in $\Delta$ as defeasible rule. This rises a contradiction which this is forbidden in DeLP. If we consider other approaches such as the argumentation systems like GK¹⁰ contradictions are removed automatically from $P$ - with an eye on computational costs by performing recursive checks with time limits. We can overcome this kind of contradictions by introducing preferences.

dialectic tree and preferences

The argumentation is the process of deciding the status of all arguments in a program. The verdict whether an argument is valid or not is declared by defeated ($D$) or undefeated ($U$) statuses. Since we tend to avoid binary discrimination, there are also here plenty of flavors of acceptance in arguments. In ASPIC+ are denoted 3 different types of attacks:

• Undercut: An argument undercuts another argument by questioning its premises or assumptions.
• Rebut: An argument rebuts another argument by directly contradicting its conclusion.
• Undermine: An argument undermines another argument by weakening its overall force or persuasiveness, without necessarily contradicting its conclusion directly.

An argument can be directly attacked to its core claim, or indirectly by attacking warrant’s backing. Indirect attack happens when an argument attack a sub-argument (which is a smaller argument, with its own premises, within a larger argument). As previously mentioned for defeasible logic, argumentation suffers from conditions of stalemate among arguments. The Nixon Diamond provides the proverbial example of blocking defeaters. Considering the following argument structure:\(\begin{aligned} \langle A_1, L_1\rangle &=& \langle pacifist(nixon) &\Leftarrow quacker(nixon), pacifist(nixon)\rangle \\ \langle A_2,L_2 \rangle &=& \langle \neg pacifist(nixon) &\Leftarrow republican(nixon), \neg pacifist(nixon)\rangle \end{aligned}\)

The two arguments defeat each other, therefore the answer to $pacifist(nixon)$ will be $UNDECIDED$. Whether an attack from $\langle A_1, L_1 \rangle$ to $\langle A_2, L_2 \rangle$ succeeds as a defeat, may depend on the relative strength of $A_1$ and $A_2$ , i.e. whether $A_2$ is strictly stronger than, or strictly preferred to $A_1$. Where do these preferences come from?
We have already previously ($\ref{pref}$) mentioned preferences in defeasible logic. All that an argumentation system wants is a binary ordering ($\le$) on the set of all arguments that can be constructed¹¹.
The reader may observe that the structure of attacks resembles a tree (or acyclic graph as circular attacks are prohibited). The tree is the dialectic tree where each argument’s element can be defeated by other arguments. A single path within the tree is termed an argumentation line. If a single argumentation line successfully rebuts an argument, the latter is considered defeated and removed.
Giancarlo Frison

1. unavailability of suppliers for:
   1. own internal reasons
   2. suppliers’ suppliers availability risk
   3. issues in the distribution
2. rising costs due to:
   1. distribution chain
   2. raw material
   3. lowering of competitive alternatives in the market
3. geo-political issues
   1. wars
   2. international sanctions
4. natural adversities

This short dissertation aims to analyze the architecture of a potential cognitive system designed to alleviate these supplier-related problems.

What is a cognitive system? Cognitive systems, in essence, are intelligent agents capable of learning, reasoning, adapting to new environments, and leveraging past experiences to continuously improve. The key advantage? They can deliver high-value functionalities with minimal human intervention.

What is a cognitive system architecture? In the realm of software, designing an architecture means to trace the invariant aspects and components that meet expected requirements and functionalities. Two well-established cognitive architectures, SOAR and ACT-R, have been under development for decades and will be the foundation for this exploration.

System goals

As it clearly appears, nearly all elements involved in the corporate’s values chain may be impacted by supply management, the procurement will influence all the other elements even top business priorities. This is why a system that fulfill the functionalities proposed here will be important for almost all companies and surely for manufacturing ones.

The goals of the system is to help analysts to mitigate risks related to the supply network and provide more suitable alternatives to lower those risks.

How would the system actually help?
The system is intended to provide probabilistic estimations on a variety of natural language queries. It will elaborate how much target forecasts diverges from current estimations according to real-time analysis. The purpose is to raise issues that may undermine the predicted costs for the business, and to raise awareness on auspicable chances to re-organize the supply chain.

Not limited to that, the system should provide estimations in the current state of affairs and also in hypothetical situations artificially setup by tweaking initial conditions.
If you are think of a simulation engine, this is exactly what I mean with that.
The tool should elaborate possible consequences and, in cascade (recursively), analyze further outcomes from them. Analysts may advance simulations on particular conditions, specifying by natural language propositions, some constraints such as:

What will be the impact on the deliveries for supplier $XYZ$ on a increase of +30% of the cargo shipping costs?

Types of simulation may involve complex aspect of related to change in demand from their customer side as in how the supply provisioning will be affect on % increase product X and % decrease of product Y?

Considering a particular simulation on changes in the manufacturing process, queries may be of interest for industrial managers where processes changes will drive to different procurement due to innovations and optimizations:

What are the costs estimations for our smartphones if we replace plastic cover with alloy masked covers?

Data & system heterogeneity

For performing the wished functionalities the system should elaborate a massive quantity of facts, analyze them, evaluate their impact on direct suppliers and their upstream suppliers. The generality of the term “analysis” hides numerous computational intensive tasks for capturing regularities in the data, so it would much easier to replicate past decisions once they have been learned¹. By impact it is intended of events that have direct effect on carrying goods around the world and their production, and also events that may be indicative of changes in mood regarding a particular technology or company or geopolitical area. Rumors usually are seriously taken into account by decision makers, but at the same time they might lead often to false positives as well.

Data awareness² is just one of the ingredients we need for this recipe. Business processes are not designed by tons of terabyte of data, rather they are intentionally crafted by experts and the decisions on setting up how things should be made up for their business are derived to general ways to conduct a specific process (therefore generalizable in templates) but they could also be unique for that business. We may generalize process modelling as:

• hierarchical. Because processes are generalizable into typical (following the prototype theory³) cases
• non-monotonic. Peculiarities are the norm, and every process may differ from the general one.
• compositional. Process models are associative and composable into macro-processes.

Data heterogeneity

Differently from expert input - used for defining how actual industrial and business processes work - all the other type of input is inherently unstructured and noisy. The evaluated architectures propose an approach that tackles the challenge of reconciling symbolic and connectionist representations for robust AI. This heterogeneity allows for diverse data formats and enables the integration of symbolic (cognitivist, based on symbolic logic) and emerging (connectionist, based on neural networks) approaches within a single system by leveraging existing architectures such as SOAR and ACT-R.

Cognitive features

Despite the high degree of uncertainty, ambiguity, and common sense – qualities even humans struggle with – automatic systems must grapple with these challenges. These processes require a set of cognitive skills. But what exactly is cognition within the realm of automatic systems? In the realm of automatic systems, Vernon⁴ has several depictions of what cognition might be:

Cognition is the process by which an autonomous system perceives its environment, learns from experience, anticipates the outcome of events, acts to pursue goals, and adapts to changing circumstances.

I would point to this more intriguing definition extracted again from his book “Artificial Cognitive Systems”:

Cognition is the means by which the system compensate for the immediate ‘here and now’ nature of perception, allowing it to anticipate events that occur over longer timescales and prepare for interaction that may be necessary for the future.

Cognition is the Swiss-knife of intelligent animals that strive to maximize their surviving chances in the environment where they live. Adapting to uncertain circumstances is not an exception, therefore the ability to self-improving is one among first principles:

Cognition is the result of a developmental process through which the system becomes progressively more skilled and acquires the ability to understand. Anticipation and sense-making are the direct result of the developmental process.

Let’s summarize the reasons why the system should pursue the virtues of cognition:

Inferences

The ability to make inferences about events that might impact a business process. Obviously the most important feature. Those events include the actions a company should ruled out according to the system recommendations. To make such inferences, the system should look back into the past and combine that experience with given processes definitions

Feedback-loop

To notice when performance are degrading, identify the reason of such low outcome and take corrective evaluation. The system should engage a feedback loop in order to minimize estimation errors. This feedback could be negative or positive according to the predictions. The feedback could be seen automatically from historic data (supervised learning) but also from experts that can help to draw a causality network of believes used by the system to infer unseen situations.

Autonomy

The automatic agent should actively trigger alarms concerning its goals and inform about risks and/or actions to take. For that, the system should not only ingest/parse/elaborate the information input, but autonomously search for those sources in order to improve its decision-making capability.

Continuous learning

If we assume that cognitive architecture forms the basis for intelligent system in uncertain environments, it should incorporate a flexible approach toward learning and skills improvements.

While narrow ML system distinguish training from inference, and expert systems take the given knowledge as granted, a cognitive system should develop an understanding of the world by constructing theories that explain and predict events and behaviors.

Continuous learning adhere to the theory-theory view⁵ by which the agents don’t passively accumulate knowledge, but they actively construct and revise (by leveraging counter-factuals) continuously what understood. This reflects a pillar of cognitive system referred as development. It states the importance of self-improvement and it is achieved by self-modification, in this view not only the environment perturb the system, but the system perturb itself ⁶.

Simulations

The system should run simulations for predicting alarms might happen in the supply chain. A simulation occurs anytime there is change in the information, even though it does not affect the tree of suppliers. This is due to the cascading consequences events may cause to apparently unrelated entities.

Interaction with other agents

For making sense of the world, cognitive agents have to transform the knowledge acquired through senses into meaningful affordances with which they can achieve their goals. The appreciations of semantic concepts into meaningful concepts is obtained not only by the interconnection with other concepts, but also by interaction with other agents and exchanging those concepts for knowledge sharing. Vernon perfectly summarize that in:

the meaning of the knowledge that is shared is negotiated and agreed by consensus through interaction

Though the supply chain agent would certainly benefit of this skills, remaining dubious its applicability in business contexts. Intellectual properties that the system is intended to conceptualize are secret by definition, therefore the propensity of sharing information that may leak confidential data (business internal structures, process models of any kind) is approximately zero. Even non confidential data may be not shareable for the risk to advantage competing agents.

Overview of main cognitive architectures

Over decades of research, the field of cognitive systems have seen the rise of a multitude of different designs for what could be defined the artificial cognitive architecture. The two more articulated approaches that have been developed are the SOAR and the ACT-R architecture. The have been inspired by diametrical different motivations. The ACT-R is definitely inclined toward modelling and replicating human cognitive processes, while the former was driven by the ambition to supersede the functionalities of an intelligent cognitive system, whether or not inspired by existing biology.
It turns out that despite the differences, the two paradigms are not so far from each other, a grossly simplified similarity is here provided.

Comparison SOAR/ACT-R

In both architectures, persists the separation of declarative memory (the set of info that we may think consciously of them when we need) from procedural memory (enacted and automated skills, such as riding a bike). Both approaches are mostly based on symbolic representation though actions (or operators in SOAR) are evaluated against continuous ranges of utilities which is adopted as reward for reinforced learning.
While the concept borrowed from cognitive psychology about working memory is a pillar of both systems, it is implemented in different ways. In SOAR, working memory is a centralized space for imminent processing, in ACT-R it is a sparse swap space between modules and modules -> central processing. Being both goal oriented, the approach taken for decision making in SOAR differs and deserve a mention.
Whenever there are multiple operators to select, SOAR attempts to fine tune the raking in order to let just one operator (action to take) to emerge. It solve such impasse setting intermediate goals with the purpose of discriminate the best operator. Among the list of differences, I find it interesting the different learning consolidation between the two, surely due to the different foundational motivations. in ACT-R a rule is consolidated (stored and actionable in procedural memory) after a series of re-learning, in SOAR the consolidation is immediate after its production.

Cognitive system structure

Data input

For performing the expected analysis the system requires different type of information: process models and sparse factual data. The first define the processes that the system is intended to optimize for what supply management concerns. The second is the flow of information that might trigger cascading events that affect evaluations and alarms. The processes models are transcribed and supervised by experts but also derived by already existing representations such as BPM models, so it is both a manual and automatic process. The data flow, on the other hand, is completely automatic, and data source may be listed as:

1. suppliers info:
   1. official financials reports. E.g.: quarterly CEO reports.
   2. news & rumors from social media/magazines.
2. central banks communications. Interest rates might strongly affect costs of debts.
3. inventory reports. Aggregated changes on company inventories can say something regarding raw material trend prices.
4. Oil prices. Anything that might affect transportation cost.
5. International political tensions. Clashes between countries/groups may result in chain disruptions.
6. important local events like Olympiads or G7 meetings may alter the usual exchange of goods and the freedom of movement of people.
7. natural disasters.
8. historic records of all above for training purposes.
   Summarizing the type of input I would list them into:
9. news text streams
10. company and supplier models that describe business and internal processes.processes are described by symbolic means through some domain specific language such as BPM or other logical and temporal language. Such representation should be denotative and connotative for objects and processes ⁷

Architecture overview

What is architecture in the context of a cognitive system? Again, Vernon⁴ comes to rescue with:

…architecture has been borrowed by many other disciplines to serve as a catch-all term for the technical specification and design of any complex artifact. Just as with architecture in the built environment, system architecture addresses both the conceptual form and the utilitarian functional aspects of the system, focusing on inner cohesion and self-contained completeness.

The forms follow functions⁸ approach adopted for this project drag attention toward the SOAR architecture for a variety of reasons such as:

1. It is adopted a completely functionalist approach. It’s irrelevant whether the set of cognitive processes are closer to the current state of knowledge on how human brain process information. While it is important to consider the implications an intelligent system might have for accomplishing its functions, it is not vital that they are adherent to the human mind.
2. The knowledge the system may handle would be always incomplete, therefore chain of reasoning will always encounter impasses. Impasse will be the natural intermediate state of any evaluation since the semantic and the procedural knowledge would never be suited for all encountered fact consequences.
   However, some ideas will be borrowed from ACT-R as well:
3. Production compilation. It is a learning task devoted to aggregate multiple rules for composing macro-functionalities. Rule re-usability is an optimization milestone for speed-up learning of complex production rules.

Modules

1. News Digester. The module transform the incoming data from news feed (but also many other sources, see above description) into propositional statements according to first order logic. Those statements will be stored in triple-store databases, which constitutes the episodic long-term memory.
2. Chunking. This module is inspired by the homonym component in SOAR. It is devoted to learn new behaviors/actions in certain conditions. Learning means creating new rules that can accomplish better estimations. Since the knowledge encoding is symbolic, chunking would comprise learning methods for discrete representations such as probabilistic inductive logic programming, that combines deterministic learning with probabilistic likelihood of the output propositions.
3. Simulator. How the system is supposed to answer questions regarding forecasts in case of hypothetical events and conditions? This is what the simulator is entitled for. It run all existing rules all over all defined processes until a number of stable models are generated by the system. Those models represent a state might occurs in a determined time step, therefore they will be used on extracting the requested metric (delivery date, prices, etc..) in that particular case. Such models are probabilistic, that means they are associated with degree of uncertainty that is reflected to the requested metrics as well.
4. Optimizer. Paired with the simulator, an optimization engine will act as a rational daemon for the chain of decisions a player involved in the business processes is supposed to behave. The simulation will involve game theoretical analysis because external agents’ decisions must be taken into account on evaluating requested metrics. For example, changing supplier might put the excluded one in the conditions to offer their products/services at discount cost to the competitors, initiating a cascading chain or eroded margin that are not welcome by shareholders. The optimizer would trigger that alarm during the simulation, informing the analysts of possible earning risks.
5. Scenario builder. This is the user interface module, for gathering constraints and given events for running simulations. The module will allow to retrieve specific evaluation metrics for the current state of events as well.
   Above the list comprehend computational modules intended to elaborate information which is stored and represented in the storage modules described below:
6. Episodic long-term memory. This database holds all propositions that regards events and facts. The information is saved in triple-store databases.
7. Semantic long-term memory. In here, the information representation is different from the episodic memory. A probabilistic likelihood is associated to proposition rules, which might have particular additional characteristics such as: symmetry and transitivity. This information module is recursive in the way that it is possible to specify recursive relations that have the advantage to be very declarative and concise in their definition. Of course, the database should be capable to deal with recursive structures and implement particular policies for optimizing the computation of (virtually infinite) depth of computation.
8. Working memory. Among the storage modules this is the most active, in the sense that it is supposed to cache the information that is temporary generated but at the same time, WM should anticipate the demand of information from long (and slow) term memories. Therefore, particular expedients should be employed to optimizing database lookups.
9. Procedural memory. This is the place for storing internal/external production and business processes models and they are provided by knowledgeable operators. This is the core of the expert system side of the entire system, where the human experience is encoded for being processed by the automatic system.

footnotes

The TOT state is also difficult to examine, because as you might imagine, it is identified only subjectively and escapes an objective scrutiny. How do we recognize to be in a TOT state? If you are unable to think of the word but you feel sure that you know it and that it is on the verge of coming back to you, then you are in a TOT state¹. However, it’s important to distinguish TOT states from a similar phenomenon known as the feeling of knowing (FOK). While during a TOT we have the sensation to be just one step away to pronounce the answer, FOKs emit different feelings. They gives us the resignation that we don’t know the target word, but we can eventually recognize the right answer among some alternatives.

Interested in language? Take a look into basic principles of language

If you feel uncomfortable experiencing a TOT, it would be pleasant to you to know that usually the feeling associated lasts for no longer than a minute, but be aware that half of the times a TOT is never solved. The sensation of being able to retrieve the word is accompanied by clues that usually appear to be correct, such as the first letter² and even the number of syllables of the target word. Psychologists found that dictionary definitions of uncommon English words would be enough for triggering a word-finding failure in their experiments. That’s why rare words, along with celebrity names, are used so widely in TOT experiments.

While we know all the sensation related to TOT, not all individuals experience this state with the same frequency. Students in their twenties might have one or two per week, while the frequency doubles for elderly people in their 80s. Age disparities seem to be the most correlated trait with this phenomenon, corroborating causal hypothesis that center their attention on cognitive degradation. While it seems a direct and intuitive explanation, this is not the end of the story. We will unfold some interesting reasons behind TOTs.

hypothesis survey

The effect has been under scrutiny since its first mention³ in the 19th century, but while early hypothesis were formulated in the thirties of last century, the TOT state is a prolific arena of research even nowadays. What makes TOT happening? Fingers are pointed against cognitive processes that involve memory and language. In particular, the linking bridge between the semantic memory - dedicated to memorize causal relation among concepts - and the phonological loop, the mental system entitled to vocalize words. How those modules may give us the sensation of TOT?

When we try to remember, the meaning of the word allows direct and immediate access to its sound, thus enabling vocalization. However, sometimes the phonological entry is not fully activated, instilling a certainty of knowing in the absence of expression ability. What could be the origin that causes those impediments between semantic and phonological systems? There are several hypothesis that have been evolved with time.

incomplete retrieval hypothesis

Let’s think of remembering a word. The more clues and reference you have the easier is to narrow down your search. When there are many paths to the final destination - the target word - it will be more likely the retrieval will be successful. Each clue acts as a signpost, pointing you closer to the target word⁴. According to this idea, if there are not so many paths for finding the right word, it would be hard to solve the search. Therefore, it is a lack of related clues that afflicts the identification of the correct answer. If you find this theory appealing, you might be disappointed by some findings⁵ that are undermining the validity of this hypothesis as solely responsible of TOT.

In the experiment, participants were fired with casual words exactly when they were experiencing TOTs. Those “bullets” (metaphorically speaking) were either related or unrelated to the target. It happened that unrelated words don’t affect the resolution of the TOT. Instead, related ones have an influence but in opposite way than predicted by this theory: more related words -> less successful resolutions.

blocking hypothesis

The experiment⁵ above may support a complementary explanation for TOTs. What if some clues instead of favoring retrieval, they prevent it? It is something that Freud mentioned⁶ when we associate a target name too early during the resolution phase, in the way:

“although immediately recognized as false, nevertheless obtrude themselves with great tenacity”

According to the blocking hypothesis, it is not just that a wrong candidate diverts attention away from the right word, it actually competes against the right target as solution. Related words obstruct the reach of the target. It is this impediment the responsible of holding back good resolutions⁷.
Similar words have such power that it is not even necessary to provide them directly. Unconscious alternative words that come to mind that are too weak to be recall may be capable of inhibiting target retrieval⁸ (somehow compatible with the knowledge hypothesis, hold on to the following chapters).

Also the blocking hypothesis present some weaknesses. According to the theory, an higher rate of TOT should be associated with a corresponding higher number of alternative words reported during the TOT state - because the theory prescribes - it is the alternatives that are blocking the target, right? Actually not. A substantial amount of TOTs are reported without any alternative word⁹. Also in this case, evidences do not clearly support the examined theory.

inhibition deficit hypothesis

It is worthy to mention that another hypothesis is closely related to the blocking hypothesis. The inhibition deficit hypothesis (IDH) emphasize the inhibitory control of speech. According to IDH, the retrieval is successful when we can get rid of all distracting information and reach the target without being overwhelmed by irrelevant details. A TOT state is due to the confusing and distracting amount of extraneous information (This is a more formal term that means not essential or necessary. May sound a bit pedant, but I like this term 😊) that impedes a straight resolution¹⁰. In support of this, the capacity of excluding noise during cognitive processes degrades with age. Aged people can less effectively shout out distractors than youngers, explaining why elderly people experience more TOTs than youngers. On the other hand, even this theory is not the whole picture. There is an amount of evidence that shows older adults produce instead fewer alternates during a TOT compared to younger people¹¹. For just this reason elderly people should be affected less, and not more by TOT. Something here does not add up.

transmission deficit hypothesis

This model suggests that TOT states arise from impairments in the communication between various cognitive modules or brain regions, rather than blaming individual systems. The retrieval of phonological traits might be particularly susceptible to linking failures because it is dependent on fragile connections between a word’s lexical node (lemma) and its phonology (Fig. 1). In contrast, the semantic and lexical systems are more connected. That’s why they are more resistant to aging decay than the lexical/phonological linking¹².
If you find many similarities with incomplete retrieval hypothesis, you are not wrong. Both point to a lack of strong connections. Interestingly, this is why some neurological evidences that enlighten aging-related issues, support all of them. Older adults exhibit reduced activity in the left insula - probably due to age-related atrophy¹³ - an important region for phonological production¹⁴.

knowledge hypothesis

This idea comes with diametrical different assumptions than the previous ones. It suggests that TOTs aren’t caused by declining brainpower, but by having too much knowledge. When experiments’ results are controlled over the type of target words, something new pop-ups from the data. The author states that in the experiments that do not require proper names (like celebrity names) there is no age difference in TOT rates⁸. This seems to be a quite incisive discovery.
If we take the transmission deficit hypothesis as granted, more you know about the target word more the semantic system compensate the aging effect. Instead, data shows that higher the knowledge, the higher the TOT rate. This aligns somewhat with the blocking hypothesis, where too many related words might get in the way.

illusion hypothesis

Among the explanations this is the most controversial one. It says that the emotional charge carried during a TOT state is an induced feeling that trigger our curiosity on the target and raise willingness on searching the right word. It seems a mental trick that tell us: “hey, for sure you know the answer, keep going!”, to push us for searching more obstinately. As we can find out in the next chapter, this is not really a disadvantage; rather it is a positive feature of TOT. In situations of great uncertainty this meta-cognitive decision can be helpful on pursuing the right action¹⁵. In support of this thesis, it has been found out that TOTs are more frequent in groups. Group magnify the feeling of the target word is on the reach, prompting TOT states more than what occurs on single individuals¹⁶.

upsides of TOT states

Remember the “tip-of-the-tongue” feeling? It might be a good thing. As mentioned above for the illusion hypothesis, TOT state is beneficial because it induces more curiosity and motivation to spend more energy to reach an achievement¹⁵. Moreover, the confidence of the apparently latent knowledge pushes us into more risky decisions. For example, participants in test quizzes are more willing to risk their grades if they are in TOT state. It turns out that such a bravery is prized by better results¹⁵and then the frustrating feelings of inability are somehow balanced by higher performances.

practical applications

This phenomenon open the way to practical applications, for example in the area of assessing students’ knowledge. If they can’t come up with a short answer, they could maybe use TOTs to know when it’s smart to ask for multiple-choice options. Quiz designers might exploit TOT effect to build adaptive tests, to let participant to demonstrate various levels of knowledge, including knowledge that might be present but momentarily inaccessible¹⁷.

conclusion

This survey of potential explanations for the TOT phenomenon hasn’t yielded a definitive answer. Instead, the evidence suggests that various hypotheses likely contribute to TOTs, each with varying degrees of influence. Tough not privileging a single explanation, we can de-emphasize the incomplete retrieval hypothesis due to the lack of compelling supporting evidence. The blocking hypothesis, however, might be encompassed by the knowledge hypothesis. Since age-related cognitive decline is a well-established factor, it could not be ignored as contributing factor in TOTs.

more curiosities

There are also some tricks could be used to mitigate those kind of retrieval failures. If the blocking hypothesis affirms that if we can’t discern useful clues from irrelevant noise we are more affected by TOT. So why not consciously attempt to ignore related but incorrect words? As Woodworth (1938) suggested:

…the wrong name recalled acquires a recency value and blocks the correct name … a rest
interval allows the recency value of the error to die away

A curious aspect of TOTs has been found out on the ethical sphere. The sensations associated with TOT is a kind of warm glove that attributes high ethical values to not able to remember memories. Curiously, when celebrities’ names induce TOT states, they are judge more ethical¹⁷.

Giancarlo Frison

Propositional Logic

PL is a simple language consisting of proposition symbols and logical connectives. Its syntax defines the allowable sentences while its semantics defines the rules for determining the truth (just true or false) of PL sentences with respect to a particular model. The semantics for propositional logic must specify how to compute the truth value of any sentence, given a model. This is done recursively. All sentences are constructed from atomic sentences and the five connectives.
A model is a truth assignment of propositions in a knowledge base (KB) which is a set of sentences (axioms) when the sentence is taken as given without being derived from other sentences. Sentences can be derived from other sentences that are logically entailed. Entailment is the idea that a sentence follows logically from another sentence. In mathematical notation, we write α $\vDash$ β if and only if, in every model in which α is true, β is also true.

Inference

Sentence derivation is done by running an inference algorithm that follows the modus ponens logic paradigm or the modus tollens. The former refers to deductive forward chaining (FC) while the latter the inductive backward chaining (BC) families of algorithms. FC is an example of the general concept of data-driven reasoning; reasoning in which the focus of attention starts with the known data. BC algorithms, as its name suggests, works backward from the query. If the query $q$ is known to be true, then no work is needed. Otherwise, the algorithm finds those implications in the knowledge base whose conclusion is $q$.

Resolution

For a sentence to be proved as true, it must be sound and complete. A sentence is valid if it is true in all models. For example, the sentence P ∨ $\neg$P is valid. Valid sentences are also known as tautologies, they are necessarily true. If a sentence is valid and its premises are true, then it is also sound. For being complete, the inference algorithm must derive any sentence that is entailed. Proofing is obtained by reductio ad absurdum (or proof by refutation or contradiction) on which α $\vDash$ β if and only if the sentence (α ∧ $\neg$β) is unsatisfiable. How resolution works for obtaining a proof?

• First, $KB \wedge \negα$ is converted in CNF. CNF stands for conjunction normal form (formula consists only of conjunction of disjunctions).
• Each pair that contains complementary literals is resolved to produce new clauses, until:
  • There are no new clauses that can be added, in which case KB does not entail $α$.
  • Two clauses resolve to yield the empty clause, in which case KB entails $α$.
    

First order logic

PL lacks the expressive power to concisely describe an environment with many objects.
The language of FOL is built around objects and relations and it assumes that the world consists of objects with certain relations among them that do or do not hold. While propositional logic commits only to the existence of facts, FOL commits to the existence of objects and relations and thereby gains expressive power. In FOL are represented objects, predicates and functions. Predicates are $n$-arity relations among objects or employed for expressing features of a single object. Functions are expressions that return a single object out of a set of arguments.
In FOL, it is natural to express properties of entire set of objects and it is done by adopting the existential quantifier ($\exists$) and the universal quantifier ($\forall$).

Natural numbers

An interesting application of FOL is the description of Peano numbers which are Peano numbers are a simple way of representing recursively the natural numbers ($Nat$) using only an axiom ($Zero$) value and a successor function $succ(Nat)$.
\(Nat(Zero). \forall x [Nat(x) \rightarrow Nat(succ(x))].\)

Unification

This is the process to make - whenever possible - different logical expressions look the same by substituting properly the value of variables. With unification it is possible to construct all queries that unifies with a given sentence, e.g.: $Employes(SAP, Giancarlo)$ and $Male(Giancarlo)$ some queries might be: is there are male employee in SAP? In FOL it might be: $\exists x [Male(x) \wedge Employes(SAP,x)]$ .

Reasoning in FOL

Reasoning in FOL works by bringing the formula into Skolem form - by removing existential and universal quantifiers - and transform it in clause normal form (which indicate a formula composed of conjunctions separated by comma). Use propositional reasoning (resolution, SAT), forward cha backward chaining.

Herbrand universe

The Herbrand Universe (HU) is a set of combinations of all ground terms (not variables) present in a formula, e.g.: in the clause formula $CapitalOf(x,y) \rightarrow IsA(x,City), IsA(y,Country), PartOf(x,y)$ we have the HU as $HU = {City,Country}$ but if there would be a function, the HU size will be infinite. The HU is useful for example to restrict the search space of Prolog programs, whenever it is possible.
The HU could play a part on resolution in FOL by applying the Herbrand expansion which is the set of formulas the results of substituting terms in the initial formula in all possible ways. Given a knowledge base: $KB = \forall x [SpecialAgent(x) \rightarrow SpiesOn(x, Danz)] \wedge SpecialAgent(MrSmith)$
it will be translated as “every special agent spies on Danz and MrSmith is a special agent”.
If we advance the hypothesis that formula $\phi=SpiesOn(MrSmith, Danz)$ in entailed by KB ($KB \vDash \phi$), the Herbrand expansion could be applied to show that $HE(KB \wedge \neg\phi)$ is unsatisfiable.

As in the established definition of agent¹, the sensation is the process of acquiring environmental information from the environment through sensors. The behaviour the agent manifests is the result of pondered actions, activated by proper strategies that aim to achieve desirable goals. Living creatures, including humans, do not deviate from this general definition. Attention could be seen as a function that help on elaborating successful strategies in efficient way by partitioning what really can affect the goals, and what it does not ².

Sensation brings its meaning directly from the raw information acquisition from the senses. They include of course the 5 canonical ones, but not limited of them, they involve also movement, position of the body, pain and temperature. It could be surprising how much sensitive are humans senses. For example, it is possible to detect a candle almost 50km away in a dark and clear night ³. On the other hand, perception is a more abstract concept.

When perception is closely tied up with sensation it is defined as bottom-up perception. In this case, perception is the interpretation of information from the environment so we can identify its meaning ⁴ and enable more or less accurate predictions about what should be there. Perception, on the other hand, emerges also from cognitive processes that involve memory and attention, completely detached from sensory circuits. This scenario is described as top-down perception, and it is a process strongly mediated by expectations and the contextual setup during which those sensations are collected ⁵. In the image below, the shadow is expected to lower the brightness of the B region, then a kind of visual compensation innately make us thinking that the region should be brighter without shadow, generating the famous visual illusion.

Attention and perception are highly correlated and it is very evident in every day life. If we are walking in the university department looking for the Cognitive Psychology professor, I think it won’t be surprising if we pay attention on people faces by visual searching in the crowd until the target pops out in the hall. The eye movement will be directed to scan the faces of the people ⁶. If expectations are violated by novel surprises (e.g.: an old mate in the classroom) these are explored extensively ⁷. The other face of the coin is inattentional blindness by which what it is not attended during the attention phase is not consciously perceived. For example, 60-80% of the observers do not attend the center point which turns into a text while a distracting cross appears in the left ⁸ in the figure below.

I’ve mentioned popping out by describing the subjective effect may have when something catches the attention with no efforts. It is established that this is an interesting topic that shows particular dynamic depending on the environmental context. If the search task involves picking a simple feature out of a context disseminated of distracting details, it is much easier than searching for complex pattern with a combination of features - the so called conjunctive search ⁹. In the experiment below, recognizing a simple red dot among all green ones is automatic, while the case in the right pane involves some degree of attentive scrutiny which is much slower and more resources demanding.

This might suggest that attention is somehow necessary for recognize non-trivial objects ¹⁰. The feature integration theory ¹⁰ affirms that automatic feature processing is followed by attentive processes to bind the features into a whole object. Objects are not a mere list of independent features and perception does not limit on pattern matching but also on identifying high order structures, with the support of controlled attention.
Just as the letter N is not simply a casual aggregation of 3 segments, things must obey to some grammar for being recognized.
A demonstration of the importance of structural relations is depicted in the image below, where a set of simple forms (geons) can easily compose complex objects ¹¹. That demonstrates that not only the relational information if needed, but it is more critical to perception than the features themselves ¹².

While some information need attention to retrieve complex structures from senses, an equal demand of effort is required to selectively ignore conflicting information while performing some tasks. This is the case raised by John Stroop in his famous experiment ¹³, that demonstrates the difficulty of partitioning contrasting information on one side, and useful one to the other. In the Stroop task, an observer reports the color of appearing words, while the words points to a different color’s name. It appears that the task is more difficult than the no-contrasting information setting (when the name matches the color). Observers activate different parts of the brain on discordant stimuli processing, those parts are in charge of executive control and selective attention functions. For the same reason, when we stop at a cross-light while driving a car, we discriminate the cross-light of our own lane from the adjacent cross-lights. Selective attention prevents us to switch from the brake to the accelerator pedal when it is not the case.

The selective attention theory states that perception is filtered before being processed by high level mechanisms, but it clashes with the notable cocktail party effect (CPE). Unfortunately for Broadbent, the CPE seems to confine the selective theory not as the only way the brain deal with attention. The CPE describes the capability of being caught by unattended stimuli once they present an important pattern. For example, while we’re on talking with friends and someone out of the interlocutor’s circle will loudly mention our name, our attention will be probably triggered by this event toward the speaker and his discourse, even though we were previously unaware of that discussion.

An interesting effect of unattended stimuli is that they interfere with attended perceptions. The experiments that put light on this effect, are the ones that enforce shadowing, by which observers are instructed to follow only one stream of perception among many. If two recorded discourses are played simultaneously, one to the left side, the other to the right side of the observers’ headset:

a: “They were standing near the bank…”
b: “the silicon valley bank has gone bankrupt…” ^[It is not exactly the example of the experiment, since that bank went bankrupt on 2023, but I guess it is equivalent to the purposes of the experiment]

Observers disambiguates the term bank with the financial bank, and not with other possible meanings ¹⁴. This experiment tell us that though attention allocates sufficient resources to spotlight some stimuli, it leaves space for unconscious mechanisms that capture information for the unattended channel, for blending it with the attended one.

Giancarlo Frison

“This is the promise of the Semantic Web – it will improve all the areas of your life where you currently use syllogisms. Which is to say, almost nowhere.”

—Clay Shirky

“Fortunately, a large majority of the information we want to express is along the lines of ‘a hex-head bolt is a type of machine bolt.’

—Berners-Lee

“Unfortunately this is not true. If one considers how humans handle concepts, the class relation structures of the Semantic Web capture only a minute part of our information about concepts”

—Peter Gärdenfors

I guess that it doesn’t come without notice that the semantic web approach isn’t the favourite of the author of the Conceptual Spaces (CS).
Two distinguished views on knowledge representation are the semantic web and the vectorized embeddings that belong to the symbolic and the connectionist schools respectively. The CS theory comes from a very different vision on how knowledge should be encoded. Concepts in CS are without doubt, close to vectorized embeddings representation though they preserve interpretability - a strength of the symbolic world.

Vector embeddings

It’s not hard to think many of you have already heard about word embeddings for knowledge graphs. Modern natural language processing tasks based on neural networks would not be in there without vectorized embeddings. We’re witnessing the immense progress of natural language processing (ex: GPT and related) in recent times, and their machine learning algorithms rely on word embeddings. A branch of deep learning named graph neural networks (GNN) have brought a similar advancement on machine learning tasks such as link prediction in ontologies. The intuition behind neural network embeddings is that words or graph nodes can be represented as a series of real numbers that embed the semantics, and programs of a very special kind - the neural networks - can use them to accomplish some specific task. Vectorized embeddings come as a byproduct of those processes where there is not contemplated human supervision in the loop. Embeddings, differently from ontologies, don’t convey any semantics that is comprehensible, and even less edited, by people.

If requested during a talk in a conference, I can hardly imagine someone raising the hand because of past experience with CS.

In the CS framework, concepts are represented as vectors of numbers that are continuous in their spectrum and convex, which means that similarity scores and distances among concepts are naturally derivable, but those vectors are not imperscrutable to human scrutiny. Concepts are built taking in account how we process and categorise them.

Properties as region spaces

Domains represent a single quality. They are convex and differentiable because they can be represented in real values. A concept is qualified via a set of domains. An apple has a round shape, a colour in the range of green, yellow and red, a taste (which is apparently related to the colour), a size, a weight. Likewise neural embeddings, similar concepts cluster together in regions. In CS, concepts are defined by vectorized properties where each of them describe a quality in a specific domain:

• The colour domain could be represented as a three dimensional space, where each dimension defines the intensity of one the three basic colours, red, blue and green. The RGB notation is convex in relation to what we perceive as colours. The two ends of RGB scale are #000000 (black) and #FFFFFF (white) and this is numerically consistent with our perception, from nothing to all colours together. Further, a slight change in one parameter implies a small visual change, as more or less reddish, bluish or greenish depending on which parameter.
• Consider now the price dimension. 43,78€ is certainly a number but also a currency, thereby the price could be encoded in 2 dimensions.
• Think of time dimension, it is a single dimension domain, where zero is the present, a positive value is projected in the future while the negative numbers are settled in the past. Its magnitude instead, can describe how far that point is from the current present.

_“The things have weight,mass, volume, size, time, shape, colour, position, texture, duration, density, smell, value,consistency, depth,boundaries, temperature, function, appearance, price, fate, age, significance.

The things have no peace.”_

—Arnaldo Antunes

The main similarity with neural embeddings and CS is that both are differentiable and convex. At the same time, concepts incorporate symbolism with their multi-facets domains; they naturally enable semantic algebra and computational problem solving. For example, consider the request: ‘show me a movie like Casablanca but scarier as Shining’ will consider the properties of Casablanca but with the scary domain similar to Shining. Put it in the frame of logic programming, and you get it in a line of code.

Cognitive affinity of cognitive spaces

CS has also some interesting psychological foundations in the regards of how people deal with inner knowledge representations and how they learn them. It has proven some validity in explaining some cognitive aspects, especially those involved in concept learning and understanding. It has been found out that when children have assimilated the meaning of a domain, it’s then easy to learn concepts that represent a flavoured materialisation of that domain. For example, once they know what the domain of ‘colour’ is, it’s easy to learn new concepts related to colour, such as ‘turquoise’. Grasping a new domain is a much more difficult step than adding new terms to an already established one. Conceptual domains are mental buckets where we place concepts based on how their properties fit into that domain and we don’t have to know how ‘turquoise’ is exactly encoded, we just need to think of it in comparison to other concepts as somewhere in the between of light blue and light green. Seems to be a provable trick we use on learning and it is justified by the principle of cognitive economy, for which our mental capabilities are limited and we favour simple and efficient ways to position new information.

I’ve written more about cognitive aspects in meeting of minds. Take a look at there!