Pioneers Insight Method Research Author
Tensor Logic "Unifies" AI Paradigms [Pedro Domingos]
Back to Episodes

Tensor Logic "Unifies" AI Paradigms [Pedro Domingos]

Summary

  • Pedro Domingos argues that TensorLogic unifies symbolic rules and tensor algebra in one construct—the tensor equation—while extending across deep networks, kernel machines, and graphical models. The central observation is that “an einsum and a rule in logic programming are the same thing,” differing chiefly in whether they operate on real numbers or Booleans. He calls TensorLogic the scaffolding for a master algorithm, not yet the algorithm itself.

  • The enterprise proposition is a controllable spectrum between guaranteed deduction and speculative analogy, potentially addressing the adoption-limiting risk of hallucination. At zero temperature, TensorLogic can restrict conclusions to what logically follows from supplied premises; higher temperatures permit looser analogical inference, potentially rule by rule. The crucial caveat, conceded under host pushback: sound deduction cannot rescue false premises—“if the premises were hallucinated, so will the conclusions be.”

  • TensorLogic’s more consequential claim is that gradient descent can learn structure, predicates, and representations rather than merely weights inside a hand-designed network. Domingos contrasts this with inefficient search in inductive logic programming and describes predicate invention—discovering relations absent from the data—as “the Holy Grail.” Tucker-style tensor decomposition is one illustrative route for learning latent factors within this broader gradient-descent story, while the host presses that useful architecture search still needs inductive bias.

  • The proposed development model reduces reliance on fully hard-coded architectures by using broad tensor equations plus strict or soft structural priors. An MLP or ConvNet can each be written as one tensor equation; researchers can then add beliefs as equations without completely fixing the topology. Domingos’s practical pitch is an iterative, more interpretable workflow: “you write one more equation or you modify an existing equation,” retrain, inspect, and refine.

  • Domingos sees weak symmetries as AI’s equivalent of physics’ foundational regularities, but he does not claim they eliminate irreducible complexity. His expected split is roughly 80/20: compact laws and reducible pieces get systems most of the way, while spontaneous symmetry breaking, new observations, and domain-specific hacks handle the rest. The operational lesson is control-oriented prediction—continually recalibrate from data, as in Kalman filtering or reinforcement learning, rather than chase Laplace’s demon.

  • The long-range goal is “universal induction”: a learning analogue of Turing’s universal machine that can infer reusable procedures from small examples and apply them at arbitrary practical scale. The host contrasts a child learning addition on tiny numbers and generalizing to any length with transformers whose fixed context may require retraining when the problem scale changes. Domingos frames this as a limitation of many transformers and a target for TensorLogic. The issue matters more than formal infinity or Turing-completeness labels.

  • The investment-relevant warning is that current AI infrastructure spending may be dramatically ahead of the algorithms capable of using it efficiently. Domingos says brute-force reasoning reflects forgotten computer-science lessons and predicts that “99.9% of that compute” may look wasted, even if the physical capacity eventually finds uses. TensorLogic’s route to market is incremental—compile equations into Python/PyTorch first, win on hallucination and opacity, seed education, and eventually threaten even “CUDA’s nice moat.”

Deep dive

1. AI needs one working language before it can fully take off

  • Domingos frames TensorLogic as the culmination of a goal pursued since his PhD: unifying AI’s competing paradigms. His analogy is calculus for physics or Boolean logic for circuit design—fields accelerate when they acquire a notation that captures their fundamental operations.

  • Earlier symbolic languages supplied automated, transparent, reliable reasoning but lacked automatic differentiation and GPU scalability. PyTorch and TensorFlow make learning and acceleration routine, yet reasoning sits above them as “all these hacks.” TensorLogic is meant to provide all three properties together.

  • He stops short of claiming one language is intrinsically best for every task: computer science offers “no one programming language that is better for everything.” Still, his experience now suggests TensorLogic captures AI’s fundamentals more broadly than he expected—“But hey, that’s me. I may be a little biased.”

2. An einsum and a logical rule are the same underlying operation

  • Tensor algebra can be reduced to Einstein summation: matrix multiplications, tensor products, and related deep-learning operations are einsums. Domingos’s “gob-smacking observation” is that an einsum and a logic-programming rule are actually the same thing, with real-valued and Boolean arithmetic as different atomic data types.

  • He reaches the equivalence through Datalog, whose database rules reduce to joins and projections. TensorLogic generalizes both into tensor joins and tensor projections: with Boolean tensors they recover symbolic database operations; with numeric tensors they encompass and extend conventional einsum.

  • A dot product is a sum of products; in Boolean form, that becomes a disjunction of conjunctions, or DNF. If multiple clauses are true, a step function maps values above one back to one. Multiple rules with the same head likewise express an implicit logical “or.”

  • The host’s NAND-gate challenge—universality alone does not make a good abstraction—draws agreement. TensorLogic can retain syntactic sugar, but its native level is far above NAND: Domingos says a transformer can be written in roughly a dozen tensor equations instead of “a vast mass of code.”

3. Better notation could also unlock better execution

  • The first advantage over direct einsum APIs is notation. Existing calls require awkward indices, arrows, and arguments; TensorLogic lets an einsum read like a rule, with an output tensor on the left and a joined expression on the right. “A good notation is half the battle.”

  • The second is performance. Domingos says einsum is used less than it might be because implementations can be slow and clunky; a language organized around one primitive could optimize tensor equations directly in CUDA and let “Einstein finally be able to reach its potential.”

  • The decisive difference, however, is semantic breadth. Conventional einsum only performs tensor algebra; TensorLogic uses the same construct for symbolic and numeric computation, mixtures of both, reasoning, and even learning the symbolic component.

4. TensorLogic supplies scaffolding for the master algorithm

  • The Master Algorithm mapped tribes including symbolic learning, Bayesian methods, kernel machines, graphical models, and neural networks. Domingos now calls TensorLogic the answer at the representation level: “a very simple language where you can do the entire gamut of AI.”

  • Graphical models fall into the framework without special machinery. Their factors and conditional distributions are tensors, while marginalization and pointwise multiplication become tensor projections and joins; kernel-style computations fit the same algebraic vocabulary.

  • TensorLogic is not itself the master algorithm because a language is not a complete learning procedure. It is the scaffolding on which one can be built, with reasoning and automatic differentiation already underneath: because there is one construct, “the gradient of a TensorLogic program is just another TensorLogic program.”

  • Domingos’s stated ambition goes beyond presenting that scaffolding: “I intend to produce it on short order.” The missing centerpiece is a universal induction procedure that uses the language to discover rather than merely express models.

5. Gradient descent is supposed to invent structure and concepts

  • A host identifies the central problem cleanly: computational universality only says a representation can express a solution; AI must adapt to novelty and construct the right representation. Domingos answers through inductive logic programming, tensor factorization, and gradient-based structure learning.

  • Classical inductive logic programming discovers rules through greedy or beam search over an enormous discrete space, making it painfully inefficient. In TensorLogic, Domingos argues, “the structure learning falls out of the gradient descent”—rules and relations become learnable within continuous tensor equations.

  • The strongest claim is predicate invention: discovering relations not explicitly present in the data but which explain it more compactly. Objects themselves are invented predicates rather than retinal pixels; scientific concepts such as force, energy, and entropy illustrate the same representational leap. Domingos calls this “the key problem in AI” and “the Holy Grail.”

  • Tucker decomposition is an illustrative mechanism in this account. As a matrix can factor into two latent matrices that compactly reproduce its contents, a data tensor can be expressed through newly learned tensors; those factors may remain numeric or be thresholded into Boolean concepts that a person can inspect.

6. Broad equations still need strong inductive bias

  • The host’s pushback—worth keeping—is that a fully connected network with excessive degrees of freedom does not become practical merely because it has a cleaner representation. Today’s “divine incantation structures” exist because unconstrained SGD generally fails to discover useful topology by itself.

  • Domingos distinguishes compact expression from unconstrained learning: an entire MLP using one nonlinearity can fit in one tensor equation, as can a whole ConvNet with its locality bias. A general equation defines a family of architectures; learned tensor values can materialize a particular network structure within that family.

  • He concedes the no-free-lunch point and does not recommend blindly brute-forcing the largest family, despite modern compute. Instead, researchers can add equations expressing partial beliefs about structure—some mandatory, others soft—so gradient descent begins with structured knowledge rather than a tabula rasa.

  • The development loop remains empirical: specify a broad structure, train, inspect results, then add or modify an equation. Domingos’s claimed gain is efficiency and transparency, both in the program and, with suitable discretization, in learned rules that a transformer “just can’t hope to” expose.

7. Symmetries may supply the 80%, not the whole world

  • Domingos embraces geometric deep learning and says the strongest weak biases are symmetries. Physics’ standard model is largely a collection of them, showing how simple regularities can constrain enormous complexity; tensor notation is already physics’ natural way to express such structure.

  • A host counters with David Krakauer’s divide between closed, reversible, symmetry-dominated classical systems and open, self-organizing, dissipative, uncertain, adaptive systems—the latter containing life and intelligence. Perhaps reality is too constructive and “gnarly” for clean factorization to recover meaningful concepts.

  • Domingos’s synthesis is “symmetries and spontaneous symmetry breakings.” Physical laws continue to apply, but accumulated breakings create biological, psychological, and social levels where those laws become practically useless; evolution and brains amplify some contingencies while others disappear.

  • His forecast is an 80/20 outcome: compact regularities deliver perhaps 80% of capability, while the remaining 20% requires accumulated exceptions and hacks. TensorLogic is meant to help with both—the clean equations and the awkward residual engineering.

8. Irreducibility changes prediction into continual control

  • Domingos accepts chaos, computational irreducibility, and the idea that “more is different.” His disagreement with stronger irreducibility claims is quantitative: the overall universe may be irreducible, but it contains many reducible pieces, and evolution and brains accumulate and exploit those pieces.

  • The right response is not to demand perfect prediction from initial conditions. Systems should reduce whatever they can, then continually incorporate observations: an aerospace engineer’s Kalman filter and reinforcement learning both update forecasts at every step instead of trusting a once-for-all trajectory.

  • This turns prediction into intervention. “You actually only need to predict things well enough to control them to make them predictable”; humans continually reshape environments to stabilize them, and Domingos argues robots will need the same closed-loop strategy.

9. TensorLogic is proposed as a meta-language across levels

  • Another host invokes Hofstadter: intelligence moves among multiple descriptive levels and often preserves several useful theories. Newtonian mechanics remains valuable at human scales even though general relativity and quantum mechanics describe different regimes; why expect one AI language to replace every layer?

  • Domingos answers more strongly than the host anticipates: “I am absolutely arguing that TensorLogic is the language to use in all these layers.” Relativity and quantum mechanics already use tensors, while TensorLogic adds a way to construct new representations and switch among them.

  • The distinction is between one representation and a meta-representation. Intelligence may require many representations at the same or different levels, but tensor equations can describe how those representations are composed, learned, and switched among; locking oneself into one fixed representation is the actual mistake.

  • He extends the claim to science. A TensorLogic program can nearly mirror equations on the page, combine numerical computation with surrounding logic, and—most importantly—make that logic learnable rather than permanently hand-coded.

10. The star index exposes a deliberately leaky abstraction

  • The host challenges the RNN example’s starred time index as an extra construct smuggled into the language. Domingos distinguishes essential semantics from implementation guidance: ordinary expressions such as T+1 can be syntactic sugar, while the star is a memory-saving hint, not additional computational power.

  • Writing a star on one index tells execution to reuse storage rather than preserve every state. In an RNN, the new state overwrites the old vector just as in a state-transition system; omit the star when history is required. “All abstractions are leaky,” and TensorLogic lets programmers expose this detail when efficiency matters.

  • For an unbounded sequence such as digits of pi, forward chaining fills successive positions in a vector. An ideal infinite vector continues forever; a real finite vector stops when memory runs out, exactly as any physical computer does.

  • TensorLogic also has both declarative and procedural semantics. An equation can be read as a statement about the world or as a function call whose left side is the call and whose right side supplies its body—Domingos says he often uses the language procedurally.

11. The Turing-completeness claim survives a sharp technical dispute

  • A host objects that the cited 1995 Siegelmann construction depends on infinite-precision rational registers and a specially structured RNN, making it a toy rather than evidence for practical, general-purpose TensorLogic. The challenge targets both the numeric field and the fixed architecture required by that proof.

  • Domingos agrees the construction has “no practical significance” and says he used it only because it establishes the formal result in one paragraph. He claims other proofs are available and, more fundamentally, says Turing completeness “doesn’t matter at all whatsoever” in a finite physical world.

  • What matters is computational universality: the ability to express any practically desired computation. Domingos argues that TensorLogic can implement finite control and, with external read/write memory operations, represent memory as a vector indexed by address and express reading, writing, and moving through index updates.

  • The host remains cautious that external read-write operations look like an extension to ordinary tensor mathematics. Domingos locates them on the logic-programming side and calls the implementation almost trivial, but the exchange ends with the host wanting concrete worked examples rather than treating the issue as fully settled.

12. Practical universality means generalizing beyond training scale

  • The host’s sharper transformer claim is not metaphysical infinity but reuse: a Turing machine can run out of memory, receive more memory, and continue without reprogramming; by contrast, the host argues that increasing a traditional transformer’s context requires retraining rather than simply resuming the same learned procedure. Domingos responds by emphasizing the broader transformer-scaling limitation that TensorLogic is meant to address.

  • Domingos wants systems that learn from small cases and transfer to arbitrary practical sizes: children learn addition using short numbers but can apply the procedure to much longer ones. The same should hold for graphs, knowledge bases, and inference problems “of any size from very small ones.”

  • Turing’s achievement, in his telling, was the counterintuitive universal machine—a device that could effectively “type with one hand and sew with the other.” Equivalent computational formalisms later showed that surface machinery mattered less than universality.

  • His research question replaces deduction with learning: “A Turing machine is a model of deduction. It’s universal deduction. What we’re missing…is universal induction.” He expects many equivalent master algorithms, just as there are many equivalent universal computing models.

13. Temperature joins exact deduction to structural analogy

  • In embedding space, unit vectors have self-dot-products of one, while random high-dimensional vectors have dot-products near zero. Their Gram matrix is therefore approximately an identity matrix; a sufficiently stiff sigmoid cleans the residual noise into Boolean identity, letting logical rules operate deductively.

  • Learning moves embeddings of objects that support similar inferences closer together. At zero temperature, a step function discretizes similarities back to zero or one for pure deduction; raising temperature permits increasingly loose analogical transfer between similar objects.

  • Domingos casts proof search as simulated annealing: begin at high temperature to explore analogies, then lower it until the result is a deductive proof. Mathematicians “notice similarities between things,” he says, but “at the end of the day you need to have a proof.”

  • The host’s sandcastle objection remains essential: formally valid reasoning can still build on a representation that does not capture the world. Domingos distinguishes this proposal from merely reimplementing an MLP or transformer, but agrees that TensorLogic does not magically certify the truth of a learned world model.

14. Structure mapping is richer than statistical similarity

  • Domingos explicitly rejects equating similarity with full analogy. Nearest-neighbor and kernel methods provide only the weakest form; powerful analogy requires structure mapping, where relations and composition in one problem are mapped onto another.

  • Dedre Gentner’s canonical example is Bohr’s mapping between the atom and the solar system: nucleus to sun, electrons to planets. “Turns out to be a bad analogy,” Domingos notes, yet it was instrumental in physics—a useful reminder that generative analogy and final correctness are different stages.

  • Case-based reasoning applies the same mechanism operationally: a help desk retrieves a structurally similar problem and adapts its solution rather than solving from scratch. TensorLogic’s claimed advantage is the combination of kernel-style similarity with symbolic compositionality, allowing nested mappings or combinations of several cases.

15. Soundness, not omniscience, is the enterprise guarantee

  • A host notes that GPT can hallucinate at temperature zero and a bad deductive system can also produce nonsense; determinism and truth are separate. Domingos agrees with the distinction but argues GPT’s failure at zero temperature is precisely why enterprises need another mode.

  • The Fortune 500 requirement is operational: business rules must be obeyed, security constraints preserved, and customers not lied to. TensorLogic could assign different reasoning temperatures to different rules, keeping mathematical or policy constraints hard while permitting softer evidence accumulation elsewhere.

  • Compared with RAG, the proposed system does more than retrieve facts: it computes their deductive closure. Domingos calls that “an exponentially more powerful thing to have,” while still restricting results to conclusions licensed by the premises.

  • His guarantee is technical soundness, not factual omniscience. TensorLogic can ensure that conclusions logically follow; it cannot ensure that the premises correspond to reality. “If the premises were hallucinated, so will the conclusions be”—model acquisition and validation remain separate problems.

16. Adoption will ride pain, interoperability, and compute economics

  • Domingos acknowledges that technical elegance cannot beat installed bases alone: “You can’t kill COBOL.” Lisp and Prolog lost despite AI advantages because general-purpose ecosystems had stronger network effects, while Java gained an opening from internet-era workloads that older languages handled poorly.

  • TensorLogic’s opening would be acute pain—hallucination, opacity, and executives unable to trust black-box systems. Migration can begin with a preprocessor that translates tensor equations one-to-one into NumPy, Python, or PyTorch, preserving existing code while adding easier reasoning.

  • Education is another wedge. One language could teach the gamut of AI without students spending their time on incidental syntax; because tensor equations also map closely to GPU work, specialists could still descend into optimization. Domingos jokes that “CUDA is a nice moat, but TensorLogic could be the end of that moat.”

  • He closes with a harsher capital-allocation critique: transformers are a genuine leap, but ignorance of established reasoning methods has encouraged brute-force spending. He predicts people may conclude “99.9% of that compute was completely wasted”—a trillion-dollar education that, paraphrasing Good Will Hunting, cost “a buck fifty in late fees at the library.”