i
Short History of the Learning Machine daniel-hromada ()


Short History of the Learning Machine

Between Archetypes & Myths: Machine as a Projection of the Human Self

Between Archetypes & Myths

Machine%20as%20a%20projection%20of%20the%20human%20self

Machine as a projection of the human self

Machine as a projection of the human self

Faciamus machinam ad imaginem et similitudinem nostram.

Let us make the machine in our image and likeness.

What Is a Machine?

Hero%20of%20Alexandria%2C%201st%20century%20AD

Hero of Alexandria, 1st century AD

Before machines became motors, engines and computers, the word already meant something broader:

an ingenious means by which something becomes possible.

The Divine Workshop

Hephaestus%20at%20the%20forge%2C%20with%20Pandora's%20urn%20and%20a%20bronze%20guardian

Hephaestus at the forge, with Pandora's urn and a bronze guardian

Greek mythology gives us an extraordinary laboratory of artificial beings.

Athena represents intelligent making.

Hephaestus represents the workshop in which artifacts begin to act.

Machines in Our Image

Jean%20Regnault%2C%20Pygmalion%20priant%20V%C3%A9nus%20d'animer%20sa%20statue%20(1786)

Jean Regnault, Pygmalion priant Vénus d'animer sa statue (1786)

Artificial beings are not merely tools.

They become surfaces onto which humans project desire, identity and self-understanding.

Machines at the Threshold

Some artificial beings do not imitate us primarily as workers or companions.

They stand between inside and outside.

They decide who may approach, enter or survive.

Machine as Oracle

The%20Antikythera%20mechanism%2C%20c.%202nd%20century%20BC

The Antikythera mechanism, c. 2nd century BC

A machine can do more than move.

It can contain knowledge unavailable to its operators.

The mythical Argo gives us an extraordinary example.

The Word Becomes Agent

The%20shem%20placed%20in%20the%20Golem's%20mouth

The shem placed in the Golem's mouth

The Golem introduces another extraordinary structure:

matter + symbolic information → agent

A Mythological Bestiary of Machines

Human — Can we manufacture ourselves?

Servant — Can we delegate labor?

Prosthesis — Can we extend ourselves?

Lover — Can we manufacture intimacy?

Protector — Can we delegate protection?

Weapon — Can artificial agency project violence?

Guardian — Can it decide who may pass?

Oracle — Can it know what we do not?

Mirror — What does it reveal about us?

Monster — What if it escapes our intentions?

In Our Image

Jean%20Raoux%2C%20Pygmalion%20amoureux%20de%20sa%20statue%20(1717)

Jean Raoux, Pygmalion amoureux de sa statue (1717)

We first imagined machines as versions of ourselves.

Then we built machines that extended parts of ourselves.

Eventually we built machines capable of modifying themselves through experience.

The artificial being is ancient.

The learning machine is the historical novelty.

Alles ist Zahl!

Pythagoras, Latin numerals, Invention of Zero, Leibniz, Decimal / Binary / Hexadecimal Codes, int8 / fp16 / bf16 &c.

Pythagoras's Claim

“All is number.” (πάντα αριθμός) — attributed to Pythagoras, 6th century BCE.

For Pythagoras and his followers, numbers were not just a tool for counting. They were the hidden structure of reality itself — the ratios that govern music, geometry, even the harmony of the cosmos.

Today we trace that claim all the way to the silicon running inside your phone.

Why It Mattered

Here is the thread we will follow: a number can be written down in many different ways — and the way you write it changes what you can do with it.

That was true for Roman numerals. It is just as true for the floating-point formats inside a modern AI chip.

Counting Before Zero

Rome counted like this: I, V, X, L, C, D, M.

No zero. No place value — the position of a symbol barely matters, only which symbols appear and in what combination.

Try multiplying MDCLXVI by XLIV in your head. Now try 1666 × 44. The second one is not just easier — it is a genuinely different kind of thinking.

An Older Theorem

Centuries before Pythagoras, the Baudhayana Sulba Sutra (India, c. 800 BCE) already states the same relationship between the sides of a right triangle.

The mathematics was not a single European discovery — it was discovered, independently and repeatedly, wherever people needed to measure, build, and trade.

The Missing Symbol

Every numeral system so far shares the same gap: no symbol for nothing.

Without a zero, there is no clean way to write a number as a position-dependent code — and without position, arithmetic stays a craft for specialists, not a procedure anyone can follow.

The Invention of Zero

Zero emerges independently more than once: among the Maya of Mesoamerica, and in India, where the mathematician Brahmagupta (7th century CE) is the first to give it formal rules of arithmetic.

It reaches the Islamic world through scholars like Muhammad ibn Musa al-Khwarizmi (9th century), whose own name — Latinized as algorismus — gives us the word algorithm.

A number system and a method for calculating with it arrive together. That is not a coincidence.

Zero Reaches Europe

In 1202, Leonardo of Pisa — better known as Fibonacci — publishes Liber Abaci, introducing Hindu-Arabic numerals (zero included) to European merchants.

The pitch was practical: this notation makes bookkeeping and trade dramatically easier than the Roman system merchants were already using.

Positional Notation

Here is the real leap: the same ten symbols, re-used, mean something different depending on where they sit.

106 and 160 use identical digits. Position alone carries the meaning.

This single idea — a small, finite alphabet whose meaning depends on position — is the one that modern computing was built on top of.

Leibniz's Dream

Gottfried Wilhelm Leibniz takes positional notation somewhere further: in 1703, he formally describes arithmetic in base 2 — binary.

He was reportedly delighted to find the same two-symbol logic echoed in the hexagrams of the I Ching, and saw in binary something close to a theological image: everything created from nothing (0) and unity (1).

Three centuries later, his once-mystical number system runs every computer on Earth.

Why Base Two?

Binary is just another positional system — base 2 instead of base 10. Only two symbols: 0 and 1.

That turns out to be the whole point. Building a reliable physical device with ten stable, distinguishable states is hard. Building one with two — voltage on, voltage off — is comparatively easy, cheap, and extremely reliable at enormous scale.

Every transistor in every chip is, at heart, a two-state switch pretending to be one of Leibniz's digits.

From Numbers to Logic

George Boole (1854) notices that 0 and 1 can represent more than quantity — they can represent truth: false and true.

Boolean algebra — AND, OR, NOT — becomes the logical skeleton underneath every circuit in every digital machine.

The same two symbols now carry both arithmetic and logic. That overlap is why a computer can calculate and also decide.

Bits, Bytes, and Beyond

One binary digit is a bit. Group eight of them and you get a byte — 256 possible values.

Hexadecimal (base 16) exists purely as a convenience: one hex digit covers exactly 4 bits, so two hex digits describe a whole byte cleanly — far easier for a human to read than a string of eight zeros and ones.

You already used one today, probably without noticing: a colour code, a MAC address, a cryptographic key — all hexadecimal shorthand for binary underneath.

A New Question

So far: how a culture chooses to write a number. Now: how a machine chooses to store one — and why, in a modern AI system, that choice is no longer a historical curiosity.

It is an engineering decision with real, measurable consequences.

Integers vs Floats

Integer formats store whole numbers exactly. int8: one byte, 256 distinct values — typically -128 to 127.

Floating-point formats trade some of that exactness for enormous range: a sign bit, an exponent, and a mantissa (the significant digits) — the same idea as scientific notation, just in binary.

More bits generally buys you more precision, more range, or both. Fewer bits buys you speed and memory — at a cost.

f16 vs bf16

Both fp16 and bf16 use 16 bits total — half the size of ordinary fp32 — but split those bits differently.

fp16 (IEEE half precision): 1 sign + 5 exponent + 10 mantissa bits. More precision, a narrower range — values that are too large or too small simply overflow to infinity or zero.

bf16 ("brain float", from Google): 1 sign + 8 exponent + 7 mantissa bits. The same exponent range as full 32-bit floats, less precision — far more forgiving during the unstable arithmetic of training a neural network.

Same bit budget. Different trade-off between "how exact" and "how big."

Why int8 for Inference

Models are usually trained in fp16, bf16, or fp32 — the precision that stability during training needs.

Once trained, they are often quantized down to int8 for inference (actually running the model): roughly a quarter of the memory of fp32, and hardware built for int8 can push through far more operations per second than the same silicon running floating point.

The accuracy cost is usually small. The speed and energy savings, at the scale a real deployment runs at, are not.

Case Study: NVIDIA Jetson Orin

Take a real, current piece of edge-AI hardware: NVIDIA's Jetson AGX Orin (64GB module).

NVIDIA's own published figure: up to 275 TOPS (trillion operations per second) running INT8, with structured sparsity.

Switch the same chip to FP16 / BF16 and achievable throughput drops — roughly by half, since each floating-point operation occupies more of the tensor core's datapath than an 8-bit integer one.

Same transistors. Same chip. The number format you choose to compute in changes your real-world throughput by a factor of two or more.

Closing: Same Question, New Stakes

Pythagoras claimed number was the hidden structure of reality.

Three thousand years later, the question "what exactly is this number, and how many bits does it cost to represent it" is no longer philosophy — it decides how fast your model runs, how much it costs to train, and how much energy it burns doing so.

The history of numerals is, in a very literal sense, the history of computation.

Unplugged #0

Cogs, Steam, Algorithm

In which the reader is asked, quite simply, What is an Algorithm? — and finds the answer rather older, and rather stranger, than expected.

Being an account of cogs and looms and engines of brass; of Signor Pascal's calculating box and Mr. Babbage's unbuilt Engines; of a certain Lady Lovelace who saw further than most; and of sundry other contrivances got up by ingenious persons long before any machine was called a computer.

Wherein two short films are watched, in the company of a partner to be consulted not once but twice; wherein old Euclid's method is recalled, some two thousand years and more since first he wrote it down; and wherein, at the last, the curious Persian origin of a word now spoken a thousand times a day — one al-Khwarizmi's own — is at length discovered.

Question of the Day

Question of the day: What is an algorithm?

You already use the word casually — "the algorithm" recommends your videos, "the algorithm" decides your feed. But what, precisely, is one?

By the end of today's session you'll have a real definition — and you'll know where the word itself came from.

First Thoughts

Turn to the person next to you. Two minutes, no googling.

Together, try to answer: what is an algorithm? Give one example each, if you can.

Don't worry about being "right" — we'll come back to this twice more today, and compare notes.

Watch: Early Computing

Before the word, there were the machines. Crash Course Computer Science #1: Early Computing — from the abacus to Herman Hollerith's punch cards.

What Did You Notice?

What surprised you?

Notice how many of these — the abacus, Pascal's calculator, the Jacquard loom, Babbage's engines — are not computers in our sense at all. They're physical machines for carrying out a fixed, repeatable procedure.

That procedure, independent of whatever metal or wood or silicon carries it out, is the thing we're actually interested in today.

From Cogs to Concept

Strip away the brass gears, the steam, the punch cards — what's left in each of those machines is the same abstract thing: a precise, repeatable sequence of steps that turns some input into some output.

That sequence of steps is an algorithm. The machine is just one possible body for it to live in.

A Working Definition

An algorithm is a finite, well-defined sequence of steps that transforms a given input into a desired output.

Finite — it has to actually end, eventually.

Well-defined — each step is unambiguous; anyone, or anything, following it correctly gets the same result.

Input / output — it does something to something.

An Everyday Algorithm

You run algorithms all the time without a computer in sight.

A recipe: fixed steps, specific ingredients (input), a finished dish (output).

Long division, tying your shoelaces, the instructions for assembling furniture — all algorithms, in the strict sense.

Computers didn't invent algorithms. They just got extremely good at running them fast.

Older Than Computers

Euclid described an algorithm for finding the greatest common divisor of two numbers around 300 BCE — still taught today, still called Euclid's algorithm.

Long division as you learned it in school is centuries old. Knitting patterns are algorithms. So is a court protocol, or a recipe passed down for generations.

The concept is ancient. Only the name is comparatively recent — and it has a surprisingly specific origin.

Second Thoughts

Back to your partner. Compare: how close was your first guess to the definition we just built together?

This time, try to name an algorithm from your own daily life that has nothing to do with computers.

Where Does the Word Come From?

"Algorithm" doesn't come from Greek, like most technical vocabulary in mathematics. It comes from a name.

Whose?

Watch: Why Algorithms Are Called Algorithms

A short answer, in under three minutes — BBC Ideas: Why algorithms are called algorithms.

Al-Khwarizmi and the Birth of a Word

Muhammad ibn Musa al-Khwarizmi, a 9th-century Persian scholar working in Baghdad's House of Wisdom, wrote a treatise on systematic methods for solving equations.

His name, Latinized centuries later as "algorithmi," became attached to the step-by-step methods he described — and eventually to the general concept itself.

(His other book's title gave us a second word: al-jabr — "algebra.")

Interlude: Pascal, Leibniz, and the Baroque

Rewind two centuries. Long before Babbage, long before steam, the same ancient appetite for mechanizing the mind was already stirring — in the Baroque.

In 1642, at nineteen, Blaise Pascal built the Pascaline to help his father with tax calculations — a gear-driven adding machine, among the first of its kind ever built and sold.

A generation later, Gottfried Wilhelm Leibniz — one of the last true polymaths, equally at home in philosophy, law, and mathematics — went further still, designing the Stepped Reckoner: a machine that could multiply and divide, not merely add and subtract.

Leibniz wanted more than a calculator. He dreamed of a calculus ratiocinator — a universal symbolic method by which any dispute, of any kind, could in principle be settled by calculation alone: "Let us calculate."

Cogs and steam would eventually belong to Babbage's century. But the dream — that reasoning itself might be mechanized — is, properly speaking, Baroque.

Steam and the Jacquard Loom

By the early 1800s, the Industrial Revolution had already proven something remarkable: steam and gearwork could replace human muscle at scale.

Joseph-Marie Jacquard's loom (1801) went one step further — it replaced not just labor, but a kind of decision-making. A chain of punched cards, read mechanically, told the loom exactly which threads to raise for every single pass of the shuttle.

Change the cards, change the pattern. The procedure — the algorithm — was now separate from the machine that executed it, encoded on something you could hold in your hands.

Babbage noticed. He planned to use these very same punched cards to feed instructions into his own Analytical Engine.

Charles Babbage's Engines

Charles Babbage (1791–1871) — mathematician, philosopher, mechanical engineer, and by most accounts a thoroughly restless Victorian gentleman.

His Difference Engine was a mechanical calculator, designed to compute polynomial values automatically — and to print the results, removing human error from mathematical tables.

His far more ambitious Analytical Engine was something else entirely: a general-purpose, programmable computer — a century before electronics made one practical. It had a "mill" (processor), a "store" (memory), and was to be fed instructions on Jacquard-style punched cards.

He never finished building it. But he had, in effect, designed a computer.

Ada Lovelace, the First Programmer

Augusta Ada King, Countess of Lovelace (1815–1852), daughter of Lord Byron, was trained rigorously in mathematics from childhood — and became Babbage's closest intellectual collaborator.

In 1843, translating an Italian paper on the Analytical Engine, she added her own notes — three times longer than the original text. Among them: a step-by-step method for the Engine to compute Bernoulli numbers.

That method is widely considered the first published algorithm written specifically for a machine — making Lovelace, by most accounts, the first computer programmer, over a century before there was a computer to run it on.

She also saw further than Babbage himself: the Engine, she wrote, might one day manipulate not just numbers, but music and symbols of any kind. Others called her the Enchantress of Numbers.

One More Name: George Boole

One more figure worth knowing, even briefly: George Boole (1815–1864), who in 1854 published The Laws of Thought — reducing logical reasoning itself to a kind of algebra, with only two values: true and false.

Boolean algebra sat unused for decades as a philosophical curiosity. In the 20th century it became the literal switching logic inside every digital circuit ever built.

Babbage's engine, Lovelace's program, Boole's logic — three separate threads in the 1800s, waiting nearly a hundred years for electronics to braid them together. We'll pick that thread up again later in this course.

Closing: What's Next

Today: a definition, a history, a name.

Next week: Alan and his Enigma — the story of Alan Turing, his machine(s), and his test(s).

We'll meet the Turing machine that finally gave "algorithm" a single, rigorous, universal meaning — and ask what happens when the body running one never gets tired, never gets bored, and never stops.

Alan and his Enigma

The story of Alan Matthison Turing, his machine(s) and his test(s)

Watch: Alan Turing's Machines

We start with a 1966 BBC programme, made twelve years after his death.

While you watch, keep one question in mind: what is this “machine” made of?

Alan Matthison Turing (1912–1954)

Born in London, 23 June 1912. Mathematician: Cambridge, then Princeton.

1936, aged 23: On Computable Numbers — the paper that defines the computer before any computer exists.

1939–45: Bletchley Park, breaking German naval Enigma.

1945–50: designs an electronic computer; 1950: Computing Machinery and Intelligence.

1952: convicted for homosexual acts, chemically castrated. 7 June 1954: dead of cyanide poisoning, aged 41. Royal pardon: 2013.

The man who asked whether machines can think was not allowed to be what he was.

The imitation game

The game as Turing wrote it

Turing refuses to define “think”. He calls the question “too meaningless to deserve discussion” and replaces it with a game.

Three players: A (a man), B (a woman), C (the interrogator, in another room). C types questions and must tell who is the woman. A tries to deceive. B tries to help.

Now let a machine play A. Does C get it wrong as often as before?

His prediction: around the year 2000, an average interrogator would have no more than a 70% chance of a right identification after five minutes.

The game is called imitation — and it is about gender, deception and performance.

Turing test

"A TT is a way how to address the question "Can machines think?" in a scientifically plausible ye deeply empathic way. More concretely, Turing (1950) proposes that the performance of an AA under question shall be evaluated by a human judge J whose objective is to determine which among two entities - with whom J is in real-time interaction - is of human and which is of artificial nature." (Hromada, 2012)

Watch: Can machines think?

The famous question, in video form.

While watching, note the one objection you would raise.

Turing answers his critics

Section 6 of the paper lists nine objections, among them theological, mathematical, from consciousness — and Lady Lovelace's.

Lovelace (1843): the Analytical Engine “has no pretensions to originate anything. It can do whatever we know how to order it to perform.”

Turing: machines surprise me all the time. And instead of programming an adult mind, build a child machine and educate it.

Keep that sentence. It is the first mention of a learning machine.

Does the test still work?

ELIZA (Weizenbaum, 1966): a few pattern rules, and some users confided in it anyway.

The Chinese Room (Searle, 1980): symbols shuffled correctly, nothing understood.

Recent controlled studies report that large language models are judged to be human about as often as, or more often than, real humans (Jones & Bergen).

The test measures our credulity as much as the machine's intelligence.

Exercise · Meet Alan

Alan is a Matrix bot (@alan:udk.ai) running on our own server. You will chat with “Alan” — but behind him may be an AI persona or another student. Alan relays every message, so you are never told which is which.

How to reach him: in Element (with your account on udk.ai or medienhaus.udk-berlin.de) start a direct message with @alan:udk.ai and type start.

Alan pairs you with a waiting fellow student or with an AI persona. After a few minutes he asks: “Am I a human or AI?” Then comes the reveal: which persona (and model) it was — or, if you spoke with a human, who.

Good to know: Alan cannot read encrypted chats, so use a normal unencrypted DM (if you invite him to an encrypted one, he leaves and opens a fresh DM with you). The conversations are stored for the course evaluation — don't share anything private.

Try Turing's own questions: “Please write me a sonnet on the Forth Bridge.” “Add 34957 to 70764.”

Decide: human or machine? Write down the three cues that decided it.

Turing's machine answered 105621. The right answer is 105721.

Debrief · What gave him away?

Which cues did you use? Speed? Spelling? Politeness? Never saying “I don't know”? A body, a weather, a bad day?

Humans make typos and slip. Turing wrote that a machine should deliberately introduce mistakes to be convincing.

Which cues tell you about the machine, and which only about your expectations of humans?

Back to your criterion from the Question of the Day: did Alan meet it?

Enigma

The Imitation Game (2014): the interview scene

Enigma: a German cipher machine. Press a key, a different letter lights up — and the rotors turn, so the next key is scrambled differently.

Bletchley Park: Turing designs the Bombe, a machine that hunts for the day's settings (building on Polish work).

A machine breaking a machine.

And enigma is also what we call a riddle. Keep both meanings in mind.

The question Hilbert asked

1928, David Hilbert: is there a mechanical procedure that decides, for any mathematical statement, whether it is provable?

Leibniz had dreamed of it: let us calculate.

1931, Kurt Gödel: every consistent formal system has true statements it cannot prove.

1936, Alonzo Church and Alan Turing, independently: no such procedure exists.

But to prove that nothing can be done, you first need to say exactly what “a procedure” is.

Computers were people

Until the 1940s a “computer” was a person who computes — often a woman with pencil and paper at a desk.

Turing asks: what does such a person actually do?

Looks at a symbol. Writes a symbol. Changes their state of mind. Moves on to the next square.

Strip away everything else:

a tape, a head, a finite list of rules.

a Turing machine #1

A "theoretical" machine consisting of
  • tape with discrete symbols
  • reading/writing header
  • table of rules
  • tape-moving mechanism
Visualization of the Turing Machine (Rosario van Tulpe, 2007, CC BY-SA)

Exercise · Be the machine

The tape holds a binary number: 1 0 1 1. The head stands on the rightmost digit. State: CARRY.

CARRY reads 1 → write 0, move left, stay in CARRY

CARRY reads 0 or blank → write 1, STOP

Run it with pencil and paper. What is on the tape when it stops? Now try 1 1 1.

It adds one: 1011 → 1100, 111 → 1000. Three rules, no understanding.

Anything you would call an algorithm can be written as such a table (Church–Turing thesis).

Exercise · Your hand is a Turing machine

Counting from 0 to 31 on the fingers of one hand is a Turing machine at work. Here is its anatomy:

Tape — your five fingers: thumb = rightmost cell, little finger = leftmost. Symbols — 1 finger up, 0 finger down.

Head — the finger you are looking at; it starts on the thumb. States — CARRY (start) and HALT.

Transition rules
CARRY reads 1 → write 0 (fold it down), move left, stay in CARRY
CARRY reads 0 → write 1 (raise it), HALT

One run = one count. Take 7 → 8: tape 0 0 1 1 1. Thumb 1→0, index 1→0, middle 1→0, ring 0→1, halt: 0 1 0 0 0.

Counting to 31 means starting the machine 31 times (57 steps in all). Same two rules every time; nothing in it “knows” what a number is. Adding two numbers on two hands is the same idea with a bigger rule table.

Now count up from 31: the head falls off the left end. Turing's tape is infinite; your hand is not. Borrow your other hand — or paper — and keep going.

Universal Turing machine

"It is possible to invent a single machine which can be used to compute any computable sequence. If this machine U is supplied with a tape on the beginning of which is written the S.D ["standard description" of an action table] of some computing machine M, then U will compute the same sequence as M." (Turing, 1936)

The tape can hold a machine

A universal machine reads the rules of another machine from its tape and behaves like it.

Program = data. Your laptop is such a machine; an app is just a long tape.

And a program that treats programs as data can also change them. Hold on to this for the learning machines.

Halting Problem

  • halting problem is the problem of determining, from a description of an arbitrary computer program and an input, whether the program will finish running (i.e., halt) or continue to run forever
  • Turing proved that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist
  • to prove this, he described a paradox involving a concept now known as "Turing machine"

Halting · the trick in four lines

Suppose a program H(p, x) always answers correctly: “p halts on x” or “p loops forever”.

Build D(p): ask H(p, p). If it says halts, loop forever. If it says loops, stop.

Run D on itself. Halts if and only if it loops.

Contradiction: H cannot exist. Some questions about programs no program can answer.

Question of the Day · Can machines think?

Turn to your neighbour. Two minutes.

What would a machine have to do for you to say “yes, it thinks”?

Write the criterion down.

We will test it at the end of the session.

philosopher's definition of "thinking"

“We may say that thinking is essentially the activity of operating with signs.”

— Ludwig Wittgenstein, The Blue Book (1933–1934), p. 6 of the commonly used edition

Computational machinery and intelligence (1950)

The enigma remains

Turing machine: what can be computed — defined by what a procedure does, not what it is made of.

Turing test: what we will call intelligent — defined by how it behaves toward us.

The question moves from “what is it?” to “what does it do with us?”

Next: Claude, Marvin, John &c. — from information theory to Dartmouth.

Logic, Gates, Machine

George%20Boole%20(1815%E2%80%931864)

George Boole (1815–1864)

In which the reader is made acquainted with a most singular Algebra, wherein there are but two numbers, and the sum of one and one is nothing at all.

Being an account of Mr. Boole’s Laws of Thought, set down in the year 1854 by a Professor of Queen’s College, Cork; of young Mr. Shannon’s relays, and the little gates that came of them; and of how one humble operation — the adding of One to a number — may be performed in three several fashions: by Logic, by Wire, and by the Machine of Mr. Turing.

Wherein the reader is required to lend a thumb, and to consult a partner upon sundry occasions; wherein it is demonstrated that the Carry may very conveniently be kept upon the thumb of the other hand; and wherein, at the last, Logic, Circuit and Machine are discovered to be one and the same contrivance in three different coats.

philosopher's definition of "thinking"

“We may say that thinking is essentially the activity of operating with signs.”

— Ludwig Wittgenstein, The Blue Book (1933–1934), p. 6 of the commonly used edition

Who Does the Thinking?

It is misleading then to talk of thinking as of a “mental activity”. We may say that thinking is essentially the activity of operating with signs. This activity is performed by the hand, when we think by writing; by the mouth and larynx, when we think by speaking; and, if we think by imagining signs or pictures I can give you no agent that thinks.
If then you say that in such cases the mind thinks, I would only draw your attention to the fact that you are using a metaphor, that here the mind is an agent in a different sense from that in which the hand can be said to be the agent in writing.
If again we talk about the locality where thinking takes place we have a right to say that this locality is the paper on which we write; or the mouth which speaks.

— Ludwig Wittgenstein, The Blue Book (1933–1934)

Keep this in mind today: in our exercises, whose hand does the thinking, and where is the carry kept?

Question of the Day

Question of the day: why does counting in binary work — and what is a machine doing when it counts?

You already know how: fold every 1 down, carry to the left, raise the first 0. You did it with a hand and with a Turing machine.

Today we ask why. One algorithm — add 1 — seen in three languages: Logic (Boole), Wire (Shannon), Machine (Turing).

Bring a thumb. Bring a partner. Both will be needed.

Warm-up · Count Together

In pairs. One of you counts in binary on one hand, from 0 up to 11. The other watches and says “stop!” at the first mistake.

Swap. Then count from 0 1 1 1 (7) — what happens, and who remembers the carry while the fingers fold down?

Keep that question. The rest of the session is its answer.

Two Values

Title%20page%20of%20Boole%26rsquo%3Bs%20Laws%20of%20Thought%2C%20London%201854

Title page of Boole’s Laws of Thought, London 1854

1854: George Boole, professor at Queen’s College Cork, publishes An Investigation of the Laws of Thought.

His claim: reasoning itself can be calculated — like algebra — if every statement is reduced to just two values: 1 (true) and 0 (false).

Leibniz’s old dream (calculemus!), now with rules.

A thumb is a perfect Boolean variable: up = 1, down = 0.

Three Operations

NOT flips: NOT 1 = 0, NOT 0 = 1.

AND: true only if both are true (both thumbs up).

OR: true if at least one is true.

aba AND ba OR b
0000
0101
1001
1111

The Fourth: XOR

XOR (exclusive or): true if the two inputs differ — one thumb up, not both.

aba XOR b
000
011
101
110

Note the last row: 1 XOR 1 = 0. Remember it — it will carry the whole session.

Which “or” Do You Mean?

In everyday speech “or” is ambiguous. Logic must not be.

sentencereading
“Soup or salad?” (the menu)XOR — not both (unless you pay extra)
“Smoke or heat sets off the alarm.”OR — and certainly if both
“Are you leaving today or tomorrow?”XOR — you cannot leave twice
“Bring a note from your doctor or your parents.”OR — two notes are no problem
“Milk or sugar in your coffee?”unclear! Both, surely?

Latin had two words for it: vel (inclusive: “one or the other or both”) and aut (exclusive: aut Caesar aut nihil). English has just one — and relies on you to guess.

Ambiguity by Design

Lawyers and programmers cannot afford to guess:

Contracts write “and/or” (inclusive) and “either … or …, but not both” (exclusive), because courts have argued over a bare “or”.

Programming languages give the two meanings different symbols: a || b (or) and a ^ b (xor).

Linguists note that the exclusive reading is often not part of the word at all: we infer “not both” (“if she could say ‘and’, she would have”).

This is Boole’s move: give each meaning its own symbol, and the ambiguity disappears — for a machine, there is no “you know what I mean”.

Exercise · Or or Xor?

In pairs. Take turns reading these sentences aloud; your partner decides OR or XOR and defends the choice with one sentence about the situation:

• “You can pay by card or cash.” • “Students or staff may enter.” • “A fine of fifty euros or three days in jail.” • “She speaks German or French.” • “Take the lift or the stairs.”

Which ones did you disagree on? Notice what you used to decide: not the word “or”, but knowledge of the world.

Now write one real sentence of your own in which the two readings lead to different consequences — a rule, a promise, a threat. Swap, and rewrite your partner’s so it can only be read one way.

Exercise · Thumb Logic

In pairs. Each of you hides a thumb behind your back (up = 1, down = 0) and shows it on three. The teacher calls the gate:

abANDORXORNAND
000001
010111
100111
111100

Show thumbs. As a pair, say the output before checking the table — then check it.

Reverse round: the teacher says gate and output (“XOR gives 1”); you show a pair of thumbs that produces it. Is the answer always unique? For which calls is there only one?

Laws of Algebra

Boole discovered that these operations obey laws, like numbers do:

lawin words
a AND 1 = aAND with 1 changes nothing
a AND 0 = 0AND with 0 kills everything
a OR 0 = aOR with 0 changes nothing
a AND a = arepeating yourself adds nothing
a AND (NOT a) = 0nothing is both true and false
NOT(a AND b) = (NOT a) OR (NOT b)De Morgan (Boole’s contemporary)

Laws like these let you simplify circuits and programs by pure calculation — no thumbs needed.

Algebra, Really

Why is it called algebra? Because the operations behave like arithmetic on 0 and 1:

AND is multiplication: a · b. NOT is 1 − a.

XOR is addition — but one that forgets the carry (addition modulo 2): 1 + 1 = 0.

Add two single bits properly and look at the two digits of the answer:

a + b= carry, sumcarrysum
0 + 00000
0 + 10101
1 + 00101
1 + 11010

The sum digit is XOR. The carry digit is AND. Arithmetic is logic.

A Relay Is a Logical Switch

AND%2C%20OR%2C%20NOT%2C%20XOR%2C%20NAND

AND, OR, NOT, XOR, NAND

1937: Claude Shannon, 21, writes his master’s thesis at MIT: A Symbolic Analysis of Relay and Switching Circuits.

His discovery: electrical switches obey Boole’s algebra. Two switches in series = AND (current flows only if both are closed). In parallel = OR. A relay that opens when energised = NOT.

So any Boolean formula can be built out of hardware — and any circuit can be checked by algebra.

The little symbols for such components are called gates:

NAND Is Enough

One gate is enough to build all the others: NAND (NOT(a AND b)), Sheffer 1913.

NOT a = a NAND a

a AND b = NOT (a NAND b) — i.e. a NAND followed by a NAND with itself.

a OR b = (NOT a) NAND (NOT b) — De Morgan at work.

A NAND gate in a chip costs four transistors. Billions of them, wired by algebra, are every computer you know.

Exercise · Build It from NAND

In pairs. One of you is the architect, the other the inspector.

Architect: on paper, draw XOR using only NAND gates (hint: four NANDs suffice; one of them is shared).

Inspector: feed all four input pairs through the drawing with your thumbs — one gate at a time, out loud. Compare with the XOR table.

If a row fails, find which gate lied. Then swap roles and build OR.

The Half-Adder

The%20half-adder%3A%20XOR%20gives%20the%20sum%20digit%2C%20AND%20gives%20the%20carry

The half-adder: XOR gives the sum digit, AND gives the carry

Put the two observations together: to add two bits b and c you need exactly two gates.

sum = b XOR c   carry = b AND c

This tiny circuit is called the half-adder. All arithmetic in your phone is built from it and its relatives.

Adding One Is Adding a Carry

Our algorithm increment is not a special case: to add 1, put a carry of 1 into the lowest position, then let every position run a half-adder.

c0 = 1 (that is the “+1”)

for every bit bi, from right to left:
outi = bi XOR ci
ci+1 = bi AND ci

The lowest bit has it easiest: b XOR 1 = NOT b — it always flips. Every higher bit flips only if the carry reached it — and the carry survives only through a chain of 1s.

Exercise · Paper Circuit

In pairs. Four numbers: 0101, 0111, 1011, 1111. Increment each using only the two formulas — no counting.

Split the labour like a circuit: one of you computes every XOR, the other every AND and keeps the carry. Fill in a table with columns b, c, out, c′ for each position, right to left.

Check against finger counting. Which number made the carry travel farthest? What is c after the last position of 1111?

Space: A Chain of Gates

Five%20half-adders%3A%20the%20algorithm%20laid%20out%20in%20space

Five half-adders: the algorithm laid out in space

Build the circuit literally: one half-adder per bit, the carry wired from each to its left neighbour.

It is called a ripple-carry circuit: change the input and the carry ripples through the chain like a row of dominoes.

A 64-bit number needs 64 adders. The algorithm is spread out in space.

Time: One Gate Pair, Many Steps

The%20same%20algorithm%20laid%20out%20in%20time

The same algorithm laid out in time

Now the other idea. Do not build five adders. Build one and walk it along the number.

Each step it reads one bit from a tape, computes XOR and AND, writes the result, and moves one cell to the left.

But then the carry from this step must wait for the next one. Something must remember it. That memory is the machine’s state.

The Thumb Is the State

Look again at the rule table of our Turing machine — the one from the Alan session — and write the state as a bit: CARRY = 1, HALT = 0.

state creads bwritesnext statemove
1 (CARRY)010 (HALT)left
1 (CARRY)101 (CARRY)left
0 (HALT)000—
0 (HALT)110—

Writes = b XOR c. Next state = b AND c. The rule table of the machine is the half-adder’s truth table.

A Turing machine is a half-adder with a memory and a moving head. The carry is the thumb of the other hand.

Exercise · Two Hands, One Machine

Alone, then compare. Right hand = tape (thumb is the lowest bit). Left thumb = the state c.

Raise your left thumb: you are adding 1. Now walk along the right hand, thumb to little finger. At every finger say out loud: “out = b XOR c” (set the finger to that value) and “c = b AND c” (set your left thumb to that).

Once the left thumb is down you may stop — nothing further can change.

Count from 0 to 31 this way. Then 31 + 1: the left thumb is still up after the little finger. That is the carry that fell off the tape — overflow. Turing’s tape is infinite; your hand is not.

Exercise · Human Turing Machine

In pairs. Two roles. TAPE: write six cells on paper, e.g. 0 1 0 1 1 1, and cover all but the cell under the head with your hand. MACHINE: you have only a rule card (the table of the previous folio) and your left thumb as the state. You may never see the whole number.

Loop: MACHINE says “read!” — TAPE reveals the bit under the head. MACHINE looks up the row and says “write w, move left”, updating the thumb. TAPE writes and moves the head one cell left. Stop when the thumb is down.

Run 010111, 001111, 111111. Convert to decimal to check. What happens in the last run when the head falls off the tape?

Swap roles. Afterwards discuss: did either of you ever need to know what number it was? Where, physically, was the carry stored?

Exercise · Break the Machine

In pairs. First: the forgetful machine. It has no thumb: at every cell it behaves as if c = 1 (it always flips the bit and always moves on).

Run it on 0101. What should the answer be? Which inputs does the forgetful machine happen to get right, among the sixteen four-bit numbers?

Then the challenge: run it backwards. Design a rule table for subtract 1 (state BORROW) — hint: it is the increment table with the roles of 0 and 1 exchanged. Test it by counting down from 8 on your hands.

Check yourselves: BORROW reads 0 → write 1, move left, stay in BORROW; BORROW reads 1 → write 0, HALT. In algebra: out = b XOR w, w′ = (NOT b) AND w. (And the forgetful machine is right on 0111 and 1111 only.)

Three Languages, One Algorithm

Logic (Boole 1854)Wire (Shannon 1937)Machine (Turing 1936)
what it isformulascircuit of gatestape, head, states
sum digitout = b XOR cXOR gatewrite rule
carryc′ = b AND cAND gate + wirenext state (thumb)
many bitssame formula for each ia chain: spaceone gate pair, repeated: time

Same algorithm, three notations. Algebra says what, gates say with what, the machine says in what order.

And the translation between them is mechanical — which is exactly why computers can exist.

Why Does a Machine That Learns Care?

Every neural network, every language model, runs on chips made of billions of gates just like the ones you drew today — XOR, AND, NAND, wired by algebra and clocked through time.

When people say “the machine computes”, this is all it means: logic, laid out in space, unrolled in time.

Learning does not change that. It changes only which numbers sit in the tape. Next time: how those numbers came to be tuned — Claude, Marvin, John and the others.

Debrief

In pairs, three minutes: explain to each other, without notes, why the sum digit is XOR and the carry is AND.

Then answer the question of the day in two sentences: why does counting in binary work, and what is a machine doing when it counts?

Write down where your thumb was needed. That is the whole difference between a calculator and a table of formulas: memory.

Feedback · Checkout Time

Is there something you like about this seminar?

Is there something you don’t like?

Is there something new and useful you learned during this seminar?

Claude, Marvin, John &c.

From information theory to Darmouth conference

Machine Learning #0

Features, Classifiers, Decision Trees, Random Forests, Training & Testing, 

The Latent Space

N-dimensional Spaces, Semantic Spaces, Feature Spaces, Embeddings

Unplugged #1

The Omega Point

The Chrismas talk on Noosphere, Datasphere, Infosphere, Singularity, Novacene &c.

Language models

Large & Small Language Models

Artificial Agents

Agentic AI

Central Problem of AI ethics

Central Problem of Roboethics, Alignment

Human-Machine Peer Learning

Humans and machines can learn together
Machines and humans can learn with each other