i
Short History of the Learning Machine daniel-hromada ()


Short History of the Learning Machine

Between Archetypes & Myths: Machine as a Projection of the Human Self

Between Archetypes & Myths

Machine%20as%20a%20projection%20of%20the%20human%20self

Machine as a projection of the human self

Machine as a projection of the human self

Faciamus machinam ad imaginem et similitudinem nostram.

Let us make the machine in our image and likeness.

What Is a Machine?

Hero%20of%20Alexandria%2C%201st%20century%20AD

Hero of Alexandria, 1st century AD

Before machines became motors, engines and computers, the word already meant something broader:

an ingenious means by which something becomes possible.

The Divine Workshop

Hephaestus%20at%20the%20forge%2C%20with%20Pandora's%20urn%20and%20a%20bronze%20guardian

Hephaestus at the forge, with Pandora's urn and a bronze guardian

Greek mythology gives us an extraordinary laboratory of artificial beings.

Athena represents intelligent making.

Hephaestus represents the workshop in which artifacts begin to act.

Machines in Our Image

Jean%20Regnault%2C%20Pygmalion%20priant%20V%C3%A9nus%20d'animer%20sa%20statue%20(1786)

Jean Regnault, Pygmalion priant Vénus d'animer sa statue (1786)

Artificial beings are not merely tools.

They become surfaces onto which humans project desire, identity and self-understanding.

Machines at the Threshold

Some artificial beings do not imitate us primarily as workers or companions.

They stand between inside and outside.

They decide who may approach, enter or survive.

Machine as Oracle

The%20Antikythera%20mechanism%2C%20c.%202nd%20century%20BC

The Antikythera mechanism, c. 2nd century BC

A machine can do more than move.

It can contain knowledge unavailable to its operators.

The mythical Argo gives us an extraordinary example.

The Word Becomes Agent

The%20shem%20placed%20in%20the%20Golem's%20mouth

The shem placed in the Golem's mouth

The Golem introduces another extraordinary structure:

matter + symbolic information → agent

A Mythological Bestiary of Machines

Human — Can we manufacture ourselves?

Servant — Can we delegate labor?

Prosthesis — Can we extend ourselves?

Lover — Can we manufacture intimacy?

Protector — Can we delegate protection?

Weapon — Can artificial agency project violence?

Guardian — Can it decide who may pass?

Oracle — Can it know what we do not?

Mirror — What does it reveal about us?

Monster — What if it escapes our intentions?

In Our Image

Jean%20Raoux%2C%20Pygmalion%20amoureux%20de%20sa%20statue%20(1717)

Jean Raoux, Pygmalion amoureux de sa statue (1717)

We first imagined machines as versions of ourselves.

Then we built machines that extended parts of ourselves.

Eventually we built machines capable of modifying themselves through experience.

The artificial being is ancient.

The learning machine is the historical novelty.

Alles ist Zahl!

Pythagoras, Latin numerals, Invention of Zero, Leibniz, Decimal / Binary / Hexadecimal Codes, int8 / fp16 / bf16 &c.

Pythagoras's Claim

“All is number.” (πάντα αριθμός) — attributed to Pythagoras, 6th century BCE.

For Pythagoras and his followers, numbers were not just a tool for counting. They were the hidden structure of reality itself — the ratios that govern music, geometry, even the harmony of the cosmos.

Today we trace that claim all the way to the silicon running inside your phone.

Why It Mattered

Here is the thread we will follow: a number can be written down in many different ways — and the way you write it changes what you can do with it.

That was true for Roman numerals. It is just as true for the floating-point formats inside a modern AI chip.

Counting Before Zero

Rome counted like this: I, V, X, L, C, D, M.

No zero. No place value — the position of a symbol barely matters, only which symbols appear and in what combination.

Try multiplying MDCLXVI by XLIV in your head. Now try 1666 × 44. The second one is not just easier — it is a genuinely different kind of thinking.

An Older Theorem

Centuries before Pythagoras, the Baudhayana Sulba Sutra (India, c. 800 BCE) already states the same relationship between the sides of a right triangle.

The mathematics was not a single European discovery — it was discovered, independently and repeatedly, wherever people needed to measure, build, and trade.

The Missing Symbol

Every numeral system so far shares the same gap: no symbol for nothing.

Without a zero, there is no clean way to write a number as a position-dependent code — and without position, arithmetic stays a craft for specialists, not a procedure anyone can follow.

The Invention of Zero

Zero emerges independently more than once: among the Maya of Mesoamerica, and in India, where the mathematician Brahmagupta (7th century CE) is the first to give it formal rules of arithmetic.

It reaches the Islamic world through scholars like Muhammad ibn Musa al-Khwarizmi (9th century), whose own name — Latinized as algorismus — gives us the word algorithm.

A number system and a method for calculating with it arrive together. That is not a coincidence.

Zero Reaches Europe

In 1202, Leonardo of Pisa — better known as Fibonacci — publishes Liber Abaci, introducing Hindu-Arabic numerals (zero included) to European merchants.

The pitch was practical: this notation makes bookkeeping and trade dramatically easier than the Roman system merchants were already using.

Positional Notation

Here is the real leap: the same ten symbols, re-used, mean something different depending on where they sit.

106 and 160 use identical digits. Position alone carries the meaning.

This single idea — a small, finite alphabet whose meaning depends on position — is the one that modern computing was built on top of.

Leibniz's Dream

Gottfried Wilhelm Leibniz takes positional notation somewhere further: in 1703, he formally describes arithmetic in base 2 — binary.

He was reportedly delighted to find the same two-symbol logic echoed in the hexagrams of the I Ching, and saw in binary something close to a theological image: everything created from nothing (0) and unity (1).

Three centuries later, his once-mystical number system runs every computer on Earth.

Why Base Two?

Binary is just another positional system — base 2 instead of base 10. Only two symbols: 0 and 1.

That turns out to be the whole point. Building a reliable physical device with ten stable, distinguishable states is hard. Building one with two — voltage on, voltage off — is comparatively easy, cheap, and extremely reliable at enormous scale.

Every transistor in every chip is, at heart, a two-state switch pretending to be one of Leibniz's digits.

From Numbers to Logic

George Boole (1854) notices that 0 and 1 can represent more than quantity — they can represent truth: false and true.

Boolean algebra — AND, OR, NOT — becomes the logical skeleton underneath every circuit in every digital machine.

The same two symbols now carry both arithmetic and logic. That overlap is why a computer can calculate and also decide.

Bits, Bytes, and Beyond

One binary digit is a bit. Group eight of them and you get a byte — 256 possible values.

Hexadecimal (base 16) exists purely as a convenience: one hex digit covers exactly 4 bits, so two hex digits describe a whole byte cleanly — far easier for a human to read than a string of eight zeros and ones.

You already used one today, probably without noticing: a colour code, a MAC address, a cryptographic key — all hexadecimal shorthand for binary underneath.

A New Question

So far: how a culture chooses to write a number. Now: how a machine chooses to store one — and why, in a modern AI system, that choice is no longer a historical curiosity.

It is an engineering decision with real, measurable consequences.

Integers vs Floats

Integer formats store whole numbers exactly. int8: one byte, 256 distinct values — typically -128 to 127.

Floating-point formats trade some of that exactness for enormous range: a sign bit, an exponent, and a mantissa (the significant digits) — the same idea as scientific notation, just in binary.

More bits generally buys you more precision, more range, or both. Fewer bits buys you speed and memory — at a cost.

f16 vs bf16

Both fp16 and bf16 use 16 bits total — half the size of ordinary fp32 — but split those bits differently.

fp16 (IEEE half precision): 1 sign + 5 exponent + 10 mantissa bits. More precision, a narrower range — values that are too large or too small simply overflow to infinity or zero.

bf16 ("brain float", from Google): 1 sign + 8 exponent + 7 mantissa bits. The same exponent range as full 32-bit floats, less precision — far more forgiving during the unstable arithmetic of training a neural network.

Same bit budget. Different trade-off between "how exact" and "how big."

Why int8 for Inference

Models are usually trained in fp16, bf16, or fp32 — the precision that stability during training needs.

Once trained, they are often quantized down to int8 for inference (actually running the model): roughly a quarter of the memory of fp32, and hardware built for int8 can push through far more operations per second than the same silicon running floating point.

The accuracy cost is usually small. The speed and energy savings, at the scale a real deployment runs at, are not.

Case Study: NVIDIA Jetson Orin

Take a real, current piece of edge-AI hardware: NVIDIA's Jetson AGX Orin (64GB module).

NVIDIA's own published figure: up to 275 TOPS (trillion operations per second) running INT8, with structured sparsity.

Switch the same chip to FP16 / BF16 and achievable throughput drops — roughly by half, since each floating-point operation occupies more of the tensor core's datapath than an 8-bit integer one.

Same transistors. Same chip. The number format you choose to compute in changes your real-world throughput by a factor of two or more.

Closing: Same Question, New Stakes

Pythagoras claimed number was the hidden structure of reality.

Three thousand years later, the question "what exactly is this number, and how many bits does it cost to represent it" is no longer philosophy — it decides how fast your model runs, how much it costs to train, and how much energy it burns doing so.

The history of numerals is, in a very literal sense, the history of computation.

Unplugged #0

Cogs, Steam, Algorithm

Al-Khwarizmi, Fibonacci, Pascal, Babbage, Lovelace

Alan and his Enigma

The story of Alan Matthison Turing, his machine(s) and his test(s)

Claude, Marvin, John &c.

From information theory to Darmouth conference

Machine Learning #0

Features, Classifiers, Decision Trees, Random Forests, Training & Testing, 

The Latent Space

N-dimensional Spaces, Semantic Spaces, Feature Spaces, Embeddings

Unplugged #1

The Omega Point

The Chrismas talk on Noosphere, Datasphere, Infosphere, Singularity, Novacene &c.

Language models

Large & Small Language Models

Artificial Agents

Agentic AI

Central Problem of AI ethics

Central Problem of Roboethics, Alignment

Human-Machine Peer Learning

Humans and machines can learn together
Machines and humans can learn with each other