i
5. As a large language model ... DDH ()


5. As a large language model ...

The turning point

With the advent of the Epoch, humanity mastered the art of quantifying the "meaning of the Word".

Geometrization of meaning

Even entities like "word meanings" or "concepts" can be geometrically represented, either as points, vectors or sub-spaces of the enveloping vector space S.

One can subsequently measure "distances" and "angles" between such representations, e.g. distance of the meaning of theword "dog" from the meaning of "wolf" or "cat" etc.

Geometrization of one’s data-set once effectuated, space S can be subsequently partitioned into a set R of |C| regions R = R1, R2, ..., R|C| etc.

(Prolegomena Paedagogica, page 265)

Questio: What features characterize word X ?

Let's have a sentence: "Harfa, gitara a piano sú excelentné hudobné inštrumenty."

What can the slovak word "hudobné" mean ? What features allowed You to decode it ? 

Features

Features of a word can be: 

phonetical

phonological

prosodic

semantic

contextual

syntactic

Distributional Hypothesis

"The DISTRIBUTION of an element is the total of all environments in which it occurs, i.e. the sum of all the (different) positions (or occurrences) of an element relative to the occurrence of other elements."

"a word is characterized by the company it keeps"

"the more semantically similar two words are, the more distributionally similar they will be in turn, and thus the more that they will tend to occur in similar linguistic contexts"

Word embeddings


word context: wings context: engine context: sky
bee 3 0 2
eagle 3 0 3
goose 2 0 4
helicopter 0 2 4
drone 0 3 3
rocket 0 4 2
jet 1 1 1