Hey guys. Fully open to more experiments. The tool chain and analysis system is set up so tweaks are easy.
While developing this I got a feel for whats happening with PCA. The language vectors really are 10,000 dimensional so it's impossible to visualize in 3D before PCA. You've just got to remember the letters a-z and space are all orthogonal. That means any given trigram will be unique in the 10K dim space. Summing the trigrams for a language produces a unique fingerprint. If enough trigrams are similar the language vectors will be -mathematically- similar.
The PCA plot as projects a 10K dim point cloud (vector endpoints) on to a 2D surface. Like shining a flashlight through a snow globe and looking at shadows on the wall. As long as you always use the same PCA axes the plot gives meaningful data from run to run.
To an earlier point, summing binary hypervectors always requires a threshold function
(1 1 0)
(1 0 1)
(1 1 0)
(1 1 0)
----------
(4 3 1) pre threshold
thr=nvec/2=2
If (bit-n >= thr) then (bit-n=1) else (bit-n=0)
(1 1 0) post threshold
Vector multiplication is bitwise XOR
0 0 = 0
0 1 = 1
1 0 = 1
1 1 = 0
D
While developing this I got a feel for whats happening with PCA. The language vectors really are 10,000 dimensional so it's impossible to visualize in 3D before PCA. You've just got to remember the letters a-z and space are all orthogonal. That means any given trigram will be unique in the 10K dim space. Summing the trigrams for a language produces a unique fingerprint. If enough trigrams are similar the language vectors will be -mathematically- similar.
The PCA plot as projects a 10K dim point cloud (vector endpoints) on to a 2D surface. Like shining a flashlight through a snow globe and looking at shadows on the wall. As long as you always use the same PCA axes the plot gives meaningful data from run to run.
To an earlier point, summing binary hypervectors always requires a threshold function
(1 1 0)
(1 0 1)
(1 1 0)
(1 1 0)
----------
(4 3 1) pre threshold
thr=nvec/2=2
If (bit-n >= thr) then (bit-n=1) else (bit-n=0)
(1 1 0) post threshold
Vector multiplication is bitwise XOR
0 0 = 0
0 1 = 1
1 0 = 1
1 1 = 0
D