27-03-2025, 10:22 PM
On the other hand, maybe random sampling is not enough to find a good approximation for the distribution. Because it still should be possible to use 3 sets to perfectly mimic the word length distribution by only allowing 1 character tokens in two of three sets and setting the target distribution shifted by two places for the third set. If random search missed this solution, who knows what else it missed 