Quote:When you say "unique" in the first paragraph, do you mean unique to the page or to the manuscript? (I'm assuming MS)
I mean unique to the manuscript, in other words its frequency count is 1.
Quote:To save us looking it up, what's the average chance any Word would be unique, about 50%?
Not sure if I understood your question. If you mean the VMS, then (as Rene suggests) roughly a half of all vords are unique, i.e. they are encountered only once in the VMS. If you mean "for a given text, what is the probability of a given word to be unique in that text", this would depend - as I guess, it would primarily depend on the category and style of the text, rather that on the language of the text. (I may be wrong in this). Actually that's the clue to what I am proposing.
For a formalized task, this would look something like:
1) Build the vord frequency histogram for the VMS
2) Select a set of contemporary documents of various categories and styles for comparison: such as herbals, astrological books, philosophical treatises, general narratives etc. Include versions with high degree of word abbreviation, if exist. (Each different abbreviation of the same word should be considered as a different word for the sake of the calculation!). Calculate their word frequency histograms.
3) Compare results of 1) and 2) and make conclusions.
4) Repeat steps 1) - 3) for word entropy calculations.
Actually, I'm not sure if anyone has not done all this before, that's why I am asking.