[Other] The Voynich Garden

[asteckley]

To add a bit more detail to how it was done:

We...invoke AI Models (e.g. ChatGPT)  to augment each one with scientific names, other common names, ranges, etc....

First: great idea for a project, and I look forward to seeing it develop.

Second: do not -- do not, do not, DO NOT! -- trust any factual assertion made by ChatGPT/etc. without independently confirming its correctness. ChatGPT fabricates "facts" far too frequently to trust without verification. If you don't check what it's telling you you risk spreading misinformation, which is that last thing the study of MS 408 needs more of...

Karl

Lol.. Thanks for waving that red flag Karl, and you are right, in general, to do so.

I am in fact an AI professional, who has been outspoken on that very subject -- particularly when LLM's got so much public attention when ChatGPT was first released. At that time, I was especially concerned that people -- even some prominent well-known experts -- were not understanding the limitations of LLM's which are ultimately just "statistical predictors of the next most likely word" (to put it in simple terms.)
ChatGPT and other LLM's can indeed hallucinate and they cannot reason (again despite the claims of some very well-respected people like Andrew Ng, and Geoffrey Hinton), and they should never be used to produce text whose factual accuracy must be depended on.

That is not how it is used in relation to this application, however.  Both the devil and the angel are in the details.

First, the tendency for an LLM to hallucinate is greatly reduced if you deal with short questions whose answers are both short and highly likely to be found in the training corpus. (By short, we mean hundreds as opposed to thousands of tokens.)  The more likely the correct answer exists in an explicit form in the data that was used to train the model, the more likely the model will "predict" the right words of that answer and provide a correct response.  But the more likely that the facts of what you are asking for are unavailable to it or must be synthesized from a lot of data spread across large portions of source text, the more likely the moddel will simply cobble together words to generate something that is just made up -- that is "hallucinate".

Second, the use of it here is to provide some additional, non-critical, information to the data from historical researchers. And the important adjective there is "non-critical". (The consequence of a piece of data being factually wrong is minor and inconsequential to the overall purposes.) We actually inspected the quality of the information it was providing for our purposes. It is not always correct or optimal. (Ironically GPT-4 often produces less useful answers than GPT-3.5). However, it was apparent that the quality of the information that it added far exceeded the quality of the proposed information from historical researchers, which was often misspelled, incomplete, or just referred to plant names that do not exist. So including it to provide background information on what a historical researcher had proposed was a net plus. (And provided a pragmatic starting point from which corrections can be reported and made.)

This, by the way, is one reason why, within the application itself, to provide helpful information to a person making a proposal, we have chosen not to use an LLM. Instead, we have provided a button that they can use to check plant names against a conventional database -- the Global Biodiversity Information Facility. And they can opt to ignore or adopt that information. (This is found on the proposal form when one clicks "Do you recognize this plant?").