• Options
• Login
• Desktop Style

Using AI to do research - Specifically online searching

• Index

  • The Voynich Ninja
  • Voynich Research
  • Voynich Talk
  • Using AI to do research - Specifically online searching
• Using AI to do research - Specifically online searching

• Using AI to do research - Specifically online searching

  Mark Knowles > 7 hours ago

  I am always looking for ways to find new examples of early 15th century ciphers. As those who are familiar with my online OneDrive archive know I have collected many images of examples of those and there are still some that I haven’t uploaded to my OneDrive. However, I continue to try to increase my collection. So, I have been experimenting with tools like Gemini and ChatGPT to see if they can help generate any new leads. I haven't found anything definitely interesting yet, although they have given me food for thought. However, I wonder if anyone else has experience doing this kind of research.

• RE: Using AI to do research - Specifically online searching

  DG97EEB > 7 hours ago

  (7 hours ago)Mark Knowles Wrote: You are not allowed to view links. Register or Login to view.I am always looking for ways to find new examples of early 15th century ciphers. As those who are familiar with my online OneDrive archive know I have collected many images of examples of those and there are still some that I haven’t uploaded to my OneDrive. However, I continue to try to increase my collection. So, I have been experimenting with tools like Gemini and ChatGPT to see if they can help generate any new leads. I haven't found anything definitely interesting yet, although they have given me food for thought. However, I wonder if anyone else has experience doing this kind of research.

  
  Hi Mark,
  
  I use Claude, GPT and Gemini deep research capabilities very often and find them mostly excellent. The only one I find frequently problematic is Gemini, which hallucinates constantly. For example, I was searching for published research on a particular folio of a different manuscript the other day. Claude correctly identified that the article referenced the Manuscript and another Manuscript started to give me some data on the folio and then realised that the f420v in the article was actually from the other one, and self corrected. I then ran the same research with Gemini and it presented it to me as fact until I challenged it. 
  
  So my preference for these things is Claude, with one exception which is that GPT has something called Agent mode. Here it behaves as if it is a real user, and can bypass captcha and other robots.txt limitations that prevent others from accessing. I often run a deep research and then and agent pass to see if there are sites that can only be access that way.
  
  For ciphers, I've quite recently run extensive searches using all 4 approaches and I've come up empty unfortunately. There's nothing freely available that you haven't already found. That said, the models are getting better and better, and OpenAI released 5.5 a few days ago which is said to be the best at uncovering information, but the reality is if it's not digitised, it won't find it.
  
  Thanks 
  
  Ed

• RE: Using AI to do research - Specifically online searching

  Mark Knowles > 7 hours ago

  (7 hours ago)DG97EEB Wrote: You are not allowed to view links. Register or Login to view.

  (7 hours ago)Mark Knowles Wrote: You are not allowed to view links. Register or Login to view.I am always looking for ways to find new examples of early 15th century ciphers. As those who are familiar with my online OneDrive archive know I have collected many images of examples of those and there are still some that I haven’t uploaded to my OneDrive. However, I continue to try to increase my collection. So, I have been experimenting with tools like Gemini and ChatGPT to see if they can help generate any new leads. I haven't found anything definitely interesting yet, although they have given me food for thought. However, I wonder if anyone else has experience doing this kind of research.

  
  Hi Mark,
  
  I use Claude, GPT and Gemini deep research capabilities very often and find them mostly excellent. The only one I find frequently problematic is Gemini, which hallucinates constantly. For example, I was searching for published research on a particular folio of a different manuscript the other day. Claude correctly identified that the article referenced the Manuscript and another Manuscript started to give me some data on the folio and then realised that the f420v in the article was actually from the other one, and self corrected. I then ran the same research with Gemini and it presented it to me as fact until I challenged it. 
  
  So my preference for these things is Claude, with one exception which is that GPT has something called Agent mode. Here it behaves as if it is a real user, and can bypass captcha and other robots.txt limitations that prevent others from accessing. I often run a deep research and then and agent pass to see if there are sites that can only be access that way.
  
  For ciphers, I've quite recently run extensive searches using all 4 approaches and I've come up empty unfortunately. There's nothing freely available that you haven't already found. That said, the models are getting better and better, and OpenAI released 5.5 a few days ago which is said to be the best at uncovering information, but the reality is if it's not digitised, it won't find it.
  
  Thanks 
  
  Ed

  
  If I find a reference to a cipher in an archive I can contact the archive and request a photoreproduction of that document. So, I don't need the cipher to have been digitised itself. Basically, if I can find a clue to where I might find a cipher or even a clue to an archive which might be worth manually searching then that's worth knowing about. I obviously have a schedule of archives that I want to manually search or for which I know someone who is happy to search them.
  
  I was suspicious that Gemini might be hallucinating when I used it. So, I will look into Claude and GPT Agent Mode.
  
  Having done research in the Milan archives I am now thinking about biting the bullet and going to the Vatican Apostolic Archives, but that might require a long visit and good plan of action to make sure I use the time productively. I have a friend from Milan who is happy to do research in the Pavia Archives.

• RE: Using AI to do research - Specifically online searching

  Rafal > 5 hours ago

  I suppose you know this site: You are not allowed to view links. Register or Login to view.
  
  Do you have access to it? Is there anything helpful?

• RE: Using AI to do research - Specifically online searching

  Mark Knowles > 5 hours ago

  (5 hours ago)Rafal Wrote: You are not allowed to view links. Register or Login to view.I suppose you know this site: You are not allowed to view links. Register or Login to view.
  
  Do you have access to it? Is there anything helpful?

  
  Yes, I know them very well. I was meaning to send Beata Megyesi another email at some time to see if they have any research trips planned. I have provided them with a lot of material that I have uncovered, although I should update them with any more recent discoveries of mine. Beata has done research before in the Vatican library. I don't think they have anything new that I don't know about, but I suppose I could check.
  
  I have struggled so far to find any researcher alive who has done any research whether into ciphers or anything else in the early 15th century records in the Vatican Apostolic Archives.

• RE: Using AI to do research - Specifically online searching

  Mark Knowles > 5 hours ago

  Some time ago I thought the following below useful, but in these days if ChatGPT and the like there may be better approaches->
  
  You are not allowed to view links. Register or Login to view.

• RE: Using AI to do research - Specifically online searching

  Mark Knowles > 5 hours ago

  I want to avoid language specific searching. Whilst the default references to ciphers from the time would likely be in Italian there are other languages that they could be in e.g.:
  
  Latin, English, German, French and others
  
  
  So, searching with all variants of appropriate vocabulary in each language to track down references is fairly laborious.
  

  ---

  As an example I list these possible keywords in my comment to Nick Pelling's post:
  
  “cryptologia”
  
  “cypher”
  “cipher ledger”
  “despatches”
  “Duke Philip Maria”
  “cipher key”
  
  “zifera”
  “zifere”
  “zyphera”
  “zyphera”
  “duca Filippo”
  “duca Philippo”
  “duca de/di Milano”
  “chiave/i di cifra”
  “cifrario”
  “scritture segrete/segreta”
  “criptografiche”
  “scritto crittografico”
  “crittografia”
  “codificare/codifica/codi/codificato”
  “nel codice/in codice”
  “scrivere oscuro”
  “ciffra”
  “cifre”
  “cifrante”
  “intercetta”
  “zifre”
  “chiavi di cifra”
  “messaggio”
  “lettera/messagi/dispacci”
  “decifrazione”
  “decifrazioni”
  “in cifra”
  “cifrato”
  “registro”
  “carteggio”
  “codici segreti”
  “scritture occulte”
  “inventari/cataloghi”
  “archivio”
  “cifratura”
  “cifrista”
  “quattrocento”
  “primo quattrocento”
  
  “ziffrette”
  
  “zifferatas”
  “cipharis”
  “cyfris”
  “ziferis”
  “Philippi Mariae Vicecomitis ducis Mediolani”
  
  “chiffre/chiffrer/dechiffrees”
  “crypter”
  “duc”
  “Philippe Marie”
  
  “verschlüsseler/verschlüsseln”
  “entziffern”
  “entschlüsseln”
  “kryptographie”
  “geheimschrift”
  “Herzog Philip”
  “Mailand”
  
  Español: “Felipe Maria Visconti”

• RE: Using AI to do research - Specifically online searching

  Mark Knowles > 4 hours ago

  I suppose for example that I would like to search for all references to ciphers in all languages and all spellings variants in construction with another parameter in all its variants that would link to the time period that I am interested in. I guess this could in theory be done using some very very long logical statement in Google, but that seems somewhat unwieldy and impractical.

• RE: Using AI to do research - Specifically online searching

  Mark Knowles > 4 hours ago

  (7 hours ago)DG97EEB Wrote: You are not allowed to view links. Register or Login to view.For ciphers, I've quite recently run extensive searches using all 4 approaches and I've come up empty unfortunately. There's nothing freely available that you haven't already found. That said, the models are getting better and better, and OpenAI released 5.5 a few days ago which is said to be the best at uncovering information, but the reality is if it's not digitised, it won't find it.
  
  Thanks 
  
  Ed

  
  I must confess that I doubt that there is "nothing freely available that I haven't already found". However, knowing where to look and what to look for is hard.
  
  I agree there is still an issue of digitisation, as many archives don't have their online inventories digitised which is why I was interested in a previous thread in what was the best OCR software as manually reading through long inventories is not realistic. Hopefully improvements in OCR technology may lead to more documents being digitised.

• RE: Using AI to do research - Specifically online searching

  DG97EEB > 4 hours ago

  Hi Mark,
  
  I ran a combination GPT and Claude.. Hope it helps
  
  Ed
• Next Oldest Next Newest