TEST * YOUR * THEORY * HERE

[gaugamela]

Internet chat bots are just one very specific implementation of AI.

In fact they are not "Intelligence" at all, not even "Artificial Intelligence" as it has been understood since the 1960s. They use practically nothing of the tools that AI research produced up to the first ChatGPT.  

What they do is grind billions (literally) of documents scraped from the internet and other sources, choose bits from those documents based on certan frequencies and weights, and string them together into a superficially correct English, with correct grammar and topic chaining.  

They can do basic arithmetic in word problems, but without understanding the setting at all. Once you know how they work, it is trivial to create questions that elicit hilariously stupid answers.  Like:

  Q. A customer ordered a 20 kg dumbbell from an online store, but the store 
  sent a 15 kg one by mistake.  How can the store make good on that order?

  A. Send the customer a 5 kg dumbbell.

The providers try to patch their models to avoid such stupid mistakes, but, because of their nature, they cannot really fix them.  

Months ago I asked ChatGPT to write a program that would print all even prime numbers between 1 and 1000.  It gave me a correct but very stupid program, that looped n from 1 to 1000, tested whether each n was prime by trying all divisors from 2 to n-1, then, if it passed, tested whether n was even.  The intelligent answer would have been, of course, just "print(2)".

I tried this same question months later, and this time ChatGPT gave me the "intelligent" answer "print(2)".  But then I asked for a program that printed all prime numbers from 1 to 1000 that ended with "5".  Not only it again gave me the correct but stupid brute-force algorithm, but it also added "The  output of this program could be, for example, 15, 35, 125, "...

And of course they will happily reproduce any nonsense that happened to find its way into their "model".  And, for obvious business reasons, they are programmed to tune their answers according to what they infer about your prejudices...

All the best, if possible, --jorge

That seems like a gross misrepresentation of the current state of the technology. 
I tried both of your examples and got very different solutions. No brute forcing at all, rather checking divisibility, which would seem like the easiest way. 

As for the context, it depends a lot on the prompt and context you give it. I've gotten usable and very accurate solutions for diffusion and convection problems, fits, and all sorts of other not so trivial questions. Obvious one can not simply rely on these but a human has to check. 

AI isn't a magic wand that can make sense of anything you throw at it, in fact it technically can't make meaning of things at all. I agree, it basically works like a large predictive search engine, trawling it's database. There are gonna be blind spots and misinformation. Nonetheless it's a powerful tool in the right context. It can give people the ability to engage with things they have little knowledge in.
Coding is a great example. Is the code right? 90% of the time. Is the code good? 50/50. Is the code usable? Almost always. For many people that is good enough, it's only a problem when vibe coding becomes the norm and isn't properly implemented. 

The biggest problem is the opaque inner mechanism. We can only really study the outputs and make guesses and tweaks to the system. We have no real clue what's really going on under the hood. Hallucinations are also a persisting problem where the gravity depends on the familiarity of the user with the subject.

The biggest problem is users taking AI generated content at face value and not checking it against actual sources. It's a tool like any other, no more and no less useful than the user operating it. As evident by some of the ludicrous theories about the Voynich Manuscript that get posted regularly.