Yes, you can use an locally hosted LLM, but these never can be so powerfull as an AI with a lot of servers supporting. We have to difference it, between mere Chatbots, relying on its own database, often outdated and with the tendece to invent answers, and AI supported search engines, like Andi, You, DDG AI and others, which retrieve informations in realtime from the Web. But there is important that these are capable to difference trustworth sources from those which are not. Andi does a very good job here, because compare informations from several sites for the answer.
The problem is privacythis don’t change if the AI uses sources from big bompany - Google GPTs. I tested several free AIs from Hugging face, Futurepedia and Futuretools, and since over two years I use Andi as main search, I didn’t find anything better. It use an own LLM, because Andi was the first AI search ever, former LazyWeb, long before the others released AI searches.