Google Assistant May Soon Be 'Supercharged' With Bard-Like AI

Google Assistant might soon get advanced AI-powered capabilities akin to those of chatbots like Bard and ChatGPT. In an email seen by Axios, a senior Google executive told employees that the company sees "a huge opportunity to explore what a supercharged Assistant, powered by the latest LLM technology, would look like." Now, the company didn't go into detail about what those "supercharged" capabilities might look like, but a quick look at Bard's own feature set offers a good idea of what we can reasonably expect.

Bard is built atop the Pathways Language Model 2 (PaLM 2) language model and it is powered by Google's Language Model for Dialogue Applications (LaMDA) technology. Aside from being able to answer questions based on information scraped from the web, Bard recently gained the ability to analyze images using the same tech that powers Google Lens. It can also pull citations, and will soon make its way to Adobe Express by integration with the Firefly generative AI.

But these capabilities have little to do with Google Assistant, which can currently pull web search results at best, or perform app-connected tasks on a device such as setting alarms or playing music. Bard, on the other hand, could be the smartest AI chatbot out there, but it can't really perform any meaningful task on your phone, like playing music or setting an alarm — but in an integrated avatar, it can enhance the Google Assistant's capabilities dramatically. Interestingly, Google has already given a teaser of what's to come.

In May 2023, Google's AI team released a report titled "Enabling conversational interaction on mobile with LLMs," which involved testing large language model prompts against a phone's UI. It talks about integrating large language models with graphical user interfaces (GUIs) — a.k.a., the apps and software running on the phone's screen. It broadly discusses four application areas that include summarizing on-screen content, answering questions based on the content you see on the display, and most importantly, assigning UI functions to language prompts. 

For example, the language model can skim through the UI to automatically generate contextual questions and the information they convey. Once it gleans the details, it can convert them into questions, so that when a user asks, the language model answers them promptly. Another notable capability is "screen question answering." For example, when a blog post is open in a web browser, the AI can provide details such as headline, author name, publishing date, and more.

But the most promising area of application is "mapping instruction to UI action." Essentially, it translates to controlling your phone using prompts (both voice and text). The virtual assistant can be asked to open an app, tweak phone settings like cellular network mode, and more, with enhanced conversational abilities in tow. It's not clear when exactly a supercharged Google Assistant will arrive, but it would be quite a leap in its capabilities. Interestingly, Apple is also said to be toying with generative AI tools — reportedly internally dubbed AppleGPT — to improve Siri.