Google's Life-Swallowing Plans For Gemini AI Will Either Delight Or Disgust You

Update 12/8/2023: Regarding this story, a Google spokesperson has reached out to SlashGear with this statement: "Google Photos has always used AI to help people search their photos and videos, and we're excited about the potential of LLMs to unlock even more helpful experiences. This was an early internal exploration and, as always, should we decide to roll out new features, we would take the time needed to ensure they were helpful to people, and designed to protect users' privacy and safety as our top priority."

The best, and the worst, of AI are yet to come. Google's next endeavor, internally called Project Ellmann, could fall in the latter category. According to internal documents seen by CNBC, Google is working on a new project leveraging the new Gemini AI model that will learn from your photos and web search history. The end product will be an AI chatbot that will answer questions about your own life after gleaning every bit of information from your Google Photos directory and web search history.

"Imagine opening ChatGPT but it already knows everything about your life. What would you ask it?" reads one of Google's own presentation slides at an internal meeting. The overarching idea is to create a chatbot that acts as "Your Life Story Teller" by reading more than just the metadata for your photos. Instead, it will read them contextually like a human being, with the goal of describing them with a nostalgic, informative flavor.

The AI will reportedly identify key moments, faces, and objects in photos, and save them as a memory, to be recounted later when needed. "When we step back and understand your life in its entirety, your overarching story becomes clear," describes another slide about the goals of Project Ellmann. It will have a dedicated chat feature that will act as a personal raconteur of your life on your phone, but it's unclear if this chatbot will be integrated directly into the Google Photos app.

One of the internal slides showed a conversation example where someone asked, "Do I have a pet?" The AI's response explained that the person once owned a dog. However, it then went on to describe additional details such as the pet's name, the dog wearing a red coat, and even the names of the family members frequently captured with the animal. As implausible as that might sound, Google's own demo of Gemini's real-time image comprehension skills suggests that Project Ellmann is not too far into the future.

Seeing some qs on what Gemini *is* (beyond the zodiac :). Best way to understand Gemini's underlying amazing capabilities is to see them in action, take a look ⬇️ pic.twitter.com/OiCZSsOnCc

— Sundar Pichai (@sundarpichai) December 6, 2023

The AI is capable of telling what food items a person prefers, the products they are looking to purchase, potential travel plans based on information gleaned from screenshots saved in the gallery, and more. And based on the web search history, it will even vomit out details such as a person's favorite sites and apps. Google Photos already does that to a certain extent. If I enter my bank's name in the app's search field, it automatically pulls up images of my credit card and banking screenshots saved on the phone.

Moreover, the foundation for implementing Project Ellmann locally on a smartphone has already been laid. The Gemini Nano model is already powering a couple of features on the Google Pixel 8 Pro, thanks to the dedicated AI engine on its Tensor G3 chip. It's a breakthrough achievement because on-device AI processing means you don't need to connect with Google's servers over an internet lane. Plus, it significantly speeds up the task at hand.