I Tried Gemini For My Email. Here's Why I Don't Trust It

Over the course of the past three months, I have experimented with three AI stacks, each priced at $20 per month for the base tier. I started with ChatGPT Plus, then moved to Perplexity Pro, and finally bought into the promise of Gemini Advanced. I stuck with the latter for the longest spell, primarily owing to its deep integration with other Google products such as Gmail and Docs, which are an integral part of my workflow.

Now, my journey with Gemini hasn't exactly been glorious, and like a majority of rival generative AI products, it has run into its fair share of hiccups. The inaccuracy woes got so bad that Google chief Sundar Pichai had to apologize for them. Researchers have also independently showcased how it can be manipulated to generate misleading content, spill sensitive data, and even go ahead with malicious tasks. Let's, for a moment, assume that those are high-level hiccups and that an average user likely won't run into such problems.

Google is pushing Gemini as a more capable alternative to Google Assistant. The reality, however, is different. I pushed Gemini predominantly for basic tasks, like speeding up my inbox duties, handling my calendar schedule, and just keeping an eye on my Workspace activities. However, the pace at which it served downright misleading information — which merely required a look at my own data instead of an expansive web search — made me question Gemini's reliability and whether it can be trusted with anything beyond its usual chatbot duties.

Gmail was my first test bed of experimenting with Gemini, hoping to integrate it with my workflow. However, what I found was that it can blatantly lie. As can be seen in the image above, I asked Gemini about the status of my most recent FedEx package. It pulled up information about shipments from last year, but couldn't pick up a single detail from over two dozen emails from FedEx in my inbox, all of which arrived within a span of one week.

The information it served in the chat box, though outdated, was not inaccurate, down to the tracking number. Where it missed the mark was confidently telling me that the "latest update for your FedEx package" was custom clearance roughly three months ago, and not a series of fresh updates that arrived merely three minutes ago, with a frequency of at least three emails daily, dating back to at least a week.

Nadeem Sarwar/SlashGear

Likewise, I asked Gemini about "the most recent Calendar entry." Instead of telling me about the three meetings I closed in the second week of January, it simply replied with "I don't see any events on your calendar." This is not only a contextually inaccurate reply, but also downright wrong because instead of looking at "recent" events, Gemini tried to find events in the future.

My Calendar entries are tied inherently to my inbox. I send and receive Google Meet invites directly via my inbox dashboard. It's just surprising that despite appearing as a standalone tool prominently in the mobile and desktop versions of Gemini, the AI can flub at something as basic as checking for events, and instead going in the opposite direction and reversing the query's context.

Gemini integrates with other Google Workspace apps (and their data) via a system of extensions. A similar pipeline is also in place for crosstalk with other apps, such as WhatsApp. Yet, the experience leaves a lot of room for improvement. Actually, scratch that. Gemini can be infuriatingly dumb at occasions, despite Google touting its natural language chops as one of the best out there.

Let's start at the most basic level. Just like chat apps, where you can use the "@" shortcut for addressing a person or group, Gemini also relies on the same shortcut for selecting the right Workspace platform to get the job done. Well, it doesn't seem to work even for the most basic queries. Moreover, comprehension disparities across different platforms only make matters worse.

I summoned Gemini and told it to send a "hi" to my sister. I even used the "@" shortcut and picked Gmail as the destination to execute the task. The AI assistant simply refused to do so in the Gemini app for iPhones, even though it worked just fine moments ago, pulling up minute details from a long Gmail chain about a research paper. In the Android app, Gemini simply couldn't decide what to do.

Within a span of a few minutes, its responses made a 180-degree flip. In the first attempt, it asked me which "Saba" from my inbox was I referring to, before the message could be sent. On the second try, using the exact same prompt, Gemini outright rejected the chore, citing an inability to do so. Also, the extra information it provided about communication history was downright false.

At the moment, trying to get even the simplest tasks done with Gemini is like playing an AI whack-a-mole, one where you also have to pore over support pages to check whether Gemini will work on your phone. It's quite vexing that, despite paying $20 a month, Gemini isn't capable of doing a task as basic as making a call. On an Android phone, I was able to make a call with a simple "Call XYZ @Phone" command.

On iOS, the Gemini app doesn't support the "@Phone" extension that would allow it to make a call. Alright, let us, for a moment, assume that Apple will never allow an AI access to the Phone app, owing to privacy and security reasons. Also, Apple already has a new avatar of Siri ready, one that is juiced up on OpenAI's tech stack, so it makes sense to keep such fundamental capabilities locked to its own assistant.

Nadeem Sarwar/SlashGear

But what about third-party communication platforms like WhatsApp? Well, the ability to send a text on WhatsApp is limited to Android, where you can freely use the "@Whatsapp" shortcut in the chatbot. On the iOS app, you don't have that luxury. Heck, even the "@Gmail" extension returns an "I can't assist you with that" response.

Nadeem Sarwar/SlashGear

These are no small failures. If Google's assistant can't even pull off a task as simple as sending an email and runs into platform gates even with third-party apps, there is little point in paying $20 for Gemini on the hollow promises of seamless Workspace access and collaboration.