Apple has sent the tech industry into non-stop chat mode with its hot Google deal. After missing the early AI hype and the stopgap solution with ChatGPT, Apple is finally putting Gemini (a customized version without any branding) in the driver’s seat. The idea is to fix Siri, Apple Intelligence and the entire Foundation Models package.
So what does this mean for you? First of all, Siri will be just as chatty and intelligent as ChatGPT or Gemini. There will no longer be situations where you are asked to outsource your requests to ChatGPT. And hopefully it will finally gain actionable insights for other apps. Google just did that with personal intelligence.
It’s quite ironic that Apple advertised the “more personal Siri” concept long before Google, but then withdrew the ad and later claimed that perfecting an all-knowing Siri would take time. With the launch of Gemini, Apple can do a lot more, and I’m pretty excited about it. But what I really want is for Apple to fix the Shortcuts app.
So much wasted potential
The Shortcuts app is one of the most underrated corners of your iPhone. It is a powerful tool that will surprise you with the variety of multi-step tasks it can handle. And now that Apple Intelligence is built into the app, it can do some extremely cool tricks.
With a single tap, it can take a screenshot of the video playing on my screen, identify the movie or TV series, and tell me where to stream it. A single tap is enough to convert a screenshot into a reminder with custom search tags, a brief summary of the content, and the source URL in the Notes app.
These AI-powered tasks can be completed by ChatGPT or even Apple’s own on-device models that run completely offline. You see the immense potential, right? Well, this is where the rosy picture ends. Creating shortcuts in the app of the same name is a nerve-wracking task for beginners.
And even for power users, creating a long automation is a test of patience. The scripting system, with its variables and vague control flow, is quite confusing. You’ll almost certainly have to go through a long explanation to keep track. And even after that, you’ll be lost when you actually start creating a shortcut.
The entire system is based on codes and hotwords that you don’t even notice exist. And it’s only when you look for the specific app where the task needs to be done that you realize, “Oh, there’s a related action available.” Why not use ChatGPT to describe the steps and simply repeat them in the Shortcuts app?
Well, I wish I could tell you it works. However, the reality is that ChatGPT will plunge you into even greater confusion with vague steps and often misleading instructions that simply aren’t present in the Shortcuts app. It’s not surprising that most of the reader emails I receive in my inbox are about shortcuts, asking for guidelines, or sharing the direct iCloud link.
The ingredients are already there
“Hey Siri, create a shortcut that will automatically enter Focus mode at 4 p.m., mute all notifications except Slack, and send me a summarized version of the group conversations.” Imagine saying that and Siri creating a shortcut for you. In the current state of the Shortcuts app (and Siri), it’s a pipe dream.
But there is good news building on the Gemini deal for Siri. Apple already has a system that allows Siri and Apple Intelligence to interact with apps. To put it simply, the built-in AI is aware of the actions it can perform within apps without actually opening those apps.
“With Apple Intelligence and improvements to App Intents, Siri suggests your app’s actions to help people discover your app’s features and gains the ability to take actions within and between apps,” Apple explains.
Gemini can already do this, not only in Google’s own apps (like Gmail or Calendar), but also in third-party apps like WhatsApp. You can simply tell Gemini to perform a task and it will do it as described. Siri can do that too. Technically it is.
But what I really want is this “conversation-to-action” system applied in the Shortcuts app. Or to put it more precisely: let Siri create shortcuts directly with natural language descriptions. If there is a frame, this shouldn’t be a big hassle.
I think Siri has been the more challenging so far, not least thanks to its language comprehension and reasoning abilities, which are far behind what you’d experience with ChatGPT or Gemini. Or Apple can offer an agent that can help create and change shortcuts using text or voice commands.
We are already in an era where AI agents enable voice-based video editing on phones, building websites, playing mini-games, and much more. With a strong foundation already in place, it’s a fantastic opportunity for Apple to finally give Shortcuts its due by simply simplifying the workflow.
A better (and more ambitious) idea
I just came across Replit’s latest announcement. The company has announced an AI-powered tool that can create a fully functional app and push it to the App Store – with just a text description. It’s almost like launching an app.
The idea is ambitious, but not every iPhone user wants to go to the trouble of having an app reviewed and published in the Apple Store. But shortcuts can definitely benefit from something like this. Look no further than Android newcomer Nothing.
The company recently launched a tool called Playground, an AI-powered toolkit that allows you to easily describe and build an app. It is a no-code platform that works entirely on your phone. You don’t have to deal with complex app creation tools on a desktop.
These mini apps can be shared with the community and you can freely choose one from the public dashboard and customize it as you wish. Google, on the other hand, offers a similar tool called Opal to create web-based apps using Gemini.
Apple can take this concept and give users the ability to create shortcuts that can also be instantly converted into an app instance. Shortcuts already has a large global community, particularly where users share helpful automations as iCloud links.
However, for many users, getting these shortcuts to work can be a hassle, especially if they want to customize them. A shortcut shared as a mini-app that allows for conversation building (and customizability) could prove to be a game-changer.
Instead of relying on third-party apps that run on a cloud server or are tied to a subscription model, users can simply do this through shortcuts (masquerading as apps). The key advantage is that these shortcuts can either run completely offline or leverage Apple’s secure private cloud computing solution for AI-based tasks.
When Gemini comes into play, the possibilities are endless. The iPhone experience will change dramatically moving on from Android. But if Apple could do a similar AI-powered overhaul of the Shortcuts app, it would not only solve an annoying problem, but also give iPhone users a unique advantage that doesn’t exist elsewhere at a native level on smartphones.
