Siri is reborn in iOS 18 — everything Apple's voice assistant will be able to do

By Marko Zivkovic

We have exclusively discovered how much iOS 18 and macOS 15 will upgrade Siri and improve everything you do. Here's what the new-and-improved Siri will be able to do.

Siri will be able to perform complex actions in core system apps

In one of our earlier exclusive reports, we said that Siri would get a big upgrade. We explained that the company's virtual assistant would be able to open and edit specific photos, set the device wallpaper, and so on.

Since then, AppleInsider has received the exact details of Siri's new functionality, as well as prompts Apple used to test the software. Coming from people familiar with Apple's AI initiative, this new info gives us our most detailed look yet at the virtual assistant's new in-app capabilities.

Although one might expect the virtual assistant only to follow direct instructions and orders like "Open Safari," Apple has tested a wide variety of different prompts for the new AI. Many of these prompts contain "natural" language, as would be spoken by a real human being.

This means that Apple's test prompts often contain descriptions of what the user is trying to do, along with some arguably superfluous details.

Instead of containing only direct commands like "Show me pictures of my cat," the company's testing prompts mention that the user wants to make a blog, or that they're feeling lazy/nostalgic in some instances.

In doing so, Apple wants to get its AI accustomed to natural language, speech patterns, sentence structures, and instruction types Siri is likely to receive from end users -- some of whom may not be all that familiar with new technology.

What core apps are going to get enhanced Siri features

Siri improvements in Apple Books

Through Siri, users will be able to open specific books or sections of the app, flip through book pages, and so on.

Image Credit: Apple

These features will ultimately assist users who may be unfamiliar with the app's user interface or who have difficulty navigating through the UI. Siri will make it significantly easier to find things like the audiobook store or the user's PDF files and downloaded books.

Within the Books app, Siri will be able to:

Siri improvements in Camera

Siri's ability to control device cameras will make it a lot easier to take photos and videos. Users could tell Siri to set their camera to a certain mode, and set a timer.

Siri will gain the ability to take pictures, videos, and more with iOS 18

This could potentially benefit users who are unfamiliar with the various options and settings within Apple's default Camera application. It would also give users the option to easily switch between the front-facing and rear cameras through Siri commands.

With the Camera application, Siri will gain the ability to:

Siri improvements in Keynote

Siri's new Keynote-related features will benefit users in the education, marketing, or business sectors. Through Siri, users will easily be able to see who edited a Keynote presentation and will be able to make changes of their own.

Keynote is part of Apple's iWork Suite of productivity applications

Educators and students will now have the option to insert audio, images, and video via Apple's new-and-improved, LLM-powered virtual assistant. Based on what we've heard, researchers and analysts will likely find this feature useful, as it would make the creation of presentations significantly easier.

In Keynote, Apple's new-and-improved visual assistant will gain the ability to:

Siri improvements in Mail

Apple's Mail application will get a significant overhaul this year, as we revealed in our exclusive report on Project BlackPearl. The company will enhance its built-in email app with machine learning, which will allow it to automatically classify emails based on their text content.

Apple will upgrade the Mail app with machine learning as part of Project BlackPearl

This means that Mail will be able to separate emails into categories such as Commerce, News, Promotions, Social, Transactions, and Other. Siri will be able to use the information to perform different actions within the app.

For instance, Apple's AI-powered virtual assistant will be able to identify emails with promo codes, or those from a specific company and label them as junk.

Siri will be able to perform a variety of actions within Mail, all of which will streamline email communication. With Siri, users will have the option to reply to all senders of an email.

For instance, users could say something like "Reply to everyone on this email with the body text 'Yes, I'll be there.'" The virtual assistant will also have the ability to forward emails, block email senders, and much more.

In the Mail app, Siri will be able to:

Apple's Mail application is also getting a feature known as Smart Replies, through which users will be able to send off an AI-generated response to an email. Apple's response generation is handled by the on-device Ajax LLM and cloud-based processing in more complex situations.

All of this is sure to benefit business users and members of large companies, who routinely deal with massive amounts of emails. Customer service representatives would also likely make use of the new Siri functionality.

Siri improvements in Notes

Apple's Siri enhancements for Notes will make it significantly easier to navigate between individual notes, tag them, and move them around. Users will have the option to utilize key features through the company's virtual assistant Siri.

Apple's Notes application will receive in-app audio recording, audio transcription and LLM-powered summarization with iOS 18

Within Notes, Apple's new LLM-powered Siri will have the ability to:

The built-in Notes app is also expected to feature in-app audio recording, audio transcription as well as LLM-powered summarization. This will allow users to store audio recordings, summaries, and transcriptions alongside other material such as text or images - all in one note.

Math Notes is another upgrade Apple has in store for the Notes app, which will offer integration with the all-new GreyParrot Calculator. With Math Notes, users will be able to include complex mathematical equations and graphs as part of individual notes.

Also in the works, Keyboard Math Predictions will allow Apple's software to recognize mathematical expressions as part of text input. As a result, users will have the option to auto-complete mathematical equations through solutions generated by AI and machine learning.

As a whole, these enhancements will turn Notes into the go-to note-taking application, making it a true powerhouse for students as well as engineers and business professionals.

Siri improvements in Photos

As we revealed in one of our earlier reports, Siri will have significantly enhanced functionality within the built-in Photos application. Within the app, users will have the option to edit, move, and hide photos, albums, collections and memories through Apple's virtual assistant.

The Photos app is getting a new Clean Up tool powered by generative AI

Siri will also have the ability to identify specific objects, landscapes or people within a photo through Visual Look Up. This means that users could request to see photos of their loved ones, like friends and relatives - or they could ask to see things like sunsets and nature photography.

The latter would help users who might be looking to create a nature-themed blog or scrapbook. Users will also have the option to edit things like vacation photos, videos, screenshots - all through Apple's virtual assistant, Siri.

In Photos, Siri will be able to:

Users who recently moved from an Android or Windows device to a Mac or iPhone would possibly have difficulty navigating the user interface. This is where Siri's in-app functionality would truly shine, as users would no longer need to memorize convoluted settings menus and various options.

Apple's default Photos app will get some minor UI changes, along with new AI-powered editing features. The Clean Up tool will facilitate the removal of objects from an image through the use of generative AI.

Siri improvements in Reminders

Siri's functionality within Reminders will assist users in creating things like lists of chores, errands, groceries, books to read, school supplies, travel arrangements, and so on. The virtual assistant will also be able to add or remove hashtags from specific reminders and generate overviews of reminders.

The Calendar and Reminders applications will become integrated with iOS 18 and macOS 15

This would ultimately help users who regularly deal with large volumes of information and who need to remember specific details. Business users could create reminders of people they need to meet, while students could make a list of supplies needed for the new school year.

In Reminders, the new AI-infused assistant will be able to:

Reminders is also expected to receive integration with Apple's default Calendar application, as AppleInsider exclusively revealed. This means that users will be able to create and edit reminders directly from the built-in Calendar app.

Siri improvements in Safari

Having Siri read or summarize the on-screen page contents would assist users who have poor eyesight or visual impairments. This would also help those who are dyslexic, or have other difficulties with reading and reading comprehension.

Safari 18 will introduce a new Intelligent Browsing feature along with a built-in content blocker called Web Eraser

Managing tab groups through Apple's upgraded virtual assistant would make organizing and navigating significantly easier. Users would be able to more easily conduct research, plan travel arrangements, and open specific websites.

As is consistent with Apple's focus on privacy, Siri will also have the option to create private tabs and private tab groups. This ensures that the Safari browser does not keep a record of browsing activity for those tabs and windows.

In Safari 18, Siri will gain the ability to:

Safari 18 will also receive an all-new Intelligent Browsing feature, which will enhance the search functionality and provide summaries of articles. The next-gen browser could also receive a built-in content blocker called "Web Eraser."

Web Eraser has generated controversy, with the UK's News Media Assocation and a group of French publishers both sending Apple complaints about the feature. Although AppleInsider was told the feature is complete and functional, Apple may opt to remove it ahead of WWDC to avoid controversy.

Siri improvements in Stocks

The in-app capabilities of Siri within the Stocks app will greatly benefit less tech-savvy long-time investors. Through Siri, users could request an overview of business-related news or create watchlists with specific stocks.

Siri will be able to perform actions within the Stocks app on iOS 18 and macOS 15

This would help users keep track of specific stock types (eg. Blue chips) while also keeping them updated in regard to the latest events in the investment world. It would also aid those who are unfamiliar with the app's user interface, or who may not have the time to scroll through different menus and settings.

In Stocks, Apple's virtual assistant will have the ability to:

Siri improvements in Voice Memos

Apple's upgraded virtual assistant will offer much-needed functionality within the Voice Memos app. Through Siri, users will be able to create, pause, delete and move individual recordings or folders of recordings.

Voice Memos is getting a new audio transcription feature in iOS 18 and macOS 15

This would ultimately serve to benefit a truly wide array of users, ranging from those in the business sector to students, engineers, and creatives. Users will be able to record meetings and lectures, shopping and to-do lists, singing and much more - all through Siri.

With the Voice Memos application, Siri will be able to:

The new in-app functionality will give users a way of more easily locating specific files they might be looking for - as Siri will be able to find recordings by name, date, or location. The Voice Memos app is also expected to receive in-app audio transcription.

Siri's new capabilities in other system applications

Siri will also be able to open different areas of System Settings, such as Brightness Settings, Home Screen, and App Library Settings, along with Multi-tasking and Gesture Settings. The System Settings application is also expected to receive a reorganized layout with iOS 18 and macOS 15.

The System Settings application is getting reorganized with iOS 18 and macOS 15

Apple's virtual assistant will be able to open new boards in Freeform, scan documents with the Files app, and move specific files to the trash. In the Contacts application, Siri will have the ability to edit contact details.

In the Magnifier app, Siri will be able to describe objects, plus identify people and doors in view of the camera. This would presumably work similarly to how products like Rabbit R1 identify objects, while making use of Apple's existing frameworks and features like Visual Lookup.

Siri is also capable of generating a variety of different images via Apple's internal Generative Playground application, though it remains unclear if this functionality will become available to the end user.

What else will Siri be able to do?

As part of Project Greymatter, Siri will receive the ability to generate summaries of articles, emails and email threads, messages, notifications, and notifications stacks as well as posts. The Greymatter Catch-Up feature will provide users with a recap of their most recent notifications.

Siri, Spotlight, Mail, and Messages will all get significant upgrades thanks to Apple's Ajax LLM and generative AI software

Apple's AI software is capable of generating different levels of summarization, ranging from just three topic words all the way up to three sentences. These different summarization levels will likely be used for specific UI elements within Apple's new operating systems.

Siri, Spotlight, and Mail will all receive advanced search functionality and greater contextual understanding through Apple's AI software. When creating replies, Siri will be able to take into consideration different entities like people and companies, places, and calendar events.

According to people familiar with the matter, generative AI software will also allow for the instant creation of Smart Replies, which will be available in Mail, Siri and Messages. These AI-generated responses will make sending messages significantly quicker.

What other updates can we expect in iOS 18 and macOS 15?

As far as non-AI upgrades are concerned, Apple will revamp the Calculator application on macOS 15. The new-and-improved Calculator will also be available on iPad for the very first time, as it's a "universal application"

The calculator app is getting a redesign for macOS 15 as part of Project GreyParrot

Developed under the codename Project GreyParrot, the new application will enable an iOS-style Calculator on Mac, with rounded buttons and darker shades of black. It will feature a new history tape sidebar and an improved unit-conversion system.

Apple's Maps application is also expected to receive an upgrade which will allow users to add notes to specific locations. With these new place notes, users can add the key information about a place that could only be relevant to them.

AppleInsider was told that these notes would only be accessible to the users who created them, within the Maps application. This would make them different from something like the built-in Notes app, which features iCloud compatibility.

It is important to remember, however, that not all features make it to release. The information in this report comes from people familiar with pre-release versions of Apple's operating systems and test environments, and may not reflect the final release version.

Apple is expected to debut its next generation of operating systems at its annual Worldwide Developers Conference on June 10, which is when we'll likely see some of the improvements mentioned here.