Apple Researchers Reveal New AI System That Can Beat GPT-4 - MacRumorsOpen MenuShow RoundupsShow Forums menuVisit ForumsOpen Sidebar
Skip to Content

Apple Researchers Reveal New AI System That Can Beat GPT-4

Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.

hey siri banner apple
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.

Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.

ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.

ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.

Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.

Popular Stories

apple pro tell logo

Apple Launches Two Products Today

Friday November 13, 2026 2:37 am PST by
Apple is releasing two new products today, Friday, November 13: the all-new Apple HomeView and the fourth-generation Apple TV 4K, both of which have been available to pre-order since mid-October. Apple HomeView: Apple's new smart home display, powered by Apple Intelligence with Siri AI, for entertainment and managing the home. Available in tabletop and wall-mounted models, it can recognize up...
johny srouji

Apple's Johny Srouji Eyed to Replace Nvidia CEO Jensen Huang

Thursday November 12, 2026 4:29 am PST by
Apple's chief hardware officer Johny Srouji is being considered as a potential successor to Nvidia CEO Jensen Huang as the chipmaker's board prepares to remove its co-founder, according to Semafor. Citing a person familiar with the matter, the publication identified Srouji as one of the people under consideration for Nvidia's top job. It is unclear if Srouji has yet been approached or...
Apple iPhone Duo Early Feature

Apple Continues to Move Up iPhone Duo Delivery Dates

Thursday November 12, 2026 12:28 pm PST by
It's a Christmas miracle: some customers who ordered the iPhone Duo through Apple and received an estimated January 25 delivery date saw their delivery windows move up to December 14 to December 21 today, according to comments posted in the MacRumors Forums and on X. A few customers have even seen their delivery date move from January 25 to December 12 to December 14. Apple's online store...

Top Rated Comments

34 months ago
Can‘t wait for it to show me what it found on the web…
Score: 38 Votes (Like | Disagree)
34 months ago

enabling users to use pronouns and other indirect references in conversation without confusion.
oh boy

I'm going to stand on the sidelines to see what group A and group B says about this.
Score: 14 Votes (Like | Disagree)
34 months ago

It's good if AI understands "Can you repeat that?" properly.

/thread
Me: Remind me about this later.
Siri: Tell me what you'd like to be reminded about.
Me: This.
Siri: Okay. I've added a reminder called 'this' to your reminders.
Score: 13 Votes (Like | Disagree)
magicschoolbus Avatar
34 months ago
Big claim from the same company that introduced Siri :rolleyes:
Score: 13 Votes (Like | Disagree)
coffeemilktea Avatar
34 months ago
Does this mean SiriGPT won't rely on Google Gemini? Not only is Gemini behind its competitors like OpenAI's models or Anthropic's, but having less Google in Apple products is always a relief. 👌
Score: 9 Votes (Like | Disagree)
aknabi Avatar
34 months ago
I assume anything their current research is talking about won't impact their offerings for several years and in the meantime they'll do what they did with outsourcing Maps until they got their solution "ready" (of course then there was the bumps until it was a competitive offering, which will likely be more so with AI)
Score: 9 Votes (Like | Disagree)
Latest Stories
Early Apple Watch Black Friday Deals Arrive With Series 12 Hitting New Record Lows
Early Apple Watch Black Friday Deals Arrive With Series 12 Hitting New Record Lows
20 minutes ago
Apple's Smart Glasses Strategy Hinges on Siri and iPhone Apps
Apple's Smart Glasses Strategy Hinges on Siri and iPhone Apps
25 minutes ago
AirPods Pro 3 Discounted to $179.99 in Early Black Friday Sale
AirPods Pro 3 Discounted to $179.99 in Early Black Friday Sale
53 minutes ago
Apple Aiming to Upgrade the iPhone Duo in Four Ways
Apple Aiming to Upgrade the iPhone Duo in Four Ways
2 hours ago
Apple Black Friday Starts Early: Save on Apple Watch, MacBook Air, and More
Apple Black Friday Starts Early: Save on Apple Watch, MacBook Air, and More
18 minutes ago
Top Stories: Apple's New Software Approach, iPhone Duo-Optimized Apps, and More
Top Stories: Apple's New Software Approach, iPhone Duo-Optimized Apps, and More
1 day ago
HomeView Tidbits: Mirror, Sleep Timer, and Other Details
HomeView Tidbits: Mirror, Sleep Timer, and Other Details
2 days ago
Siri AI Still Not Enabled on HomePod Mini 2 With Latest Software Update
Siri AI Still Not Enabled on HomePod Mini 2 With Latest Software Update
2 days ago
Apple Releases tvOS 27.2.1 With Siri AI for Older Apple TVs
Apple Releases tvOS 27.2.1 With Siri AI for Older Apple TVs
2 days ago
Microsoft 365 Outage Resolving After Nearly 24 Hours
Microsoft 365 Outage Resolving After Nearly 24 Hours
2 days ago
iPhone Duo Survives 450,000 Folds, but Hinge is Weakening
iPhone Duo Survives 450,000 Folds, but Hinge is Weakening
2 days ago
Apple TV 4K Launch Day Deal Arrives at Amazon
Apple TV 4K Launch Day Deal Arrives at Amazon
2 days ago
The MacRumors Show: HomeView and Apple TV 4K First Impressions
The MacRumors Show: HomeView and Apple TV 4K First Impressions
2 days ago
YouTube App Not Working on Your New Apple TV? Here's the Fix
YouTube App Not Working on Your New Apple TV? Here's the Fix
2 days ago
Apple Meets Thai Officials Over iPhone Spyware Warnings
Apple Meets Thai Officials Over iPhone Spyware Warnings
2 days ago
Apple Shares HomeView 27.2 Release Notes With List of Features
Apple Shares HomeView 27.2 Release Notes With List of Features
2 days ago
HomeView in Québec: Apple Says Canadian French Support to Arrive in Software Update
HomeView in Québec: Apple Says Canadian French Support to Arrive in Software Update
2 days ago
Apple Watch Ultra 3 Gets Big $250 Discounts Ahead of Black Friday
Apple Watch Ultra 3 Gets Big $250 Discounts Ahead of Black Friday
2 days ago
Apple Says HomeView Sets New Record in Design-Focused Interview
Apple Says HomeView Sets New Record in Design-Focused Interview
2 days ago
Best Apple Deals of the Week: AirTag 2 Hits Record Low Price, Plus Early Black Friday Sales
Best Apple Deals of the Week: AirTag 2 Hits Record Low Price, Plus Early Black Friday Sales
2 days ago
iPhone 18 and 18e Coming in Spring 2027 With A20 Chip
iPhone 18 and 18e Coming in Spring 2027 With A20 Chip
2 days ago
King Charles to Join Elton John's Apple Music Radio Show
King Charles to Join Elton John's Apple Music Radio Show
2 days ago
Select 'Apple Home Collection' Accessories Now Available to Order
Select 'Apple Home Collection' Accessories Now Available to Order
2 days ago
Major Microsoft 365 Outage Continues
Major Microsoft 365 Outage Continues
2 days ago
Apple Launches Two Products Today
Apple Launches Two Products Today
2 days ago
Hands-On With the New Apple TV 4K and Siri AI
Hands-On With the New Apple TV 4K and Siri AI
3 days ago
Siri AI Coming to Older Apple TV 4K Models in tvOS 27.2.1
Siri AI Coming to Older Apple TV 4K Models in tvOS 27.2.1
3 days ago
HomeView Launch Kicks Off as First Orders Reach Customers
HomeView Launch Kicks Off as First Orders Reach Customers
3 days ago
Apple Continues to Move Up iPhone Duo Delivery Dates
Apple Continues to Move Up iPhone Duo Delivery Dates
3 days ago
Apple Explains How HomeView Keeps Camera Data Private
Apple Explains How HomeView Keeps Camera Data Private
3 days ago

🔗 Related Apple News & Rumors

Stay updated with the latest Apple ecosystem news and verified rumors