Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
iOS 26 was released last month, but the software train never stops, and iOS 26.1 beta testing is already underway. So far, iOS 26.1 makes both Apple Intelligence and Live Translation on compatible AirPods available in additional languages, and it includes some other minor changes across the Apple Music, Calendar, Photos, Clock, and Safari apps.
More features and changes will follow in future ...
Apple plans to cut production of the iPhone Air amid underwhelming sales performance, Japan's Mizuho Securities believes (via The Elec).
The Japanese investment banking and securities firm claims that the iPhone 17 Pro and iPhone 17 Pro Max are seeing higher sales than their predecessors during the same period last year, while the standard iPhone 17 is a major success, performing...
With the fourth betas of iOS 26.1, iPadOS 26.1, and macOS 26.1, Apple has introduced a new setting that's designed to allow users to customize the look of Liquid Glass.
The toggle lets users select from a clear look for Liquid Glass, or a tinted look. Clear is the current Liquid Glass design, which is more transparent and shows the background underneath buttons, bars, and menus, while tinted ...
Apple's software engineers continue to internally test iOS 26.0.2, according to MacRumors logs, which have been a reliable indicator of upcoming iOS versions.
iOS 26.0.2 will be a minor update that addresses bugs and/or security vulnerabilities, but we do not know any specific details yet.
The update will likely be released by the end of next week.
Last month, Apple released iOS 26.0.1,...
iOS 26.4 is expected to introduce a revamped version of Siri powered by Apple Intelligence, but not everyone is satisfied with how well it works.
In his Power On newsletter today, Bloomberg's Mark Gurman said some of Apple's software engineers have "concerns" about the overhauled Siri's performance. However, he did not provide any specific details about the shortcomings.
iOS 26.4 will...
While the new iPad Pro's headline feature is the M5 chip, the device has some other changes, including N1 and C1X chips, faster storage speeds, and more.
With the M5 chip, the new iPad Pro has up to a 20% faster CPU and up to a 40% faster GPU compared to the previous model with the M4 chip, according to Geekbench 6 results. Keep in mind that 256GB and 512GB configurations have a 9-core CPU,...
Apple on Wednesday updated the 14-inch MacBook Pro, iPad Pro, and Vision Pro with its next-generation M5 chip, but previous rumors have indicated that the company still plans to announce at least a few additional products before the end of the year.
The following Apple products have at one point been rumored to be updated in 2025, although it is unclear if the timeframe for any of them has...
Apple plans to launch MacBook Air models equipped with the new M5 chip in spring 2026, according to Bloomberg's Mark Gurman. Apple is also working on M5 Pro and M5 Max MacBook Pro models that will come early in the year.
Neither the MacBook Pro models nor the MacBook Air models are expected to get design changes, with Apple focusing on simple chip upgrades. In the case of the MacBook Pro, a m...
Apple on Wednesday updated the 14-inch MacBook Pro base model with an M5 chip, and there are two key storage-related upgrades beyond that chip bump.
First, Apple says the new 14-inch MacBook Pro offers up to 2× faster SSD performance than the equivalent previous-generation model, so read and write speeds should get a significant boost. Apple says it is using "the latest storage technology," ...
