Honor’s New AI Agent Can Read and Understand Your Screen

It selected a restaurant, however then could not full the method because the spot it selected required a bank card to substantiate a reservation, at which level the person needed to take over. You might be versatile in your question—in one other instance, asking it to ebook a “extremely rated” restaurant meant it might have a look at critiques with excessive scores, although the agent does not do any extra analysis than that. It isn’t cross-referencing OpenTable critiques with knowledge from different elements of the net, particularly since all of this knowledge is processed on system and is not despatched to the cloud.

This type of agentic synthetic intelligence is the present buzzword within the tech sphere. My colleague Will Knight lately examined an AI assistant that would browse the net and carry out duties on-line. Google late final 12 months unveiled its Gemini 2 AI mannequin skilled to take actions in your behalf. It additionally renews the concept of a generative person interface for smartphones—at MWC 2024, we noticed just a few corporations engaged on methods to work together with apps with out utilizing apps in any respect, as an alternative leaning on AI assistants to generate a person interface as you issued a command.

Honor’s method feels considerably like what Rabbit—of the notorious Rabbit R1—is doing with Educate Mode, the place you prepare its assistant manually to finish a activity. There isn’t any have to entry an app’s Utility Programming Interface (API), which is the standard approach apps or companies talk with one another. The agent memorizes the method, permitting you to then subject the command and have it execute the duty.

However Honor says its self-reliant AI execution mannequin is not skilled to comply with strict steps—it is able to multimodal display screen context recognition to carry out duties autonomously. As a substitute of getting to coach the assistant to study each single a part of the OpenTable app, it’s able to understanding the semantic components of the person interface and can follow-through with a multi-step course of to execute your request. Honor highlighted that this course of was less expensive: “In contrast to opponents corresponding to Apple, Samsung, and Google, which depend on exterior APIs—leading to increased operational prices—Honor’s AI Agent independently manages a variety of duties.”