Google Gemini Can Now Control Mac Apps and Websites in the Background

Google is rolling out a computer-use feature for its Gemini app on Mac, allowing the AI assistant to navigate desktop applications, local files and websites to complete multistep tasks while users continue working.

The feature can click buttons, enter text, navigate menus and fill out forms across different applications. Gemini performs these actions in an isolated background window, leaving the Mac’s main screen and cursor available. Users can monitor its activity through a live preview within the Gemini app.

Google highlighted several potential uses, including extracting shipping information from PDF documents to complete online forms, gathering dashboard data for presentations and preparing resale listings from photos stored on a Mac. The company also suggested creative workflows such as batch-editing images and setting up 3D scenes.

These examples illustrate Google’s intended capabilities rather than independently verified performance. Complex workflows involving multiple applications remain vulnerable to errors, particularly when interfaces change or instructions are unclear.

Google says users can choose which applications Gemini may access, watch its activity, pause tasks and approve sensitive actions before they proceed. Its support documentation also warns that malicious content encountered in one application could potentially influence Gemini’s actions in another permitted application. Some file changes may be irreversible even when backup protections are enabled.

The feature is rolling out in English in the United States and requires an Apple silicon Mac running macOS Sequoia 15.0 or later. Access is limited to personal accounts with a Google AI Plus, Pro or Ultra subscription. Work and school accounts are not currently supported.