(Don't) Use Computer

Anthropic has just released a new product called "Use Computer" which allows Claude to, erm, use a computer. They provide the Claude, you provide the computer.

With the existence of multimodal models, this was quite an obvious step - and I'd be surprised if someone in the world of open source hadn't already rigged up something similar. If you just wanted a browser, you could probably wrangle something together with an API key and puppeteer - the innovation here is the interface that lets Claude interact with any element on your computer desktop. And the productisation.

Anyway - the exciting thing about this release is that it allowed people to demonstrate how giving an LLM control over your computer is incredibly risky. These demonstrations are almost redundant, as it's trivially obvious that such vulnerabilities exist and will be exploited. Anthropic knows this, and provides a lengthy disclaimer where they tell you just how dangerous this thing is:

Computer use is a beta feature. Please be aware that computer use poses unique risks that are distinct from standard API features or chat interfaces. These risks are heightened when using computer use to interact with the internet... In some circumstances, Claude will follow commands found in content even if it conflicts with the user's instructions. For example, Claude instructions on webpages or contained in images may override instructions or cause Claude to make mistakes. We suggest taking precautions to isolate Claude from sensitive data and actions to avoid risks related to prompt injection.

by Unknown

Anthropic Docs

Go to text →

I would go further and say that if you let Claude use your computer, you should assume that it's owned. Wipe and reset any passwords and credentials associated with it.¹

What's wrong with LLMs

LLMs can't distinguish the operations they are tasked with from the data that they are operating on. Tellingly, some of the most commonly exploited vulnerabilities in traditional software have occurred when programmers have accidentally allowed the data a program operates on to be interpreted as instructions. Think of SQL injection attacks, or all the damage caused by people running JavaScript's eval on user input. Consider that when building an LLM system, you are always operating in this mode.

It's critical to remember that when you use an LLM to build systems that operate with the outside world or accept user input then you are in a very hazardous environment. The more capabilities you give the LLM, the more dangerous things become. Giving an LLM access to your own computer, and letting it browse the internet is pretty much the worst case scenario.

I really cannot state this firmly enough. You should never let an LLM drive a computer with any data you would prefer kept secret, or with access to anything even vaguely important. If you do, you will be compromised, lose data, be hacked and you will only have yourself to blame.

Don't use Computer²

¹ Why owned? Well consider the following: Any content the LLM processes could contain hidden instructions. The LLM has access to everything the desktop session can access. It's nearly impossible to audit what the LLM might have accessed or modified. The attack surface is enormous - any website, document, or image could contain malicious prompts. It's almost impossible to tell if any given action taken by the LLM is malicious. Rendering an image, for example, is a classic chat interface vulnerability.

² unless it's in a locked down sandbox. Maybe a docker container with access to a browser? Maybe a VM running on a cloud? IDK.