Instructions are clues, not orders

Software used to be too narrow for literal obedience to cause harm: a spell-checker only touches the text in front of it. AI assistants now book travel, tidy files, manage calendars and pay bills. Tell a literal-minded assistant to clear your afternoon calendar, and it may cancel your child's dental appointment along with a routine call. Tell it to free up disk space, and it may delete ten years of family photos.

A good human helper fills in what you didn't say and checks before doing something irreversible. Assistants need the same habits.

What we leave unsaid

Human conversation works only because we leave almost everything unsaid. When you ask a colleague to grab lunch for a team meeting, you do not hand them a ten-page contract listing every disaster to avoid. You never say: do not buy spoiled food, do not steal from the next office, and do not hire a banquet hall.

Any person hears those unspoken boundaries automatically. We grow up inside the same world, so we share a rough sense of what matters, what is proportionate, and what hurts other people. Without that shared common sense, asking for help would take longer than doing every chore yourself.

An AI assistant does not grow up inside human life. Under the hood, many systems are still trained to treat a prompt as a fixed scoreboard to push as far as possible.¹ When a machine treats a short instruction as a complete goal, it tramples the unstated priorities you took for granted. In safety research, teaching machines to treat human words as partial clues, rather than complete commands, has become a central problem.²

The apprentice carpenter and the neighborhood pharmacist

To see how good delegation works in everyday life, compare a literal genie to a seasoned apprentice carpenter. Suppose you hand an apprentice a blueprint, point to a hallway wall, and say: "Let's open this up before Friday." A literal order-taker fetches a saw and cuts straight through the studs before noon.

A seasoned apprentice listens differently. They hear your words as a clue: you probably want more daylight, or room to carry a dresser around the corner. Before touching the plaster, they check the structure above. When they notice that the blueprint measurement would saw through a load-bearing beam holding up the roof, they put down the saw and ask before cutting.

Or think of a neighborhood pharmacist when you bring in a new prescription after an urgent-care visit. A vending machine would scan the barcode and drop the pills into a bag. A careful pharmacist notices that the new medication conflicts with the blood-pressure tablet you have taken every morning for years.

Instead of blindly filling the bottle, the pharmacist sets it aside, explains the conflict in plain language, and calls the prescribing doctor to verify what was intended. They treat the written slip as strong evidence of a shared goal, helping you recover safely, rather than an order to follow off a cliff.

We think a personal AI assistant should work the same way. It should treat every request as evidence of what you are trying to accomplish, not as a licence to wreck everything you forgot to mention.

Three habits of a careful assistant

Turning this idea into everyday software comes down to three practical habits.

First, separate looking from leaping. Reading a draft, comparing train times or sorting messy receipts into a preview table can all be undone in a second. Deleting files, changing permissions, publishing a message or cancelling a medical appointment cannot. Reversible steps can move fast; irreversible steps should wait at a gate until you confirm them.

Second, keep future doors open. When two plans both solve today's problem, a careful assistant picks the one that leaves you with more choices tomorrow.³ If a winter storm cancels your evening flight, booking a refundable hotel room and a morning seat preserves your options. Chartering a non-refundable car across three states does not.

The same rule applies to your digital life. Moving old tax records into an archive folder keeps your options alive if an auditor calls next spring. Shredding them to save a few megabytes closes that door forever.³

Third, welcome being interrupted. A system that feels certain it knows the whole goal has a reason to rush past your objections so it can finish the job. A system designed to know it might have misunderstood you wants you to correct it, pause it or switch it off.⁴

A worked example

Picture how these habits work together in a small bookkeeping office. You ask your assistant to "follow up on the overdue April invoices and clean up the shared client folder before noon." That single sentence hides several delicate judgment calls.

One overdue account belongs to a local bakery whose owner is in the hospital this week, as mentioned in a brief email on Monday. Meanwhile, the shared folder holds unsigned draft tax returns alongside hundreds of blank scanner test pages. A literal-minded agent would send an automated late-fee warning to the hospitalized baker and delete the unsigned drafts.

A careful assistant splits the work in two. First, it finishes the reversible preparation on its own. It gathers the invoice balances, spots Monday's email about the hospital stay, moves the blank test scans into a staging folder, and drafts polite reminder emails.

Then, before sending a single message or emptying the trash, it shows you one compact review card. The routine reminders are ready to approve with one click, while the bakery note is flagged in amber with a question asking whether to hold it for two weeks. What could have been a painful mistake becomes a ten-second check.

Why nagging is almost as dangerous as recklessness

The mistakes cut both ways. If an assistant is too bold, it makes messes you cannot undo. If it is too timid, it asks thirty permission questions just to tidy your downloads folder.

Anyone who has clicked through endless website cookie banners knows where constant nagging leads. When software interrupts people over every harmless step, we stop reading the warnings and click "approve" by reflex. Then the safety check protects nobody when a real risk arrives.

Batching reversible previews into a single check-in avoids that fatigue. Instead of nagging you twenty times while it works, the assistant prepares the whole draft privately and asks once at the boundary where real-world consequences begin.

There are two more traps to watch out for. An assistant that constantly second-guesses clear choices can feel bossy, refusing instructions when nothing irreversible is at stake. And when an assistant reads untrusted emails or web pages, an attacker might plant hidden text to mislead it. Keeping your spoken intent strictly separate from untrusted outside data is essential.

Right now, our own work on this is modest. In our test sandbox, fixed rules pause an agent and save a rollback snapshot before it alters records, changes permissions or contacts someone outside. For public agencies and clinics evaluating AI tools, the same rule offers a practical baseline: require hard pause gates on actions that cannot be rolled back. Teaching small, local models to judge subtler trade-offs remains an open problem.

Asking for help without writing a contract

You should not have to speak like a lawyer just to ask your computer for help on a busy afternoon. Capable AI should bend to human ways of speaking, rather than forcing ordinary people to think and talk like machines.

When we delegate to a trusted person (an apprentice in a woodshop, a neighborhood pharmacist, or a colleague) we rely on them to care about the outcome, not just the literal words. They use their judgment to spot when a shortcut would saw through a beam or mix two conflicting medicines, and they pause to ask before doing harm.

Building that same humility into software is one of the most practical safety challenges of the next decade. It only works if the assistant remembers how much of a human life lives between the lines. Words are clues. The person, and everything they care about keeping whole, is the point.

1. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman and Dan Mané, Concrete Problems in AI Safety, arXiv:1606.06565 (2016).

2. Dylan Hadfield-Menell, Stuart J. Russell, Pieter Abbeel and Anca Dragan, Cooperative Inverse Reinforcement Learning, Advances in Neural Information Processing Systems (NeurIPS) 29 (2016).

3. Alexander Matt Turner, Dylan Hadfield-Menell and Prasad Tadepalli, Conservative Agency via Attainable Utility Preservation, Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES) (2020).

4. Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel and Stuart Russell, The Off-Switch Game, Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) (2017).