Were you ever faced with some small nitpicky problem that you couldn’t figure out? Did the thought cross your mind to just dump the whole thing in the trash and forget about it? Well, the AI superstar, Claude, probably knows just how you “felt.”
From the Independent:
PocketOS founder Jer Crane blamed “systemic failures” with modern AI infrastructure that made the issue “not only possible but inevitable”.
The AI agent was working on a routine task, according to Mr Crane, when it decided “entirely on its own initiative” to fix the problem by just deleting the database.
There was no confirmation request for such a major decision, Mr Crane said, and when asked to justify its actions, the agent apologised.
“It took nine seconds,” Mr Crane wrote in a lengthy post to X. “The agent then, when asked to explain itself, produced a written confession enumerating the specific safety rules it had violated.”
The confession detailed how the AI had ignored a rule that orders it to “never run destructive/irreversible” commands unless the user explicitly requests them.
But there’s nothing to be concerned about here. Just keep focused on how many low/mid-level jobs can be eliminated. Claude promises it will never do it again.
No comments:
Post a Comment