The discussion at today's tech coffee meetup turned philosophical. At one point we asked whether it was even possible to perfectly determine which course of action was best under utilitarianism. I argued no, because to do so would require calculating the complete and exact consequences of all available actions, performing that calculation would require running something like a fully-accurate simulation of the complete future of the universe, and a computer big enough to run such a simulation probably wouldn't fit inside the universe. I think that's probably true, but it's a bit hand-wavy, and I can at least conceive of a universe that could contain a full simulation of itself. Is there a less hand-wavy foundation for the argument?
Suppose that a genie has created an immortal rat and a Turing machine, both inside an indestructible box. Tomorrow the machine will start running at one operation per second. Each second, the machine dispenses a pellet of delicious rat food (+1 util/s), except if it's in a halt state, it will give the rat an electric shock instead (-1 util/s).
You are given a button. If you push it before the machine starts, the rat will instead receive shocks while the machine runs (-1 util/s) and food while it's halted (+1 util/s). The Turing machine's program and initial tape state have been provided to you.
| Rat's net utility | Machine halts | Machine runs forever |
|---|---|---|
| Button not pushed | −∞ | +∞ |
| Button pushed | +∞ | −∞ |
You should press the button iff the Turing machine will halt. Therefore, determining the best course of action, in the general case, requires solving the halting problem.
2025-08-16
↑ Home