This post is going to assume some knowledge of AI safety as a field. The short explanation is that some people think that artificial general intelligence (AGI) has the potential to cause human extinction, or close to it, because of the difficulty of correctly specifying human goals. To try to get a sense of this, imagine designing a fully autonomous, superintelligent cleaning robot and trying to design a numerical reward function that it can use to learn how to clean stuff. Now imagine a baby goes into the room it’s trying to clean, or that the room has priceless ming vase, or frayed electricity wires. AI safety is the study of how to try to make sure that any very powerful AI systems we design are good for the world.
When is AI safety research bad?
When is AI safety research bad?
When is AI safety research bad?
This post is going to assume some knowledge of AI safety as a field. The short explanation is that some people think that artificial general intelligence (AGI) has the potential to cause human extinction, or close to it, because of the difficulty of correctly specifying human goals. To try to get a sense of this, imagine designing a fully autonomous, superintelligent cleaning robot and trying to design a numerical reward function that it can use to learn how to clean stuff. Now imagine a baby goes into the room it’s trying to clean, or that the room has priceless ming vase, or frayed electricity wires. AI safety is the study of how to try to make sure that any very powerful AI systems we design are good for the world.