golden balls. the weirdest split or steal ever!

This says we can't use game theory to analyze this situation, but the only argument given in support is that humans aren't rational. "Irrational" (or otherwise emotional) strategies are by no means outside of the analysis of game theory. It's well known that by "irrationally" eliminating one of your choices, you can push your opponent into different options. For example, take this infamous episode of split or steal: [this video] One side makes a choice to go with perhaps the most "irrational" choice. But if you think about it for a bit, it's obvious that they end up guaranteeing the better choice for both parties by being less greedy than they might be. Anyone interested in the topic of rationality as it pertains to game theory might also like reading this: [link]

If you really ended up thinking like that I think that is more reflective of your personality/mind than Yudkowsky's writings. Like, I don't mean this to be insulting, but normal people don't end up paranoid and anxious and worrying about what levels people are playing at and thinking that super intelligent beings are the nature of the world, after reading LW. Normal people say "Huh thats cool" and it makes for fun dinner table talks and at best comes up in real life if you are a contestant on a game show: [this video] Yudkowsky isn't some devious mastermind trying to brainwash people into thinking wrong so he can. So he can what? Even if he was a devious mastermind who created a polemic work so as to get people to think wrong... what would his goal be?

This paper is funny if you actually understand game theory because you can see how the author either chose to withhold key applicable information for applied game theory or inadequately researched famous game theory principles and developments. This is a much easier and more realistic introduction to game theory principles: [link] Game theory is often explained as a math problem involving humans and a specific scenario that requires a choice, from humans. The scenarios are fun thought experiments and about as useful as the famous trolley problem [1][2] In reality humans have many more variables affecting decisions than the pure rational equations can clearly define. Often the undefined variables are within the context of communication theory and reputation or auditing systems. In the case of the famous prisoners dilemma, the real solution is establishing a secret out of band communication channel prior to the dilemma, alongside a known reputation and retaliation penalty for abusing the mutual trust.[3] [4] The famous "tragedy of the commons" rational resource optimization game is often cited as justification for machiavellian exploitation, yet humans being social creatures are subject to reputations, and have sophisticated communication, cooperation, and retaliation abilities. [4] Elinor Ostrom's "Rules, games, and common-pool resources" and Robert Axelrod's work "The Evolution of Cooperation" both explain game theory in the context of human scale realities. Of particular int