One of the things that drives me up the wall in Internet stories about rationality is their lack of rationality, logic, and general understanding of the world. Sadly, Yudkowsky is one of those bad examples.
Chapter 14 of Harry Potter and the Methods of Rationality has a long discourse on the word “mystery,” and it just reveals that he doesn’t understand its definitions. It is possible that some of his sources shared this ignorance, but we have dictionaries for this purpose.
So let’s talk about this.
The root word, the Greek “mysterion,” is from the Greek word “mystes,” initiate, one who has been initiated. A mysterion is what the mystes has been initiated into. So it could be a religious ceremony or a religious secret teaching. It was also what you called the religious artifacts used to conduct a mysterion religious ceremony.
Later (but still in classical/ancient times), the word was extended to mean things that had to be taught to students of the sciences by someone who knew those things. So every science textbook is a book of mysteries. (Which is hilarious, but sadly I didn’t know it when I began this post.)
Jesus talked about teaching His apostles “the mysteries of the kingdom of Heaven;” and He was referencing the Septuagint translation of Daniel, which talks extensively about God as “the revealer of mysteries.” Paul also talks about God’s “mysteries” in many places. But the point is that these things are being revealed by God, even if people can only understand them partially now, in this life. Again and again, “I show you a mystery” and “He made known to us a mystery.”
If one does not understand a religious mystery, one is supposed to think about it and keep working on it. That’s the whole point of saying that something is a mystery. If the existence of a mystery hadn’t been revealed, you wouldn’t be able to think about it. Duh.
Moving along… in Latin, “mysterion” became “mysterium.” Again, the primary meaning was a religious ceremony, which eventually was more commonly known in Christian circles as “sacramentum,” an oathtaking, which referenced God’s covenant. The secondary meaning was a religious secret teaching, or any kind of secret.
From there we come to the English word “mystery,” which incorporates pretty much every previous meaning. The “mysteries” of a trade were proprietary techniques that you had to be taught. “Mystery novels” are stories about secret activities which must be riddled out.
The main problem with Yudkowsky’s understanding seems to be that some denominations, or teachers, seem to use “mystery” to mean an insoluble puzzle or question, whereas the normal theological meaning is that an answer exists but cannot presently be understood.
There’s even an assumption that parts of the answer can be figured out, in this life, because otherwise theology classes would be really short.
So if something is considered “a mystery beyond the reach of science,” does the person mean “right now, but not later”? Does the person mean “I think this is properly part of another field, and requires investigation with other techniques”? Is the person just a pessimist? Or what?
This is what Yudkowsky doesn’t seem to get. He throws everything together in a heap, and complains about things as if they were all the same.
One of the reasons that Western science got the jump on everyone else was that, unlike the Greeks, Romans, Chinese, etc., the early Christians and medieval Catholics all believed in an orderly Creation by God, Who is Logos. If all Creation was created with number, weight, and measure and with all the elements arranged in order (cf. Wis. 11:21), then there are indeed orderly and discoverable answers to all questions about the created universe.
The real methods of rationality are methods of Logos (or Dabar), and there’s nothing to complain about. If God were not running an ordered and discoverable universe with beings of free will, there’d be nothing to say or discover.
All that said… it’s still a good story, albeit with some indulgent bits.
Aliens in This World 
“There’s even an assumption that parts of the answer can be figured out, in this life, because otherwise theology classes would be really short.”
Not to mention that such short classes would highly dissatisfying. It’s one thing to be told, “We don’t have all the details yet – we know this much, no more for certain. Maybe if you think about it you’ll find another piece of the puzzle. Just don’t teach your theories as fact until you can reason them out to the point they don’t contradict anything else we know for sure.” It’s another thing entirely to be told: “This is it, no you can’t think about or study it, shut up and don’t ask questions.” The number of people who think “mystery” means the latter and not the former when it comes to religious instruction is honestly sad. :,(
Not having done ‘the homework’ (though reading with interest such things as Hermann Gruber’s 1910 Catholic Encyclopedia articles “Illuminati” and “Masonry (Freemasonry)”, Vernon Stauffer’s New England and the Bavarian Illuminati, and John Roberts’ The Mythology of the Secret Societies), I lazily wonder how impudently the language of ‘Enlightenment’ may have been deliberately coined with full knowledge of the apprehension of Baptism as ‘Photismos’ and ‘Illuminatio’.