A nice essay by Leslie Lamport: . (Ironic that this is by the guy who wrote LaTeX, of all things, but it's a good essay nonetheless.)
Bob Colwell: "Design Fragility" 
- Complexity breeds fragility.
- Fragility breeds surprises.
- Surprises are bad.
Good quote from Anders Hejlsberg : "There's one kind of simplicity that I like to call simplexity. When you take something incredibly complex and try to wrap it in something simpler, you often just shroud the complexity. You don't actually design a truly simple system. And in some ways you make it even more complex, because now the user has to understand what was omitted that they might sometimes need. That's simplexity. So to me, simplicity has to be true, in the sense that the further down you go the simpler it gets. It shouldn't get more complicated as you delve down."
"We have simple solutions, what we need are simple problems." (Benjamin Franklin) Objection: But our problems are so frightfully complex... Reply: Well, then simplify them... RS
"The mistake frequency of programmers and coders increases with the quantity of coding to be written. Short pieces of program can be written error-free, whereas long routines will inevitably contain mistakes." (Grace Murray Hopper, 1958)
I can't help but add: "Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary" . Personally, I think the dodekalogue hits this mark much better than (at least the later) Scheme reports.(TCV notes that to be fair to the Schemers, there are many who feel that the latest Scheme standard (R6RS) went somewhat against this grain of simplicity. The ratification results page  contains some interesting rationales from both sides.)
The above can be taken as a paraphrase of Antoine de Saint-Exupery:"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."
dbohdan 2015-09-09: The talk Simple Made Easy (transcript) by Rich Hickey of Clojure fame is relevant here. It pays to not confuse the simple with the merely easy.