As Philomel, Sinfonia, Gruppen, and Piccola Musica Notturna show, even 12-tone organization is not the issue. It strikes me that the deciding factor is whether or not the listener senses that there is some organizational factor that you’re supposed to be hearing that can’t be located by ear, whether the meaning of the piece is buried somewhere underneath the surface. That quality seems to be more what Holland objects to about Perle than the mere lack of tonality. I was dumbfounded by the quotation Alex Ross in his book unearthed from Boulez; asked why the serial pieces of the ’50s never became standard repertoire, the meister admitted, “Perhaps we didn’t pay enough attention to how people listen.” In general, and as evinced by a thousand film scores, atonality tends to express anxiety, and much of the music, like Sun-Treader, that freely acquiesces to that is extremely effective. But Wolpe’s output is Exhibit A that music can be relentlessly atonal and also whimsical, jaunty, and attractive.
The way Leonard Meyer put it half a century ago was that the problem total serial music was facing was that understanding the rules of the precompositional process was not the same thing as understanding the end result. You can't any more appreciate a piece of serial music because you have grasped the tone row than you will appreciate Beethoven's Eroica simply because you know what the key of E flat is.
Gann's pointed out the obvious, but as a teacher of mine used to put it, don't underestimate the obvious. Atonality has been great as expressing emotions like dread and anxiety and fear. Erwartung isn't exactly camping out in the emotional/social realms of My Little Pony: Friendship is Magic.
As polemics go, though, there's another polemic that can be made and has been made about when atonality does and doesn't stick but we may save that for some other post.