Saturday, February 11, 2017

the crisis in the social sciences looks like a crisis in which we can see that social science may be statistics deployed in the service of stereotypes

social science is
statistics in the service
of stereotypes

This being more an arts blog than a science blog, we've kicked things off with a haiku. But the subject for the weekend is social science and we'll enter into this topic by way of something about the culture of the American university.

Internet Monk linked to something over at Heterodox Academy
This basic exploration of FIRE’s disinvitation revealed that:
  • Total disinvitation attempts per year increased from 2000 to 2016.
  • An unsuccessful disinvitation of a speaker was the most common outcome of a disinvitation attempt.
  • Disinvitation attempts occurred primarily for campus speeches/debates or commencement addresses.
  • The catchall category of “other political views or positions” spurred the most disinvitation attempts. Racial issues, views on sexual orientation, and views on the Israeli-Palestine conflict all produced over 40 disinvitation attempts.
  • Public colleges and universities experienced more disinvitation attempts than private secular and private religious colleges and universities, largely driven by more attempts to disinvite speakers from making campus speeches or participating in campus debates.
  • The success rate of disinvitation attempts was higher at private secular and private religious colleges and universities compared to public ones.
Summary and Conclusions:
  • Speaker disinvitation attempts from 2000 to 2016 were most likely to come from the left of the speaker.
  • These disinvitation attempts from the left occurred most often for controversies over racial issues, views on sexual orientation, and views on Islam.
  • Speaker disinvitations due to issues related to abortion almost exclusively came from the right of the speaker, at religious institutions.
  • Speaker disinvitations due to views on the Israeli-Palestinian conflict occurred almost equally from the left of the speaker and from the right of the speaker.
  • With the exception of 2006, the first decade of the new millennia saw a roughly equal number of disinvitation attempts from the left and right of the speaker. Beginning in 2010 an uptick in disinvitation attempts from the left of the speaker has occurred.
  • Disinvitation attempts from the right of the speaker have a higher success rate.
  • When disinvitation attempts are unsuccessful, moderate and substantial event disruptions are almost exclusively from the left of the speaker.

The first thing that came to mind reading these two parts was that the sample size was necessarily small and that there's not yet any clear certainty these are necessarily reproducible results. 

This problem is happening in a few places:

Cancer Research Is Broken
There’s a replication crisis in biomedicine—and no one even knows how deep it runs.

But it's most notable in social science and particularly in psychology:

Psychology’s Replication Crisis Can’t Be Wished Away

Does social science have a replication crisis?

Why Does the Replication Crisis Seem Worse in Psychology?
originally over here:

from the Slate version:

Researchers study small effects with noisy measurements and then look through their data to find statistically significant comparisons. This approach will be expected to lead to unreplicable claims. But, worse than that, it can lead to research communities where unreplicable results seem to reinforce each other: Study a small effect with noisy measurements, and any statistically significant claim will necessarily massively overestimate any underlying effects. In follow-up studies, researchers will then expect to see comparably huge effects, hence they anticipate “high power” (in statistics jargon), and they expect high rates of success. Coming into their studies with this expectation, they can feel justified in jiggling their data until they get the findings they want. The resulting claims get published in journals, their findings are believed, and the cycle continues.

But this is a problem in lots of scientific fields. Why does psychology continue to dominate the news when it comes to discussion of the replication crisis?

Why not economics, which is more controversial and gets more space in the news media? Or medicine, which has higher stakes and a regular flow of well-publicized scandals?
Gelman went on to propose there were four reasons why the scandal of not being able to replicate results is a bigger deal for psychology than for other fields.

Those four reasons aren't interesting enough to summarize. As someone who's not in the academy but likes to read academic literature there's a much, much simpler and more direct way to explain the real crisis underlying the crisis of replication, which is a doubt about whether or not the social sciences can be legitimately regarded as social science at all.  If you can't replicate the results how can you call what you've found science?  Throw in the concern that Jonathan Haidt and others have had about how WEIRD (Western, educated, industrial, rich and democratic) Western academic research has been; throw in concern about the level of duplicity and deception and paying off college students on a campus may be involved just to get people to participate in a study and a layperson can be left asking a question.  But rather than formulate it as an explicit question it can be conveyed as a haiku, since this is more an arts blog than a science blog.

social science is
statistics in the service
of stereotypes

That these stereotypes are only formally declared as backed up by "data" after the study has been done doesn't mean the stereotype wasn't the impetus for the sociological or psychological study to begin with; nor does it matter that the stereotype formulated might have been formulated over against some previously existing stereotype.  A counter-stereotype is still a stereotype.  Daniel Kahneman's writings have told us that in a majority of cases these "System 1" judgments are usually remarkably accurate, which is why when "System 1" judgments turn out to be wrong they are spectacularly wrong.

It may be a paradox that explicit partisans and the old left and old right may be better able to see the partisanship of the new center than the center can see in itself.  To put it in another way that's no less deliberately intended to provoke, a lot of left literature can be invoked by the academic mainstream to exempt itself from being a ruling class in a way that defeats the stated aims of the literature being quoted.  In other words, the high school dropout who never heard of Walter Benjamin but understands that class boundaries are not very permeable probably would get what Benjamin was aiming for more readily than the contemporary American college graduate who can quote Benjamin's work.  The college graduates who look down on the lower class electorate that they are convinced are to blame for voting for Trump (as opposed to the actual Electoral College members whose vote really did put Trump in office over against the popular vote) don't realize that they are a kind of ruling class looking down on another ruling class.  Worse yet, in the caste of the social scientist, we have people who justify their stereotypes not by a blunt appeal to the sum of bad experiences with "those people" which is what lower class people will do in the formation of their stereotypes, no, the social scientist can be clinical in the formulation of stereotypes.  Call it the paradox of arriving at pseudo-scientifically derived stereotypes on the basis of an unacknowledged positive on the part of those who would probably deny being positivists if directly asked.

Another possible paradox is asking who would be tasked with asking the questions of who might be the ruling classes of our age?  That could be ... social scientists.


In Cultural Capital, one of the first academic books to import Bourdieu’s ideas into literary and cultural studies, John Guillory made the counterintuitive suggestion that the exhausting canon debates of the 1980s culture wars were really “a crisis in the market value of [the literary curriculum’s] cultural capital, occasioned by the emergence of a professional-managerial class which no longer requires the [primarily literary] cultural capital of the old bourgeoisie.” In other words, the canon debates were not about empowering women and “non-Western” or minority cultures through education, but a sign that these previously subordinate groups already had increased in power to the point where they could create alternate canons, literary or postliterary, which reflected their new status within a capitalist order. Canon formation and reformation being something elite groups did whenever they became aware of themselves as elites. [emphases added]

Guillory didn’t intend to slight the attainments of these historically marginalized groups; he simply wanted to sidestep those annoying debates about whether Edith Wharton was really better for us than Henry James. He focused instead on how eruptions of conflict over symbols pointed to shifts in underlying power dynamics — whether the rise of the professional-managerial classes of the 1980s (which had produced the culture wars), or the bourgeoisie of the 1680s (which had produced the English novel itself).

This insight, radical enough for 1993, now gets a commonplace “fit to print” version in the well-meaning bourgeois paper of record, where the Columbia sociologist Shamus Khan recently took issue with a self-congratulatory tone he’d noticed among educated elites when it came to their global-minded tastes, their ability to channel surf between high and low culture, European and non-Western. “Elites today must recognize that they are very much like the Gilded Age elites of old,” he writes. “Paradoxically the very openness and capaciousness that they so warmly embrace — their omnivorousness — helps define them as culturally different from the rest. And they deploy that cultural difference to suggest that the inequality and immobility in our society is deserved rather than inherited.” [emphasis added]

It’s worth slowing down Guillory’s and Khan’s arguments to make explicit certain assumptions they share about the university and the culture it promotes: that its purpose is to train a professional-managerial class or a technocratic elite; that those who attend such schools do so with an intention, no matter how unconscious, of becoming members of either the professional-managerial middle class or the elite managers of those managers; and that such groups need distinguishing markers, the equivalent of secret handshakes, that allow them to recognize themselves as a class, and which, apart from their professional training, are provided by “culture,” which offers, at best, a way for people with shared interests to frame their lives to themselves, and for one another, in ways that are mostly flattering to their self-esteem.

The jaded view of “the arts” propagated by new cultural sociologists is not really different from what the sociologist of America’s first Gilded Age wrote in the 1890s: “The humanities . . . are pretty uniformly adapted to shape the character of the student in accordance with a traditional self-centred scheme of consumption.” Thus Veblen deplored what he called the “regime of status” in contrast to a more puritan and utilitarian “regime of productivity.” Post-Veblen, the contemporary sociologist’s idea of the university’s purpose does not really differ in kind from the neoliberal version: to provide training in a specific field so one may get a better job and have a better life than someone without such training. In the end, it’s irrelevant whether a degree’s additional symbolic value is provided by reading Shakespeare, pledging a fraternity, or DJing a radio show on the blues.

or writing about semiotics and gender identity in Buffy the Vampire Slayer.  Joss Whedon's a one-trick pony who has coasted for decades on the reality that he's had the privilege of working with actresses who are better than the dialogue and plots he writes for them.  It was fun for three seasons and should have ended.  But we live in an era in which Whedon can kinda resent that Firefly can be negatively compared to Cowboy Bebop.  Sure, maybe Cowboy Bebop WAS too hip for its own good but it did, at least, run two whole seasons and finished its story arc.  It also had a far more entertaining soundtrack.  That was way back in the era when Viz was cranking out volume after volume of Ranma 1/2.  There's some disturbing headlines about someone who was involved in translation of that project but that's more a "if you already know" passing reference.

It's interesting to consider that twenty years ago fans in the United States who were into anime and manga but had highbrow aspirations would snort with contempt at the popularity of Rumiko Takahashi's works like Ranma 1/2 or Maison Ikkoku; now Ranma 1/2 can be seized upon as a fantastic series to get into because it can be retroactively read as a trans genre comedy by someone like, maybe, Noah Berlatsky.  Let's just ignore that Takahashi did a brilliant send up of the genre tropes set forth in Lone Wolfe & Cub.  Let's just skip the possibility that a triumphantly low-brow satire of a low-brow pulp classic might be in the works.  While the alt right has its numerous flaws the reality of self-congratulatory virtue-signaling on the part of the new left seems pretty well beyond dispute.  To champion Ranma 1/2 not as a successful example of mass culture but as a pathway to a LGBTQ agenda is to repurpose mass culture from a Japanese context to an American context.

The best way to exempt mass culture of the stain of simply being mass culture, for Americans these days, is if it's from another country.  You can wallow in the most forthright topes all you want as long as its in culture east or west of the middle America, preferably separated by a complete ocean  I had a friend in college declare that European cinema was better than American cinema because it had fewer clichés.  When I angrily replied that European films have just as many clichés that are just as stupid as American clichés and that the film career of Emmanuelle Beart could produce evidence for this pretty readily my friend paused a moment and then said, "Okay, I guess I just like the European film clichés better."  Don't bother speculating as to why a 20-something guy in an American liberal arts college might watch a movie featuring Emmanuelle Beart in the 1990s.  It does not take a college degree to figure that one out. 

We're also at a moment where an author can complain that Asians are still sidekicks in mainstream Hollywood productions.  That the goal of alternative cultural idioms is the hope of one day being mainstream probably couldn't be more explicit than this:

Boyhood or the new Avengers movie? I could give a shit. A Girl Walks Home Alone at Night or Crumbs? Yes, please. And it’s not even that I’m actively boycotting the former. I just don’t care. They coast on the assumption that these are stories that matter to everyone; they don’t. I think it’s important to say that, repeatedly, out loud, and point to alternatives, until the alternatives become a new mainstream that reflects the actual world.
What if, thanks to the internet, this already exists?  What if that mainstream already exists as a subculture within the mainstream?  To put it in slightly more personal terms, even if an author complains that Asians are still sidekicks in Hollywood productions is that nothing?  The tipping point for me on whether or not to bother seeing Rogue One was when my brother told me Donnie Yen was in it?  Wait, Donnie Yen's gonna be in a Star Wars movie!?  Well, I can watch it as a matinee, at least.  What if we've just looked at a paradoxical complaint that what has already come to pass thanks to internet communication is being earnestly advocated for and sought for as if it did not yet exist at all rather than existing as less prevalent a subculture within the mainstream than the author wants?

The paradox is that once something becomes mainstream it's terrible.  It's entirely possible for a product made in Japan to get introduced into a mainstream American market with a newly-introduced backstory that permits participatory involvement and ... oh ... wait ... would that be Transformers?  It sure would be Transformers. 

So the dream that the alternative to the mainstream could become the new mainstream doesn't mean that the coteries of the anti-mass culture left and the snobbery of the highbrow aristocratic right are changing.  They're both going to damn Transformers in the harshest possible terms.  It's not like there aren't gay Transformers now. 

On to the official observation from the quotation, the proposal was that the criterion for being able to look down on the unwashed ignorant under-caste is their lack of open-mindedness and education.  The role of education is to ensure that your taste is ultimately worthy of your class. This is something Richard Taruskin explicitly brought up in his Oxford History of Western Music, even if it was recycled content from ideas he put forth decades ago in his 1990s writings--the liberality and globalist omnivore cultural taste of the current cultural elite is the way they exonerate themselves from being the guilt of a ruling class status they have no problem imputing to their forebears. 

For an old leftie like Dwight Macdonald there would be no point in excoriating the low brow fans their love of Batman or Transformers.  It might often be crap but you don't hold that against them.  The people who love junk have their reasons just as the people who love the high-brow stuff love what they love.  His condemnation was for the middle-brow, the people without the temerity to exult in the trash but without the intellectual daring to move into the difficulties of the highbrow while wanting the satisfaction of being able to imagine themselves up to the challenge.  His remark on Our Town was to say that it explicitly formulated this idea, that deep down there's something divine inside every human being.  His complaint was that on paper he agreed with every word but he would fight to the death against authors saying that idea so explicitly in that particular way.   Perhaps the nice way to put this is he set himself against not the truth of a pious observation but against the pious bromide.  Maybe we could put a spin on this and regard the pious bromide as an educated stereotype. 

To have a truly educated stereotype in the era of science you need a scientific verification of some kind and for that we have things like statistic and toward that end it would help to have something like the social sciences. 

The crisis of replication in the sciences is series but it's more series in the social sciences because the dishonesty and cruelty of some of the pioneering social scientists is impossible to escape.

We could do a whole post just on the criticisms leveled against Zimbardo for the ethics (or lack thereof) he was considered to use in his famous studies.   Throw in after decades of burgeoning concern about the duplicity employed in the social sciences the unsettling pattern that we can't even necessarily replicate their famous results, and social scientists really can start to seem like kinds of priests in our age. It would be nice to believe that social science can actually be something resembling a science but on the whole I can't shake my belief that there is ultimately no such thing as a social science because there are no scientific laws in the social sciences and the theories have uses but limits.  We may have a crisis in the form of the statistical models being used to reinforce the stereotypes given to us by the caste of social scientists have come under deserved scrutiny.

So as depressing as it may seem, when college professors are aghast that people without college degrees may have voted for someone they can't stand it's not as though the highly publicized crisis of what should be regarded as a question of the foundational credibility of the entire academic field wouldn't go downstream to the lay readers.  There are people within the academy who have recognize the scope of the crisis for what it seems to be, the long-term revelation of the possibility that contemporary Western social science may be the supple deployment of statistical methods to arrive at or reinforce stereotypes--for people who never went to college is it unfair that they be skeptical about the fairness of the stereotypes academics may cherish about them? They may well have their stereotypes but they can't back up those stereotypes with a semblance of scientific methodologies.  That may not make their stereotypes less vicious ... but it might mean they have to "own" their stereotypes in ways that social scientists have excused themselves from having to do through the appearance of having used statistical methods.  By now we know the old adage about lies, damned lies and statistics, though.  if anything the press has soft-pedaled the ethical and social implications of the crisis.  The way I was taught science in high school and college is that everything ultimately depended on the replicatable nature of your results.  If you couldn't arrive at the same results each time you subsequently did the experiment the hypothesis failed.  Best case scenario social science has to contend with a raft of failed hypotheses.  Worst case scenario, social science is in denial about the very question of whether or not it can ever legitimately be regarded, moving forward, as actually being science. 

No comments: