(Originally Posted on July 17, 2023)
Hey gang! I’d like to bring to life a fuzzy thought that has been bouncing around my head for the past few weeks — infusing the lab with the perspectives/lessons that I’ve internalized over the years working in the disciplines of psychology (and the social sciences more broadly), the humanities, and being a frequent dabbler in the dark arts of computational methods.
That is to say: every now and again — with no regular schedule or specific goals in mind — I’m going to share what I will call “Ryan’s Important Papers” or “RIPs”. Wow, the acronyms write themselves.
I’ll admit — 80% of the time, when someone sends a paper my way, I tell them “Wow, looks interesting! I look forward to reading it!” … and then I don’t read it. I’m a busy guy. We’re all busy, and nobody likes getting “vaguely interesting” papers dropped on their desk — that just feels like homework.
As such, whenever I do post one of these papers in the chat, you can rest comfortably at night knowing that I’m only sharing papers that I think are really, really meaningful for everyone to dedicate some time to reading and studying closely. Sometimes they’ll be deeply relevant to projects that some (or all) of you are working on. Sometimes they’ll just be papers that virtually every (good) psychologist knows about and has read, but seldom makes its way outside of the field, which means that it’s something that might help you think differently about the very nature of human beings. And sometimes, it’ll simply be a paper that is really important and will help you better understand why psychologists do/believe what they do, even if they’re completely wrong.
So, for our first RIP — let’s do something that is broadly applicable to everyone here. A bit of context — often, I’m surprised to learn how little people know about the Harvard General Inquirer. There isn’t even a wikipedia page on it, yet it was THE most important development in early psychological text analysis. Here is a (still unpublished1) snippet that I’ve written about the General Inquirer:
In the early 1960’s, the first formalized, computer-automated method for text analysis in the social sciences made its debut in the form of the Harvard General Inquirer (Stone et al., 1962), a complex system intended to surpass various human shortcomings that plagued manual efforts at text analysis. The General Inquirer was recognized instantly for its incredible potential; widespread enthusiasm over its adoption spread throughout virtually every social science, including political science (Brody et al., 1965), education (Bhushan & Ginther, 1968), anthropology and sociology (McClelland et al., 1966), and psychotherapy (Miller et al., 1968). However, the system plummeted out of favor within a few short years, with commonly-cited causes including its cost, complexity and difficulty, and a perceived failure to harmonize with established theory and practitioners of content analysis in its day (Kadushin et al., 1968; Psathas, 1969).
An important take-home point here: a number of really, really smart people have been working on text analysis questions in the social sciences for over 60 years. These people weren’t hacks — they were quite literally the top scientific geniuses of their time, bringing theories, lessons, and ideas to text analysis that, in many ways, have completely changed the shape of culture and society — some of the basic “truths” that we still have in fields like the study of propaganda, racism, and basic human motives were formalized by people who worked on, or with, the General Inquirer. The take-home lesson here is that, for many of the questions that people are trying to answer with text analysis today: these are not even remotely new questions, and the idea of using text analysis to answer these questions is hardly innovative — most researchers just don’t realize it.
The General Inquirer came on the scene and make a huge splash, but a decade later it was all but abandoned, and no method/approach really took its place in the social sciences until LIWC came along. The reasons why are complex and worth discussing, but not here, and not now. However, people kept talking about computerized text analysis methods and applications for those several decades in-between the General Inquirer and LIWC. Again, I’m often stunned to see people publishing papers claiming to be the first to talk about using text analysis for X or Y, completely unaware that someone else did the same thing nearly 50 years ago, and did it pretty well considering the tools that were available to them.
As such, the first RIP that I’d like to share is one of my favorites from the early 80’s. This is an important read for several reasons. One of the things that I hope everyone will notice is that, while some of the things that Woodrum is saying might seem quaint by today’s standards, several of the key issues that he raises remain just as critical and important today as they were 40 years ago. He mentions that Markoff et al. (1975) referred to text analysis as a “methodological ghetto” (and for good reason) — and this is still true today for a large portion of NLP research happening in (and beyond) the social sciences. Thankfully, the HLAB is one of a few outstanding groups whose research does not fall into this category — while we might take it for granted that text analysis should be combined with as many other methods as possible for conducting good social science research, it is important to appreciate that this is still not the norm in many ways, and in many fields of scholarly research.
So, with that elaborate history lesson — please do read (and enjoy) Woodrum’s classic (and completely forgotten) 1984 paper:
Woodrum, E. (1984). “Mainstreaming” content analysis in social science: Methodological advantages, obstacles, and solutions. Social Science Research, 13(1), 1–19. https://doi.org/10.1016/0049-089X(84)90001-2
P.S. — future “RIPs” will seldom come with such an elaborate background. But, in this case, it’s really worth repeating: there have been innumerable brilliant thinkers grappling with computerized text analysis and the human condition for decades, and it would be a huge mistake for us to pass up an opportunity to learn from their experiences, research, and the unsung history of the ideas that we so often find ourselves exploring over a half-century later.
P.P.S. — I am an amateur collector of General Inquirer papers and have a healthy stockpile of classic publications from the 1960’s-1970’s that used this tool. Happy to share anything that I might have when relevant.
- Since the time of writing the original post, this paper has now been published:
Boyd, R. L., & Markowitz, D. M. (2024). Verbal behavior and the future of social science. American Psychologist. https://dx.doi.org/10.1037/amp0001319 ↩︎