Acceptability judgements tell us about social norms, not about internal systems

Since the 1960s, many works on syntax have primarily relied on acceptability judgements, rather than on examples attested in corpora, as was common in earlier times. In Jespersen’s Essentials of English grammar (1933), there were many invented examples, but also quite a few observed corpus attestations (from authors such as Shakespeare, Austen, Thackeray, Carlyle). But over the last five decades, syntacticians have relied much more on experimental methods, which allowed them to make great progress in exploring the incredibly rich patterns of the major languages. (Note that I include acceptability judgements of all kinds under “experiments” here, because they all go beyond pure observation of naturally occurring speech.)

But it seems to me that much of the discussion around acceptability judgements (e.g. Schütze 1996; Bard et al. 1996; and much subsequent work) is based on a misunderstanding: namely that acceptability judgement (should) somehow give us access to internal knowledge systems of the speakers, in a way that corpus data don’t. It has often been said that earlier linguistics was concerned with “external language” and corpora, while the more recent linguistics (in the wake of the “cognitive revolution”) focuses on the truly interesting cognitive questions: Rather than describing corpora, linguists now want to describe mental systems that allow us to use languages in rich and creative ways. The connection between acceptability experiments and the mentalist goals is described well in §1 of Scholz et al.’s (2015) article on philosophy of linguistics (in the Stanford Encyclopedia of Philosophy).

This supposed (but illusory, as I will argue) connection is also reflected in the term “introspection” that is often used for acceptability judgements: When linguists carry out the experiments on themselves (as they are routinely taught in syntax classes), they are said to use their “intuitions”, and this is often equated in with “introspection” (for example, Weskott & Fanselow (2011: 249) use both “intuitive judgments” and “introspective judgments” in the intro paragraph of their paper, apparently in the same sense). In contrast to the vague word “intuition”, the term “introspection” has clear mentalist implications, reflecting a debate in psychology that goes back to the 19th century and probably earlier. The idea of introspection is that one can examine one’s own mental states in a scientific context, and apparently, the mentalist linguists have thought since the 1960s that the sense of (un)acceptability of a sentence is a mental state that gives us privileged access to our mental grammars (= our knowledge systems for the languages we can use). Even authors such as Branigan & Pickering (2017) who are very critical of acceptability judgements do not question the idea that they might give access to mental representations.

In contrast to these very widespread views, I would argue that an acceptability judgement is an assessment of the normative status of a potential expression: When we accept or reject a possible sentence, we make judgements about the social norms of our community, not about our mental grammars. Grammars are systems of social conventions for communicating linguistically, and a well-formed sentence is a sentence that conforms to the norms. An ill-formed sentence is outside these norms. I have long thought that this goes without saying and should be uncontroversial. But for some reason, I do not find this idea reflected in the literature on acceptability judgements. Over the last 25 years, this literature has been persistently treating acceptability judgements as if they were trying to measure a purely psychological phenomenon (e.g. Bard et al. 1998; Sprouse & Almeida 2012; Gibson & Fedorenko 2013; Linzen & Oseki 2018).

It has long been known, of course, that acceptability cannot be directly equated with grammaticality (= generatability by a mental grammar) – and that “grammaticality judgements” are not possible (even though we often use this expression colloquially). Mental grammars are tacit knowledge, and acceptability can be influenced by all kinds of factors (especially semantic-pragmatic oddness, but also processing difficulties). Acceptability judgements are an aspect of linguistic performance (Bard et al. 1996: 33), not of competence, and are not that different from naturally occurring speech in this regard. This does not seem to be controversial.

So why should acceptability judgements give better access to mental grammars than corpus data? I have not seen a good reason for this, and one might venture the hypothesis that the connection is accidental: There was a shift to cognitive questions about human language in the 1950s and 1960s, away from historical  and cultural questions (probably also as a result of the new dominance of the United States, whose national identity is not rooted in language), and around the same time, English syntax became more and more prominent as mathematicians, logicians and computer programmers got increasingly interested in natural language as well. The idea of a complete specification of the generative rules of English syntax fascinated many linguists in the 1960s, and it was clear that only an experimental approach (based on acceptability judgements) could supply enough material for this goal. It may seem far-fetched to suggest that there was no intrinsic connection between these two ideas (both embodied by the famous figure Noam A. Chomsky), but I would like to see a persuasive case for it. I can easily imagine a cognitive linguistics that is rooted in corpus data (as in the fascinating work of my former Leipzig colleague Natalia Levshina, and many others), and I can also easily imagine an experimentally-based linguistics of community grammars (as in the practice of fieldworkers of the Boasian tradition, where elicitation/experimentation has long been a key method, without any claims of mental reality of the resulting rules). Eventually, of course, every linguist needs both: We need experiments to complete our paradigms, and we need corpora to get a thorough sense of actual language use, in addition to experimental results.

When I say that linguistic regularities are social norms (or conventions), I don’t say anything radical or controversial – linguists and philosophers have been saying this for a long time, and it is evidently true. Linguists have tended to downplay it, and have often talked about the “social dimension” primarily in connection with social variation within a speech community. But different languages are spoken by different speech communities, reflecting different social norms. Languages may be a bit different from other systems of conventions (or norms) such as politeness systems, government systems, economic systems, religious systems etc, because we have an inborn instinct to acquire a language (an innate linguisticality, with no clear counterparts in other domains, apart from music). But this is not a direct talking instinct: It is an instinct to acquire a specific set of social norms that allow us to talk to other people who can follow the same norms. So when judge whether a particular sentence is well-formed or not, what I ask myself is whether its use would conform to the social norms – I make a judgement about normative acceptability. I ask myself: Would this sentence be acceptable in my speech community?

It is sometimes thought that “community norms” are somehow epiphenomenal to language and that internal cognitive representations are the primary locus of language. But in fact, I would say, it is the opposite – when we talk about (and study) “a language” (such as German or Guarani), we mean the set of community norms, not the mental representations of particular speakers. Language acquisition is possible only if a set of community norms exists that can be acquired. It is clear that the community norms are fairly uniform, and that we have intuitions concerning these community norms (because speakers’ judgements are extremely similar, as has been shown repeatedly, e.g. by Sprouse & Almeida 2012). Whether we have intuitions concerning our own mental representations is quite unclear.

That language systems are primarily systems of social conventions is of course familiar from de Saussure’s (1916) Cours de linguistique générale:

“The language (la langue) <we speak> is at the same time a social product of the language faculty (le langage) and an ensemble of necessary conventions, adopted by the social body in order to allow for the exercise of this faculty by each individual” (Cours, 30; cited and discussed by Seuren 2016: 8)

And even though generative linguists often say that they want to study individual competence grammars, the very fact that authors such as Sprouse & Almeida (2012) base their study of consistency in acceptability judgements on hundreds of speakers shows that they are interested in community norms after all, not merely in individual judgements. (In the discussion of acceptability judgement reliability, a few mentalist authors have taken a consistent position and noted that pools of informants are irrelevant to the Chomskyan enterprise, e.g. den Dikken et al. 2007; but interestingly, this seems to be a minority position).

One important point that is often forgotten by linguists is that the same normative results may be achieved in rather different ways. Zwicky (1992: 340) discusses partial regularities like English keep/kept, which could be treated as constructed by a rule or as suppletive (like buy/bought). Speaker judgements of acceptability are uniform here, but which of these is the right analysis? Zwicky observes:

“The methodological problem arises when we try to decide whether the weight of the evidence warrants positing a morphological rule in situations like this one. I take it for granted that speakers might differ considerably with respect to the generalizations captured in their grammars, but there is still the problem what the system is for any particular speaker.” (1992: 340)

So while individual grammars may well differ, this is not reflected in different usage or different judgements. There seems to be no way of telling what a speaker’s grammar is based on their conscious behaviour (though maybe psycholinguistic measures would help). Some linguists have said that when there is no empirical evidence to decide between two analyses, the general framework should decide, but it has never been shown that this works in practice – linguistic analyses are full of indeterminacies, and unclear cases like keep/kept are rampant in all languages, in phonology, morphology and syntax. I take this as another important argument for an interpretation of judgements as telling us about norms, not about internal knowledge.

Perhaps the most original recent discussion of acceptability judgements has been offered by Pullum (2017). Pullum argues that we need a middle ground between the extremes of “corpus fetishism” (the idea that experimental approaches are useless) and “intuitional solipsism” (the idea that only a single speaker’s intuitions are relevant). And he ends up comparing grammatical rules to moral principles, and acceptability judgements to moral judgements of right or wrong. This is not more than an analogy (there is of course no moral problem with a deviation from the grammatical norm), but it highlights the need to think of acceptability judgements in terms that are different from the long-standing (and often implicit) idea that such judgements are necessarily about internal cognitive systems.

References

Bard, Ellen Gurman & Robertson, Dan & Sorace, Antonella. 1996. Magnitude estimation of linguistic acceptability. Language. Linguistic Society of America 72(1). 32–68. (doi:10.2307/416793)

Branigan, Holly P. & Pickering, Martin J. 2017. An experimental approach to linguistic representation. Behavioral and Brain Sciences. Cambridge University Press 40. (doi:10.1017/S0140525X16002028) (https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/an-experimental-approach-to-linguistic-representation/56398BE6CDD90731063F352A6C65AAB7) (Accessed December 5, 2020.)

de Saussure, Ferdinand. 1916. Cours de linguistique générale. (Ed. Bally, Charles & Sechehaye, Albert & Riedlinger, Albert.)

den Dikken, Marcel & Bernstein, Judy B. & Tortora, Christina & Zanuttini, Raffaella. 2007. Data and grammar: Means and individuals. Theoretical Linguistics. De Gruyter Mouton 33(3). 335–352. (doi:10.1515/TL.2007.022)

Gibson, Edward & Fedorenko, Evelina. 2013. The need for quantitative methods in syntax and semantics research. Language and Cognitive Processes. Routledge 28(1–2). 88–124. (doi:10.1080/01690965.2010.515080)

Jespersen, Otto. 1933. Essentials of English grammar. London: Allen & Unwin.

Pullum, Geoffrey K. 2017. Theory, data, and the epistemology of syntax. In Konopka, Marek & Wöllstein, Angelika (eds.), Grammatische Variation: Empirische Zugänge und theorische Modellierung (Institut Für Deutsche Sprache, Jahrbuch 2016), 283–298. Berlin: De Gruyter Mouton.

Scholz, Barbara C. & Pelletier, Francis Jeffry & Pullum, Geoffrey K. 2015. Philosophy of linguistics. In Zalta, Edward N. (ed.), The Stanford Encyclopedia of Philosophy. Stanford: Stanford University. (https://plato.stanford.edu/archives/win2016/entries/linguistics/) (Accessed July 30, 2019.)

Schütze, Carson T. 1996. The empirical base of linguistics: Grammaticality judgments and linguistic methodology (Classics in Linguistics). Chicago: University of Chicago Press. (https://langsci-press.org/catalog/book/89)

Seuren, Pieter A. M. 2016. Saussure and his intellectual environment. History of European Ideas. Routledge 42(6). 819–847. (doi:10.1080/01916599.2016.1154398)

Sprouse, Jon & Almeida, Diogo. 2012. Assessing the reliability of textbook data in syntax: Adger’s Core syntax. Journal of Linguistics. Cambridge University Press 48(3). 609–652. (doi:10.1017/S0022226712000011)

Weskott, Thomas & Fanselow, Gisbert. 2011. On the informativity of different measures of linguistic acceptability. Language 87(2). 249–273.

Zwicky, Arnold M. 1992. Some choices in the theory of morphology. In Levine, Robert D. (ed.), Formal grammar: Theory and implementation, 327–371. New York: Oxford University Press.


6 thoughts on “Acceptability judgements tell us about social norms, not about internal systems

  1. Martin, I think you are expressing something that is basically sound, but when I’m trying to understand what you are saying my head starts spinning. Here is my attempt at an exegesis.
    What is a language?
    Languages are sets of social norms/conventions which we need to acquire in order to communicate:
    “we talk about (and study) “a language” (such as German or Guarani), we mean the set of community norms”
    “the community norms are fairly uniform, and […] we have intuitions concerning these community norms”
    “Language acquisition is possible only if a set of community norms exists that can be acquired”
    What is a grammar?
    “Grammars are systems of social conventions for communicating linguistically”
    What is a mental grammar?
    Mental grammars are “our knowledge systems for the languages we can use”.
    But “[w]hen we accept or reject a possible sentence, we make judgements about the social norms of our community, not about our mental grammars”.
    So it appears that the knowledge system that makes up our mental grammars is not knowledge of the social norms of our community? Could you explain why and how?

    • Yes, it’s difficult, and I have no ready solution (do you have one?). I think the main point that I wanted to make is that we should think more about conventions and norms, because the goal of finding “the mental grammar” behind speaker behaviour seems to be at least premature, and perhaps pointless, given that different speakers may well have different grammars. I went through some of the literature on acceptability judgements and was struck by how little awareness of these problems there is in this literature.

      • Trying to find “the mental grammar” or the “mental representations” behind speaker behaviour may not only be premature or pointless – it is not clear what it even means. Your central nervous system and mine are “wired up” in different ways making us native speakers of different languages, but we know too little about the neural correlates of languages to be able to say if there is anything you can call “mental representations” there. On the other hand, it does make sense to think of languages as systems of conventions and/or norms, and I certainly agree that you cannot get a full grip on these systems by just asking speakers about their intuitions. I think “acceptability judgement” is only one form of what one could call a “meta-reaction” to utterances. For instance, if you listen to someone speaking your language with an accent or dialect that differs from your own, you may not judge it as acceptable or unacceptable, even if you register that it sounds funny or deviant. Meta-reactions of all kinds shed light on the norms and conventions that an individual has internalized but they have clear limitations. One is that they are not necessarily verbalizable in the same sense that even if you know a person well, you cannot necessarily describe their face in a way that makes it possible for someone else to pick them out of a crowd. Another is that we mostly cannot sort out the different factors behind our linguistic meta-reactions or even sort them out from our reactions to extra-linguistic features of the situation. You may not be able to tell if someone sounds “funny” because of their dialect or because of their mood or their personality. Similarly, people (including linguists) usually cannot directly say if they find a sentence deviant because it violates a grammatical rule or because it describes an impossible state of affairs.

        • OK, so what you are saying is even more radical than what I was saying: I was assuming that somehow we have “mental representations” of our linguistic systems, only that acceptability judgements are in no way introspections – we judge social well-formedness, not well-formedness in terms of our mental structures. – But note that I was not saying that acceptability judgements are not enough to describe the social conventions. This is a point that corpus linguists have often made, but I don’t find it 100% convincing. On the contrary, I think that acceptability judgements are more important than corpora, because we can make negative judgements, and we can make judgements about structures that occur very rarely. That such judgements may not have to do with grammatical conventions but with pragmatic oddness (or other factors) is also something that is widely recognized. It is your view, that we should “think of languages as systems of conventions and/or norms” that is a distinct minority view.

  2. Thanks, Avery, but note that I was not in any way arguing against acceptability judgements – they are absolutely essential for completing inflectional paradigms (fieldworkers have always worked with elicitation, which is a kind of acceptability judgement), and also for completing syntactic paradigms (as you note with your Icelandic example). Arguments against acceptability judgements have typically come from people who do not know much about linguistics (psychologists like Gibson & Fedorenko, and Branigan & Pickering), or people who only work on English. My argument was that there is no particular connection between acceptability and mental reality, and that what we judge to be (un)acceptable is social.

  3. The problem I see with this is that in many cases, judgements are used to give information about structures that are too complex, and therefore rare, to get much information about from corpora, especially in the days before computers got cheap, and digitized texts were widely available. My favorite example of course being constructions with missing oblique subjects in Icelandic and other Germanic languages, which only became investigable with corpora during this century (and there are still questions that don’t seem to be accessible with them, which is what happens when something is supposed to agree with a missing oblique subject).

    I agree however that judgements don’t provide privileged access to the underlying system, and that corpora are important. In particular, because only corpora can provide information about how frequent things are, which is essential to work out what, if anything, is in UG. And both corpus and judgement data present their own difficulties of interpretation.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.