Since the 1960s, many works on syntax have primarily relied on acceptability judgements, rather than on examples attested in corpora, as was common in earlier times. In Jespersen’s Essentials of English grammar (1933), there were many invented examples, but also quite a few observed corpus attestations (from authors such as Shakespeare, Austen, Thackeray, Carlyle). But over the last five decades, syntacticians have relied much more on experimental methods, which allowed them to make great progress in exploring the incredibly rich patterns of the major languages. (Note that I include acceptability judgements of all kinds under “experiments” here, because they all go beyond pure observation of naturally occurring speech.)
But it seems to me that much of the discussion around acceptability judgements (e.g. Schütze 1996; Bard et al. 1996; and much subsequent work) is based on a misunderstanding: namely that acceptability judgement (should) somehow give us access to internal knowledge systems of the speakers, in a way that corpus data don’t. It has often been said that earlier linguistics was concerned with “external language” and corpora, while the more recent linguistics (in the wake of the “cognitive revolution”) focuses on the truly interesting cognitive questions: Rather than describing corpora, linguists now want to describe mental systems that allow us to use languages in rich and creative ways. The connection between acceptability experiments and the mentalist goals is described well in §1 of Scholz et al.’s (2015) article on philosophy of linguistics (in the Stanford Encyclopedia of Philosophy).
This supposed (but illusory, as I will argue) connection is also reflected in the term “introspection” that is often used for acceptability judgements: When linguists carry out the experiments on themselves (as they are routinely taught in syntax classes), they are said to use their “intuitions”, and this is often equated in with “introspection” (for example, Weskott & Fanselow (2011: 249) use both “intuitive judgments” and “introspective judgments” in the intro paragraph of their paper, apparently in the same sense). In contrast to the vague word “intuition”, the term “introspection” has clear mentalist implications, reflecting a debate in psychology that goes back to the 19th century and probably earlier. The idea of introspection is that one can examine one’s own mental states in a scientific context, and apparently, the mentalist linguists have thought since the 1960s that the sense of (un)acceptability of a sentence is a mental state that gives us privileged access to our mental grammars (= our knowledge systems for the languages we can use). Even authors such as Branigan & Pickering (2017) who are very critical of acceptability judgements do not question the idea that they might give access to mental representations.
In contrast to these very widespread views, I would argue that an acceptability judgement is an assessment of the normative status of a potential expression: When we accept or reject a possible sentence, we make judgements about the social norms of our community, not about our mental grammars. Grammars are systems of social conventions for communicating linguistically, and a well-formed sentence is a sentence that conforms to the norms. An ill-formed sentence is outside these norms. I have long thought that this goes without saying and should be uncontroversial. But for some reason, I do not find this idea reflected in the literature on acceptability judgements. Over the last 25 years, this literature has been persistently treating acceptability judgements as if they were trying to measure a purely psychological phenomenon (e.g. Bard et al. 1998; Sprouse & Almeida 2012; Gibson & Fedorenko 2013; Linzen & Oseki 2018).
It has long been known, of course, that acceptability cannot be directly equated with grammaticality (= generatability by a mental grammar) – and that “grammaticality judgements” are not possible (even though we often use this expression colloquially). Mental grammars are tacit knowledge, and acceptability can be influenced by all kinds of factors (especially semantic-pragmatic oddness, but also processing difficulties). Acceptability judgements are an aspect of linguistic performance (Bard et al. 1996: 33), not of competence, and are not that different from naturally occurring speech in this regard. This does not seem to be controversial.
So why should acceptability judgements give better access to mental grammars than corpus data? I have not seen a good reason for this, and one might venture the hypothesis that the connection is accidental: There was a shift to cognitive questions about human language in the 1950s and 1960s, away from historical and cultural questions (probably also as a result of the new dominance of the United States, whose national identity is not rooted in language), and around the same time, English syntax became more and more prominent as mathematicians, logicians and computer programmers got increasingly interested in natural language as well. The idea of a complete specification of the generative rules of English syntax fascinated many linguists in the 1960s, and it was clear that only an experimental approach (based on acceptability judgements) could supply enough material for this goal. It may seem far-fetched to suggest that there was no intrinsic connection between these two ideas (both embodied by the famous figure Noam A. Chomsky), but I would like to see a persuasive case for it. I can easily imagine a cognitive linguistics that is rooted in corpus data (as in the fascinating work of my former Leipzig colleague Natalia Levshina, and many others), and I can also easily imagine an experimentally-based linguistics of community grammars (as in the practice of fieldworkers of the Boasian tradition, where elicitation/experimentation has long been a key method, without any claims of mental reality of the resulting rules). Eventually, of course, every linguist needs both: We need experiments to complete our paradigms, and we need corpora to get a thorough sense of actual language use, in addition to experimental results.
When I say that linguistic regularities are social norms (or conventions), I don’t say anything radical or controversial – linguists and philosophers have been saying this for a long time, and it is evidently true. Linguists have tended to downplay it, and have often talked about the “social dimension” primarily in connection with social variation within a speech community. But different languages are spoken by different speech communities, reflecting different social norms. Languages may be a bit different from other systems of conventions (or norms) such as politeness systems, government systems, economic systems, religious systems etc, because we have an inborn instinct to acquire a language (an innate linguisticality, with no clear counterparts in other domains, apart from music). But this is not a direct talking instinct: It is an instinct to acquire a specific set of social norms that allow us to talk to other people who can follow the same norms. So when judge whether a particular sentence is well-formed or not, what I ask myself is whether its use would conform to the social norms – I make a judgement about normative acceptability. I ask myself: Would this sentence be acceptable in my speech community?
It is sometimes thought that “community norms” are somehow epiphenomenal to language and that internal cognitive representations are the primary locus of language. But in fact, I would say, it is the opposite – when we talk about (and study) “a language” (such as German or Guarani), we mean the set of community norms, not the mental representations of particular speakers. Language acquisition is possible only if a set of community norms exists that can be acquired. It is clear that the community norms are fairly uniform, and that we have intuitions concerning these community norms (because speakers’ judgements are extremely similar, as has been shown repeatedly, e.g. by Sprouse & Almeida 2012). Whether we have intuitions concerning our own mental representations is quite unclear.
That language systems are primarily systems of social conventions is of course familiar from de Saussure’s (1916) Cours de linguistique générale:
“The language (la langue) <we speak> is at the same time a social product of the language faculty (le langage) and an ensemble of necessary conventions, adopted by the social body in order to allow for the exercise of this faculty by each individual” (Cours, 30; cited and discussed by Seuren 2016: 8)
And even though generative linguists often say that they want to study individual competence grammars, the very fact that authors such as Sprouse & Almeida (2012) base their study of consistency in acceptability judgements on hundreds of speakers shows that they are interested in community norms after all, not merely in individual judgements. (In the discussion of acceptability judgement reliability, a few mentalist authors have taken a consistent position and noted that pools of informants are irrelevant to the Chomskyan enterprise, e.g. den Dikken et al. 2007; but interestingly, this seems to be a minority position).
One important point that is often forgotten by linguists is that the same normative results may be achieved in rather different ways. Zwicky (1992: 340) discusses partial regularities like English keep/kept, which could be treated as constructed by a rule or as suppletive (like buy/bought). Speaker judgements of acceptability are uniform here, but which of these is the right analysis? Zwicky observes:
“The methodological problem arises when we try to decide whether the weight of the evidence warrants positing a morphological rule in situations like this one. I take it for granted that speakers might differ considerably with respect to the generalizations captured in their grammars, but there is still the problem what the system is for any particular speaker.” (1992: 340)
So while individual grammars may well differ, this is not reflected in different usage or different judgements. There seems to be no way of telling what a speaker’s grammar is based on their conscious behaviour (though maybe psycholinguistic measures would help). Some linguists have said that when there is no empirical evidence to decide between two analyses, the general framework should decide, but it has never been shown that this works in practice – linguistic analyses are full of indeterminacies, and unclear cases like keep/kept are rampant in all languages, in phonology, morphology and syntax. I take this as another important argument for an interpretation of judgements as telling us about norms, not about internal knowledge.
Perhaps the most original recent discussion of acceptability judgements has been offered by Pullum (2017). Pullum argues that we need a middle ground between the extremes of “corpus fetishism” (the idea that experimental approaches are useless) and “intuitional solipsism” (the idea that only a single speaker’s intuitions are relevant). And he ends up comparing grammatical rules to moral principles, and acceptability judgements to moral judgements of right or wrong. This is not more than an analogy (there is of course no moral problem with a deviation from the grammatical norm), but it highlights the need to think of acceptability judgements in terms that are different from the long-standing (and often implicit) idea that such judgements are necessarily about internal cognitive systems.
Bard, Ellen Gurman & Robertson, Dan & Sorace, Antonella. 1996. Magnitude estimation of linguistic acceptability. Language. Linguistic Society of America 72(1). 32–68. (doi:10.2307/416793)
Branigan, Holly P. & Pickering, Martin J. 2017. An experimental approach to linguistic representation. Behavioral and Brain Sciences. Cambridge University Press 40. (doi:10.1017/S0140525X16002028) (https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/an-experimental-approach-to-linguistic-representation/56398BE6CDD90731063F352A6C65AAB7) (Accessed December 5, 2020.)
de Saussure, Ferdinand. 1916. Cours de linguistique générale. (Ed. Bally, Charles & Sechehaye, Albert & Riedlinger, Albert.)
den Dikken, Marcel & Bernstein, Judy B. & Tortora, Christina & Zanuttini, Raffaella. 2007. Data and grammar: Means and individuals. Theoretical Linguistics. De Gruyter Mouton 33(3). 335–352. (doi:10.1515/TL.2007.022)
Gibson, Edward & Fedorenko, Evelina. 2013. The need for quantitative methods in syntax and semantics research. Language and Cognitive Processes. Routledge 28(1–2). 88–124. (doi:10.1080/01690965.2010.515080)
Jespersen, Otto. 1933. Essentials of English grammar. London: Allen & Unwin.
Pullum, Geoffrey K. 2017. Theory, data, and the epistemology of syntax. In Konopka, Marek & Wöllstein, Angelika (eds.), Grammatische Variation: Empirische Zugänge und theorische Modellierung (Institut Für Deutsche Sprache, Jahrbuch 2016), 283–298. Berlin: De Gruyter Mouton.
Scholz, Barbara C. & Pelletier, Francis Jeffry & Pullum, Geoffrey K. 2015. Philosophy of linguistics. In Zalta, Edward N. (ed.), The Stanford Encyclopedia of Philosophy. Stanford: Stanford University. (https://plato.stanford.edu/archives/win2016/entries/linguistics/) (Accessed July 30, 2019.)
Schütze, Carson T. 1996. The empirical base of linguistics: Grammaticality judgments and linguistic methodology (Classics in Linguistics). Chicago: University of Chicago Press. (https://langsci-press.org/catalog/book/89)
Seuren, Pieter A. M. 2016. Saussure and his intellectual environment. History of European Ideas. Routledge 42(6). 819–847. (doi:10.1080/01916599.2016.1154398)
Sprouse, Jon & Almeida, Diogo. 2012. Assessing the reliability of textbook data in syntax: Adger’s Core syntax. Journal of Linguistics. Cambridge University Press 48(3). 609–652. (doi:10.1017/S0022226712000011)
Weskott, Thomas & Fanselow, Gisbert. 2011. On the informativity of different measures of linguistic acceptability. Language 87(2). 249–273.
Zwicky, Arnold M. 1992. Some choices in the theory of morphology. In Levine, Robert D. (ed.), Formal grammar: Theory and implementation, 327–371. New York: Oxford University Press.