A few months after their (2014) target article with comments from de Reuse, Dryer and me were published in the “Perspectives” section of Language, Henry Davis, Carrie Gillon and Lisa Matthewson have published their response (DGM 2015). Their original claim was that a UG-oriented approach is better suited to the discovery of linguistic diversity, in contrast to some of the claims made by Levinson & Evans (2010). In particular, they stressed the need for systematic hypothesis-testing, which turned out to be uncontroversial.
In my commentary (Haspelmath 2014b), I noted that the main difference between two types of approaches is not between “C-linguists” and “D-linguists” (a strange distinction that they adopt from Levinson & Evans and continue to use in their response), but between linguists whose main goal is the discovery of UG (i.e. continuing the Chomskyan tradition) and linguists who want to chart all the existing diversity and eventually explain it in whatever way may be appropriate (i.e. continuing the Boasian and Greenbergian traditions).
DGM (2015) explain that they continue to disagree with the Boasian/Greenbergian linguists (even though they acknowledge the more conciliatory tone of the replies by de Reuse, Dryer and Haspelmath). But I was very happy to note that with respect to my own commentary, they at least agree on the nature of the disagreements. This is a first step toward resolving the disagreements, so I’d like to comment on these points a bit further.
A large number of theoretical linguists talk about “linguistic theory” as if there were only one way of doing it, so in my recent overview chapter on comparative syntax (Haspelmath 2014a) I contrasted two starkly different approaches: the restrictivist approach and the non-aprioristic approach. The former assumes that the features and categories of grammar are part of the innate UG, so that language description and language comparison are part of the same enterprise. This explains that in the original target article, DGM make no distinction between cross-linguistic comparison (as in the World Atlas of Language Structures, which they criticize) and language-specific generalizations, as discovered in original fieldwork. And in their response, they now confirm this: “there is no principled difference between descriptive and comparative fieldwork … for us, as for generative linguists generally, fieldwork is certainly part of a cross-linguistic investigation” (p. e135-e136).
In the non-aprioristic approach, as I described it, description is language-specific (using descriptive categories which may be unique to a language), while comparison makes use of special comparative concepts which may not be relevant to individual languages (see also Haspelmath 2010).
The crucial argument that DGM (2015) are making in the response is that while they adopt categories from another language when doing fieldwork (or even categories postulated as universal), these categories are not adopted “as analyses about other languages, but only as initial hypotheses, which are often disproven” (p. e135)
The mistake here is that categories cannot be disproven. The target article mentions a number of cases where claims about universals have been disproven by their work (e.g. the fact that Condition C of the binding theory is not always observed), but there is no way in which a category that is assumed as universal can be disproven. One can perhaps fail to find evidence in favour of a category (e.g. evidence in favour of a VP, or evidence in favour of binary branching), but if one assumes that all languages have binary branching and a VP, there is no problem. The resulting analysis may not be fully motivated by the language-specific phenomena, but this has never been stated as a requirement in generative linguistics.
Non-apriorists, by contrast, are often uncomfortable with analyses that seem to be motivated primarily by comparison with some other, better-known language – for example, the idea that a language like Latin has subject agreement with a null subject pronoun in a sentence like venit ‘he is coming’ strikes me as eurocentric (cf. Haspelmath 2013). The null pronoun is not motivated by anything in Latin, only by the comparison with English and German (which are VERY unusual in requiring both a pronoun and a person marker on the verb). Since categories cannot be disproven, one cannot show that it is wrong to say that Latin has a null pronoun, but this is what Levinson & Evans mean when they say that generative linguists take “off-the-shelf categories arising from specific grammatical traditions”, thus doing “procrustean violence” to the languages.
Another example of this is the claim that Pacific Northwest languages have a noun-verb distinction just like English (§4.1 of the target article), despite earlier claims by Swadesh and others. Of course, one CAN describe these languages in this way, just as one CAN describe Latin with null pronouns and Hungarian with a VP. But since we are talking about categories, one can never disprove such analyses – they just do not look pretty to a non-apriorist. Perhaps Swadesh was too much focused on the amazing and exotic predicate-argument flexibility of these languages that he failed to see the subtle ways in which subclasses can be established within the unitary verb-noun category of these languages. And if these subclasses are semantically close to the English verbs and nouns, the temptation is great to say that the Pacific Northwest languages have verbs and nouns in the same way (similarly, Sandra Chung succumbed to the temptation of saying that Chamorro has verbs and nouns like English, as discussed elsewhere). Now it is certainly interesting that these subclasses have been found, and that they match the English classes, but since the class-defining properties are quite different in the two types of languages, one cannot really say that one has discovered the same categories. This is somewhat like saying that Germany has representatives of the universal Hindu-Muslim distinction just because there are two different religions with a tradition of fighting against each other (and where one religion emphasizes scripture and lacks pictures in its mosques, while the other religion worships a whole range of male and female deities with various attributes in its temples). Such a claim cannot be disproven, and it might not even be wrong, who knows.
The larger moral of the basic point that categories cannot be disproven is that we should focus more on claims that CAN be disproven, e.g. universals of the Greenbergian type (some of which have been disproven, cf. Dryer 1988). One such universal is Condition C, and DGM (2014) nicely showed that it does not hold exceptionlessly (though it may still hold with overwhelmingly greater than chance frequency). But in order to formulate testable universals, one needs categories that uncontroversially apply to all languages – i.e., categories such as VP or null pronouns cannot be part of testable claims. We thus need special comparative concepts, and this has been for me one of the main methodological lessons of the WALS and APiCS enterprises.
References
Davis, Henry, Carrie Gillon & Lisa Matthewson. 2014. How to investigate linguistic diversity: Lessons from the Pacific Northwest. Language 90(4). e180–e226.
Davis, Henry, Carrie Gillon & Lisa Matthewson. 2015. Diversity driven but cognitively constrained: Boas meets Chomsky (Response to commentators). Language 91(3). e127–e143.
Dryer, Matthew S. 1988. Object-verb order and adjective-noun order: Dispelling a myth. Lingua 74(2-3). 185–217.
Haspelmath, Martin. 2010. Comparative concepts and descriptive categories in crosslinguistic studies. Language 86(3). 663–687.
Haspelmath, Martin. 2013. Argument indexing: A conceptual framework for the syntax of bound person forms. In Dik Bakker & Martin Haspelmath (eds.), Languages across boundaries: Studies in memory of Anna Siewierska, 197–226. Berlin: De Gruyter Mouton.
Haspelmath, Martin. 2014. Comparative syntax. In Andrew Carnie, Yosuke Sato & Dan Siddiqi (eds.), The Routledge handbook of syntax, 490–508. London: Routledge.
Haspelmath, Martin. 2014b. Descriptive hypothesis testing is distinct from comparative hypothesis testing: Commentary on Davis, Gillon, and Matthewson. Language 90(4). e250–e257.
Levinson, Stephen C. & Nicholas Evans. 2010. Time for a sea-change in linguistics: Response to comments on “The myth of language universals.” Lingua 120(12). 2733–2758.
OpenEdition suggests that you cite this post as follows:
Martin Haspelmath (October 18, 2015). Hypothesis-testing in comparative linguistics: Aprioristic categories cannot be disproven! Diversity Linguistics Comment. Retrieved December 6, 2024 from https://doi.org/10.58079/nstg
Pingback: Facing the challenge of general linguistics when nature doesn’t help us | Diversity Linguistics Comment
Thanks a lot for this interesting methodological discussion. It’s interesting that you first describe your position as Popperian, but then say that “empirical researchers never ‘disprove’ a hypothesis”. In my experience, this is not the case – some typological hypotheses (e.g. Kayne’s Generalization that clitic-doubling of an object is only possible if it has special prepositional marking) HAVE BEEN DISPROVEN to everyone’s satisfaction. But I would agree that a descriptive hypothesis cannot be disproven easily, because one can always add a little stipulation here and there. Language description is more like a legal case (you have to persuade the reader, who is the judge), and it’s difficult to make it rigorous. Zwicky has compared decisions on categories with diagnostic judgments made by doctors, which is another apt metaphor for non-rigorous (and typically theory-dependent) decisions made by grammarians (see my 2015 paper on defining vs. diagnosing categories). Hence my insistence on comparative concepts, which are not theory-dependent and allow the formulation and testing of rigorous hypotheses. – Note that I am not at all against saying that Salishan and Wakashan languages have nouns and verbs, but I don’t think it’s a major discovery. The discovery of “nouns” and “verbs” in Wakashan is important only if one believes that categories of this type are innate, but I don’t see any strong reasons for this belief, other than the universalist zeitgeist (see my blog post about the history of categorial distinctions: http://hiphilangsci.net/2014/10/08/non-universality-of-word-classes-and-words-the-mid-20th-century-shift/).
Thanks for giving us the opportunity to reply to your blog post on our ‘Response’ paper. Here in turn is our response.
First, the falsifiability issue. This issue here is really a philosophical one. The position we adopted in our ‘Language’ article (which might fairly be termed ‘crudely falsificationist’) is straightforwardly Popperian, and as such subject to the same criticisms as Popper’s own work: to whit, there isn’t a simple relationship between theory and data, such that a single counterexample is sufficient to invalidate a whole theory. The reason is that a theory consists of a complex series of linked hypotheses, some more integral than others, whose predictions can sometimes be internally inconsistent or even empirically wrong. The point is, as long as no other theory is more consistent and less wrong, that’s the best theory we have – and certainly better than no theory at all.
When it comes to a particular issue, such as categories, the objection you raise is what ‘counts’ as evidence. In particular, what you are objecting to is our claim that *any* evidence for categorial distinctions will count as positive evidence, whereas the absence of evidence will *never* count as negative evidence, because the researcher may just not have looked hard enough.
We might call this claim the ‘Rumsfeld Gambit’, after the latter’s famous quote about WMD in Iraq: “Absence of evidence is not evidence for absence”. (Rumsfeld is remarkable in accidentally stumbling on the truth while trying hard to avoid it.) The Rumsfeld Gambit also has a counterpart in first language acquisition, in the form of models which assume the language learner only has access to positive evidence; under this view, the absence of some phenomenon in the input will never lead a child to change his or her grammar.
Now of course, Rumsfeld was (cataclysmically) wrong about WMD; but not because his premise was false. What was wrong was that there were no WMD facilities in any of the places most likely to house them, and Rumsfeld knew it: this was what made his statement an example of sophistry, not wisdom. So though the hypothesis that WMDs existed in Iraq has never been ‘disproven’, it has nevertheless long been abandoned by even its staunchest proponents.
How does this relate to the debate about category distinctions? In our original paper, we (correctly, we think) argued that for each question, we must take the most falsifiable hypothesis as the initial one. For categories, that is the hypothesis that syntactic category distinctions DO NOT exist. Now suppose we had in fact found no evidence for them (like Swadesh, Kinkade, Jelinek, etc.): then it is true that we would not have PROVED they don’t exist, even if we had made the most exhaustive search possible. So in that sense you are correct – the hypothesis that categories don’t exist isn’t technically falsifiable, in the same way that the WMD hypothesis is not falsifiable, and it follows that we are guilty of employing the Rumsfeld Gambit.
But of course, that’s not actually how evidence relates to hypothesis formation, whether one is searching for lexical categories or for chemical weapons. Empirical researchers never ‘disprove’ a hypothesis, any more than they ‘prove’ one. What actually happens is that they accumulate evidence (including negative evidence) that allows them to build a case for or against a hypothesis, more in the manner of a legal investigator than a logician. Thus, in the case of categories, convergent results from a number of diagnostic tests, replicable between speakers and often across languages, lead us to the abandonment of the null hypothesis that there are no lexical category distinctions in languages of the Pacific Northwest.
More broadly, then, the dispute comes down to the far trickier issue of exactly what counts as ‘sufficient’ or ‘insufficient’ evidence for a given hypothesis, rather than abstract issues of falsifiability. And that question is often heavily theory-dependent. For example, subject/non-subject asymmetries count as evidence for hierarchical structure only in theories where grammatical relations are defined configurationally, as in e.g. the Principles and Parameters model, but not where they occupy a separate level of analysis (as in e.g., LFG). And of course, the ‘configurational’ theory of grammatical relations is itself a hypothesis – but one so deeply entrenched in the fabric of theories that contain it that no single piece of counter-evidence would ever be likely to ‘refute’ it.
Thus in the case of syntactic categories, it is not simply a matter of finding some categorial distinction and then declaring the matter closed. Rather, good argumentation consists of developing a set of converging (and hopefully crosslinguistically valid) diagnostics, which furthermore cannot be explained equally well or better in another way – say, by using independently motivated semantic criteria. Over the last couple of decades of research on Pacific Northwest languages, it is fair to say that the weight of evidence for categorial distinctions in the syntax has become overwhelming, which is why there is now a consensus amongst researchers who work on the languages, and why the most prominent advocates of category neutrality (Kinkade and Jelinek) publicly withdrew their earlier claims. We see this as embodying real scientific progress.
Henry Davis, Carrie Gillon and Lisa Matthewson