Over the last few years, psychologists and scientists in some other field have been talking (and worrying) a lot about problems of replicability and reproducibility. In psychology, people even talk about a “replication crisis”, and linguists have been thinking about replicability (e.g. of cross-linguistic generalizations, Plank (ed.) 2006) and reproducibility as well (see Berez-Kroeker et al. 2018, a position paper on reproducibility of linguistic data).

But some people are warning that solving the replication problem (e.g. by rigorous preregistration) may not be enough, because research may be biased or nonconclusive in all kinds of other ways. Munafò & Smith (2018) have recently argued that for truly robust research findings, one needs triangulation:

Triangulation is the strategic use of multiple approaches to address one question. Each approach has its own unrelated assumptions, strengths and weaknesses. Results that agree across different methodologies are less likely to be artefacts. Isn’t this how science is meant to operate? Perhaps so, but scientists in today’s hyper-competitive environment often lose sight of the need to pursue distinct strands of evidence.

The problem was aptly described last year, when cancer researcher William Kaelin lamented that the goal of the scientific paper had shifted from testing narrow conclusions in multiple ways to making a broadening series of assertions, each based on limited evidence^. Consequently, he said, “papers are increasingly like grand mansions of straw, rather than sturdy houses of brick”.

These authors are talking about other disciplines, but linguists are certainly not immune from the temptation to build “grand mansions of straw” – since the neo-grammarians, “revolutionary” new ideas have had a lot of prestige in our field, but often it is not clear how to tell whether the ideas are anywhere close to correct. So would triangulation help linguists?

The linguist Seán Roberts has a recent paper (Roberts 2018) in which he makes some very similar points, with respect to large-scale cross-linguistic studies that link structural features of languages (especially sound systems) to features of the natural environments. He advocates a “maximum robustness” approach in which we “run tests with as many specific assumptions and sources of data as possible” and see whether the hypothesis is not only confirmed, but also robust.

However, in less quantitatively oriented areas of linguistics, e.g. in my area (cross-linguistic grammar research), it is often not even clear what we mean by research findings or research results. Is it a research result if we have described one grammatical pattern in one language? Well, yes, but this is descriptive linguistics, not general linguistics. So is it a research result if we have identified a cross-linguistic pattern, e.g. an areal pattern or an implicational universal? Or do we need to find a correct explanation before we have a research result that is worth reporting on (in a major publication)?

Moreover, many linguists seem to presuppose that we have already found the basic building blocks of grammar – categories and features such as C, T, v, DP, specifier, and operations such as Agree and Move, which are everyday terms for syntacticians working in Mainstream Generative Grammar. These linguists typically describe particular languages in terms of these categories hypothesized to be universal, but often these descriptions are very complicated and not comprehensible by other linguists – even though we are all dealing with (more or less) the same phenomena. Thus, it’s quite impossible for outsiders to see whether the research findings are robust, or even whether the research is sound.

This lack of comprehension leads to social isolation – generative grammarians tend to attend only their own conferences (and refer only to papers from their own community), and similar for non-generative grammarians. The social isolation reinforces the mutual incomprehension.

There is thus no unity of morphosyntactic research. This is immediately apparent to anyone who looks at papers in journals such as Linguistic Typology and Studies in Language, compared to journals such as Linguistic Inquiry or Syntax.

What is going on here? What happened to the ideal of unity of science? Does everyone have their own truth? No, this is not possible – we all agree that we are engaged in science in the sense of pursuit of objective truth. So what is going wrong?

I don’t know what it is that is going wrong, but it is clear that something is not good. The field of linguistics – or rather, more narrowly, the subfield of general research on grammar – is in trouble. And we won’t solve the problem by looking at how psychologists are dealing with their own problems. (One thing that I started to do is to write some blogposts criticizing work by some of my colleagues, but as I noted, this is not a nice kind of activity, and I fear that it may lead to even more polarization, as currently seen in American politics and elsewhere).

It could be, of course, that the MGG researchers are already quite close to finding the basic building blocks of grammar, and that all the others have simply not realized this yet because they do not make the effort to understand “modern syntax”. But when one looks at the soul-searching statements of some of the leading generative linguists at the Athens 2015 conference on the future of generative linguistics, one does not get this impression.

So it seems clear that we have a problem. A few authors such as Mendívil-Giró (2018) are trying to find ways of reconciling the different approaches, but as long as the social isolation of the communities persists and only a few valiant philosophers try to build bridges, linguistics will not solve its problems and will not impress the neighbouring disciplines, I fear.

P.S. Some generative grammarians have realized that they are being sidelined or attacked by prominent psychologists (e.g. Ibbotson & Tomasello 2016) (e.g. here and here), and they have partly responded by trying to explain things better to nonlinguists. (Adger & Svenonius 2015 is a particularly nice example of this kind.) In these discussions with nonlinguists and philosophers, I find myself generally agreeing with the generative grammarians, but I see no basis for their extensive use of universal features and categories for which there is extremely little evidence.

References

Adger, David & Peter Svenonius. 2015. Linguistic explanation and domain specialization: A case study in bound variable anaphora. Frontiers in Psychology 6. doi:10.3389/fpsyg.2015.01421. https://www.frontiersin.org/articles/10.3389/fpsyg.2015.01421/full (15 November, 2018).

Berez-Kroeker, Andrea L., Lauren Gawne, Susan Smythe Kung, Barbara F. Kelly, Tyler Heston, Gary Holton, Peter Pulsifer, David I. Beaver, Shobhana Chelliah & Stanley Dubinsky. 2018. Reproducible research in linguistics: A position statement on data citation and attribution in our field. Linguistics 56(1). 1–18. doi:https://doi.org/10.1515/ling-2017-0032.

Ibbotson, Paul & Michael Tomasello. 2016. What’s universal grammar? Evidence rebuts Chomsky’s theory of language learning. Scientific American.

Mendívil-Giró, José-Luis. 2018. Is Universal Grammar ready for retirement? A short review of a longstanding misinterpretation. Journal of Linguistics 54(4). 859–888. doi:10.1017/S0022226718000166.

Munafò, Marcus R. & George Davey Smith. 2018. Robust research needs many lines of evidence. Nature 553(7689). 399. doi:10.1038/d41586-018-01023-3.

Plank, Frans (ed.). 2006. Re-doing typology. Linguistic Typology 10(1). 67–128. doi:10.1515/LINGTY.2006.004.

 

 

 

 

---

---