Unity of science: Why linguistics faces bigger challenges than the replication crisis

Over the last few years, psychologists and scientists in some other field have been talking (and worrying) a lot about problems of replicability and reproducibility. In psychology, people even talk about a “replication crisis”, and linguists have been thinking about replicability (e.g. of cross-linguistic generalizations, Plank (ed.) 2006) and reproducibility as well (see Berez-Kroeker et al. 2018, a position paper on reproducibility of linguistic data).

But some people are warning that solving the replication problem (e.g. by rigorous preregistration) may not be enough, because research may be biased or nonconclusive in all kinds of other ways. Munafò & Smith (2018) have recently argued that for truly robust research findings, one needs triangulation:

Triangulation is the strategic use of multiple approaches to address one question. Each approach has its own unrelated assumptions, strengths and weaknesses. Results that agree across different methodologies are less likely to be artefacts. Isn’t this how science is meant to operate? Perhaps so, but scientists in today’s hyper-competitive environment often lose sight of the need to pursue distinct strands of evidence.

The problem was aptly described last year, when cancer researcher William Kaelin lamented that the goal of the scientific paper had shifted from testing narrow conclusions in multiple ways to making a broadening series of assertions, each based on limited evidence. Consequently, he said, “papers are increasingly like grand mansions of straw, rather than sturdy houses of brick”.

These authors are talking about other disciplines, but linguists are certainly not immune from the temptation to build “grand mansions of straw” – since the neo-grammarians, “revolutionary” new ideas have had a lot of prestige in our field, but often it is not clear how to tell whether the ideas are anywhere close to correct. So would triangulation help linguists?

The linguist Seán Roberts has a recent paper (Roberts 2018) in which he makes some very similar points, with respect to large-scale cross-linguistic studies that link structural features of languages (especially sound systems) to features of the natural environments. He advocates a “maximum robustness” approach in which we “run tests with as many specific assumptions and sources of data as possible” and see whether the hypothesis is not only confirmed, but also robust.

However, in less quantitatively oriented areas of linguistics, e.g. in my area (cross-linguistic grammar research), it is often not even clear what we mean by research findings or research results. Is it a research result if we have described one grammatical pattern in one language? Well, yes, but this is descriptive linguistics, not general linguistics. So is it a research result if we have identified a cross-linguistic pattern, e.g. an areal pattern or an implicational universal? Or do we need to find a correct explanation before we have a research result that is worth reporting on (in a major publication)?

Moreover, many linguists seem to presuppose that we have already found the basic building blocks of grammar – categories and features such as C, T, v, DP, specifier, and operations such as Agree and Move, which are everyday terms for syntacticians working in Mainstream Generative Grammar. These linguists typically describe particular languages in terms of these categories hypothesized to be universal, but often these descriptions are very complicated and not comprehensible by other linguists – even though we are all dealing with (more or less) the same phenomena. Thus, it’s quite impossible for outsiders to see whether the research findings are robust, or even whether the research is sound.

This lack of comprehension leads to social isolation – generative grammarians tend to attend only their own conferences (and refer only to papers from their own community), and similar for non-generative grammarians. The social isolation reinforces the mutual incomprehension.

There is thus no unity of morphosyntactic research. This is immediately apparent to anyone who looks at papers in journals such as Linguistic Typology and Studies in Language, compared to journals such as Linguistic Inquiry or Syntax.

What is going on here? What happened to the ideal of unity of science? Does everyone have their own truth? No, this is not possible – we all agree that we are engaged in science in the sense of pursuit of objective truth. So what is going wrong?

I don’t know what it is that is going wrong, but it is clear that something is not good. The field of linguistics – or rather, more narrowly, the subfield of general research on grammar – is in trouble. And we won’t solve the problem by looking at how psychologists are dealing with their own problems. (One thing that I started to do is to write some blogposts criticizing work by some of my colleagues, but as I noted, this is not a nice kind of activity, and I fear that it may lead to even more polarization, as currently seen in American politics and elsewhere).

It could be, of course, that the MGG researchers are already quite close to finding the basic building blocks of grammar, and that all the others have simply not realized this yet because they do not make the effort to understand “modern syntax”. But when one looks at the soul-searching statements of some of the leading generative linguists at the Athens 2015 conference on the future of generative linguistics, one does not get this impression.

So it seems clear that we have a problem. A few authors such as Mendívil-Giró (2018) are trying to find ways of reconciling the different approaches, but as long as the social isolation of the communities persists and only a few valiant philosophers try to build bridges, linguistics will not solve its problems and will not impress the neighbouring disciplines, I fear.

P.S. Some generative grammarians have realized that they are being sidelined or attacked by prominent psychologists (e.g. Ibbotson & Tomasello 2016) (e.g. here and here), and they have partly responded by trying to explain things better to nonlinguists. (Adger & Svenonius 2015 is a particularly nice example of this kind.) In these discussions with nonlinguists and philosophers, I find myself generally agreeing with the generative grammarians, but I see no basis for their extensive use of universal features and categories for which there is extremely little evidence.

References

Adger, David & Peter Svenonius. 2015. Linguistic explanation and domain specialization: A case study in bound variable anaphora. Frontiers in Psychology 6. doi:10.3389/fpsyg.2015.01421. https://www.frontiersin.org/articles/10.3389/fpsyg.2015.01421/full (15 November, 2018).

Berez-Kroeker, Andrea L., Lauren Gawne, Susan Smythe Kung, Barbara F. Kelly, Tyler Heston, Gary Holton, Peter Pulsifer, David I. Beaver, Shobhana Chelliah & Stanley Dubinsky. 2018. Reproducible research in linguistics: A position statement on data citation and attribution in our field. Linguistics 56(1). 1–18. doi:https://doi.org/10.1515/ling-2017-0032.

Ibbotson, Paul & Michael Tomasello. 2016. What’s universal grammar? Evidence rebuts Chomsky’s theory of language learning. Scientific American.

Mendívil-Giró, José-Luis. 2018. Is Universal Grammar ready for retirement? A short review of a longstanding misinterpretation. Journal of Linguistics 54(4). 859–888. doi:10.1017/S0022226718000166.

Munafò, Marcus R. & George Davey Smith. 2018. Robust research needs many lines of evidence. Nature 553(7689). 399. doi:10.1038/d41586-018-01023-3.

Plank, Frans (ed.). 2006. Re-doing typology. Linguistic Typology 10(1). 67–128. doi:10.1515/LINGTY.2006.004.

 

 

 

 


3 thoughts on “Unity of science: Why linguistics faces bigger challenges than the replication crisis

  1. I broadly agree with Crom daba. I don’t think the generative paradigm has had anything interesting to say about language for a long time.

    Many individuals have important insights and they do useful work (better than nothing) on particular languages. But as a whole, they don’t talk about language in interesting or useful ways.

  2. Generativism and typological approaches operate on completely different assumptions and I think it’s no use trying to reconcile them.

    It certainly seems like we can’t solve the issue from within linguistics, both offer good models for how languages operate, only advancements in neuroscience or maybe NLP and game/information theory can show which assumptions hold true.

  3. I don’t think this sort of unity is worth pursuing, generative grammar seems to operate on completely different postulates that can’t be reconciled with your functionalist approach.

    These postulates will either be confirmed or falsified by advances in NLP, Neuroscience or even Information Theory, but I doubt they can be tested from within linguistics – both approaches can describe language well enough, choosing one over the other comes down to which methodological machinery appears more parsimonious, which itself depends on extralinguistic priors.

    In any case I think it’s a good thing to have multiple independent paradigms combing the solution space, there is less danger of being blindsided by theoretical limitations.

Leave a Reply to Dominik Lukes Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.