Annotated corpora of small languages as refereed publications: a vision

It has often been said that the job of linguists working on a small language is not complete until they publish three major works on the language: a grammar, a dictionary, and a volume of texts.  This threefold result of extended fieldwork has been described as the “Boasian triad” (e.g. Grinevald 2001, Tsunoda 2006: 245), and ever since Bloomfield’s (1917) “Tagalog texts”, quite a few prominent (and not so prominent) field linguists published text collections (e.g. Heath 1980 on Nunggubuyu, Bollée & Rosalie 1994 on Seychelles Creole, Wichmann 1996 on Texistepec Popoluca, Noonan 1999 on Chantyal, to name but a few).

But more recently, a different view of annotated texts seems to have become prevalent. On the one hand, corpus-based research is becoming more and more important, also for smaller languages, and elicitation-based research seems to be on the defensive (e.g. Davis et al. 2014). But on the other hand, the creation of annotated texts now tends to be subsumed under the larger enterprise of “language documentation”, and the place for a documentation is not the published volume, but the archive. Authors tend to emphasize the use of new technologies for recording (such as video, which has become very cheap), annotating (such as ELAN, which alllows time-aligned annotation) and retrieval via sophisticated metadata. The idea of publication seems to have somehow disappeared, and replaced by a vague notion of access.

The alternative that we would like to discuss here is basically to go back to the old model, where annotated texts/corpora were published by the linguists who recorded and annotated them. Publication has three crucial advantages over simple archiving: (1) It gives the author prestige and thus contributes to career-building, (2) it creates a powerful incentive for annotating as many texts as possible, and (3) it ensures that widely usable high-quality annotations are produced. Moreover, once a corpus is published, it is much more likely that it will actually be used by people other than those who were immediately involved in its creation, including the community of speakers. At least in their current form, archives such as the DoBeS archive, the ELAR archive or the AILLA archive do not make it very easy to access their corpora.

By corpus publication, we do not mean old-style paper publication, of course. Corpora need to be searchable, so modern corpus publication must be electronic, open-access, easily browsable and easily downloadable, with a clear Creative Commons license (e.g. CC-NC-ND) (somewhat along the lines of the examples in APiCS Online. Maybe one could think of supplying a PDF version, so that if readers want a paper version, they can use a print-on-demand service of their choice. We will now discuss each of the advantages of corpus publication in turn.

By publishing an annotated corpus, in a book-series-like or journal-like corpus-publication outlet (henceforth, “corpus journal”), the author gets credit, just like she gets credit for any other (book or) journal publication. This is different from archiving – at least in spirit, the purpose of archiving is preservation, not career-building. The idea of language archives arose in response to the realization that there are not enough resources to publish all documents on small languages, and for endangered languages (which provided much of the impetus for the archiving endeavors), also materials that are not of publishable quality are valuable enough to be archived. But this does not mean that archiving can replace publication. Only publication in a selective peer-reviewed journal or series can create the kind of prestige that is necessary for career-building. So all junior researchers will be very interested in this option, and established linguists should not ignore these concerns either. (It is sometimes suggested that one should build some kind of recognition process on top of the archives, but it is very uncertain whether this will work. It is much easier to use the primary existing process for scientific recognition, namely journal publication, because it is far easier to create a new journal than to create a new type of recognition process.)

Once corpora can be published in a way that counts for the corpus creator’s prestige, we will see a lot of new annotated corpora. Fieldworkers generally have a lot more text material that they could annotate, beyond what they deposit in the archives. If they had the additional incentive of a peer-reviewed publication, many of them would find the resources to do more annotations. Since the archives may contain anything at all, even completely unannotated texts, a well-annotated text in an archive is not particularly visible, and hence fieldworkers are likely to give preference to other activities.

Most importantly, since the corpus journal will be selective, it can insist on certain quality standards for annotations. The annotations found in archives are often annotations created primarily with a small group of users in mind, and there is nobody who enforces particular annotation standards, because even unannotated texts are accepted. A corpus journal would accept only submissions that fulfill certain mimimum standards, and as more and more corpus creators compete for the limited space, the quality of submissions would rise and rise. The submitters should probably be called “editors”, because they are not really “authors of the annotated texts.

How difficult is it to create such a corpus journal? It seems that the technological prerequisites already exist. The CLLD framework allows for publishing annotated example sentences, as seen most clearly in APiCS. The APiCS example sentences do not generally contain sound, but some do. In addition, we need a way to show all the text units of a single text on one page, and to concatenate the individual sound files so that the text can be listened to in its entirety.

The corpus journal would thus have an interface quite similar to LACITO’s Pangloss (for example, this text in Araki), or to Nick Thieberger’s EOPAS (for example, this text in the South Efate language). But in addition, it would have a unique page for each text unit, so that each text unit has a unique “address”, i.e. a unique URL on the web. This unique addressability allows linguists to link easily to example sentences that are found on the web. A vision for the future is that each sentence cited in a linguistics article dealing with a small language will have a link to a web page that shows the actual example in its context, with sound and perhaps even video. (This is much more realistic than the idea of embedding sound files in PDF files, which is technically problematic. Scientific data on the web should be “linked data”, as is generally the case in the CLLD framework.)

Thus, the main challenge would be to set up an editorial board, find an editor, and devise a methodology for evaluating (“peer-reviewing”) corpora.

One of the concerns that we sometimes hear is that publishing texts may be problematic because the speakers or the speaker communities may object to this. This is indeed a problem for some communities, but there are also many communities that have no problem at all with web publication of their texts – on the contrary, many speakers of smaller African or Asian languages are thrilled to see that their language is prominently represented on the web. It seems that the more restrictive attitude is characteristic especially of some speakers of North American and Australian languages, and linguists working on these kinds of languages have traditionally been quite influential. But it is doubtful that this restrictive attitude is characteristic of the majority of language communities. In any event, many texts of small languages are being made public without any serious restrictions, through the archives, through sites like Pangloss and EOPAS, and as appendixes of grammars, so it is clear that many speakers are happy to have their texts published. If these texts were published as open-access linked data (as in the CLLD framework), in a peer-reviewed journal, it seems that substantial progress could be made.

Recently David Nathan has argued against the open-access principle in language documentation, because

“the most linguistically authentic resources – recordings of spontaneous language events in social contexts – often raise questions of privacy, safety, or of controversial or secret/sacred content”

This may be true, and such records may be very valuable to those few people who are allowed to access them. But precisely because so few people can use them, they are not very valuable to linguistic science. Science is an activity that thrives on openness, and in contrast to David Nathan (“language documentation materials are not scholarly research”), we do think that annotating corpora is a scientific activity. The outputs of this research should therefore be published with open access, to the extent that this is possible, i.e. whenever the speakers are happy with this.


Bloomfield, Leonard. 1917. Tagalog texts with grammatical analysis. Part 1: Texts and translation. Part 2: Grammatical analysis. University of Illinois Studies in Language and Literature 2.

Bollée, Annegret & Marcel Rosalie. 1994. Parol ek memwar: récits de vie des Seychelles. (Kreolische Bibliothek 13). Hamburg: Buske.

Drude, Sebastian. 2012. Prospects for e-grammars and endangered languages corpora. Potentials of Language Documentation: Methods, Analyses, and Utilization, ed. by Frank Seifart, Geoffrey Haig, Nikolaus P. Himmelmann, Dagmar Jung, Anna Margetts, and Paul Trilsbeek, pp. 7–16

Grinevald, Colette. 2001. Encounters at the brink: Linguistic fieldwork among speakers of endangered languages. In Osamu Sakiyama (ed.), Lectures on Endangered Languages 2, 285–313. Osaka: ELPR Publications.

Heath, Jeffrey. 1980. Nunggubuyu myths and ethnographic texts. (23). Canberra: Australian Institute of Aboriginal Studies.

Noonan, Michael. 1999. Chantyal dictionary and texts. (Trends in Linguistics: Documentation 17). Berlin: Mouton de Gruyter.

Tsunoda, Tasaku. 2006. Language endangerment and language revitalization: An introduction. Berlin: Mouton de Gruyter.

Wichmann, Søren. 1996. Cuentos y colorados en popoluca de Texistepec. København: C.A. Reitzel.

8 thoughts on “Annotated corpora of small languages as refereed publications: a vision

  1. Three years down the road, and I tried to access the links given in Martin’s original post:
    This link is 404
    This links requires checking a box and clicking a button.

    That shows two main challenges for such a journal. The first is of course persistence. Arguably, items like this should be hosted in a dedicated repository which can guarantee the persistence of links over decades and more. Zenodo comes to mind.

    Second, I will start with the disclaimer that I do understand the peculiar nature of language data in Australia. This being said, in order for re-use of the data to thrive, it is important that the data be accessible via a straight URL without further customisation. In the case of EOPAS, this does not work. I understand that in this particular case, the hurdle is not a bug, but a feature, but it illustrates one of the problems third parties can have when aggregating or enriching data. I can of course fake that I clicked the button when accessing the content automatically, but this requires customization of the access procedure, which does not scale.

    So, if ever such a journal were to be set up, two main requirements will have to be:

    – access via a simple HTTP request

    – persistence of the URLs


  2. Treasuring annotated corpora of small languages: is this a universal of linguistic research? I hope so. I won’t attempt the typological survey, just point out a neat Chinese equivalent of the “Boasian triad” — a grammar, a dictionary, and a volume of texts. Jackson Sun (2007:103) describes these as “the three treasures” (三件宝), playing on the analogy with the “writer’s four essentials” (文房四宝) of Tang poetry: brush, ink stick, paper, and ink slab. He also emphasizes that in this day and age, multimedia content available through the Internet is a necessary complement to the “three treasures”.

    Sun, Jackson T.-S. (孙天心). 2007. 藏缅语的调查 (Fieldwork on Tibeto-Burman). Yuyanxue Lucong (语言学论丛) 36. 98–107.

  3. I think there is no contrast in positions, especially since Martin qualifies open access with “whenever speakers are happy with this”.

    In fact, I gave a paper titled “Archives as publishers of language documentation: experiences from ELAR” at the ICLDC in 2011 which made many of the same points (although less eloquently) as Martin has here, and in 2012 Peter Austin also wrote a blog post in a similar vein. ELAR’s “mission” states concisely “Preserving and Publishing Endangered Languages Documentation” (see And many people, such as Heidi Johnson of AILLA, worked hard on the “prestige” side, in terms of scholars receiving appropriate recognition and credit for their documentation/annotation/corpus work.

    I particularly agree that until language documentation and associated work does become “published” – as well as reviewed, selected, edited, discussed, critiqued, and the other activities that we normally associate with scientific and critical pursuits – then language documentation will not further develop and flower as it should.

    In pursuing these important matters, we shouldn’t get bogged down in polarised positions regarding open access (or, rather, perhaps “Open Access”). While of course science thrives on openness, it gets its legitimacy from methodology, and it is not at all controversial that there are many scientific areas where protecting confidentiality is a fundamental methodology, or that most research institutions have methodological and ethical protocols for human subjects research. As I wrote earlier, access is simply not a binary or a technical setting, and the notion of “negotiated access” is going to prove a touchstone of success in moving this topic forward.

    I might also correct the slight misquote of me above ie “language documentation materials are not scholarly research”, since my original post’s comment focused on the contrasting genres of conventional published research outputs vs “language documentation materials, as recordings and representations of the range of ‘the linguistic practices characteristic of a given speech community’ …”. In this sense I am, again, fully in synch with Martin in that we do want to see recognition of the intellectual input and we do need to develop new genres to publish language documentation corpora; presumably we agree that our new genres should have the properties that best reflect our discipline and its goals.

  4. Martin and Susanne — your post raises many issues, not all of which I can address here, but I would like to remind you that around 15 months ago I pointed out (in this blog post) that what I called “the ecology of publishing and dissemination” in language documentation is rather more complex now than it has been in the past, or indeed as you seem to see it as being now, and that archives like ELAR (and DoBeS) are part of that ecology. Rather than thinking that “the purpose of archiving is preservation”, a number of us have been arguing that archives also participate in this complex ecology (see also David Nathan’s conference paper from 2.5 years ago, among other sources). Material in archives has been professionally curated (to say anything less is to insult our colleagues who work so hard with documenters to make their materials robust, portable (in the sense of Bird and Simons 2003) and preservable). It may not have been peer-reviewed, but that’s because it holds a different place in the dissemination ecology.

    It seems to me what you are arguing for (with or without the standardisation you seem to demand) is surely just another sub-genre of traditional publication, and one which has been deprecated as too uninteresting (and therefore not saleable) by the analogue guardians of that area. It is good that you are in favour of opening up a digital space for it, and if you can make it happen then documenters can decide whether they want to add it to their inventory of outputs. However, I believe there is an intellectual gap in your presentation that is crying out for elaboration and explication, namely how to “devise a methodology for evaluating (“peer-reviewing”) corpora”. This has been the stumbling block so far to the implementation of such things as the LSA resolution on the scholarly merit of language documentation that was passed over 4 years ago. Can we hear some thoughts from you about the “development of appropriate means of review of such works so that their functionality, import, and scope can be assessed relative to other language resources and to more traditional publications”? Without these your proposal for a new sub-type of publishing is dead in the water.

    As for your claim that the “idea of publication seems to have somehow disappeared, and replaced by a vague notion of access” — you seem to have missed the point. These are two cross-cutting dimensions (as I tried to point out in the blog post referenced above) that create a refined dissemination space in a Web 2.0 world. Access in the archiving world is not “vague”, indeed the opposite is true for the major digital endangered languages archives where access is clearly defined and implemented (see ELAR and DoBeS specifications for explicit statements of what access to their archival material is all about (see also Heidi Johnson’s paper about access for the AILLA archive). In a social-networking world access is managed by individual users themselves as they decide who to follow, friend, share or email with — the same is true of academics as much as non-academic users, it seems to me.

    • Here are some thoughts regarding Peter’s comments on our post:

      People working in language documentation want to get the same credit for their time-consuming and laborious work as they want to get for their other linguistic work. Even more so as they have surely gone through weeks, months and often years of most demanding transcriptions, annotations, translations, time-alignment etc. Archives like ELAR or the DoBeS archive cannot give this credit because they have been designed for different purposes. Their primary aim is long-term storage of non-published material, as is true for any other archive. Even with the more modern “social publication” mode (termed after Peter Austin) via social networks similar to the model, ELAR lacks the most important mechanism for giving full scientific credit to the depositors, that is peer selection.

      When we talk about “publication” of corpora in a newly designed online journal, we have something in mind that could be called “accredited publication”. It’s exactly the peer-selection process which makes this kind of publication different from the social publication format. Furthermore corpus texts under accredited publication conditions will have (i) a fixed form (i.e. the texts cannot be removed or changed), and (ii) a stable URL, they are therefore citable and subject to critique as is standard for any other accredited published work (again not relevant for social publishing in the format). Annotated corpora which will be published in such a format have the potential to eventually raise the prestige of documentary work quickly and effectively.

      Now, as for the methodology for evaluating (peer-reviewing) corpora: What do we have in mind? Contributions to a corpus journal should show the following features obligatorily:
      (i) the primary text, (ii) an analyzed version with morpheme breaks (if the language is morphologically complex), (iii) glosses/annotations for grammatical categories and/or parts of speech, (iv) translation into English, (v) metadata

      Other features are optional:
      (vi) translations into languages that users may find convenient for various reasons, e.g. lexifiers for creole languages, major national languages (Russian, Chinese, Spanish etc.), (vii) sound and video files, (viii) time-alignment, (ix) other annotation tiers (intonational information, reference tracking, IPA, etc.)
      Reviewers will check the consistency of the glossing/annotations and the translation.

      We don’t see such a project as competing with existing archives, to the contrary. Not all texts collected by a fieldworker will be suitable for publication, but a corpus journal would add a missing option: the readily recognizable, accredited publication of selected well-annotated corpora.

  5. Proposal: Journal editors should reject out of hand articles that cite data taken from texts that are unpublished; this practice would force authors to publish their texts along side or prior to citing them in journal articles.

  6. A superb and timely observation. I would like to add a further plea. It is the primary data that is most important in ‘documentation’. As a historical linguist it irritates me immensely that some people publish a string of articles on specific morphemes or constructions in a language, or even a whole reference grammar without a text collection. Proper historical linguistics cannot be done without extended textual passages; word lists and example sentences is not enough. We need things like Homer and the Vedas. I often worry that for documentation projects too much efforts is spent fussing over things like glossing and time alignment. Even a pretty poor phonemic approximation of a language in enough quantity is better than one or two pear stories perfectly done. There is a thriving linguistics of Sumerian, but its corpus was compiled without the benefits of an alphabet, phonemes, or Elan. In the interests of science it is getting the data out there that matters most.

    • Nathan — if you want the digital equivalent of Sumerian “primary data” to work on there are shed loads of video and audio recordings in ELAR and DoBeS totally lacking in “alphabet[ic transcriptions], phonemes, or Elan” that I am sure the (overworked) depositors and communities who helped create them would be only too glad for assistance to transcibe, translate, annotate and analyse.

Leave a Reply

Your email address will not be published. Required fields are marked *