Beyond the Zipf-Mandelbrot law in quantitative linguistics

Research output: Contribution to journalArticlepeer-review

  • Authors:
  • Marcelo A. Montemurro

Abstract

In this paper the Zipf-Mandelbrot law is revisited in the context of linguistics. Despite its widespread popularity the Zipf-Mandelbrot law can only describe the statistical behaviour of a rather restricted fraction of the total number of words contained in some given corpus. In particular, we focus our attention on the important deviations that become statistically relevant as larger corpora are considered and that ultimately could be understood as salient features of the underlying complex process of language generation. Finally, it is shown that all the different observed regimes can be accurately encompassed within a single mathematical framework recently introduced by C. Tsallis. © 2001 Elsevier Science B.V. All rights reserved.

Bibliographical metadata

Original languageEnglish
Pages (from-to)567-578
Number of pages11
JournalPhysica A: Statistical Mechanics and its Applications
Volume300
Issue number3-4
DOIs
Publication statusPublished - 15 Nov 2001