Skip Navigation

The Computer Journal 1972 15(3):259-262; doi:10.1093/comjnl/15.3.259
© 1972 by British Computer Society
This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (33)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Clare, A. C.
Right arrow Articles by Lynch, M. F.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

The identification of variable-length, equifrequent character strings in a natural language data base

A. C. Clare *, E. M. Cook * and M. F. Lynch *

Postgraduate School of Librarianship and Information Science, University of Sheffield, Sheffield, UK

The words of natural language texts exhibit a Poisson (or Zipfian) rank-frequency relationship, i.e., a small number of common words accounts for a large proportion of word occurrences, while a large number of the words occur as singletons or only infrequently. Inverted-file retrieval systems using free text data bases commonly identify words as the keys or index terms about which the file is inverted, and through which access is provided. They therefore involve large and growing dictionaries and may entail inefficient utilisation of storage because of the distribution characteristics.

An alternative approach may be used on the analysis of text in terms of sets of variable-length character strings, the frequency distributions of which are much less disparate than those of words. This could lead to substantial reductions in dictionary size, and increased efficiency both in dictionary look-up times and storage utilisation.

Received October 1971.

* Postgraduate School of Librarianship and Information Science, University of Sheffield


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Journal of Information ScienceHome page
M. F. Lynch and P. Willett
Information retrieval research in the Department of Information Studies, University of Sheffield: 1965-1985
Journal of Information Science, January 1, 1987; 13(4): 221 - 234.
[Abstract] [PDF]


Disclaimer:
Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.