Understanding Artificial Intelligence
through Algorithmic Information Theory
Zipf’s law
The law was first discovered in 1928 by Edward Condon 
and then by George Zipf.
What does the "Condon-Zipf law" say?
Zipf’s law: \(r_f(w) \approx \frac{k}{f(w)}\)
or: \(log(f(w)) \approx -log(r_f(w)) + c\) \( f(w) \) is the frequency of the word \( w \) in the language, \( r_f(w) \) is the rank of \( w \) in the vocabulary sorted by frequency and \( c \) is a constant.
Zipf’s law: \(r_f(w) \approx \frac{k}{f(w)}\)or: \(log(f(w)) \approx -log(r_f(w)) + c\)
This means that the 10th most frequent word is 10 times more frequent than the 100th most frequent word, which is itself 10 times more frequent than the 1000th most frequent word, which is itself 10 times more frequent than the 10 000th most frequent word, and so on. Or the 10th most frequent word is twice more frequent than the 20th most frequent word, 4 times more frequent than the 40th most frequent word, 8 times more frequent than the 80th most frequent word, and so on.
Why is it so?
Zipf’s law: \(r_f(w) \approx \frac{k}{f(w)}\)or: \(log(f(w)) \approx -log(r_f(w)) + c\)
This plot has been drawn based on the frequency of words in Alice’s adventures in Wonderland.
Several similar plots are available in this compressed file (look into texts/_Zipf). Yes, it works for Chinese too!
You may try on any corpus in any language you may choose using this small program.
Zipf’s law: \(r_f(w) \approx \frac{k}{f(w)}\)or: \(log(f(w)) \approx -log(r_f(w)) + c\)
If we plot the number of hits of a few words in a search engine against their frequency rank in the language, we get a linear tendency in log-log coordinates.
This conformity with Zipf’s law suggests that the Web, as seen through search engines, offers an accurate image of language use. The data displayed above are available in this table
(format .xls, format .csv).