No. Grandparent has picked out a few pathological examples out of many good ones.
Here is a uniform, random sample of documents:
faststretch details
honourable judge
oriental museum http
london wc 1n 3 bg pdsg seminar
1857-8 indian mutiny
vera bell
katherine r.b
rare books collection
novy arbat street
universitys case
The data set is unique terms that occur in a crawl of .uk.
Here is how I constructed the data set:
I took the UKWAC web-as-corpus crawl (2 billion words, crawled in 2008), ran it through the splitta sentence splitter, removed all funny characters, ran the Penn treebank word tokenizer, and perform term extraction with topia.termextract, discarding terms that will single words.
I then lowercased the terms, sorted them, and uniqued them, to give the dataset.
Here is a uniform, random sample of documents:
The data set is unique terms that occur in a crawl of .uk. Here is how I constructed the data set:I took the UKWAC web-as-corpus crawl (2 billion words, crawled in 2008), ran it through the splitta sentence splitter, removed all funny characters, ran the Penn treebank word tokenizer, and perform term extraction with topia.termextract, discarding terms that will single words. I then lowercased the terms, sorted them, and uniqued them, to give the dataset.