Genealogical Trees on the Web: A Search Engine User Perspective

This paper presents an extensive study about the evolution of textual content on the Web. That is, how some new pages are created from scratch while others are created using already existing content. We show that a significant fraction of the Web is a byproduct of the later case. We present an estimation of how our finds can be applied to other Web data sets. We introduce the concept of Web genealogical tree, in which every page in a Web snapshot is classified into a component. We study in detail these components, characterizing the copies and identifying the relation between a source of content and a search engine, by comparing page relevance measures, documents returned by real queries performed in the past, and click-through data. We observe that sources of copies are more frequently returned by queries and more clicked than documents in general.
Published in 2008