Understanding the processes that lead to societal change involves exploring how ideas are generated, disseminated, and discussed; how opinions form, and most importantly, how groups of people who hold similar opinions and positions are interacting with others and changing over time. Although these questions have long been studied at smaller scales in the real world, an increasing amount of human interaction is occurring in new online spaces that we can observe through the immense data available on user behavior in online forums. Yet as Jon Kleinberg also pointed out in a recent blog post, despite the outpouring of data on user actions on the web, we still don’t understand collective human behavior online.
Current methods offer partial snapshots of various structures present in the data, thus the reference to the old parable in the title of our paper: “Blind Men and the Elephant”. Of course, no method can be expected to provide a “complete” picture (as there is no such thing as a complete picture), but we should seek methods that provide a context while laying out a path to investigate data at different granularities and in increments. The ability to zoom in to specific phenomena and then zoom out to observe the big picture is particularly important for scholars and scientists. It allows them to see context, produce frameworks that explain the regularities, and identify irregularities and concentrate on them further.
The idea in our work is to first determine significant macroscopic structures, then add on what they mean while organizing data in layers so there is a way to observe it from high-level and selective to fine-grained and general. Our proposed methodology produces a temporal map of evolving groups in a social news website, represents each group through automatically-identified content preferred by their members, then quantifies its characteristics. The method is widely applicable to different contexts, requires no expert knowledge of the forum under study, and allows for both high-level and fine-grained inspection of groups over time.
Applying this method to a popular Persian-language social news website called Balatarin, we find evolving groups with distinct political preferences. We demonstrate the immense effect of a contentious political event, a structural rearrangement of groups following the event, an abrupt and enduring shift in the focus of groups, and a near-complete extinction of certain interests.
For more, including visualizations of the evolving groups, see our full paper, Blind Men and the Elephant: Detecting Evolving Groups In Social News.