A visual story behind words on the web

Every day a large amount of content is being generated continuously on the internet. There are numerous blog posts, social media posts, reviews, ratings, comments, websites, images and videos about people, products, companies, regions, countries etc. This large content generated, targeted and consumed every day, affects public perception. In turn, it can strongly influence entities or events in either positive or a negative way. Given the sensitivity, there is a strong need to understand underlying patterns, trends and attribution in near real-time and use this information for corrective actions. For example, a news analytics service need to detect bias, fake news, analyze sentiments and public perception of various entities and topics of interest. A marketing agency need to understand sources of media bias and how to manage the perception.

Below I have described steps to analyze the unstructured data and provide actionable insights through visualization.

Most of the content is unstructured in the form of text, pictures and videos. A daily crawler can capture data from relevant websites, social media accounts, blogs etc. and store for analytics.

Below are the various ways you can extract and associate meta-data from the captured content.

The collected meta-data on each source can be aggregated and analyzed on different dimensions based on duration, sources, clusters, entities, keywords, authors, sentiments, topics, aspects etc. This aggregation can run daily or as often as needed to reflect real-time information. Such aggregations help in discovering patterns and trends which can then be visualized easily and drilled down to narrow details.

Here are some of the interesting ways you can visualize meta-data extracted from unstructured data to show relationships, clusters, measures and trends.

This graph shows positive and negative sentiments of top media outlets publishing the stories about a particular entity. This is shown as a bar graph for each media outlet (0–5 in the graph below) showing average sentiment score of important entities within all the articles the media outlet created on the keyword.

This scatterplot plots subjectivity against polarity. Each bubble represents an entity, with size denoting the frequency of occurrence and color represents clusters in which it is found. In this case, the outliers shows entities with highly bias (highly subjective and polarized sentiments).

This graph shows association between articles and entities related to a keyword. This force-directed graph shows keyword at the center connected to all the articles which in turn are connected to entities found inside the article. This is a nice way to visualize various relationships between keywords, articles and entities.

Subjectivity by source

This chart shows how subjective or opinionated is the source for a particular keyword. This can tell whether a source might be inherently biased towards a position with respect to an entity or event.

Tag cloud of important words

This visualization can show current topics of importance related to a keyword. For example, in the chart below, you can see that “immigration” is an important topic for the keyword search related to “Trump”.

Subjectivity and Polarity Trends

A candlestick chart typically used to track stock movement, is a nice way of showing recent subjectivity and polarity trends related to a keyword. Based on these trend lines, you can find if a particular person, event, company is developing into a highly emotional and biased story and whether the bias is positive or negative.

Some other interesting visualizations could be:

Dashboards

Various combinations of above visualizations can be combined together in a dashboard. Such a dashboard can provide an interactive way to visualize public perception and impact of on-going stories: A true visual story behind words on the web!

Related posts:

23 participants have come together to make this happen! The total prize is valued to be over 30ETH!! Only 3 lucky winners. Winners will get the prize in-game assets and become “EARLY ADOPTER” —…

Efficient management of air tickets is essential for businesses operating in the UAE, especially in sectors that require frequent travel. With the increasing complexity and volume of travel…