Consumer Click Behavior at a Search Engine: The Role of Keyword Popularity

Kinshuk Jerath, PhD, Ma Liye, PhD, and Young-Hoon Park, PhD

Consumers use search engines every day. What they might not realize is that with each search and each click, they are giving information to researchers about their behavior. Search engine tools such as Yahoo!, Google, and Bing, as well as international versions of these such as Baidu in China or Yandex in Russia all provide consumers with answers to any of their questions in a matter of milliseconds.

When a user conducts a search, results consisting of sponsored links as well as organic links are returned. Sponsored results are links that are paid for via an auction process. This type of advertising is also referred to as paid search or sponsored search advertising. What our research explores is how consumers respond to the search results they are given, how their clicks vary with the use of different keyword searches, and if there are patterns in click behavior.  Consumers use search engines to find answers to life’s questions every day, and it is important that firms who wish to reach their markets analyze the click behavior of consumers.

Sponsored advertising is a growing practice that has developed into an art of choosing just the right keywords to bid on to obtain the most effective use of sponsored advertisements.  In this study, we examined click behavior on organic and sponsored links generated by a search engine after a keyword search is conducted. We use this data to examine consumer activity and decipher their click behavior.

A key question in evaluating the data from 1.63 million keyword searches over a one-month period for 120 keywords is which characteristics should be considered as good indicators of consumer response after a search? Previous studies have considered whether or not a search has a particular retailer’s name in the search, or the length of the search phrase as an important characteristic to consider. However, we also included the measure of popularity of a keyword. This measurement is determined by examining how many users searched for one particular keyword relative to others.

Our research shows that popularity is a key cause of consumer clicks on the search results page. The more popular keywords had proportionally more clicks on sponsored links, while less popular keyword searches had more clicks on organic results.

Additionally, we determined that consumers can be grouped into segments based on their click behavior and how involved they are with the topic they are searching. For example, a user who is searching a topic that they have a low level of involvement with will use more popular keywords, and therefore click on more sponsored links, comparatively. A user who is more highly involved with his or her search will use less popular and more specific keywords, and therefore tend to click on organic links.

Data Overview

Data for this research was collected through a leading search engine firm in Korea.  The page layout that is produced when a search is conducted is similar to the layout produced by many U.S. search engines like Google or Bing. Sponsored links (up to 5) are displayed at the top of the page, followed by organic links below. The organic links that are produced by the Korean search engine are grouped based on the source of the content—news, images, video, etc.—similar to how users can search for images or video specifically on Google. Given these similarities, the conclusions that are drawn from this study can be applied to other search engines around the world. 

The data that was collected centered around the click activity of 1,200 different keywords over a 28-day period in 2011. Keywords can refer to either a single word or a phrase of a few words.  After the 28-day period was over, the dataset consisted of more than 30 million search occurrences. Given this large amount of data, a random sample of 120 keywords was chosen from the original 1200 keywords.  The researchers ensured that there is exactly one search occurrence per IP address to make certain that there is no more than one search instance per user in the data. There is an average of 13,595 searches per keyword, and a total number of searches of 1,631,336 across the 120 keywords. An average of 4.39 advertisements are shown with each search with a standard deviation of 1.21.

Model Development and Estimation

The model that was developed with this data is governed by two components: the tendency to click, and the likelihood to utilize the sponsored or organic listings. Additionally, the model accounts for observed heterogeneity in keywords, the observed heterogeneity in consumers, as well as unobserved heterogeneity in consumers.

Even after narrowing down the selection of keywords down to 120, we were left with over 1.5 million search instances. A random sample of 20% of these instances was selected to narrow down the data set to 326,080 search instances, and the model was estimated on these data. Consumers were segmented into four different segments. Within those four segments, the expected number of organic and sponsored link clicks for each keyword were calculated, and these calculations were compared with the actual numbers. The percentage errors weighted by search volume are reported to be 2.33%, 1.08%, and 2.09% for organic, sponsored, and total clicks, respectively. These percentages provide evidence that this model with four segments is an appropriate model for analyzing click behavior.

Results

Characteristics of Segments

We discovered that the average number of clicks immediately after a keyword search is relatively low, and the share of sponsored clicks is also comparatively small. In segments 3 and 4, consumers are more likely to click links after their search, and they are also more likely to click sponsored links than in segments 1 and 2. This information leads the researchers to conclude that segments 1 and 2 are lower-involved, while segments 3 and 4 are more highly involved segments. A consumer in segment 4 clicks almost 10 times more often than a consumer in segment 1, and is more than 3 times more likely to click on a sponsored link.

The sizes of the consumer segments are 49.11%, 44.96%, 4.20%, and 1.73%, respectively, meaning that segments 1 and 2 make up more than 94% of all searches. Keywords that are not used as often are more likely to be classified into segments 3 and 4. The lower-involvement segments have a higher search volume of keywords that are more common than higher-involvement segments.

As consumers move through a purchase inquiry process, often referred to as a purchase funnel in marketing, their level of involvement changes. The segments identified in this research are parallel to the purchase funnel model. The consumers representing a higher level of involvement are a significantly smaller group of individuals than the consumers in segments representing a lower level of involvement.

Effects of Popularity

The data show that the effect of popularity on click behavior occurs through the different stages of involvement of consumers with purchases.

Company information, brand information, and the length of search results also have an impact on the propensity for consumers to click links. The tendency for consumers to click is actually lower if there is a specific company identifier in the search criteria. Additionally, if more sponsored links are shown, the likelihood for a consumer to click a sponsored link increases. In other words, there is a positive correlation between the number of sponsored links and the propensity for a consumer to click.

Conclusion

The central finding of this study is that the total clicks and the proportion of sponsored clicks after a keyword search is greater for less popular keywords. Additionally, consumers can be grouped into segments that correlate with how high or low their involvement is with their product or service search. Segments with lower-involved consumers are usually focused on more popular keywords. Segments with more highly-involved consumers typically have more clicks per search, as well as more clicks on sponsored links.

In this information age, potential homeowners are turning to the Internet when searching for homes. This trend is especially prevalent for homebuyers who are moving longer distances; seeing the homes in person is not typically feasible in these cases. In fact, according to a study done by the National Association of Realtors and Google, 90% of homebuyers searched online during their home-buying process, and real estate-related searches on Google have grown 253% over the past 4 years. A movement towards digital home buying makes this research in consumer click behavior that much more vital for real estate professionals. Consumers searching for homes via search engines are conducting high-involvement searches. According to this study, those consumers will be using less popular keywords. For example, they could be searching for homes in particular neighborhoods, school zones, or apartment complexes.

What these implications mean for real estate agents is that it may be smart for them to develop their strategies and tactics to reach the online home buying community as potential customers. Agents may choose to focus on particular neighborhoods and place that information on their personal webpages, making their page more likely to pull with these less popular keywords.  After all, home shoppers using search engines are 9% more likely to take an action on a real estate brand website than those who do not search (Digital House Hunt 2013). Having a presence on the web has become more important than ever in this age of information, especially in real estate.

. . . . . . . . . . . . . . . . . . .

Recommended Reading

Jerath, K., L.M. Ma, & Y.H. Park (2014). “Consumer Click Behavior at a Search Engine: The Role of Keyword Popularity” Journal of Marketing Research, 51(4), 480–486.

References

Digital House Hunt (2013), https://www.realtor.org/sites/default/files/Study-Digital-House-Hunt-2013-01_1.pdf, accessed on October 29, 2014.

. . . . . . . . . . . . . . . . . . .

About the Authors

Kinshuk Jerath, PhD
Associate Professor of Business, Columbia University
Kinshuk Jerath is Associate Professor of Marketing at Columbia Business School. His research interests are two-fold: (1) theoretical models that help to obtain deeper understanding of marketing phenomena, especially phenomena related to retailing and online advertising, and (2) applied statistical models for data-based analysis to inform marketing strategy. His research has appeared in top-tier marketing journals, such as Marketing Science, Management Science, Journal of Marketing Research, and Journal of Interactive Marketing. He received a B.Tech. degree in Computer Science and Engineering from the Indian Institute of Technology Bombay and a Ph.D. degree in Marketing from the Wharton School of the University of Pennsylvania. Prior to being on the Columbia faculty, he was on the Marketing faculty at the Tepper School of Business at Carnegie Mellon University.

Liye Ma, PhD
Assistant Professor, University of Maryland
Liye Ma teaches the undergraduate Marketing Research Methods course. His research focuses on issues at the intersection of marketing and technology, with the current emphasis on Internet and Social Media. His work has been published at Marketing Science. He joined the Marketing Department in Fall 2011 after obtaining the PhD degree from the Tepper School of Business at Carnegie Mellon University.

Young-Hoon Park, PhD
Associate Professor of Marketing, Cornell University
Professor Park’s expertise centers around the analysis of behavioral data to understand and forecast customer shopping/purchasing activities, and to conduct customer relationship management. His research has been published in various leading academic journals such as the Journal of Marketing Research, Management Science and Marketing Science. He was a finalist for the 2008 John D.C. Little Award for the best marketing paper published in Marketing Science or Management Science for his research on Internet auctions. He currently serves on the editorial board of Marketing Science. At Johnson, Park teaches in the Strategic Brand Immersion course, as well as Customer Relationship Management and Marketing Research. During 2009-2010, he was a visiting associate professor at New York University Stern School of Business. He has been recognized for exceptional teaching on multiple occasions. The 2008 graduating MBA class at Johnson selected him to receive the Apple Award for Teaching Excellence. Additionally, he taught marketing for three years at the University of Pennsylvania’s Wharton School of Business, where he received the Wharton Advisory Board Outstanding Teaching Award and the Graduate Student Association Council Teaching Award.