Keyword research tools are costly to access and complicated to navigate. With this blog post, I want to peel back the curtain for anyone that wants to understand these tools and how marketers like ourselves typically use them to create strategies.
What is a keyword research tool?
It’s a software that provides data on the language, or keywords, people are using on search engines.
The data typically answer these 3 fundamental questions:
- What keywords are people using to search for a particular topic?
- How many people are searching for those keywords? That is keyword volume.
- Which websites come up for those keywords? That is keyword competitiveness.
It can get granular down to the monthly searches, or number of searches in a country and city.
For example, a keyword that I watch like a hawk for the agency is “seo for interior designers”.
Using the keyword research tool SEMrush, I can see that an average of 650 people search for the keyword every month worldwide with small-sized websites competing for it. I can also see the keyword’s seasonality, cost-per-click and related keywords.
This data helps me evaluate whether or not this is a keyword I want to bother with and how much effort or resources I need to fork over in order to rank on the first page of Google.
There are hundreds of keyword research tools for every need and niche. Some of the most popular are Google Keyword Planner, SEMrush, Ahrefs, Moz Keyword Explorer, and UberSuggest.
Alternative keyword tools include Google Trends or Answer the Public. I call these “alternatives” because these tools don’t provide all the usual keyword data, like search volume, but instead provide other useful insights that can be just as valuable.
How do keyword research tools get their data?
Frankly, keyword data is elusive. Where keyword research tools source their data and how they go about consolidating and analyzing it isn’t always transparently shared. From what we do know, there are 3 common data sources:
Search engine APIs
Most keyword research tools have partnerships or access to search engine APIs, such as Google Search API, Bing API, or YouTube API. These APIs provide developers with access to search data, including search volume, related keywords, and other relevant metrics.
Web crawlers
There are a lot of ways to call this data source – web crawlers, clickstream data, web analytics or click path. This data follows the sequence of hyperlinks users take on or between websites, and we consent to the collection and sharing of this data when we accept the cookies on every website we enter.
It’s pretty common for keyword research tools to purchase and use web analytics as a data source to measure keyword competitiveness and rankings.
User data and surveys
Although less common, keyword research tools also collect data from their own user base. They may gather information on search queries and user behavior within their own platform to gain insights into keyword popularity and performance. Some tools even conduct surveys or studies on user preferences and search patterns.
If you’re interested in reading what the most popular keyword research tools share about their data, here are some places to start:
What are the advantages of keyword research tools?
Here are some advantages …
Learn how your customers speak
Keyword data shows us the language the general public uses to describe or look for something online.
The language you use in your marketing matters because like attracts like. Speak like yourself and you’ll attract more people like you. Speak like your clients and you’ll attract more people like them. And who is hiring your services – your peers or your clients?
Name it wisely
With 4.4k monthly services for “apartment painting” versus 170 for “condo painting”, which one do you use to name your services or describe your business? Common but competitive? Or specialized yet attainable?
Take it a step further and use keyword data to help you name the page titles and subheadings to your blog posts. Keyword data can show you what is most commonly searched as well as what’s the most competitive.
Catch your blindspots
On a similar note, keyword research tools can suggest variations, related phrases, or questions that you didn’t think of because they might not be in your vocabulary or be top of mind.
These may be keywords with fewer monthly searches, but the intent to purchase is high, or it could support the entire topic cluster.
Put data behind your content plan
Put all the points above together, and keyword data can be the backbone of an entire content marketing strategy so that you can be found for your expertise and not just your business name.
Keyword volume, competitiveness, and variations can help decide what blog posts, podcast episodes or videos to create and how to go about creating them.
Inform how you go about creating and distributing content
Say you’re filming a video commercial. If you know the video will be published on mobile apps like Instagram or TikTok, then your team will know to film and edit the videos vertically.
The same is true for SEO.
It’s easier to use keywords before writing your next blog post, recording a podcast episode, and filming a video.
A well-researched list of keywords will help the writer or speaker structure their content and title their work while organically using keywords instead of robotically plugging them in afterward.
Creating content is one half of the equation; distributing the content is the other. With keywords, your content can do both at the same time.
Set expectations and evaluate progress
When creating content or going after a target customer, keywords can help you choose and prioritize topics based on which will have the most ROI.
With this data, you can estimate the size of a market or demand for it and, in turn, answer these questions …
- Should I do this? Is it worth it?
- And if I do, what expectations should I have?
- Without data, it’s hard to create marketing strategies that answer those questions.
What are the disadvantages of keyword research tools?
Here are some disadvantages
Keyword volume is wildly inaccurate
How many monthly searches does “interior design marketing agency” get in the USA?
SEMrush says 30 and UberSuggest says 40.
Comparing those numbers to our own website analytics, the keyword saw at least 279 searches – only ranking third. That’s 9 times more than what keyword research tools estimated.
Keyword research tools are notorious for inaccurate data. Discrepancies exist between keyword research tools themselves. Even Google Ads’s Keyword Planner, which has access to its own search database, has been inaccurate for several years.
We see the largest discrepancies among low-volume keywords. In his seminar at Brighton SEO 2022, Mark Williams-Cook presented an experiment going after “0 volume keywords” but seeing up to thousands of clicks a year from those very same keywords.
With 2.4 million searches a minute on Google, a keyword that gets hundreds of searches a month typically goes unnoticed and uncaptured by software tools.
As marketers, our campaigns lose validity when the data behind them is inaccurate. We’re left guessing how or where to spend our time and money around content.
For the internet, 300 searches are equivalent to 0. For a small business, 300 searches for “interior design marketing agency” represents our exact target customers. Even if 5% of those monthly searches turn into customers, that’s a meaningful amount of revenue.
Keyword volume is a sophisticated estimate. Not facts.
Data looks at the past, not the future
Keyword data are updated monthly and are averages from the past six months – meaning that they average out seasonality and don’t predict future demand or trends.
Of the 2.4 million searches a minute on Google, up to 20% are new (source). That’s 480,000 new keywords a minute. Some industries move too fast to rely wholeheartedly on historical data like keyword data.
Scores are ambiguous
Keyword tools often assign keyword metrics like a competitive score or density score. Each tool calculates these scores differently or even names them differently.
It’s hard to gauge what is a reasonable metric for you. When a keyword has a volume of 50,000 and a competitive score of 50, what does that mean for you? Is that a reasonable volume and score to go after? Or is it out of your league?
A website like Architectural Digest can rank for a keyword much easier than a mom-and-pop website.
In reality, the website’s age, the number of backlinks, and current content output are all factors that determine how easily a website can compete for a keyword.
Lacks context
Research tools tell you the numerical metrics behind a keyword, but they can’t tell you the nuances or intent of a keyword.
Say you’re brainstorming names for a new brand. A keyword research tool won’t tell you “Hey, there is a century-old corporation somewhere in Germany that is already named this,” or “Watch out, this is spelled very similarly to the surname of an infamous criminal from the 80s that will soon star in a Netflix documentary”.
They also don’t tell you about intent. Using AI, some keyword research tools have been able to categorize keywords as informational or commercial, but the algorithms to understand the nuances are still learning. It still requires a human to look at a keyword, use some empathy and decipher the intent ourselves.
This is the largest disadvantage and the most widely discussed in the SEO community.
Limited to non-USA regions and languages
With the most popular keyword research software located in the US and serving US-based companies, much of the data is US-centric.
Often, data is limited for:
- Less populated countries
- Non-English keywords
- Specific cities or regions, even in the USA
If you’re doing international SEO and marketing, you’ll need a specialized tool for that kind of information.
The more you pay, the more reliable it can be
This is changing, but it used to be that the cheaper the tool, the less reliable the data. Google’s Keyword Planner, for example, is, in theory, free to access but only gives you more refined data if you are paying actively for ads.
Should I use keyword data in my marketing?
Keyword data is nuanced and flawed and needs to be used with discernment. Despite all its problems, I still rely on keyword software. Just not too much.
If we rely on the data too much, our approach may lack empathy or miss out on nuances and opportunities that only a human can spot.
Yet if we ignore the data together for our intuition, then we risk making decisions that feel comfortable to us but not our target customer.
Here are some guiding principles I’ve learned over the years using keyword research tools.
Use search intent more than volume
This means scanning through every keyword on your list and asking yourself: why would someone search this? Are they looking for an answer to their question? Or are they looking for a product or service to their desire? Shortlisting keywords by numbers is efficient, but it doesn’t help you understand how to give the user what they want.
Use a mix of data
Your marketing analytics, like Google Analytics, Google Search Console, or even the analytics of your social media accounts, can provide insight and observations. The conversations you have on social media, the ones you overhear at conferences, and the topics popularizing industry publications are all informal forms of surveys and qualitative data.
Look at the numbers as ratios
According to Google Trends, there were 0 searches for “interior designer near me” in 2011. Today, there are 60,000 searches worldwide. Do I believe the numbers are true and exact? Common, worldwide?! No. But do I believe the ratios are on to something? Yes.
Use the 40-30-20 rule
This is one I like to use when creating content strategies. Back up 40% of your content with clear data, 30% with semi-clear, and 20% with pure intuition.
To me, semi-clear data is when one data source says “yes,” and another says “not sure”. For example, the keyword data may say there are 0 searches for any given keyword but other data, like industry forecasts or surveying, suggests that people are or will soon be searching for those keywords.
Marketing is a dance between data and intuition. Frankly, you need to experience dancing in both waters and even drowning in one to know where the delicate balance lies.
