How to get daily Google Trends data for any period with R

Last updated in June 2021

Recently, I needed some seven years of Google Trends daily data. It turned out that by default it’s not possible to get it neither through the web interface nor via API. So I wrote a tiny script that pulls daily Google Trends data for any period using gtrendsR package

What’s the problem with Google Trends?

Google Trends returns data in daily granularity only if the timeframe is less than 9 months. If the timeframe is between 9 months and 5 years, you’ll get weekly data, and if it’s longer than 5 years – you’ll get monthly data.

A trivial solution like querying the data month by month and then tieing it together won’t work in this case, because Google Trends assess interest in relative values within the given time period. It means that for a given keyword and month, Google Trend will estimate interest identically – with a local minimum of 0 and a local maximum of 100 – event in one month it had twice as many searches than in the other.

Querying Google Trend daily data properly

To get proper daily estimates, I do the following:

  1. Query daily estimates for each month in the specified timeframe;
  2. Queries monthly data for the whole timeframe;
  3. Multiply daily estimates for each month from step 1 by its weight from step 2.

Here is the R code:

library(gtrendsR)
library(tidyverse)
library(lubridate)

get_daily_gtrend <- function(keyword = c('Taylor Swift', 'Kim Kardashian'), geo = 'US', from = '2013-01-01', to = '2019-08-15') {
  if (ymd(to) >= floor_date(Sys.Date(), 'month')) {
    to <- floor_date(ymd(to), 'month') - days(1)
    
    if (to < from) {
      stop("Specifying \'to\' date in the current month is not allowed")
    }
  }
  
  mult_m <- gtrends(keyword = keyword, geo = geo, time = paste(from, to))$interest_over_time %>%
    mutate(hits = as.integer(ifelse(hits == '<1', '0', hits))) %>%
    group_by(month = floor_date(date, 'month'), keyword) %>%
    summarise(hits = sum(hits)) %>%
    ungroup() %>%
    mutate(ym = format(month, '%Y-%m'),
           mult = hits / max(hits)) %>%
    select(month, ym, keyword, mult) %>%
    as_tibble()
  
  pm <- tibble(s = seq(ymd(from), ymd(to), by = 'month'), 
               e = seq(ymd(from), ymd(to), by = 'month') + months(1) - days(1))
  
  raw_trends_m <- tibble()
  
  for (i in seq(1, nrow(pm), 1)) {
    curr <- gtrends(keyword, geo = geo, time = paste(pm$s[i], pm$e[i]))
    print(paste('for', pm$s[i], pm$e[i], 'retrieved', count(curr$interest_over_time), 'days of data (all keywords)'))
    raw_trends_m <- rbind(raw_trends_m,
                         curr$interest_over_time)
  }
  
  trend_m <- raw_trends_m %>%
    select(date, keyword, hits) %>%
    mutate(ym = format(date, '%Y-%m'),
           hits = as.integer(ifelse(hits == '<1', '0', hits))) %>%
    as_tibble()
  
  trend_res <- trend_m %>%
    left_join(mult_m) %>%
    mutate(est_hits = hits * mult) %>%
    select(date, keyword, est_hits) %>%
    as_tibble() %>%
    mutate(date = as.Date(date))
  
  return(trend_res)
}

get_daily_gtrend(keyword = c('Taylor Swift', 'Kim Kardashian'), geo = 'US', from = '2013-01-01', to = '2013-09-01')

get_daily_gtrend function should return a tibble with daily trend. Now you can plot it nicely or use in some analysis

 1284   2019   R
12 comments
mary azari 6 mon

Hi Alex
Many thanks for sharing the code
I applied it but found an error like this:
Error in UseMethod(“count”) :
no applicable method for ‘count’ applied to an object of class “NULL”
I’d greatly appreciated if you kindly guide me.

Alex Dyachenko 6 mon

Hi Mary, I just double-checked the script and it seemed to work fine. Most likely it’s something on your side. I just replied you via email.

Giovanni da Rosa 6 mon

Hi Alex!

Thanks for sharing this code.
Do you have any idea of how can I get daily city level data?
I’ve never used R, but  I really need to work with daily data grouped by day, especeally applied to Brazil.
I was trying to do that in python, but I could not find a way to do that with pytrends, so here I am.

If you have any ideas, it would be of great help!

Alex Dyachenko 4 mon

Hi Giovanni,

As far as I can see, there is no simple way to implement this. Google Trends web UI -- as well as gtrendR library -- do not provide trends broke down by geo. You can have either, but not both.

The only bypath I can think of is to query daily data separately for each city and than normalize it by aggregated popularity of the query in those cities.

I’d suggest you to take a look at gtrendR documentation https://cran.r-project.org/web/packages/gtrendsR/gtrendsR.pdf. It has a function called ‘gtrends’ that return an object with multiple params, including ‘interest_over_time’ (trend) and ‘interest_by_city’ (snapshot). Combine the two and you’ll get what you want.

Thoraya R 4 mon

Hi Alex,
Thanks for this, it’s very useful! I got the same error as Mary, any thoughts on how to resolve this?

Alex Dyachenko 4 mon

Hi Thoraya,

I wasn’t able to reproduce Mary’s case. If you still have this problem, you can send me your code via email and I’ll try to take a look at it.

Yoan Aleksandrov 4 mon

Hi Alex,
Thank you for providing this solution. I tried the code and it works for me, except I am not an expert in R and I am not able to view or plot the data I obtained. Plot() does not work in displaying the data from the get_google_trends function I obtained in the environment with your code and gives an error. So in short, my question is how do I access the complete daily data for use in further analysis.

Alex Dyachenko 4 mon

Hi Yoan,

I’d suggest you to go over some basics books on R, such as https://r4ds.had.co.nz/. If you need to plot the data ASAP and don’t have time to read the whole book, try this: https://uoftcoders.github.io/rcourse/lec04-dplyr.html

mt 4 mon

Hi Alex,
Thank you so much for your codes! It helps me a lot!
When I ran the codes:
trend_data_plywood = get_daily_gtrend(keyword = c(‘fire’,’plywood’), geo = ‘US’, from = ‘2011-05-01’, to = ‘2012-01-30’) ,
I got an error:
Error: Problem with `mutate()` column `est_hits`.
i `est_hits = hits * mult`.
x non-numeric argument to binary operator

Also, when I ran :
trend_data_forestry = get_daily_gtrend(keyword = c(‘fire’,’forestry’), geo = ‘US’, from = ‘2011-05-01’, to = ‘2012-01-31’) ,
I got this error:
Error: Problem with `summarise()` column `hits`.
i `hits = sum(hits)`.
x invalid ‘type’ (character) of argument
i The error occurred in group 1: month = 2011-05-01, keyword = “fire”.

I would appreciate a lot if you would give me the reply.

Best regards,
Ming

Alex Dyachenko 4 mon

Hi Ming,

Thanks for pointing out the issue! Turns out that gtrendsR’s API might return non-numeric value of hits (such as ’<1'). I've modified the script to handle that so both of your queries return proper results now.

Let me know if you have other questions!

Ben Murphy 3 mon

Hi Alex,

I am receiving the same error as Mary and Thoraya with null objects from the usemethod (count.)

It seems like it only arises with very low monthly values, in my case, it is for consistent zeros. I was curious if you were able to figure this out? Any help is greatly appreciated.

Thanks!
Ben

Alex Dyachenko 3 mon

Hi Ben,

Could you please share the exact query and timeframe so I could reproduce the error?

Radu G Cristea 3 mon

Hi Alex,

Thanks for this super useful script. Got a quick question – please don’t take this as me being lazy, I just don’t have the time right now to get a good grasp of R and implement what I’m about to ask myself. Was wondering is there’s a quick tweak to the code you shared to retrieve weekly data (not daily) for periods longer than five years. Is the code amenable to such requests? It didn’t seem straightforward to me when looking over it.

Many thanks in advance for your help!

Cheers,
Radu

Alex Dyachenko 2 mon

Hi Radu, the simplest way to get weekly data for a period over 5 years would be to get daily data and then group it weekly.

Radu G Cristea 3 mon

Hi Alex,

If I may, I’ve got another relevant question – apologies for spamming. Would appreciate any suggestion from your side.

I’ve been trying to pool trends data for categories also, using your script but I couldn’t tweak the code in an effective way for that. Although it runs, I’m not retrieving the data, but instead a bunch of zeros. See my code below, thanks!

get_daily_gtrend_category <- function(category=20, geo = ‘US’, from = ‘2013-01-01’, to = ‘2019-08-15’) {
if (ymd(to) >= floor_date(Sys.Date(), ‘month’)) {
to <- floor_date(ymd(to), ‘month’) – days(1)

if (to < from) {
stop(“Specifying \’to\’ date in the current month is not allowed”)
}
}

mult_m <- gtrends(category = category, geo = geo, time = paste(from, to))$interest_over_time %>%
mutate(hits = as.integer(ifelse(hits == ‘<1’, ‘0’, hits))) %>%
group_by(month = floor_date(date, ‘month’), category) %>%
summarise(hits = sum(hits)) %>%
ungroup() %>%
mutate(ym = format(month, ‘%Y-%m’),
mult = hits / max(hits)) %>%
select(month, ym, category, mult) %>%
as_tibble()

pm <- tibble(s = seq(ymd(from), ymd(to), by = ‘month’),
e = seq(ymd(from), ymd(to), by = ‘month’) + months(1) – days(1))

raw_trends_m <- tibble()

for (i in seq(1, nrow(pm), 1)) {
curr <- gtrends(category, geo = geo, time = paste(pm$s[i], pm$e[i]))
print(paste(‘for’, pm$s[i], pm$e[i], ‘retrieved’, count(curr$interest_over_time), ‘days of data (all keywords)’))
raw_trends_m <- rbind(raw_trends_m,
curr$interest_over_time)
}

trend_m <- raw_trends_m %>%
select(date, category, hits) %>%
mutate(ym = format(date, ‘%Y-%m’),
hits = as.integer(ifelse(hits == ‘<1’, ‘0’, hits))) %>%
as_tibble()

trend_res <- trend_m %>%
left_join(mult_m) %>%
mutate(est_hits = hits * mult) %>%
select(date, category, est_hits) %>%
as_tibble() %>%
mutate(date = as.Date(date))

return(trend_res)
}

trendsAV_DE = get_daily_gtrend_category(category = 47, geo = ‘DE’, from = ‘2007-01-01’, to = ‘2007-01-30’)

Florian Schäfer 2 mon

Hello Alex,
Many thanks for this great Help. As many in this forum I am by no means a professional in r and as I have to do linear regression in r using gtrends and I have only ever done data analysis with data frames, I would like to ask you if you have got a Code or maybe even just a line of code that would convert the tibble into a data frame.
Any help is greatly appreciated.
Thanks in advance
Florian

Alex Dyachenko 2 mon

Hi Florian, you may consider as.data.frame function https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/as.data.frame

Siem Haile 24 d

Hi Alex. Great code. Thanks for helping out!

When I cross check if the returned by the code against the actual data Google Trends (i.e. ‘Taylor Swift’ between 2020-01-01 and 2020-01-31), the index for the individual days do not seem to match. Is there a particular reason as to why this is the case?

Thanks for taking the time to reply!

Siem Haile 24 d

*misspelling in the first post, sorry.

Hi Alex. Great code. Thanks for helping out!

When I cross check if the tibble returned by the code against the actual data Google Trends (i.e. ‘Taylor Swift’ between 2020-01-01 and 2020-01-31), the index for the individual days do not seem to match. Is there a particular reason as to why this is the case?

Thanks for taking the time to reply!

Alex Dyachenko 19 d

Hi Siem,

I’ve double-checked if the script return the same result as Google Trend itself and it seems to match.

Here is the params I used in the script:
get_daily_gtrend(keyword = c(‘Taylor Swift’), geo = ‘US’, from = ‘2020-01-01’, to = ‘2020-01-31’)

And here is how I retrieved that same data from Google Trends (gtrend library is the official Google’s API to Google Trends):
gtrends(keyword = c(‘Taylor Swift’), geo = ‘US’, time = paste(‘2020-01-01’, ‘2020-01-31’))

The result seems to match. Could you please provide the two queries you compare so that I could reproduce it?

matti heino 3 d

Hi!

I’m getting the same error as some others:

Error in UseMethod(“count”) : no applicable method for ‘count’ applied to an object of class “NULL”

Here’s the query:

get_daily_gtrend(keyword = “koronatilanne”,
geo = “FI”,
from = “2020-01-01”,
to = Sys.Date())

Sys.Date was “2021-10-20”.

Thanks for any insights!

Matti