How To Predict Your Traffic Volumes
Data on social phenomena may be volatile, particularly in the era of the internet. New social movements, memes, and trends come and go. The internet traffic reflects this. The amount of visitors to a website displays erratic fluctuations that coincide with long-term trends and seasonal patterns.
In this post, I demonstrate how to anticipate time series that are inherently uncertain. You'll learn how to utilize meteorological data to forecast human activity, with a focus on the internet. I'll teach you how to use domain knowledge of the target variable to enhance your forecasts. We'll go through feature engineering and model assessment briefly. Finally, we create a web application that trains, tests, and evaluates numerous prediction models.
Assume you work for a hosting firm as an analyst. A state-owned corporation uses your servers to operate a network of local news sites. Because public firms are among the most economical and cost-effective organizations on the globe, the concept of assigning server resources (hardware and network bandwidth) according to predicted demand in a specific city was born.
Publishers anticipate page views in order to allocate resources and project future income and advertising growth. The estimates will also be used by the sales divisions to anticipate future revenues and possible commitments to advertising.
Publishers offer advertising space and must estimate the number of page views available before signing a contract with an advertising agency. You have access to server logs as a hosting provider, which includes the number of requests arriving from each city.
The daily number of site visitors from each city having a local news site is included in the dataset. Seasonalities are seen in time series. For daily page visits, there is frequently a weekly seasonality. It appears reasonable to suppose that holidays and weekends have a significant influence on the number of individuals who go online.
We need to give a framework for dealing with basic seasonality before we get into the specific peculiarities of each place (such as local weather). It's also worth mentioning that most websites follow a long-term pattern. There is generally a linear increasing tendency for active firms.
Spectacular increases do occur, but not in the case of our customers. The trend is reversing in March 2020. Poland was the first country to use lockdowns at the time. There are days with outlier observations, as you can see. DDoS assaults, a viral post, or testing in a production setting might all be causes for this.
The Prophet's creators say that removing outliers is the best method to deal with them (see the “Forecasting” section). Prophet has no issues with data that is missing. Outliers are simply assigned n/a values.
Does The Weather Influence Your Urge To Go Online?
It is self-evident that weather conditions influence people's activities. If you want to buy tickets to an amusement park, for example, you presumably consider the weather prediction for that day. Let's check whether this rationale shows up in collective behaviour forecasts.
API For Weather
WeatherAPI.com provided me with access to their API. They provide free weather and geolocation API plans in real-time and prediction. It's critical for our purposes that they give historical weather data for a variety of areas.
All of the results are saved. Because this is historical data, we may keep it indefinitely without having to update it. This will aid in improving performance and lowering expenses. Then, for future usage, we download daily weather to distinct JSON files.
We'll take advantage of Facebook's Prophet library. The prophet is an additive model that fits non-linear trends with annual, weekly, and daily seasonality, as well as holiday impacts, to predict time series data. Prophet is ideal for developing prediction models for time series with significant seasonal characteristics. Because it captures weekly recurrences and includes holiday dates, it is even more suited to forecasting social behaviour than physical data (e.g. solar energy output).
The overall result is that weather data enhances site visit forecast accuracy. Predictive models that are aware of programmatic advertising supply and demand, as well as projections of page views, may provide suggestions to website publishers on how to behave in the market.
This also enables the use of artificial intelligence to maximize the output from each visitor. This may provide publishers with extra income options. The program may also be used to identify irregularities in real-time communications, such as a DDoS assault early warning system.
When SEO professionals and other parties discuss a planned SEO strategy, the subject of how to estimate organic traffic comes up regularly.
Because SEO isn't an exact science, and because there aren't any universal facts (number of words per page, for example) or specific figures (cost per click, for example), it's impossible to create mathematical forecasts.
Why Should You Predict Organic Traffic?
The CEO of a firm, the head of a department, and many other decision-makers may request SEO traffic forecasts for a variety of reasons:
- To have confidence in the investment. (SEO is first and foremost a marketing channel investment.)
- To strike a balance between the SEO budget and paid search spending (Google Ads, Shopping, etc.).
Should You Agree To Make Prognoses?
When presented with demanding management or customer, every SEO expert must answer this question sooner or later. Because SEO is an inexact science, it may seem dangerous to try to forecast outcomes.
Sometimes the individual you're working with will see this and recognize the complexity of SEO right away. In other cases, though, presenting a prognosis will be a condition precedent to receiving approval for any SEO plan. However, before you can start computing a forecast, you'll need adequate data:
- Last year's monthly organic sessions: I believe this is the bare minimum of time that enables you to smooth forecasted data across a full year, allowing for a true grasp of what's underlying the data.
- Monthly sessions from various channels during the same time period to get a clearer view of overall website traffic. This data won't be utilized in any computations.
- Significant occurrences that may need greater sponsored search spending.
- Seasonality (high and low activity cycles) as well as critical times for the website's sector.
This data must be a “good candidate” for making accurate and relevant forecasts. In other words, data that is stochastic or incomplete cannot be utilized.
Are You Tired Of Scams?
How Can Organic Traffic Be Predicted?
Several ways for forecasting expected traffic exist, depending on the tools you employ. For the sake of this post, we'll look at two ways that are both simple to implement and explain to your superiors.
The Holt-Winters Approach
Despite being an exponential smoothing approach, the Holt-Winters method has a significant benefit in that it considers patterns in a sequence of data as well as seasonality. As a result, it can provide accurate estimates based on data particular to the website for which we want to make a forecast.
Using Search Console To Calculate CTR
Because it does not enable you to smooth the prediction over the following 12 months, this second technique takes a more short-term approach to analysis. It does, however, have the benefit of allowing you to target certain sites based on additional, bespoke criteria, such as a significance score you give to them.
This exercise may be done with any crawler that can link to other data sources and any tool that offers keyword data, but we'll utilize OnCrawl, SEMrush, and Search Console in this example. In this case, we'll be looking at data visualization using our keywords (excluding the brand name). We might also use a more granular segmentation to focus on a certain set of pages, for example.
These data will be connected with crawl and Search Console data to generate the following graphic once they are linked to URLs. The goal here is to look at sites that are on Page 1 of Google's search results, between positions 4 and 10, and have little or no competition.
For the time being, we'll presume that this KPI is a factor in the effectiveness of our optimization efforts. As an alternative, we may utilize keyword difficulty as our underlying KPI. In this case, we have 27 pages ranked between 4 and 10 with a low degree of competitiveness and 120 pages with a very low level of competition.
We can now generate a projection based on the current average CTR of sites listed in the top three places in the search results using the table below, which was built using a cross-analysis of Search Console and crawl data. We may also make both positive and negative estimates based on the pages with better or lower CTRs than the whole site's average CTR.
You've just built two distinct sorts of organic traffic forecasts for a website. It's worth noting that various forecasts may be made depending on additional information about competing websites (for example, the presence or absence of structured data on ranking pages, etc.).
There are plenty of headlines these days about how unrecognizable the world seems in the era of coronavirus; we live in unpredictable times, as practically anybody would tell you. However, the times aren't as bleak as they seem in certain aspects. It could be simpler than you think to figure out what kind of business to anticipate on your website.
We experienced the weakest week of traffic on the Ten4 website in recent history as the week lockdown began in the United Kingdom. (We searched for a slower one for 18 months before giving up.) Similarly, March was the slowest month we've ever seen, with just 707 unique visitors. Many other websites, apart from those that reserve grocery delivery slots, are likely to have seen comparable visitor drops. It seemed catastrophic at the moment.
But, if you'd told us a year ago that we'd have 707 website views in March, we probably wouldn't have been concerned, much less astonished. Instead, we'd say, “That sounds about right.” We're not claiming to have predicted the coronavirus.
However, we are familiar enough with fundamental statistics to see that all firms fluctuate over time. While the coronavirus has had many serious and tragic repercussions so far, preparing to get through this difficult moment requires distinguishing between those actual implications and the normal ebb and flow. That is precisely what we did.
To Forecast The Remainder Of The Year, Use One Week's Traffic
So, how did we anticipate a drop in traffic in March? The quick and perhaps deceptive response is that March is usually quiet – the January rush of people thinking new year, a new website has gone, and funds are already spent till April as we approach the conclusion of the financial year.
The lengthier and more honest response is that we didn't anticipate a slowdown in March. However, even a year ago, we could have predicted that the number of users in March would range between 590 and 1,500, based only on week-to-week changes. In that sense, 707 is still low, but it no longer seems so doomsday.
Of course, one might argue that presenting a range of over 900 possible outcomes isn't actually forecasting the future. That's true enough, and we really have a far more thorough internal model that considerably reduces the range. What's amazing about the 590 to 1,500 statistic is how it was calculated: it was based on only one week of daily traffic from February 2019.
That is, we took only seven statistics and utilized them to accurately predict our total monthly traffic for the next 15 months. Similarly, the same seven numbers allowed us to precisely anticipate total weekly traffic for 58 of the next 60 weeks.
Although the week lockdown began was one of just two weeks we got incorrect, we only missed it by 11 users. When we're predicting from a year out, that's a margin of error we're comfortable with.
Set The Scene For Your Present Traffic
Monte Carlo modelling, which employs millions of random data points to bulk out your genuine ones, is the not-so-secret approach underlying these forecasts. We produced around 14 years' worth of false traffic data following the same pattern since those 7 days in February were enough to infer a pattern — a very rough pattern, to be sure, but a pattern nonetheless.
We then analyzed the fictitious data as if it were true, and it turned out that it might have been. That's how we managed to nail so many weeks.
We were fortunate in various ways. When extrapolating from just 7 days of data, slight deviations may have a big impact on your estimate, which is why the range is so broad at 900. It may have been larger, smaller, taller, or lower if we had chosen a different week.
That's why our internal model employs much more than seven data points to arrive at a considerably more accurate result.
Even a large range has importance because considering what would have happened if we only had 200 visits in March. Being so far outside of even the most expansive range would be cause for alarm, a clear indicator that business was grinding to a standstill and severe actions were required.
We were able to stop, take a big breath, and hold our nerve knowing that even the rapidly plummeting traffic in March was still within the range of normalcy. We carried on as usual, although from home as if nothing had happened. That was also a good thing since traffic in April and May returned to typical levels, and business continued to flow in slowly but steadily.
We were fortunate in that regard as well, and we're aware of it, and there's no way of predicting what the world will look like in a few months. With the figures, we'll be able to tell if our company is operating normally – and whether we need to be concerned.
And if the low months start to outnumber the high ones, those data points will be fed into the model, and we'll change our projections accordingly, so at the very least, we'll always know which way we're going.
Predict Conversion Rates And Revenue, Too, And Combine The Results
Because we don't sell anything on our website, visitor volumes are less important than they would be if we were running a full-fledged e-commerce site. But a lot of people own e-commerce businesses – a lot more people since lockdown began — and this type of prediction may be quite useful to them.
The beauty of Monte Carlo models is that they can be used for almost anything and can be combined simply. So, if you know the range of probable visits to your site, the range of likely conversion rates, and the range of likely order prices, you can predict your range of potential income for any given time rather accurately.
While the month-to-month range may still be significant, the other amazing thing about Monte Carlo models is that the farther out you attempt to forecast, the more accurate they get.
Our 7-day model's monthly projections had a spread of 900, but it only missed the total number of visits to our site by 884 over 14 months, a difference of less than one month. If you can accurately forecast your company's revenue for the next year, you'll be in an excellent position to determine whether to make cutbacks, investments, or neither.
It can also show you how modest changes to your e-commerce firm may affect your bottom line — for example, if a digital advertising company claims to increase your traffic by 2%, you can quickly understand what that means in terms of income and if it's worth it.
Even if you don't believe the final estimate, the extremes of the range will give you a solid indication of what to expect in the worst- and best-case situations. And in these uncertain times, a sliver of assurance might seem like a breath of fresh air.
I trust you enjoyed this article on How To Predict Your Traffic Volumes. Would you please stay tuned for more articles to come? Take care!
Want to Learn How to Build Your Own Home-Based Online Business & Start Making Money Online From Your Comfortable Couch?
Try Wealthy Affiliate!
Your Opinion Is Important To Me
Thoughts? Ideas? Questions? I would love to hear from you. Please leave me your questions, experiences, remarks, and suggestions about How To Predict Your Traffic Volumes in the comments below. You can also contact me by email at Jeannette@WorkFromAnywhereInTheWorld.com.