Less is more

As mentioned in my previous post. I really looked forward to testing Google Analytics Intelligence out and here it happens. It is an amazing tool.  Avinash Kaushik, the author of the web analytics best- selling book, nailed it as “identifying the unknown unknowns!” The intelligence function is truly a great breakthrough for online marketers, check the intro video by Google out if you want to know more.

I particularly like the flexibility of the tool, details as following.

1. I can choose the time period for data entry as “by day”, “ by week”  and “by month” . It tailored both my needs for real time campaign tracking and historically data mining.

2. It allows me to tell how sensitive of the changes I want to track.  In a particular case, if I get several alerts per time period, I always want to rank them by the extension of the changes first.

3. Customize my alerts to the metric that I cared most about.

4. Another thing that I particular like is the segmentation tab. After helped me identifying the unusual pattern, Analytics Intelligence easily opens a door for me to go to further analysis for the segment that comes up with a significant changes, for understanding why.

With all the good stuff that are exciting, here comes the critique: Is Google Analytics Intelligence intelligent enough?

Here are the things that need to be paid attention to.

1. Custom alerts not show up for past data

Like the goal and funnel setting, custom alerts don’t apply for historical data. Though it is not hard to draw a historical graph and found the changes I want to track, it would be another manually task for historical data miner who wants to do so.

2. After create the “alert” segment, so what?

I tried and found I could not adopt Intelligence to sub-segments in Google Analytics,  For example, if I identified 173% increases in New York visitors on my site. It would be perfect if Intelligence could tell me why. For instance, is it because a sudden increase in PPC or referring channel?  What keywords leading to that and what pages do they visit most?  It would be a huge time-saver if Intelligence could use its scientific method again to tell me that, other than I dig into them by myself.

3. Similarity in the massive alerts

In web analytics, we always found metrics are very much correlated with each other. A group of consumers that convert better always demonstrate a higher time on site, a lower bounce rate and more page views/session at the same time. Same happens in the alerts. If I got a 143% increase in visits, it could apply to a multiple geographic areas, different channels together.  If in the case, there is a total 143% increase for all visits but only 68% increase in Chicago, or 34% increase in PPC, actually the particular area and channel is below the average. But the alerts will still shows in “green”, while in some situation, could not be a good indicator.

4. How to calculate the expected value range.

In the alerts, GA identify the changes based on expected value. But the question is, how it calculate the value? There is no information in its help center, so I tried to pull out the data by myself , just curious to see how it works.  Take the New York sales data for example.

According to GA intelligent, sales data for New York is expected to hit between $1238 to $2754 yesterday, but reached $4322, hence caused a significant 162% increase. There are two way to come to the expected value range, one is to calculate average, find the frequency and standard deviation of historical sales data,  the other is doing predictive modeling. Let’s  look at the previous one first, I analyzed all the historical sales data in New York and here is the result generated by SPSS as descriptive analysis.

Due to the skewness of data, cause 2/3 of sales falls from 0 to $1905, apparently not the way GA is using, while the standard deviation is close to the range of expected value (it depends on what confidence level GA uses)

Now I change to another way, which is predictive modeling of time series.  Here is the result and the blue line is the forecasting value, and it was $2914 for yesterday.

Actually if I adding the standard deviation I found in the descriptive analysis, the expected sales value for  yesterday could falls somewhere between 2300 to 3500, surprise to find it is more close to real data than what GA is coming up. Hence I would love to see if GA could disclose more about their calculation methodology here.