This is why I was particularly perplexed by this chart and the corresponding article:

When I look at this chart, I think, “Wow, RIM sure does have a foothold on the market share of SmartPhones in the US. The others have some catching up to do.” But according to the article, the main finding is “Apple’s U.S. iPhone user base blew past Windows Mobile for the first time in October and it’s bearing down on Research In Motion…”

I certainly didn’t get the feeling that Apple is blowing past anything. Notice how each of the horizontal bars represents a month. Everybody knows that time is best represented as a continuum – since that is what it is. What is even more striking is that Silicon Insider thought they had improved upon the original display of this information by rank ordering the SmartPhone OS by share of the market in October (check out the original chart here. Yuk.).

FierceDeveloper was kind enough to post the original data on their website. Here’s my attempt to show the same information but more visually appealing.

You can still see that RIM has the strongest position but Apple’s surge over Microsoft in July is much more apparent than in the horizontal bar charts above.

Well, what do you think?

To perform a Correspondence Analysis, your data will need to take on a particular form (for one example, see table below). In this table,Brand represents the groups you wish to compare (e.g., males vs. females). In the Attribute, each of the numbers represents a different product being rated by the groups in the Brands column. Finally, the Top 2 Box (%) represents the proportion of respondents within that group who provided a top rating for that Attribute. In the table below, the top row (1,1,.67) means that 67% of Males gave the first attribute a rating of 4 or 5 (out of five). The second row (1,2,.54) indicates that 54% of Males gave the second attribute a rating of 4 or 5.

One thing that makes this a little tricky is that a single row no longer represents one respondent and data from one respondent can now be in multiple groups (rows). For example, both the group for females and the group for young people would represent a 20-year old female who gave a top box rating of Attribute 3.

There is more than one way to get the data into this format; the process outlined below is just one way that can be replicated in SPSS by having to make only a few adjustments to the code.

Note: I used SPSS 12.0.2 for Windows – you may have to make additional modifications if using a different version of SPSS.

1. Select the groups you want to compare

In this example we will examine how seven groups rated 10 different options of snack foods on a scale of 1 to 5.  The seven groups are males (Gender=0), females (Gender=1), young (where Age=1), middle aged (where Age=2), old (where Age=3), dog owners (Cell7=0) and cat owners (Cell7=1). The options for snack food are represented by the variables “option_1″ to “option_10″.

Here is what the initial data file might look like:

2. (Optional) Recode the data

Recode your groups to correspond to the format of the final output. In other words, recode the variables in numeric order so that once they are all combined together, there will be no duplicates or overlap with variable codes. While this step is optional, it will save time and prevent confusion in subsequent steps.

In this example, we will recode variables for gender, age, and pet owner into one new variable called Brand.

RECODE
  Gender
  (0=1) (1=2) INTO Genderr .
EXECUTE .
RECODE
  Age
  (1=3)  (2=4) (Else=5) INTO  Ager .
EXECUTE .
RECODE
 cell7
 (0=6) (1=7) INTO Pet .
EXECUTE .

Here is what your data file might look like at this stage:

3. Restructure the Data Using the Variables to Cases Function

VARSTOCASES is a function that allows you to combine information from multiple variables into one. You can also use it to simplify the working data file, keeping only the variables that you will need and dropping extraneous ones.

The newly created ‘trans1’ will contain all the information from Gender, Age and Pet (here is where you will be thankful for recoding earlier). ‘Index1’ will contain the variable label while ‘trans1’ will contain the numerical code. Because we want the proportion of Top 2 Box ratings from the 10 options, we will use the ‘/KEEP’ option to list out the variables we want to remain in the data file. All other variables will be dropped from the data file at this stage.

VARSTOCASES  /MAKE trans1 FROM Gender Age Pet
 /INDEX = Index1(trans1)
 /KEEP =  option_1 option_2 option_3 option_4 option_5 option_6
    option_7 option_8 option_9 option_10
 /NULL = KEEP.

Here’s what your data would look like at this stage:

4.Count the Number of Top 2 Box Scores

Now that we have each respondent identified to a particular group, we need to count each time they gave a top 2 box rating for each attribute (in this example, the attributes are the variables called “option_x”).

There are multiple ways to do this, one way is to create new variables and use a “do repeat” function. This code creates one new variable for each attribute and places a one for each top two box score for each of the attributes.

compute count_1=0.
compute count_2=0.
compute count_3=0.
compute count_4=0.
compute count_5=0.
compute count_6=0.
compute count_7=0.
compute count_8=0.
compute count_9=0.
compute count_10=0.

do repeat x=option_1 to option_10 / y=count_1 to count_10.
if x=5 or x=4 y=1.
end repeat.
execute.

Now, your data would look like this:

5. Aggregate the data

Once you have counted the number of times a respondent provided a top two box rating for each attribute, now you need to find the proportion of respondents within each group that gave a top two box rating. This can be easily accomplished with the Aggregate function in SPSS.

Aggregate will create a new data file containing only the information you specify. Start by giving your new file a name. Use the ‘/BREAK’ option to specify how you want to group your data, in this example, ‘trans1’ contains our grouping information. The next few lines of code specify how you want SPSS to handle the data. Because our Count variables are binary, simply taking the mean of a binary variable will return the proportion of respondents who gave a top two box rating. The last line in the code, ‘/N_BREAK=N’ is a non-mandatory option. This will return the number of respondents within each of the groups. This is an easy way to check to see if your code is working properly.

AGGREGATE
  /OUTFILE="C:\Documents and Settings\yourname\My Documents\data.sav"
  /BREAK=Brand
  /count_1 = MEAN(count_1) /count_2 = MEAN(Count_2) /count_3 = MEAN(Count_3)
  /count_4 = MEAN(Count_4) /count_5 =  MEAN(Count_5) /count_6 = MEAN(Count_6)
  /count_7 = MEAN(Count_7) /count_8 = MEAN(Count_8) /count_9 = MEAN(Count_9)
  /count_10  = MEAN(Count_10)
  /N_BREAK=N.

Your data would now look something like this:

6. Stack the Data

We are almost done! The next step is to stack the count variables on top of each other. We can use the VARSTOCASES function again to make a new variable from multiple variables and to simplify the dataset down to only the variables we need using the ‘/KEEP’ option. Notice that the values for the variable Attribute are not numeric – you will also need to convert this string variable into numeric format in order to perform a Correspondence Analysis.

VARSTOCASES  /MAKE T2B FROM count_1 count_2 count_3 count_4 count_5
   count_6 count_7 count_8 count_9 count_10
 /INDEX = Attribute(T2B)
 /KEEP =  Brand N_BREAK
 /NULL = KEEP.

RECODE
  Attribute (CONVERT)
  ('Count_1'=1)  ('Count_2'=2)  ('Count_3'=3)  ('Count_4'=4)  ('Count_5'=5)
  ('Count_6'=6)  ('Count_7'=7)  ('Count_8'=8)  ('Count_9'=9)  ('Count_10'=10)
  INTO  Attribute1 .
EXECUTE .

Your data should now look something like this:

The only variables you will need for Correspondence Analysis are Brand, Attribute1 and T2B. Add in the value labels for you Brand and you will be ready to go.

7. Correspondence Analysis

You are now ready to perform a Correspondence Analysis.

Weight by T2B .
CORRESPONDENCE TABLE=Attribute1(1 10) BY Brand (1 7)
 /DIMENSIONS=2
 /MEASURE=CHISQ
 /STANDARDIZE=RCMEAN
 /NORMALIZATION=SYMMETRICAL
 /PRINT=TABLE RPOINTS CPOINTS
 /PLOT=NDIM(1,MAX) NONE .

Here is all of the code for you:

RECODE
  s4
  (1,2,3=3)  (4=4) (Else=5) INTO  Age .
EXECUTE .

RECODE
 cell7
 (0=6) (1=7) INTO Pet .
EXECUTE .

VARSTOCASES  /MAKE trans1 FROM Gender Age Pet
 /INDEX = Index1(trans1)
 /KEEP =  option_1 option_2 option_3 option_4 option_5 option_6 option_7
  option_8 option_9 option_10
 /NULL = KEEP.

SAVE OUTFILE="C:\Documents and Settings\yourname\data.sav"
 /COMPRESSED.

compute count_1=0.
compute count_2=0.
compute count_3=0.
compute count_4=0.
compute count_5=0.
compute count_6=0.
compute count_7=0.
compute count_8=0.
compute count_9=0.
compute count_10=0.

do repeat x=option_1 to option_10 / y=count_1 to count_10.
if x=5 or x=4 y=1.
end repeat.
execute.

AGGREGATE
  /OUTFILE="C:\Documents and Settings\yourname\My Documents\data.sav"
  /BREAK=Brand
  /count_1 = MEAN(count_1) /count_2 = MEAN(Count_2) /count_3 = MEAN(Count_3)
  /count_4 = MEAN(Count_4) /count_5 =  MEAN(Count_5) /count_6 = MEAN(Count_6)
  /count_7 = MEAN(Count_7) /count_8 = MEAN(Count_8) /count_9 = MEAN(Count_9)
  /count_10  = MEAN(Count_10)
  /N_BREAK=N.

Weight by T2B .
CORRESPONDENCE TABLE=Attribute1(1 10) BY Brand (1 7)
 /DIMENSIONS=2
 /MEASURE=CHISQ
 /STANDARDIZE=RCMEAN
 /NORMALIZATION=SYMMETRICAL
 /PRINT=TABLE RPOINTS CPOINTS
 /PLOT=NDIM(1,MAX) NONE .

All it is is an introduction to yours truly and a brief foray into my favorite survey research topic, mobile web survey design.
If you don’t want to open the PDF (and I don’t blame you) here’s the bulk of what I had to say on the topic:

While some of the complications that make surveys so challenging are almost always present, we are now facing issues that were largely unimaginable just a few years ago. The increase in cell-only and cell-mostly households has lead many to question the validity of telephone surveys. The legal, ethical, sampling and measurement issues are all cause for concern but mobile phones present another challenge for survey research in that they are now being used, with great frequency, to access the web.

One concern with this is that the small size of the screen can cause a web survey to appear very differently than on a desktop. This is problematic because research has shown that the visual presentation of the survey may influence the respondent’s choices. The burden on the respondent may also increase because the user experience on a mobile is very different than a desktop or laptop. For example, where a mouse can easily select small radio buttons, they can be much harder to select with a finger on a 3×5″ touch screen. Grids can be even more problematic on a mobile device. Next time you have some time to kill, I implore you to try responding to a web survey with your smart-phone. Does this mean we should have different versions of a web survey specifically for the mobile? Should we restrict web survey access to desktop or laptop computers? What is the likelihood of responding to a web survey via a mobile device and is it likely to increase? These are just some of the questions in need of research.

Many statistical techniques are used to examine relationships between a dependent variable and independent variable(s). In that regard, Factor Analysis is different. Factor Analysis attempts to detect patterns or relationships among a set of defined variables. Basically, all Factor Analysis is, is a grouping of correlated variables into factors, or unobserved (latent) constructs. The assumption is that the latent constructs explain the correlation among the observed (not latent) variables. You can use it to takes long list of items and group them together. Thus, not surprisingly, it is commonly known as a data reduction technique.

Unlike regression techniques, you can’t use FA to make predictions about anything. For example, you could never use FA to say customers with XYZ characteristics are more likely to prefer Product A over Product B. But what you can do is attempt to define underlying constructs. This can be useful when presenting results of a survey to a client who wants to know what qualities are important to their customers or identify attributes of a product that are important, or determine the characteristics of a company’s highest performing employees.

For example, in a market research survey, you might ask respondents to rate Company X on a list of attributes. Attributes can include things like uniqueness, attractiveness, humorous, enjoyment, ease of shopping, customer service, proximity of stores, cleanliness of stores, corporate responsibility, expensive, cheap, convenient hours, luxury, whatever. Often times, attributes will be correlated, like cleanliness of stores, ease of shopping, convenient hours and customer service. These attributes might be measuring the similar ideas or constructs. In this example, these three variables may be measuring attitudes toward in-store shopping.

I won’t go deeply into the difference between Common Factor Analysis (CFA) and Principal Component Analysis (PCA). Just know that they are two similar but distinct types of factor analysis. Both are data reduction techniques but they make very different assumptions about the variance. More information can be found here:
http://www.ats.ucla.edu/stat/sas/library/factor_ut.htm

When to use Factor Analysis:

Factor Analysis can be used when your survey contains a lot of correlated variables that may be measuring similar underlying constructs. So instead of reporting scores for each individual attribute, you can distill all the attributes into multiple groupings, or factors. A large sample size is also important.

And that’s it! Not too intimidating after all, was it?

That is behavioral targeting at it’s finest.

Here’s how it works:
A company operating a website joins an ad network. When you visit a website in the network, a cookie is placed on your browser that begins collecting information. These cookies then track the pages you visit, how long you spend, the searches you make and what articles you read. All this information is then used to segment you into groups with other like-minded browsers. Once you are segmented, when you go to another website in the ad network – no matter what the content of that site is – you can see an ad targeted to your previous online behavior.

So what happened to my boyfriend was that cookies stored in his browser identified that he was shopping for Egyptian cotton sheets online. Because he visited multiple sites, read reviews and compared prices he was probably placed in a segment called “Likely buyer of Egyptian cotton” or something like that. So that a week later when we were reading other blogs in that ad network, we saw ads for good deals on high thread count Egyptian cotton sheets even though the content of the blog had nothing to do with sheets.

The benefits of Behavioral Targeting:

The benefits of behavioral targeting are plenty. First advertisers can get better ROI by improving the performance of their online advertising and secondly, consumers see ads that are relevant. AdAge recently wrote that a world without behavioral targeting or collecting information online “would be like having the same conversation” over and over again. It would also be like the billboards along the highway. Relevant for a few, irrelevant and an eyesore for many more. Why not use the information available to improve the online experience. Otherwise websites would be like Time Square at night: every sign competing for your attention, each new billboard trying to be bigger, brighter and more colorful than the one next to it and almost all of them irrelevant to you.

Benefits for advertisers:
+ Identify potential customers by their behaviors rather than demographics
+ Increase ROI by showing your ad to consumers who are more likely interested in what you have to show them

Benefits for consumers:
+ Exposed to ads that are relevant to you
+ Reduce clutter and nonsensical ads

Some people are getting all up in arms about privacy and the security of all this information being collected. But these fears are largely unfounded. Almost all of the information stored from your browser is kept anonymous and stored in highly secured servers.

Measuring website performance is complex; there are so many things to keep in mind (this will become a major theme of this blog, I can already tell). One common thing people can do is regular analysis of your Google AdWords campaign.

Basically, the idea is like this: your website sells bike saddles so you’ve bid on the keyword “saddle.”  You’ll probably quickly notice that you’re losing money and ignoring potential customers because a) a lot of your clickthrough rate (CTR) is driven by cowboys not cyclists and b) most people don’t know that bikes don’t have seats they have saddles.

Google, being the smartypants they are, has of course thought of this. So you can put in keywords to avoid like “western.” So if someone searches for “western saddle,” your ad won’t show up. You might also want to consider adding “bike seat” because not every knows that when you ride a bike your derrière sits on the saddle, not a seat.

But I’m no expert on AdWords. Andrew Goodman, is the expert; I’m just trying to play catch-up. This guy has written an entire book about it. He’s even created a shorter version of the book that he’s giving away for free over here. I just started reading it but what I like about it so far is that he’s all about measurement and paying attention to the client’s needs.

Lesson of the Day
Today’s lesson of the day came on page 32 of Goodman’s free e-book, Google AdWords, A Brave New World (A Pocket Guide to the Road Ahead).
Don’t be afraid to use less precise keywords. “When you use exact matches only, you’re exhibiting fear of clicks. You’re opting out of potential customers.”
That’s a good lesson.

Currently, this particular edition CA functions as our mouse pad, which I occasionally flip through when I’m bored or waiting for things to download. What strikes me as fascinating about it is that there doesn’t seem to be any firm criteria for which to judge the quality of the work featured in the magazine. For me, being a very quantitative person who likes to measure things, I find this very perplexing.

What I can’t seem to wrap my head around is the fact that there isn’t any concern with whether or not the website serves the needs of the client who commissioned the work. I’m not saying that I don’t think the winners in the issue are awesome – most of it is in fact pretty awesome.

But my question is: did it do anything? Did these really expensive/creative/innovative interactive websites increase sales, help to build brand equity, or bring in new customers? I think I’m a pretty good judge of when a website is cutting edge or new but to me it’s got to perform some function other than just being “creative.”

Maybe the “best” websites aren’t the most “creative”?

Today’s lesson: Residual deviance, Null deviance and Likelihood Ratio Tests (LRT).

First of all, only someone nerdy enough about logistic regression would still be reading this far, so I’m going to go ahead and make some assumptions about your background (e.g., you are at least vaguely familiar with generalized linear models).

For starters, let’s say you’ve got your response variable (Y) and explanatory variables (X, Z, etc) and you want to find the best fitting model. Naturally, the bestest (not a real word, I know) fitting model is the one with a parameter for each cell of the contingency table, we call this the “saturated” model; but this is just far too cumbersome to work with. So it becomes the baseline to which we make comparisons of other (shorter/simpler/easier) models.

The Null Deviance assesses the goodness of fit of a model with only the intercept term to the saturated model; basically, it tells you whether at least one of your βs is not equal to zero.
The hypotheses you’re testing are:

Ho: logit(π) = α
Ha: logit(π) = the saturated model: γj

The Residual Deviance assesses the goodness of fit of a specified model with k number of βs to the saturated model; this tells you whether the model you’ve deduced from model building process fits adequately compared to the saturated model.
The hypotheses are:

Ho: logit(π) = α + β1x1 + β2x2 +...+ βkxk
Ha: logit(π) = the saturated model

But what if you want to compare two simplified models to each other, not to the saturated model? You set them up normally; the new model you’re testing is the Ho, the old model is the Ha, run the glm() and simply subtract their residual deviances from each other. Ta-da!! You’ve just performed a Likelihood Ratio Test!!

I first heard of it a few years ago, tried using it but I was bogged down with the syntax programming. One of the reasons SPSS is so popular is because it is so easy to use with drop-down menus. SAS has drop-downs, too, but the syntax is so easy to write, why bother? I’m not as familiar with STATA but my impression is that it is more similar to SAS than SPSS.

R doesn’t have drop-downs, you tell it what you want it to do. The advantage is that it is extremely customizable. I haven’t used SPSS for a while so this may be irrelevant, but back when I used it, you couldn’t modify your charts and graphs. You could always pick out a graph made in SPSS because of the thick bright red fill (in other words, it was boring). R allows you to define pretty much everything.

I was forced to learn R for a statistics class in grad school. My prof liked to make us do everything the long (and most difficult) way possible. For one project we had to do a permutation to use a Pearson Chi-square test for independence.

Permutations allow you to test for independence without making assumptions about the data distribution.  For example, the Pearson Chi-square test for Independence assumes a Chi-square distribution. But what if your data isn’t chi-square? Well, then you do a permutation.

This basically takes your observed data, rearranges it a bunch of times (like 10,000 times, for example), then you look at the distribution of the data assuming the Null Hypothesis is true (i.e., your response and explanatory variables are Independent). So instead of forcing your observed data to take some assumed distribution and increase your Type I error rate, running a permutation allows you to compare the observed results to it’s own distribution. Or something like that.

There’s a one line code in R that will do all of this for you:

The short way:

chisq.test(gender.table2, correct=FALSE, simulate.p.value = TRUE, B = 1000)

And here’s the long way:

#Put data into raw form

all.data<- matrix(data=NA, nrow=0, ncol = 2)

#Put data into "raw" form
for (i in 1:nrow(gender.table2))    {
for (j in 1:ncol(gender.table2))    {
all.data<- rbind(all.data, matrix(data = c(i,j), nrow = gender.table2[i,j], ncol=2, byrow=T))
}
}
all.data
save<- xtabs(~all.data[,1]+ all.data[,2])

#First do one permutation to illustrate:
set.seed(8067)
all.data.star<- cbind(all.data[,1], sample(all.data[,2], replace = F))
all.data.star
calc.stat<- chisq.test(all.data.star[,1], all.data.star[,2], correct = F)
calc.stat$statistic

save.star<- xtabs(~all.data.star[,1] + all.data.star[,2])

#Now do this with forloop
do.it<- function(data.set)    {
all.data.star<- cbind(data.set[,1], sample(data.set[,2], replace = F))
chisq.test(all.data.star[,1], all.data.star[,2], correct=F)$statistic
}
summarize<- function(result.set, statistic, df, B)  {
par(mfrow = c(1,2))
#Histogram
hist(x= result.set, main = expression(paste("Histogram of ", X^2, " perm. dist.")))
segments(x0 = statistic, y0 = -10, x1 = statistic, y1 = 10)
#QQ Plot
chi.quant<- qchisq(p= seq(from=1/(B+1), to = 1-1/(B+1), by = 1/(B+1)), df=df)
plot(x= sort(result.set), y = chi.quant, main = expression(paste("QQ-Plot of ", X^2, " perm. dist.")))
abline(a = 0, b = 1)
par(mfrow = c(1,1))
#p-value
mean(result.set>= statistic)
}
#Do.it for 1,000
do.it(data.set = all.data)
B<- 1000
results<- matrix(data = NA, nrow = B, ncol = 1)
set.seed(8067)
for(i in 1:B)    {
results[i,1]<- do.it(all.data)
}
summarize(results, x.sq$statistic, (nrow(gender.table2) - 1) * (ncol(gender.table2)-1), B)

Here’s the easy way again:
chisq.test(gender.table2, correct=FALSE, simulate.p.value = TRUE, B = 1000)

I’m very grateful for having had the good fortune to meet Warren once.  He was a great innovator and an unabashed critic when it came to political surveys and methodology.

So this new book by Fritz J. Scheuren and Wendy Alvey brings together a collection of work by leaders in our field.

PS. Be sure to check out Appendix 7.4.2!! It’s a good one!