(electronic music) Narrator: Everyday, Yelp receives thousands of reviews from all over the world. People review everything from hairstylists to restaurants to grocery stores. And each time you write a review, Yelp gains a little more knowledge about who you are and what you’re thinking.
Matt Taddy: Sentiment classification and sentiment prediction is one of the classic settings for text analysis, and so in this setting, what you’re trying to do is read a document, machine-read a document, and predict whether the author was positive, negative, or neutral.
And the nice thing about review data is that you have different areas that people can review about. So with the We Ate There data, they were able to review the ambience. They were able to review the value. They were able to review the quality. And so what you cannot just pick out whether they were happy or positive or negative, you can pick out whether they were positive or negative about specific attributes and tease out which words are associated with some attributes and which words are associated with other attributes, right?
So both dark and well-lit were associated with good ambience, so try and solve that. Dim sum, Chinese, Mexican was associated with value propositions, and then steak was kind of a quality, highly associated with the quality word.
But the nice thing about this is that it’s proof of concept that if you get enough data, and you get rich enough data, you cannot just tease out happy or sad, you can really tease out happy about something, happy about something else, and figure out the interconnectedness between these different elements.
The company Yelp, which manages online reviews—and everybody knows Yelp—they have been so kind as to make much of their data public. So for a specific period in time, specific region—Phoenix and the surrounding areas—what they did is they threw all the reviews out there in an easily digestible format.
We go through the reviews, and we would like to predict whether this is going to be a popular review, right? So Yelp doesn’t need to know whether this is a five-star review or a four-star review. They know that because you put that in your review. But what they don’t know in advance is what the most useful reviews are gonna be. So for someone new coming onto Yelp, should this review be at the top of the website or should it be at the bottom of the page because nobody cares about this opinion?
And so what we do is we go through the reviews that were popular in the past. Yelp has this system where you call them funny, useful, or cool, and we try to predict in the future which reviews are going to be funny, useful, or cool based upon just the content of those reviews, which means that when they get a new review, they can feed it through, and they can say, “Hey, this is a review that is predicted to be both funny and cool, but not useful,” and then they can deal with it accordingly.
And then the last question is this very subtle example of using text as a control. So there we’re asking the question: Do more experienced reviewers tend to give more positive reviews? And we wanted to ask that question. If you look at the data, it looks like, hey, if you’ve been a reviewer for longer, your review average tends to be higher. You give higher stars. But we wanted to ask the question: Does that difference exist when we control for the fact that, hey, as a longtime reviewer, you reviewed different stuff, right?
The first-time reviewer is always gonna review a restaurant. As you’re a heavier user, you start to review, I don’t know, grocery stores, airports. You can see everything’s reviewed up there. And so the way that we control for the stuff that they’re reviewing is we actually say, “OK, let’s control for the language that they use.” And let’s say even given the text, so even what they say in their review: Does this reviewer who’s been around for 10 years or given 1,000 reviews tend to give a higher star rating, even given the exact same content to their review than someone else who’s brand new who had the same review content for this same item? And we do see that, yes, there is some sort of unexplainable positive bumps.
So people that say the exact same things, they tend to give a higher star rating if they’ve been on the website longer. Instead of just looking at what we would call pair-wise—so hey, you said this and you’re happy, you said that and you’re happy, OK—looking at: you said all of this in connecting that to the big set of information, everything that we know about you.
And the reason that that’s important is when we do things in that way, when we connect all of what you say to everything that we know about you, we can tease out the relationships. You said this because you’re happy and you’re in the Midwest and you bought something this morning, rather than trying to estimate each of those relationships individually and then piece them back together at the end.
So that’s to me the most interesting point right now is: How can I figure out ways to match up the statistics and the computer engineering with the realities of what people want in a language that they’re familiar with in social science, economics, and business environments?