Deconstructing Recommender Systems
- http://spectrum.ieee.org/computing/software/deconstructing-recommender-systems
- “猜你喜欢”是怎么猜中你心思的？ | 科学人 | 果壳网 科技有意思 : https://www.guokr.com/article/382553/
- 推荐系统绝对不会向你推荐什么 | 科学人 | 果壳网 科技有意思 : https://www.guokr.com/article/382681/
关于推荐算法
- what you look like to Amazon? Here is the cold, hard truth: You are a very long row of numbers in a very, very large table. This row describes everything you’ve looked at, everything you’ve clicked on, and everything you’ve purchased on the site; the rest of the table represents the millions of other Amazon shoppers. Your row changes every time you enter the site, and it changes again with every action you take while you’re there. That information in turn affects what you see on each page you visit and what e-mail and special offers you receive from the company. # 对于每个顾客来说我们就是一个向量存在表里面，表里面还存放了其他向量存储其他顾客的信息（百万级别）。这个向量非常长，反映了你的一些行为和动作。你的每一个行为和动作都会改变这个向量里面的一些值。
- Over the years, the developers of recommender systems have tried a variety of approaches to gather and parse all that data. These days, they’ve mostly settled on what is called the personalized collaborative recommender. They’re “personalized” because they track each user’s behavior—pages viewed, purchases, and ratings—to come up with recommendations; they aren’t bringing up canned sets of suggestions. And they’re “collaborative” because they treat two items as being related based on the fact that lots of other customers have purchased or stated a preference for those items, rather than by analyzing sets of product features or keywords. # 个性化协作推荐系统，所谓个性化就是通过追踪和分析单个用户行为比如页面访问，购买记录或者评价等来得到推荐内容，而不仅仅是一系列的建议；而协作是指这种分析是通过和其他用户行为或者是他们购买的东西来共同计算出来的而不仅仅是孤立地分析单个用户行为。
- GroupLens and Ringo both used a simple collaborative algorithm known as a “user-user” algorithm. This type of algorithm computes the “distance” between pairs of users based on how much they agree on items they have both rated. Users whose tastes are relatively “near” each other according to these calculations are said to share a “neighborhood.” . But the user-user approach doesn’t work that well. For one thing, it’s not always easy to form neighborhoods that make sense: Many pairs of users have only a few ratings in common or none at all, and in the case of movies, these few ratings in common tend to be of blockbusters that nearly everyone likes. Also, because the distance between users can change rapidly, user-user algorithms have to do most of their calculations on the spot, and that can take more time than someone clicking around a website is going to hang around. # 一种是user-user的算法，这种算法通过比较两个顾客之间购买物品的相似度来计算两个用户之间的距离。但是这种算法存在问题就是很多顾客可能交集非常少，但是对于一些大众的东西则交集非常多。并且因为用户行为变化非常快就会带来大量的计算，而这些计算又不能够很快地完成。
- So most recommenders today rely on an “item-item” algorithm, which calculates the distance between each pair of books or movies or what have you according to how closely users who have rated them agree. Distances between pairs of items, which may be based on the ratings of thousands or millions of users, tend to be relatively stable over time, so recommenders can precompute distances and generate recommendations more quickly. # 另外一种是item-item的算法，很多顾客对某两个东西之间联系的评价来决定两个item之间的距离。相对于user-user算法来说，因为item-item之间的关系需要参考大量的用户因此相对来说比较稳定，所以可以提前计算好。
- One problem with both user-user and item-item algorithms is the inconsistency of ratings. Users often do not rate the same item the same way if offered the chance to rate it again. Tastes change, moods change, memories fade. Researchers are trying different ways to incorporate such variables into their models; for example, some recommenders will ask users to rerate items when their original ratings seem out of sync with everything else the recommender knows about them. # 但是两个算法存在共同的问题就是一致性问题，同一个user对于同一个item在不同时间的ranting可能会变化，因为user的心情品味变化等原因。研究人员尝试多种方式将这种变化纳入模型，一些推荐系统有时候也会要求user重新对item进行rate。
- But the user-user and item-item algorithms have a bigger problem than consistency: They’re too rigid. That is, they can spot people who prefer the same item but then miss potential pairs who prefer very similar items. About a decade ago, researchers figured out a way to factor in such sets of similar items—a process called dimensionality reduction. This method is much more computationally intensive than the user-user and item-item algorithms, so its adoption has been slower. But as computers have gotten faster and cheaper, it has been gaining ground. # 比一致性更大的问题在于，没有办法发现一些潜在的可能感兴趣的item，原因就是以item作为最小计算单位。设计出的另外一种办法就是降维，就是将某一类的产品归为一类，然后计算类和类之间的联系。这种算法相对来说计算更加密集，所以在当时没有被采纳。但是之后随着计算能力的提高又开始逐渐被采用。
- It’s a pretty cool solution. But how do you find those taste dimensions? Not by asking a chef. Instead, these systems use a mathematical technique called singular value decomposition to compute the dimensions. Unlike the food example above, the dimensions that get computed are neither describable nor intuitive; they are pure abstract values, and try as you might, you’ll never identify one that represents, say, “salty.” And that’s okay, as long as those values ultimately yield accurate recommendations. # 降维使用一种奇异值分解(SVD)的算法来完成。实际上所归为的类可能在实际中没有任何意义，在算法里面仅仅是作为数字存在。
影响推荐因素
- Recommenders have two other features that dramatically affect the recommendations you see: First, beyond figuring out how similar you are to other shoppers, the recommender has to figure out what you actually like. Second, the system operates according to a set of business rules that help ensure its recommendations are both helpful to you and profitable for the retailer. # 另外两个因素也在影响推荐结果，一个是推荐系统除了计算你和其他顾客的相似程度，实际上还在分析你具体想购买什么东西(从其他角度比如用户行为) ，另外一个是使用一些商业策略（外部策略）来保证推荐内容对于你和零售商都有好处。
- Amazon’s art store assesses your preferences in a few ways. It asks you to rate particular artworks on a five-star scale, and it also notes which paintings you enlarge, which you look at multiple times, which you place on a wish list, and which you actually buy. It also tracks which paintings are on your screen at the time as well as others you look at during your session. The retailer uses the path you’ve traveled through its website—the pages you’ve viewed and items you’ve clicked on—to suggest complementary works, and it combines your purchase data with your ratings to build a profile of your long-term preferences. # 以amazon art store为例，除了会要求你评分之外，还会收集你的一些行为，针对这些行为进行更深入的分析。
- Companies like Amazon collect an immense amount of data like this about their customers. Nearly any action taken while you are logged in is stored for future use. Thanks to browser cookies, companies can even maintain records on anonymous shoppers, eventually linking the data to a customer profile when the anonymous shopper creates an account or signs in. This explosion of data collection is not unique to online vendors—Walmart is famous for its extensive mining of cash register receipt data. But an online shop is much better positioned to view and record not just your purchases but what items you considered, looked at, and rejected. Throughout much of the world, all of this activity is fair game; only in Europe do data privacy laws restrict such practices to a degree. # 行为收集可以通过cookie来完成。收集行为的方式并不是在线上才有的，比如沃尔玛也会收集一些顾客的购买记录数据，但是相对来说线上收集数据更加方便并且详细。
- Of course, regardless of the law, any customer will react badly if his or her data is used inappropriately. Amazon learned this lesson the hard way back in September 2000, when certain customers discovered they were being quoted higher prices because the website had identified them as regular customers, rather than as shoppers who had entered anonymously or from a comparison-shopping site. Amazon claimed this was just a random price test and the observed relationship to being a regular customer was coincidental, but it nevertheless stopped the practice. # 如果用户数据使用不当的话顾客也会非常反感的，amazon就有这样一个例子，有一个顾客发现对于某些商品展现价格相对与其他人更高，因为amazon后台发现他是一个常客，虽然amazon宣称这只是一个随即的price test，但是事实上他们从来没有停止过这种分析
- The business rules around these systems are designed to prevent recommenders from making foolish suggestions and also to help online retailers maximize sales without losing your trust. At their most basic level, these systems avoid what’s known as the supermarket paradox. For example, nearly everyone who walks into a supermarket likes bananas and will often buy some. So shouldn’t the recommender simply recommend bananas to every customer? The answer is no, because it wouldn’t help the customer, and it wouldn’t increase banana sales. So a smart supermarket recommender will always include a rule to explicitly exclude recommending bananas. # 避免展现一些大家常用购买的东西，这样的推荐没有任何意义。
- Most recommender rules are more subtle, of course. When John recently searched for an action movie on Netflix, for instance, he wasn’t offered The Avengers, because the blockbuster was not yet available for rental, and so the suggestion wouldn’t have profited Netflix. Instead it steered him to Iron Man 2, which was available for streaming. # 不会推荐一些还没有到货的东西
- Other business rules prevent recommenders from suggesting loss leaders—products that sell below cost to draw people into the site—or conversely encourage them to recommend products that are overstocked. # 展现一些廉价的商品以便引入流量，或者是推荐那些库存比较多的东西。
- This kind of thing quickly gets tricky, however. A system that simply pushes high-margin products isn’t going to earn the customers’ trust. It’s like going to a restaurant where the waiter steers you toward a particular fish dish. Is it really his favorite? Or did the chef urge the staff to push out the fish before its sell-by date? To build trust, the more sophisticated recommender systems strive for some degree of transparency by giving customers an idea of why a particular item was recommended and letting them correct their profiles if they don’t like the recommendations they’re getting. # 为了建立信任关系，一些比较复杂的推荐系统也会尝试对顾客透明一些，比如解释为什么给你推荐这个东西。
如何评价推荐系统
- Right now the biggest challenge for those of us who study recommender systems is to figure out how best to judge the new approaches and algorithms. It’s not as simple as benchmarking a microprocessor, because different recommenders have very different goals. # 评价推荐系统并不像做benchmark简单，因为不同的推荐系统有不同的目标。
- The easiest way to evaluate an algorithm is to look at the difference between its predictions and the actual ratings users give. For instance, if John gives the teen-romance novel Twilight one star, Amazon might note that it had predicted he would give it two stars, based on the ratings of other similar users, and so its recommender was off by a star. But sellers care much more about errors on highly rated items than errors on low-rated items, because the highly rated items are the ones users are more likely to buy; John is never going to purchase Twilight, so scoring this rating contributes little to understanding how well the recommender works. # 一种简单的方式就是对比推荐系统预测和用户实际的评分是否一致。但是对于seller来说他们更新对于high rate item的偏差，因为这些东西更加热门。(RMSE)
- Another common measure is the extent to which recommendations match actual purchases. This analysis can also be misleading, however, because it erroneously rewards the recommender for items users managed to find on their own—precisely the items they don’t need recommendations for! # 另外一种方式是对比推荐东西和用户实际购买。但是这个问题就在于它会鼓励推荐系统寻找那些用户自己找到的的东西，而这些东西实际上不需要推荐系统来推荐. (Top-K Precision)
- Given the shortcomings of these approaches, researchers have been working on new metrics that look not just at accuracy but also at other attributes, such as serendipity and diversity. # 同时引入了两个其他的评价标准(serendipity and diversity)
- Serendipity rewards unusual recommendations, particularly those that are valuable to one user but not as valuable to other similar users. An algorithm tuned to serendipity would note that the “White Album” appears to be a good recommendation for nearly everyone and would therefore look for a recommendation that’s less common—perhaps Joan Armatrading’s Love and Affection. This less-popular recommendation wouldn’t be as likely to hit its target, but when it did, it would be a much happier surprise to the user. # 人们总是希望在推荐里面看到一些不太common的东西，这样可以给他们带来惊喜。
- Looking at the diversity of a recommender’s suggestions is also revealing. For instance, a user who loves Dick Francis mysteries might nevertheless be disappointed to get a list of recommendations all written by Dick Francis. A truly diverse list of recommendations could include books by different authors and in different genres, as well as movies, games, and other products. # 另外人们也希望能够看到一些稍微存在差异性的东西