While each user has 130 friends on average, they have approximately 48000 friends-of-friends5. Even assuming only 4 bytes per posting list entry, this would still take over 178TB to store the posting lists for 1 billion users, which would require hundreds of machines.
Catalog
Facebook is an American for-profit corporation and an online social media and social networking service based in Menlo Park, California. The Facebook website was launched on February 4, 2004, by Mark Zuckerberg, along with fellow Harvard College students and roommates, Eduardo Saverin, Andrew McCollum, Dustin Moskovitz, and Chris Hughes. The founders had initially limited the website's membership to Harvard students; however, later they expanded it to higher education institutions in the Boston area, the Ivy League schools, and Stanford University. Facebook gradually added support for students at various other universities, and eventually to high school students as well. Since 2006, anyone age 13 and older has been allowed to become a registered user of Facebook, though variations exist in the minimum age requirement, depending on applicable local laws. The Facebook name comes from the face book directories often given to United States university students. Facebook may be accessed by a large range of desktops, laptops, tablet computers, and smartphones over the Internet and mobile networks. After registering to use the site, users can create a user profile indicating their name, occupation, schools attended and so on. Users can add other users as "friends", exchange messages, post status updates and digital photos, share digital videos and links, use various software applications ("apps"), and receive notifications when others update their profiles or make posts. Additionally, users may join common-interest user groups organized by workplace, school, hobbies or other topics, and categorize their friends into lists such as "People From Work" or "Close Friends
Jon is also a friend-of-friends of himself since he is—by definition—a friend of his friends.
For storing per-entity metadata, Unicorn provides a forward index, which is simply a map of id to a blob that contains metadata for the id. The forward index for an index shard only contains entries for the ids that reside on that shard.
Unicorn also supports a Difference operator, which re-turns results from the first operand that are not present in the second operand. Continuing with the example above, we could find female friends of Jon Jones who are not friends of Lea Lin by using (difference (and friend:5 gender:1) friend:6).…
Although there are many billions of nodes in the social graph, it is quite sparse: a typical node will have less than one thousand edges connecting it to other nodes. The average user has approximately 130 friends. The most popular pages and applications have tens of millions of edges, but these…
Unicorn is designed to answer billions of queries per day at latencies in the hundreds of milliseconds, and it serves as an infrastructural building block for Facebook’s Graph Search product.