>> this event is 75 minutes long. >> our subject tonight is how google works, and we'll examine that question in more than just theory. google executive chairman eric schmidt and senior vice president jonathan rosenberg are here, and they'll be discussing

their new book by the same title. "how google works" is surprising and candid and an authentic inside look at the culture and practices behind one of the world's most successful companies. and what a moderator we have to lead that discussion, race saw meyer -- marissa mayer saw a lot of this from the inside, and she's now seeing it again from the ceo's office, and i can't wait for them to get started. please join me in welcoming eric, jonathan and marissa. [applause] >> i am so delighted to be here tonight. [laughter] >> we have so many acronyms for you, marissa. >> i have gotten a chance to

work with both of these gentlemen for a daled, and they are my -- for a decade, and they are my two longstanding mentors, and i really lucked out. [laughter] literally not a day goes by when i don't think about something each of them has said in a way that's shaped me or shaped what i do next. jonathan may be the most insightful person i've ever met about people. >> what about me? >> in addition -- [laughter] and eric may be the most insightful person on scaling of all forms; communication, business, technology -- >> can we say something about marissa? >> we've got about an hour to do that. >> we've got a lot of abuse for marissa. action items, design -- >> we'll get there. >> so tonight i'm very excited to get a glimpse into more of their understanding of people, of scaling, of how google works

which, obviously, there is just immense interest in. so about the book, how'd you decide to write it? what was the process like? what inspired you? >> you built half of this stuff. why are you asking the questions? [laughter] you know the answers -- >> because you wrote the book. >> you invented it. right? >> i used to come back from everything's staff -- eric's staff meetings, and i would get bored. so i would write down rules that people came up with when they said things that were insightful, like repetition doesn't spoil the prayer. when we got into an argument about something, i could get everybody to move along more expeditiously. and then i would come to my staff and explain all of the rules. so in 2009 our head of sales said we should pass all of this tribal wisdom on to googlers, so i wrote a speech -- which you helped me with -- notes for the bestseller i'll never publish. people liked it -- [laughter] >> jonathan, you're so wrong. one, it is a bestseller and,

two, we did publish it. >> you're right. >> marissa, why are you not a co-author? anyway. you left! that's the reason. [laughter] >> so, but it is a bestseller. so tell us about the reception. you're in week three of the book tour. what surprised you about what people have taken from the book, how it's been received? >> well, people are very hungry for new ideas, and as you know because, again, you deserve a lot of the credit. i think along the way over the decade google pioneered a whole new set of principles of managing things. and whether it's the way we recruited people or the way we made decisions which was rather bizarre, as you know, it worked. and it worked remarkably well. and, of course, the reason we wrote the book is because we all benefit, i think, by making american companies stronger, creating more innovators, you know, trying this new model. i was concerned that there's a

huge gap between the world that we all live in, right, and then the normal companies. and that there's this whole generation of people coming out for which the normal companies, the traditional hierarchical companies are just not designed. it doesn't make sense to them. and they're very smart, very global, very quick, and they walk into these companies with very, very turgid management systems. >> that made sense to me. it's funny, reading the book i felt like for me there's so many things in there that are just values i hold dear that i was like, wait, i can't -- i was almost going to say it's my husband, my family, if you want to understand me and my values, read the book. there's so much in there that twines the culture. -- defines the culture. one of the things that defines the culture is hiring great people. what do you look for when you hire? how important is hiring? >> hiring is everything that you do, and we had a set of attributes which you helped us create, role-related knowledge,

basic cognitive ability. but i think the main thing that was different was we really looked for passion, and we looked for generalists, not specialists. and i think the biggest thing that i learned in in the first year was that technology was changing so quickly that if you put a specialist into a role, they're going to look at that role based on their specialty. if you put a generalist and a learning animal into that role, they're going to roll with the punches and be able to deal with change. the other thing that was fundamentally different was the way we hired with committees, which i couldn't stand at the beginning, and you and others were largely responsible for. >> and you were ultimately in charge of this process. >> i was. i couldn't hire my friends, which was a real problem, because i wanted to hire more people like me, and you wouldn't let me. [laughter] >> actually, it's important you tell the story, right? that you actually complained about this. about the hiring. >> he complained about not being able to hire his friends. [laughter] >> as marissa, you should hire

people more like marissa. >> she did. [laughter] >> which we did. and it was the right strategy. do you remember this? >> it was -- [laughter] >> i remember it quite well. >> and she created this crazy rating system. >> go ahead. >> we rated people on a scale of 1-4 which seemed easy to me, 1 is bad and 4 is good. that's how i remembered it. then she explained to me that, no, 4 meant this person so good, dear committee, that if you have the audacity to turn them down, you need to call me out of whatever i'm doing so i can come into the room and fight to make sure we bring this person into my company. and 1 was the opposite. call me, this person so bad, i don't want this person many our company. and that was how we ended up rating people for committees. >> the idea was that you took convincer or convincee roles.

>> that is such a marissa -- [laughter] now you understand how it actually works. i try to follow her logic. along the way -- [laughter] her actual most, let's take control of this. her most important accomplishment of her many at google was that one day she decided that we actually to rethink the way product d d at the time there were three product managers who now run large parts of the company or a large other company -- separate discussion. [laughter] and so the question was how do we build that model or a scale? -- on a scale? so marissa had the idea that we should hire people that had technical backgrounds but did not want to be developers. she called these the apms. and they're very, very young, right? right out of college, green, never worked in a company, so eager to do everything. and so she built that. and if you look at that group,

almost all of the interesting start-ups in the valley are headed by those people, and the major product lines within google are headed by those people. so the model worked brilliantly. so jonathan had the idea -- sorry, jonathan -- that we should have work for these people. so he created something called the chain gang. [laughter] the idea was that you had these sort of work streams, and they would take these tasks. and this is how we developed them. remember? >> yeah, absolutely. the whole program started with a bet, and, actually, i can tell it because it's an embarrassing story for jonathan. [laughter] jonathan had been at google for four months. you want to grow product management in re show roughly to engineering, 1 to 10, 1 to 8. wayne hired eight engineers a week for all ought weeks. jonathan hired two people making our total fife at the -- five at the time the program started, and he gave one person to susan and one to sala, and i said when

do i get my person to help me with my group? and jonathan was like, when you make revenue. [laughter] and i said, because sala was doing advertisement, and susan was doing partnerships, and i was doing consumers, like google.com. and i was like, if there's no google.com, then there's no revenue. and jonathan just said, like, i don't know. i said, well, can i have a bet that i can hire and grow new product managers faster than you can hire experienced ones? and jonathan said, well, sure, but when are you going to do -- [inaudible conversations] >> telling all the stories, okay? >> but they're in the book, jonathan. >> this was the moment in my career when i was most upset with you. >> yeah. >> i'm in bill campbell's office, and i confess, i had been failing. i was trying to hire people like me, and i couldn't find people like me that larry and sergei would approve. [laughter] so i'm on the couch -- >> but they were --

[inaudible conversations] >> they're literally on the couch? >> i had a couch, yes. >> i remember this. he was on the couch. >> it's my story, it's my story -- [laughter] look, marissa, my love, okay? i can say that up here and in the appropriate, avupg lahr way a male colleague can love you. [laughter] but i almost strangled you that day. i'm on bill campbell's couch ready to quit before i'd even been there a year which would have ended tragically. and i've explained to him i can't hire. and marissa's outside, and it's 2002, so she's -- how old -- 27. when you give her math questions, always think through the answer before you ask her, and you'll beat her to the answer. [laughter] i've learned that too. and she comes in, and she says, well, what's the matter? i said, well, i'm failing to hire. and bill said maybe marissa has

an idea. and she's really bubbly and excited, so she starts talking about this program. and bill says, well, what do you think? and i said, well, i doubt it will work. and then marissa looks at me and she says, well, your problem, jonathan, is you're trying to hire people like you, and larry is and -- larry and sergei don't like people like me. [laughter] then she says, they like people like me! and that's when we had bill campbell in between us as an ex-football coach, i would have reached over and strangled her. [laughter] >> well, i have actually learned to like you. >> ladies and gentlemen, i think we've exhausted this thing. the important thing about the apm program is it served as the sort of mechanism for building the creativity of the company. and it worked brilliantly. >> well, thank you. i think one of the things you talk about in the book is smart create e, and -- creative, and

apm stands for associate product manager, and we hired people right out of school to do a two-year rotational program -- >> by the way, she would make them go on these trips. >> yeah. >> these apm trips, and a number of the people in the audience went on them with them, and they would basically stay three people to a room, that kind of a thing, and eat poor quality food and go from city to city -- >> it was like camp, summer camp. >> literally like high school. >> and it worked. >> yeah. the idea was to go and actually see our products in the wild, meet with advertisers and to all that. but so talk about smart creative. why are smart creatives so important? what do they bring to a culture? >> so smart creatives are people who are highly technical, they're also business savvy, and they're creative. and they're passionately curious, they're the kind of people who don't can take no for an answer. they're the kind of people who today can build a demo out of the tools that we have off the shelf that are easy to produce

things with, and they show you as opposed to tell you what you're going to do. so you don't need to actually create rigid product plans, you can actually create demos and prototypes and test them and then see where they go. >> and by the way, one of the things that we sort of figured out in the writing of this book is almost all of the traditional business discussion is around scarcity. how do you manage your marketing, how do you manage your channels and so forth. but the internet has upended all of that. now there's global source of funds for anything you have, allows you to distribute and brand yourself in pretty much any way. new brands are popping up all the time. think uber as an example. so you have to the build phenomenal products. so if you don't have an overall location, if you will, of these smart creatives, you're not going to be competitive in such a hypercompetitive world. which is why the whole recruiting tack, kind of people you hire and the focus on culture are so important going

forward. and i would argue further that this is true, obviously, for the tech industry because it's now been largely copied by every company, but it's actually true for non-tech companies as well. >> we talked on the interview scale before, but one of the claims in the book is interviewing is one of the most important skills for any business leader. talk about that. >> well, i would often in our staff meetings tell managers, you know, you are the sum of what your people produce. and i find it interesting that managers in most companies don't spend the amount of time with recruiting that they should. if you look at the coach of a sports team, they seem to get it, right? the players that they draft onto their team are really all that matters. and i think we lose sight of that, and we dell gate recruiting to -- delegate recruiting to recruiters. and as soon as you delegate, you go in a downward spiral. you want to make sure all of the individual employees are focused on recruiting.

larry used to tell us when you and i were upset that we couldn't meet the hiring quotas and hire as quickly as we could, that we could just double every quarter if every single person hired one person. husband math was correct. -- his math was correct. [laughter] >> as it always was. we did lots of interesting things that are nontraditional, but one of the more controversial ones was the relicense on what were called hiring committees. and these hiring committees would make the hiring decision, and the manager was often not on the committee. and often the subject did not know what job they were going to work on. so it was modeled very much after how successful universities hire, because hiring is critical in universities, and ultimately jonathan and david actually ran a whole process that marissa helped design. it produced this sort of ruthless efficiency around the quality of the people. and we were heavily criticized for these sort of hiring decisions. but again, if you look over the arc of a decade, it was one of

the most important things, if not the most important thing that we did. because it gave us a work force that was intellectually flexibility for the challenges the corporation faces. >> we also have the one thing that we learned from tim armstrong; you don't want to allow the invasion of the bozos. [laughter] and what was very clear is that arks could be taught to hire as -- as could be taught to hire as, but bs will hire cs because they're afraid of people that are better than they are. so you really create a positive effect if you manage to limit the company to just hiring as and then teach them how to interview, teach them how to hire, you avoid the invasion of the bozos. >> so the opposite of bozos are great people. how do you make sure the great people, the smart creatives stay successful and happy? >> well, many people who visit google think it's because of the lava lamps, the balls, the free food. >> i like that stuff!

[laughter] >> you know, the on-site doctors, the number of benefits. but there's a secret, and the secret is the best people want to work on what they want to work on, and they want to work on things that they care a lot about, and they want to be allowed to be successful. and if you can build a culture which empowers them to do that, you will retain them. if you micromanage them, if you don't give them the resources they need in this competitive world of labor, they'll go somewhere else. >> yeah. i think you, you can't tell the people how to think, but you can manage the environment in which they think. you can be transparent with information, you can run meetings in a way that allows everybody to participate, and you can do what we call in the book limit the hippos. hippo is what we call -- >> jonathan, please define hippo for our audience -- >> highest paid perp's opinion. person's opinion. >> hippo to.

>> hippo. >> the most dangerous animal in the savannah. >> they live actually in the water, eric. >> sorry. [laughter] >> but they do can kill more people in africa, and they kill good ideas in companies. and one of the rules we had -- which i only figured out late in my ten your in google because -- tenure in google. i only had three people working for me. it would have been so easy if it was just you and susan and sala. but instead i always had 12 or 15 people working for me, and larry would say you can't re-org that way, no. and one day i was in a meeting, and there was an engineering manager who wanted to do the same thing, and larry said, oh, we can't let him do that, he'll be a hippo. if you have too many people working for you, you can't possibly micromanage them. [laughter] what is the number in your case, jonathan? >> the number is seven.

>> okay. >> and if i had eight marissas working for me, i could concur the world. [laughter] -- conquer the world. >> you talk about being overworked in a good way, happily overworked. tell us a little bit about that. i think, there was always way more work than people. you know, how do you bring that into a culture and make that a positive thing? >> well, we've actually been criticized over this sort of general term because somehow it doesn't sound right. but let's explain what we mean. the kind of people who we want to have working in our company and you want to have working in your company are people who are going to do this because they love it. and they work more or less all the time as best they can when they're not doing things that they also enjoy like being with their family, raising their kids, you know, going to the movies or what have you. in other words, they're intense people. so when we say overworked in a good way, that's the context that we mean it in. if you believe that the work should be nine to five with a half an hour lunch break, then

you should work for the government, right? and by the way, a fine job. but that's not the kind of person that we're looking for. we're looking for somebody who wakes up in the morning and just can't wait to solve some new, interesting and hard problem whether it's in sales, whether it's in policy, whether it's in legal, whether it's in management, you know, so forth. people who delight in that. and it's so much easier to manage those people. and we learned this, larry and sergei and i went to visit warren buffett. and he had a huge empire, and he had 14 employees. this is like, you know, the highest ratio of leverage to individuals ever assembled. and he explained he only bought companies where the leadership would stay in place, and they were going to do it whether he bought them or not. so he never worried about it. he just called them and talked to them. and it worked. and at google the key aspect of this sort of competition over resources was that the people are just dying to drive harder, to make things happen because they care passionately about it.

it makes it much easier to decide where the priorities are and so forth. you don't have to sit there and worry that somebody's off the reservation. >> on a related note, you have this colorful metaphor in the book about knaves and divas. jonathan, you want to -- >> did you write thatsome. >> no, jonathan. >> did i write that? >> no, you did not. you've been taking credit for this. the credit goes to alan, our co-author. >> alan's here. [applause] >> so can you explain -- >> i explain what alan meant, is that what you're asking me? >> yes. >> yes. so typically we find the most successful teams are relatively small, small groups of three or four. and one of the things you find when you have small groups is you kind of get a dynamic of some built-in social controls. people help modify that behavior in a small group. so the worst thing that can

happen is you allow a knave into the midst of a good small team. so what's a knave? a knave is someone who lies, cheats, steals, takes credit for somebody else's work, spills coffee in the kitchen and doesn't pick it up. you want to help the teams exile these knaves and get rid of them. because if you get to a tipping point in knave density, nobody wants to come to work -- >> and this is another jonathan phrase. knave density? [laughter] >> knave dense city. >> i learned this from -- >> i learned about the knaves and the divas in the book. that was new -- >> you want to minimize the number of knaves -- >> minimize knaves and help the divas. what's a diva? a diva is a brilliant person who believes that everything should be done in an exceptional way and is often dictatorial, difficult to work with but has great vision and has a better notion of where the world is going and what they're doing with the product than anybody else. in these small environments, the

teams can drive out those people. so you need to exile the knaves and fight for the divas. >> and the run you need the divas, aside from entertainment value -- [laughter] is these are the people who drive the culture forward. these are the people who say that's not good enough. these are the people who drive the average person because not everyone's going to be a dee i have, as many as you try to find, push them all to fight harder. and, often, these are the people who ultimately become the most famous in our industry. you can argue, for example, steve jobs, a brilliant, brilliant product manager, diva-esque, right? very driven, very strong, right? look at the impact of people like that. we need them. don't drive them out. >> i talked in my opening about your thoughts on scale. i've never seen anyone manage scale as effectively, to take google from 200 people when you joined to today 50,000. and i think the most remarkable

things, i'll honestly tell some of the stories and more broadly how you scale culture, but there was something that happened when you arrived. 200 people, we had way too much work, and eric saw our annual plan, and we said, well, we're going to double under larry's doubling principle. we're going to go to 400 people this year. and erik said, no, no, no, no, no. we're going to have 50 people hired between now and the end of the year. so basically he cut it to the fourth of what we hoped. who's going to do all this work? every time you tried to hire someone, you had to turn in one of the 50 numbered cards with the founders' faces -- >> i made those at home with my laminator. >> they were very impressive. and they created this whole black market for the larrys and sergeis because you'd have like the sales guys would need something built, and they'd be like, i can give you a larry and

sergei. [laughter] talk about what made you do it. when i look back at the way we scaled, there were some points where it was really important for us not to grow that fast. and i think that was one of the most critical moments, because i think if we had tried to trouble the company that year, we would have probably messed everything up. it was that commitment to quality and pacing that was so important. >> every engineering team needs twice as many engineers as it needs now. i assume people in the audience are living this. normal. [laughter] and there's a rate at which an organization can hire quality. and you can eyeball it. in our case it was the calculation. i looked at the hiring rate, and i said we're going to hire 50 people. and, again, we can try to hire more, but realistically, we're going to try to hire 50 people, so let's do it with a quality measure. so the larry and sergei cards were a simple way of putting in a management technique which ultimately became quite rigorous with respect to finances. we were using quickbooks.

we had a five-user license -- [laughter] which was the maximum on a single pc, and it was overloaded, and it took too long to get the quickbooks report. and that is really true. >> so my position on the company was we needed to build a management team that could scale, and i'm a strong believer that you can systematically innovate. in other words, innovation is not something that you do randomly. it's a culture, it's a way of hiring, it's a way of thinking and so forth. so what would happen is we would sort of figure out our growth plan, and then i would ask larry and sergei how much bigger will these be? they would take the current growth rate and just extrapolate it. and one day they were roller blading, and they roller bladed over to the sgi campus, and they came back and said we're going to need that campus. and now today we're in the former campus. etc., etc. so if you take a look at the growth rate, you can actually plot it. one of the things we talk about

in the book is companies don't really have long-term plans. they have plans for the next year. and so, ultimately, you need a five-year plan. so, ultimately, what happened was you and a set of other people said these are the growth rates, this is the hiring plan, and off we go. >> i think it was really interesting the way we were able to scale the culture. because around the same time as jonathan during 2002, there was this notion -- and i remember you put a fine point on it where you said we're a one-building company and a one-building culture, and it's not going to work if we aren't spread across multiple buildings. and i remember that really scaring me. we were down to 80 square feet per person, i was like, hang on, how are we going to get our culture to scale? what were some of the insights you guys development in terms of how to scale the culture? >> i think at the outset google had a mission that was very clear, and it created a selection bias early on so that we got people that really believe inside that same --

believe inside -- believed in that same mission. most companies kind of do the opposite. they get started, and then they have a little bit of success, and so then they can hire in some health care r person or pr person, and the person comes in and says, well, i'll write the culture out. and this person wouldn't know the culture if it jumped up and bit him. so they'll produce something like, you know, to build unrivaled partnerships with our customers through the dedication, hard work, perseverance of our -- [laughter] wonderful employees to generate shareholder value. >> so that was, let's see -- >> which, by the way, i was in that meeting. [laughter] it's what gave birth to don't be evil. >> the one you just quoted was from enron, right? [laughter] >> enron -- enron was respect, integrity, communication and excellence. [laughter] this was lehman brothers.

by the way -- [laughter] the don't be evil was interesting, and i don't know if you were in the meeting -- >> i was. >> when i first encountered it. somebody explained to me we were doing don't be evil, it probably was you, and i thought this was some stupid marketing phrase because i'd never seen marketing phrases inside a company be effective. so we were sitting in the old conference room in 2400 and as a team, and my guess is you were there. and there was a conversation about search and ads, and there was a product that was across the search and ads boundary. and halfway through the meeting one of the engineers, his name was ron, took his fist and went, what'll, that's evil. and the whole -- wmam. and so after a lengthy hour of debate, it was agreed not to move forward. all of a sudden i understand that, one, this company was

serious about this, and it gave the engineer a moral question to ask, in this case a guy, to guide his actions. >> yeah. i mean, there's so many amazing don't be evil stories. my favorite one was the hrs culture meeting when we first put it out there. they pulled a bunch of old-timers together. don't be evil, and i think it was, like, number six. and someone said, well, you can't have a negative. corporate value. can you reframe it positively? we were like, um, you could -- >> only non-evil. >> do good? it's not as catchy as don't be evil. >> we had other ones like you could have fun without wearing a suit. [laughter] >> yes. >> and the first one was focus on -- >> the user. >> focus on the user. >> and i think one of the things that helped, and we talk a lot about this in the book, is when you have a framing principle like that, it helps you a lot.

when you look at the challenges a corporation faces today, we ultimately retreat -- i think successfully -- if we can get to the end user, we can get through everything. >> i think one of the things i learned tactically from both of you in different ways was around meetings. i remember jonathan arriving and organizing all kinds of meetings. it was sort of magical because like it was funny, i think larry and sergei one night decide, you know, jonathan's arrived, and he takes notes. [laughter] and sends them out after the meeting, and then we all know what was said in the meeting. it's like magical. and then it became this whole culture of organizing a meeting, having an agenda, sending out notes afterward, making sure you have to -- >> which actually forced two projectors in the room; one to present the presentation and the notes so you could see what the notetame taker was saying --

notetaker was saying. >> i had actually been a master in my previous job, sitting in the room and writing everything down and then defining the action items after everybody else left so that you could say what they had said they agreed to. [laughter] and so when i put the second projectors in the room, i decided that i shouldn't be allowed -- i wasn't going to allow all the project managers who worked for me to do that, earn actually going to follow eric's principle and leave the meeting with buy-in about what they were supposed to be doing. >> and i remember, eric, i think a lot of people think about meetings as something that takes time out of your day. and you, i remember, would always say this phrase i work through meetings, that's how i get things done. >> managers spend their time in meetings, and having, figuring out what the rules are about your meetings turns out to be very important. when i was at novell, we had the novell nod, and everyone would agree in the meeting, and after the meeting was over, they'd do whatever they wanted. [laughter] it was highly effective, as you can see.

so when i showed up at google, we had this new idea that we should have a different model. we should try to figure out the best outcome, the best idea in the following process. we would sit around, and then somebody would make a proposal x then we would try to have a debate. and we'd try to have every person in the room have an opinion. now, there's always two or three people, usually guys, that talk a lot, and they'll dominate the conversation. so i learned to call on people who don't normally say and try to get a spark of opposition. all of a sudden you have a debate. now, you need to have a time limit. so i found that my role was to stimulate the debate and have a time limit. be you don't have a time limit, you have a university. they just argue all the time, right? but if you have a debate and a decision, then everybody can move forward, and everyone participated in the decision. and in the book we actually go through the china decision as an

example not of the china issues which have been well covered, but rather, that we followed our process thoroughly and fully in that particular and very important decision or for the company. >> jonathan, how do you feel meetings shaped the culture at google, and how did we do them differently -- >> well, you were in my weekly staff meeting. >> i was. i learned all kinds of things. >> so the, our staff meeting we actually had one of the apm assigned to do chain gang tasks even week, and -- each week, and his job or her job was to assemble all of the information so instead of everybody coming into the meeting and getting updated, we actually had all of the information go out the day before, and then we would come in, and after everybody got really good at sort of reciting their information, i would insist that we go around the table and force everyone to articulate what is it that they need from their peers, the other people in the room, to be more efficient? and that was how we started.

that was how we actually ran my staff meetings which eventually you got very good at. you would come in with your list ahead of time. >> and one of the principles, and we talk about it in this book, is why don't you start your management meetings with actual data? what a shock. so marissa would show up, and she drove this on her own, and she would have a complete study -- because she was in charge of the users, most important thing in the company -- and showing where the users were, what the trends were on each of our properties, and the moment she put that slide up, we would have an argument. is this right, where's the priorities. but you start with the data. >> i mean, i learned that -- you start with the dashboard data. >> the theorum, right, if we have data, let's look at data. if all we have are opinions, we will go with mine. [laughter] that's what we call a hippo. [laughter] >> um, so we talk a lot about people and culture, but obviously one of the most

important things companies build are products, products and services. and you talk in the book about how important it is that, basically, every great product has a key technical insight. why is that so important? what -- when you, how do you define technical insight? >> so a technical insight is the kind of thing that could end up in a journal article. it's the kind of thing that typically a researcher, a scientist is going to come up with. if you think about google and you make a list of all of our great products, a technical insight behind them. why is search so great? page length. why are prices so great? the notion of quality of the ads. why is chrome so great? it's secure and fast. why is maps so great? i think you said that one, because it's a virtual mirror of the world, right in and if you look at other products, you look at, you know, salesforce.com, they realize that you could use the cloud for contact and

relationship management or pixar, you can tell stories with computers. the great products when technology is changing very quickly leverage some insight about how technology works to usurp an existing set of products that make the product better by a factor of ten. >> and a current example for the audience has to deal with cloud computing. the fact of the matter is if these smartphones are where all the apps are and that's where all the excitement is, but they're well enough connected now that you can be connected to a server bank that can do incredibly powerful calculations, speech recognition, those things off phone and send the answer back fast enough the user thinks that the phone did it. but, in fact, there's a supercomputer literally a million computers that helped answer this question. that's another example of a technical insight. >> looking a little bit into the future, what are some of the technical insights that aren't being capitalized on today, some of the trends that you see that

are going to define products, services, five years out and what are the technical insights and trends that are appearing today that can drive that? >> the one that is the most profound is the development of a new category of artificial intelligence. and you say i don't really use a.i., you do it every day. and you do it in the kinds of search rankings that google does, in any form of voice recognition. all of those use incredibly sophisticated algorithms that, again, are often server-based to do this. we're very close to being able to do highly accurate speech recognition, image recognition and text recognition. and it means that we'll begin to start learning really profound things. you say what is that useful for? imagine an assistant that helps you figure out, you know, do you have to get up now, can you stay asleep? you want to sort of, is your plane going to be late? do i need to go to the dentist, did i forget? all the things that are annoyances to the average

person's life that a computer can help you with. so where are we with that? two or years ago google built a neuronetwork which is a form of this a.i. with no programming in it whatsoever. and not thinking what else to do, we have the neural network start to watch youtube. and it watched 11,000 hours of youtube. what did it discover? two things. it discovered the human face; eyes, nose and mouth. and it discovered cats. [laughter] now, and i mean the whole concept of a cat, you know, the tail, the running around, the furriness, the purring, all that kind of stuff. now, i wish i could report to you that this seminal moment in the history of humanity we had discovered aristotle or plato or something like that. but that's what we discovered. and there's a lot of evidence that that will now establish a very broad set of new applications and not just in what we do, but also in medicine

and so forth. >> i think probably the biggest and most exciting ones are in health care. you know, there's certainly lots of opportunities in terms of genetic research and data and designer drugs. but i think the other real exciting thing is just this notion of wearable devices and longitudinally being able to track information about physiological information that you get about people. and, you know, today we're just sort of seeing these nascent, you know, fitbit-type devices and staying fit types of things, but that's all going to move to really measuring things about you that matter, right? you could give information for an at-risk teenager from their cell phone to a psychologist, and it's going to tell you a lot more about the state of that kid's depression or anxiety than when you ask a teenager how they're doing and they sort of give you some monosill big type of answer -- monosill big type of answer.

and there are all sorts of other potential opportunities with anything that you can track longitudinally over large numbers of people. >> i'm going to open it up to the audience questions if just a -- in just a minute, but i have a few more. the buzz word of the day is millennials. and one of the things to think of is the products you just talked about are going to be used in the future by millennials. how do you think their expectations about products, technology are going the shape things? how are they going to interact in a way that's different than the way we would? and i'm also going to ask the question about how they will be in the work force and how you think google works -- >> well, my observations of millennials consist of the following. what do millennials do can? text, text, text, text, picture of themselves, text, text, text, picture of their friends, text, text, text, picture of the food they're eating. [laughter] occasionally they use important and powerful apps, and they change the world. so what do you learn from this?

they learn they're highly, highly interactive. they're highly verbal in a text way, right? they're communicating all the time. and they're very much savvier than i was at that age. the amount of information sources whether they be traditional sources or fun sources or social, they're on supercharge all the time. and my guess is that this will be true of teenagers and younger people now because, frankly, be you have a child -- if you have a child, your child is online all the time. and if they wake up at night, they're online when they wake up, they go back off, and then they go back to sleep. >> jonathan, thoughts? >> clearly, eric is more insightful about the state of millennials than i could ever hope to be. i would ask the apms who you hired for me as opposed to go in and tell them how to develop products for these millennials. i think we're certainly seeing a big transition to apps on mobile devices. i have not yet figured out what

their preferred form of communication is. i just wrote a book with eric, or alan did, and we wrote our rules of e-mail. and near as i can tell, they don't answer my e-mail. [laughter] i have yet to figure out what the best way to communicate -- >> but that is sort of their parent's activity, and i think there's clearly a much -- a new, hyperactive form of commune caution that will emerge out of this because these are are communication-centric people. how does nude spread? it's link to link to link. this is an interesting story, take a look at this and so forth. the majority of youtube i viewing, by way, is from people watching links that other people send to them and that they end counter on facebook -- encounter on facebook and yahoo! and so forth. >> i think it's one of the things that's causing there to be a lot more transparency, government transparency and products and services, how things are built.

one of the great things about google through all the years was this real commitment to all hands. right? having weekly all hands no matter how big the company got, this access to information. before i moved to the audience questions, this is my last question, but talk about the effect that has on cultures. is out important? is it not important? what role does it play? >> yeah. so i think it's hugely important. and atlantic city really modeled that from the beginning. it went to all aspects of pretty much all information. eric would get up quarterly, and we would produce the board slides which lots of you would help me with, and he would literally present the same slides to all of the employees that were presented to the entire board. so it was very clear to all of these smart creatives that they understood not only the mission, but they understood the metrics by which we were running the company, the metrics by which we were measuring the company. they had access to all this

information. they also had access to all of this information on the company's internet. and i think if you have the expectation that the ideas are going to p late up from the smart -- percolate up from the smart creatives, you have to be completely transparent about how you're sharing information with them. >> cool. so in the spirit, one of the things that always happens at google is there's a big q&a session at the end. we'll try and go through rapid fire. the first question is, what opportunities are there for google in the sharing economy? >> well, we're a big investor in uber to tart with. [laughter] >> i think a lot of it depends on where our future products end up. it's one of the sort of discussions that people have had about the self-driving cars, is that perhaps in the future self-driving cars themselves will be shared, and they'll literally show up, you'll get in it, you'll be in an uber-like model, get out and it'll go pick

up someone else. so we have the opportunity with these new technologies to really rethink the ownership model. one of the more interesting statistics about cities is that 9% of the urban landscape is devoted to parking. so imagine if you could get that back somehow, the impact in terms of ability to build buildings, getting people closer together, the strength -- >> i think our role is really probably in the middle. because what google does is help reduce information costs and, with payments, if we can also help people reduce transactions costs, then any asset in the sharing economy that at any moment is going to go fallow, we can get the synapse firing between the person who can sell that asset and the person who might want to buy that asset. and that can apply to -- [inaudible] tube assets or any other, it could be your car sitting idle, it could be your home, it could be the dress that you wore this week that you're not going to wear again next week. >> in fact, one of the most

important things that google and other companies are trying to do is make the entire business transaction friction-free. literally, the moment you want it, we get it to you in boom. that velocity, increasing the velocity of that transaction benefits the entire ecosystem of payers, obviously, the ultimate manufacturer and the consumer. >> very cool. the next question is, are there any aspects of google's culture when you look back you would do differently the next time around? >> well, for me i think all of the errors that we've made, the core error was in taking too long to do something or allowing something to go on for too long. you'll will be wave, right? a team of ours brought out wave. sounds like a great idea to me, i more or less understood it, i thought it was really neat. we launch it, every month the traffic goes down and down and down. but i was sort of attached to the product, and we kept trying

things, and traffic went down and down and down. and during that time while it was going down, the real opportunity cost was taking that team and not putting them on something which was more likely to go straight up. so businesses are not -- the highest cost, and sergei used to say this, the highest cost of our business -- which i thought was taxes, by the way, mathematically, is opportunity cost. >> cool. um, going back to smart creatives, how do you tell the smart creatives and from the crazy ones? [laughter] >> well, i tend to, i tend to interview for passion. so i actually get them off talking about the things that they've rehearsed and get them on to talking about the things that they're passionate about. and i think that's when you kind of see people let their guard down. and the farther they go in the interview and the more they tell you, the crazier they are, the more likely they're to be a

diva-like personality and the better. i would rather err on the side of bringing in more of the people that are actually crazy than having to figure out how to manage them than r on the side of -- err on the side of not allowing some of those crazy people in the door. >> very cool. the next question goes to, it's around hp. i have a son who's 2. we live in the neighborhood of the hp garage, and one of my great moments of pride was when i taught my son to recognize it. i'll say, what's that? he says hp garage. [laughter] this questioner asked how much would you ascribe cultural success of google to the example set by bill and dave? >> wow. so i worked at hp briefly for a summer. i actually read the hp way. it surprises me that most people go into companies, and they don't read these documents. i think there really was a time at hp where management was by walking around. and i think that senior managers

at google once they kind of come in and figure out that they haven't been brought in to be dictators, they've been brought in to be aggregators of viewpoints, and they've been brought in to help get synapses firing across teams, then they're finally able to be successful. and i really think that is the kind of management style that bill and dave espoused. >> a googler be highly successful and have time and energy left over for a family? [laughter] >> yes. we have a number of googlers here on the stage who manage to do that. >> and we also have a lot of our family members -- >> a lot of our family members are here. i think the most important thing to say is it's easier if i sort of net it out. think about diversity, and think about trying to get the very best people. and think about trying to have professional women who are, of course, fantastically talented people. you've got to solve the childcare problem.

so you've got to solve it in a way that works for them. so after a lengthy conversation, we ultimately decided to purchase three closed schools, and we created three slightly different daycare centers. and, again, the employees had a co-pay and so forth. that's an example of the kind of stuff you have to do. you have to solve the problem on their terms. just using that as a specific example, but one that everyone can understand, you've got to find a way. we talk in the book about how sort of, you know, middle-aged white men we would notice that the women we worked with would disappear for a few hours, presumably taking care of the kids or whatever they were up to, and they would come back late at night working very, very, very hard late into the night. obviously, by their choice. and you've got to find a way to let them get both done. so we learned don't encroach on that time. you're encroaching on their family time which is just as important not more important than time with eric and jonathan. >> um, this question says i

think i -- [inaudible] in silicon valley are crazy again at this point. do you think they're overvalued? and if i'm a ceo of a start-up, should i keep hiring aggressively now, or should i secure funding and spend it more wisely, fully and carefully over time? [laughter] >> well -- >> true question. word for word. [laughter] >> you always want to spend money wisely, but i think the questioner's really asking so are we in kind of a tech bubble? i actually think the answer is, no. i used to circulate regularly mary meekers' slide. she was at morgan stanley before. and if you go back and look at the bubble in 2000 and the nasdaq's still lower -- >> the nasdaq is not quite at the bubble level of 2000. bring it back, by the way. those were great times. [laughter] >> go to those slides and just as a proxy for the value of the set of these companies, look at e-commerce. the amount of e-commerce has

grown enormously, and we are moving to this model, which eric mentioned earlier, of eliminating friction. be you eliminate -- if you eliminate friction between people, it's seamless to create a transaction? the communations you're going to get through the roof exponentially. and i think the other thing you need to add into that equation is cell phones are getting much cheaper, we've got 2.5 billion new people in the next five years from all corners of the globe that are going to join the online medium, and they're going to be able to access information on the internet, access health care information and engage in commerce. so the number of opportunities that we have are huge, and e-commerce already far greater than it was at the beginning of the -- >> the reason it feels like a bubble is because what happened was the rounds in the financing hierarchy of the venture all of a sudden went way up, and they went up because these companies had a product, and you could see a path to modernization over a billion people.

so all of a sudden if you have a successful product, you have a much greater likelihood of success. so to answer the second part of the question, it depends on where you are in the sort of company stage. if you're still building your technical team, you want to be careful to hire very carefully, very high quality people, work very, very hard, classic start hum model. if -- start-up model. if you have revenue, right, you want to be careful to understand what the revenue growth rate is. and so depending on whether you have revenue or not, you get a different answer. >> next question is, what is a chain gang? i guess we neglected to define that well. >> well, the chain gang was actually a simple list that we could create, and any manager could define a task. larry actually once defined a task which was for a chain gang member to help him figure out by project where resources and the engineers of the companies were being allocated so we could actually figure out what all of

the people were doing. so it was basically, you know, jonathan's fun euphemism. it's the guy who was popping balloons and torturing the apms who came in. and it was just a list of random tasks that management needed some kid right out of school to do. and to be fair to marissa, what she told them was -- she recruited them, and she said you're going to get a really big and grand project straight out of school. and she gave brian rakowski gmail. and i gave him all these little chain gang tasks to keep him busy. >> what happened was jonathan was like i want to give them all these little tasks, and i said, okay, you know, look, you're managing team now. brian also went on to be the product manager for chrome, which people don't realize the same person did. a tremendously talented guy. he got this list of tasks, and i said, okay, so you're going to do jonathan's tasks, and he said

i'm being end slaved -- enslaveed. so he created this little wiki page, and on the top of it, jonathan's chain gang. and that's where it came from. [laughter] >> imagine this situation where you have these incredibly smart people that you can apply the questions about your business. we ultimately figured out what we needed were some physicist ises. and i put them in the office next to mine, and susan talked a lot because her father was a physicist, and she would talk about the physics of our revenue. and we would actually go and analyze these and say how can we make it better, how can we make it stronger, how does the company really work if and we would eyes these as intrnl checks and balances. internal checks and balances. even asking the question would cause an interesting outcome. >> you called it the physics of clicks. >> the physics of clicks. >> this person is very -- [inaudible] how does the cost fit into

google's user priority? >> did you acquire professor? >> [inaudible] >> the it's a fantastic product. and for those of you who don't know what it is, it's a fantastic photo manager. and be most of the usage of by casa has been shifted over to the cloud-based photo services. i happen to still be a huge picasa user. >> by the way, my quick story about our photo service. remember when i came back from africa, and i had all these beautiful pictures, and i made you guys look at all of my pictures? it was 2007. i then went to eric's staff meeting, and i tried to show them the pictures. and larry pooh-poohed my pictures and said, well, you know, you missed all sorts of stuff. and he said in seven, eight, nine, ten years you you won't need to take pictures like this. you'll put a camera on your head, and when you come back, software will figure out all of the best pictures for you.

and like he's almost right, right? today you can put a camera on your head, and you can come back, and google stories can practically say this is best stuff that -- >> this is an example of a recent innovation where an algorithm within google will actually pick what it believes are the best pictures of the ones you've given it in the photo stream. that's a.i.. it's also very useful. >> very cool. the next question is would the management principles that worked at bag google -- culture, hiring, decision making -- work for other start-ups and other cultures? >> i would argue the principles we talk about in the book would work very well at any tech start-up anywhere in the world. and i say this with conviction because many of them have copied all of this. many of the people came from our culture, etc. to me, the question is what happens for existing companies? ..

two pages which i actually wrote, strategy pages. they are whichever pages, but anyway -- and the largest company that i deal with, a they have -- they have not figured out what to do when their model is disrupted. what is the narrative? all business, new business, old media, new media, old

companies, new companies, change is afoot. if you believe, as i do, the growth of america, the seed of america are the entrepreneurs and the right answer is over five years each and every company should organize itself around the destruction they will face from new interest, and almost no company can answer that question in a coherent way. some examples, the obvious one, uber verses the taxi industry. what about tesla? and example after example. >> i remember, a 5-year plan, i would have missed every great thing that ever happened. sometimes you just cannot see far enough ahead. >> my point is, you have to have a vision. let's have a conversation about our 5-year plan, everyone in this room.

let's start with smart phones. we will all have them, they will all be faster, you will still sleep with him next to your bed, have really powerful applications that you will use extensively. the back end? huge new services of one kind or another, transactions will be -- making those assumptions, and then you can begin to say how i will do, what will happen from my product, how will make my revenue and transition to that new model. >> do you have thoughts? >> i think that i was sort of shorten what eric said into the car culture is hard to change if you are a start up. with cultured my suggestion would be start over. everything beyond that, strategy in a plan and fluid way where strategy is the foundation, not the naturally a documented, dictated plan applies to all

of these companies. i think all of the principles and decisionmaking about how to run meetings, structure decision around data, everything in communication, repetition, principals and innovation, focusing on the user and thinking big, all of those principles apply to any company in any industry. >> of a tumor in -- consumer executives make good decisions and rapidly changing circumstances given that they don't have all the information they need to make decisions? >> first thing is, you do make mistakes and it is important right now is that little and big companies make this -- make mistakes. the part of it is, you want to make a culture where you fail quickly, admit your failure, and move forward. we have made plenty along the way, but somehow we got enough right that it produced a good success. i think that sort of his principal number one. number two is that with the new aspects of technology that we can all use, you can

extend the value and reach of an executive far and wide read and answer your e-mail. i am a big fan of holding executives accountable for what they do. you hear, oh, there is all this bureaucracy. fix it. our products out. fix it. there is someone or a team in charge, unlike other things, like the government. i disagree with the premise there is less information available to management today, particularly in web- based. all of the decisions we made in the early years related to products were done on the basis of testing and data and statistics that told us what was going on. the same is also true in terms of inventory in existing business.

>> so in 1987, and i was working at sign and i was young and did not know very much. and marrisa mayer was 12. [laughter] >> we had a crisis, a transition in computer manufacturing, and we nearly ran out of cash, and it involves inventory planning and so forth. the ceo at the time was justifiably worried, and so he ordered all of us, the entire management staff, four out of every five days per week in a factory working on this. how many machines that remanufactured and how many did we have in place? so if you want to understand the difference between there and today, when i first joined google i was shocked we moved revenue to second. today we know revenue to the

nanosecond. that is a huge change every 25 years. >> a sense of humor with the next question which i remember one of the things eric asked me along -- eric asked me to do along the way , he was giving a keynote . he turned to me and said, you are funny. why don't you create a video and so i was usually complemented. hired writers and put together a sketch and video, and i got it back. and it was awesome. it was so awful i was in tears on the phone with eric's ad man talking to how bad it wasn't how i could tell eric how bad it was. i went, it is bad, not usable. cannot possibly -- i cannot use it. [laughter] that was really grateful.

had a sense of humor in that moment. >> their is a simple rule, at the end of the day people can leave your office smiling just a little bit you will get more work out of them. so it is just human nature that people want to be thanked a little bit, respected a little bit, and off they go. i will tell you a story, all over in building 41 at the time, and there was a review of a proposal that an engineer had to go wire of san francisco, and we did it in a different way. this gentleman was very proud of this proposal, a new executive who came in and started asking questions. and this is someone who did not know the answer to any single question, and you could watch, very proud, you

could watch the movie go from ecstatic to be presenting to this augusta board to being an about to kill. they had actually literally been distraught. so the question is, what do you do at that point? and the answer is, you felt to the engineer and saved coming your idea was brilliant, and i want you to spend some time and i want you to come back and want to tell you how you will address it. and if you don't do that you are not going to get any productivity. came back with a product that helps make the internet happen. what we really do that google for fun, as you know, is we make fun of the most senior people in management. [laughter] so eric, i know you probably

like everyone to believe that you learned most of your management techniques for michael porter and peter drucker and tom cheers. you just mentioned that when you leave my office of want to touch your heart, i want you to smile. who is the brilliant management writer and personality who you learn that from? >> oprah. [laughter] >> it's true. oprah is a very gifted woman . if you want to change human behavior you have to change people's hearts, not their heads. it is true. we call that in the book the oprah's principle, but it is true. >> to that note, you talk in the book about being in

googly. what does that mean? >> fine, witty, laugh at my jokes. [laughter] >> i think it is like marrisa mayer. the ultimate smart creative and founder of googly. [applause] >> our time here has gone way too fast, but i have one more question, his dream has been to ask erica question, so here is our panel questioned. >> come on up. >> come in all the way. >> he just jumped. here. [inaudible conversations] [laughter] >> give him your mike.

>> don't kill him. [laughter] >> so what is your name? >> of the boil. >> okay. >> i wanted to know how google is trying to deliver internet coverage to people in remote areas. i heard about all of these projects, and i wanted to know how they are delivering internet coverage. >> well. [applause] all right. another question. >> and i also want to know about google in china. is google willing to accept internet? [laughter] [applause] >> okay. stay right here. >> you can debate right now. you can invoke debate. >> the hardest questions. so, in the first place you

understand how balloons work. we invented a balloon that would sit in the stratosphere and flowed around. give him the mike. posted on him. so what happened was, we figured out a way to make a transponder on the balloon that could talk to existing smart phones. this had never been done before. the figure that it would never work. not only did it work, but it works miraculously. the people you are talking about take their normal fun and will be able to talk to lead to early balloons in the sky, 60,000 feet all around the southern enormity of northern hemisphere it is the neatest thing. and your question about china, ask it again. >> is google willing to accept internet to enter china? >> actually, no.

[laughter] >> china has the most people in the world. shouldn't -- shouldn't most companies be willing to accept almost any terms to get that market open? [laughter] [applause] >> so, we debated this a long time and talked about this in the book, which you have clearly red. [laughter] >> i have not read it. >> you have not read it? >> i have not. >> okay. well, here is my copy. [laughter] [applause] anyway, what happened in china was, we decided the level of the was counter to the principles of google and decided that we would not operate in china as long as that was part of the plan. an historic decision and a decision that most other companies would not have made. [laughter] >> go right ahead.

[laughter] >> thank you very much. [applause] >> thank you. thank you. [applause] >> ladies and gentlemen, that is "how google works," a real-life example. such an honor. >> when are you applying to work for us? [inaudible question] >> he wants to run for u.s. secretary of state. you heard it here first. good for him. we are support -- we are a supporter of u.s. a politician. [laughter] and okay. thank you very much. thank you all. [applause] thank you. thank you. [applause] >> thanks so much. >> you, to. >> thank you. thanks. >> by, everyone. thank you so much for coming. i hope to see y