Daniel Tunkelang - Endeca, Search and Choosing to Freelance (#29)

Welcome to another episode of Develomentor. Today's guest is Daniel Tunkelang. 

Daniel Tunkelang is currently an independent consultant or, in his words, a 'high-class consultant' for technology companies. Previous to this, he was a data science and engineering executive who has built and led some of the strongest teams in the software industry.

Daniel studied computer science and math at MIT and has a PhD in computer science from CMU. He was a founding employee and chief scientist of Endeca, a search pioneer that Oracle acquired for $1.1B. He led a local search team at Google. Prior to this, he was a director of data science and engineering at LinkedIn, and he established their query understanding team.

Daniel is a widely recognized writer and speaker. He is frequently invited to speak at academic and industry conferences, particularly in the areas of information retrieval, web science, and data science. He has written the definitive textbook on faceted search (now a standard for ecommerce sites), established an annual symposium on human-computer interaction and information retrieval, and authored 24 US patents. His social media posts have attracted over a million page views.

Daniel also advises and consults for companies that can benefit strategically from his expertise. His clients range from early-stage startups to "unicorn" technology companies like Etsy and Flipkart. He helps companies make decisions around algorithms, technology, product strategy, hiring, and organizational structure.

Click Here –> For more information about tech careers

Episode Summary

"Well it started with pretty much the people that reached out to me were trying to persuade me to take full-time jobs and I'd say, Hey, I have a better deal for you. You could just have me one day a week"

—Daniel Tunkelang

In this episode we’ll cover:

1. How Endeca got started? Why did the founders reach out to Daniel?
2. The pros and cons of being an independent contractor versus a full-time employee 
3. What made Daniel interested in search?
4. Why Daniel became lost while working at Google

You can find more resources and a full transcript in the show notes

To learn more about our podcast go to https://develomentor.com/

To listen to previous episodes go to https://develomentor.com/blog/

Follow Daniel Tunkelang
Twitter: @dtunkelang
LinkedIn: linkedin.com/in/dtunkelang/

Follow Develomentor:
Twitter: @develomentor

Follow Grant Ingersoll
Twitter: @gsingers
LinkedIn: linkedin.com/in/grantingersoll

Transcript

Intro: 0:19

[inaudible]

Grant Ingersoll: 0:19

welcome everyone to the development or podcast, your source for interviews and content on careers and technology. I'm your host grant Ingersoll. For those new to the show, we have two simple goals. We want to showcase interesting people in tech across a variety of roles and we want to highlight the different paths people take in their careers in tech. To listen to more episodes. Please check out our website[inaudible] dot com or subscribe to this podcast using your favorite podcast app. Today's guest is a self-described high class consultant who has long worked in the search machine learning and natural language processing space in a variety of roles. He was on the founding team of Endeca as their chief scientist before it was sold to Oracle and has had stints at both Google and LinkedIn these days. He consults for a variety of industry leading companies, including sites like eBay, Apple, Yelp, and Pinterest. Please welcome to the show, Daniel[inaudible]. Daniel, great to have you here.

D. Tunkelang: 1:18

Grant. It's my pleasure to be here today. Thanks for joining me. I know we've been trying to schedule this for some time, so I very much appreciate you taking the time out of your schedule. How about we just start off by having you introduce yourself to our audience and tell us a bit more about your background and some of the career choices that you've made. Well, you've already stolen my thunder by introducing me as a high class consultant. Uh, in terms of of no, I am and how I got here. I mean, I took the, I suppose, straight and narrow path of studying computer science and math at MIT and CMU. Uh, you know, both of my parents, where in the public sector, my mom is a professor and my dad working for the city of New York. So I imagined that was going to go down a similar path. As soon as I finished my PhD, go into academia and it didn't work out that way. I, it turned out I didn't really have the temperament for a public sector life. And the first thing I did as soon as I finished my PhD was to go into really a sort of a dot com job. This is back in in 99. And while the job itself that I took as a sort of consultant in a um, dot com company wasn't all that exciting. It turned out that during those few months I was discovered by the cofounders of what would be Endeca and joined that team. And the rest from my perspective is history. I ended up becoming a search guy, uh, being on the front lines of developments in this space and then ultimately taking what I learned there a to a variety of these really large companies now going off on my own and doing it. So I'm basically a failed academic turned successful industry professional and entrepreneur.

Grant Ingersoll: 3:11

That's really great. Let me, uh, let's go back a little bit in time and I want to understand a little bit more about the, you know, so you, so you go to MIT, frankly, like you're there back before like the real big tech boom, like myself, kind of just right on the cusp of the dotcom. Uh, where did that interest come from back then? Cause you know, like I, I recall, you know, again going to school about the same time and people were like, wait, you're going for what? Like math, they kind of got, but this whole computer science thing not so much because it just wasn't as prevalent.

D. Tunkelang: 3:46

Right. Well first I knew I was going to do basically science and technology. And so my only other school that I was seriously thinking about was Caltech and MIT was a lot closer to me in New York. So the, uh, funny enough I'd heard of Stanford but it seemed really far away. So I had a very parochial Northeastern view of the world. And this is 1988. Ah, so you're right. The software industry in the leading company at the time it was Microsoft RS plus even IBM. Yeah. Aye. I was going to a triple major I think was my goal in math computer science, and physics. And I did the first two, you know, two out of three ain't bad. Physics was a little hard for me. It still is. I under achiever. I know, I know. But I had started with computers very early. My dad had access to a mainframe computer at his work and at home we had, you know, this is l ike if you've seen the movie war games, we had a teletype O h with the kinds of of modems that, you know, the acoustic couplers, y ou k now, j ust shove the phone physically in and, you know, it was basically, i t looked like a typewriter with no screen but just, you know, everything being printed. That was my introduction to programming. I mean I learned even languages like P L o ne a nd JCL that are[ inaudible] y ou'd have to go to the computer history museum to learn more about at this point. But the breakthrough for us was we were able to get a home computer, a Commodore 64, a nd this is back in, u h, the very early eighties at the time, t hat b estselling computer of all time. But, uh, aye was so excited that I could basically give this thing commands and it could bring them to life and mean to this day. That's what's fascinated me about, uh, computer science about software is that this is a world in which we have God like powers. I mean maybe it's little scary that now we're inflicting those godlike powers onto the real world. But before there was any effect on the real world that that was what, uh, what drove me and yeah, I wanted to study that. I actually thought about becoming a theoretical computer scientist, uh, when I was in school and I luckily found myself going towards more practical tasks, uh, by the time I was done with graduate school.

Grant Ingersoll: 6:22

Right. Well so tell me a little bit about like, actually, Hey, I'm going to go do a PhD, cause that's a real time commitment, right? I mean, even if you're at the top of your game and you know exactly what you want to study, you're talking three or four more years usually on top of your undergraduate degrees. So you know, what motivated you there?

D. Tunkelang: 6:42

Uh, five years in my case. And that was considered quick by Carnegie Mellon standards. So there are a few things. First, it was a sensible default for me. Both my parents had PhD and I wasn't sure that you could get a a good job without a PhD. This shows you how narrow my conception of the world was. But I also, uh, well first I thought I was going to go into academia. Beyond that, when I finished my undergrad in 92, I spent a year I'd done, I had basically done my masters as an intern at IBM research and I extended my tour there by a full year working full time. First I thought IBM research was working in industry, which now it seems comical, but that was the closest I had gotten at the time. And that was an interesting year at IBM because that's when they had the layoffs for the first time in their company. All of this made industry feel very scary, unappealing, you know, compared to the wonderful sheltered experience of the ivory tower. So I, uh, I took it for granted. I wanted to be a professor and focus on thinking of these deep thoughts and doing research and so forth. Um, and then when I was in graduate school, I had a lot of interesting things. I actually wanted to work on a shared ride transportation, which back in 93, 94 was a little bit too early. There were no smart phones. Right. So it was the people were using CD radios for these things.[inaudible] on top of that, I thought of it as a more of a math problem. How to optimize everybody's routes. Yeah. Luckily I ended up working on network visualization instead and that was a bit of a gateway to not the social networks. I'd heard of those, but they didn't seem to have any practical applicability back then either. But rather information retrieval and search, a problem that I had never imagined working on. But it turned out that, uh, visualizing the way that words and topics are related to each other. It was a natural place to apply this network visualization that I was doing and that effectively brought me into the space of search, which is where I live now.

Grant Ingersoll: 9:08

Yeah. Very cool. That's, that's really interesting cause I actually had a very similar entry in Oh, like a a just kind of stumbled into search if you will. And it, you know, and I think both of us kind of came into it around the same time and uh, very life changing in aspects. So, so that's actually a really good segway and I don't think I've ever actually heard the founding story of Endeca. So you've been doing these stints industry and then it sounds like you, you met some of the other founders around that same time. You know, what was the Genesis of Endeca like and making that leap as a, you know, somebody who is still fairly young in their career to say, I'm going to go start this company.

D. Tunkelang: 9:51

That was wild in mid 1999 I believe in may I get an email, uh, from this guy Steve Papa, which as you might imagine, I have the last name, like Papa first attempted to dismiss this as spam and it's a short email saying something. Hey, I see that you've worked in graph theory. We are doing something. We'd love to meet you in New York. As it turns out, I'm kind of having a rough a few weeks. Uh, I was particularly dissatisfied with this, uh, dot com job I was in by coincidence. I had just broken up with my girlfriend of several years and it's like, yeah, whatever. Like I don't really have anything better to do. I'll meet this guy and he says, we'll meet at the, at the Harvard club, New York, uh, or the Princeton club, I suppose. I can't remember which. He'd gone to both. I show up. They don't let me in because I'm not dressed well. No, I'm still dressed so well that people ask me at work if I'm interviewing an EIN system going out on a date, but I was eating at burger joint. So this is, I'm eating with let's Steve Papa and Pete Bell, the two cofounders and we chat. They seem like nice enough people and then they tell me the story to try to motivate what they're doing. Which is imagine that you wanted to go into an art museum and you're interested in art, but perhaps you're interested in art based on the genre of the art or perhaps you're interested in based on the painter or the time period they get things. And how would you explore this space? Of course. I, you know, I had already the uh, uh, the cheat sheet, right? That they said that this was about graph theory. So I start drawing the graphs of navigation through the space of possibilities. And clearly I passed the sort of test on your standard remodeling, what they have in mind. And then Steve says, well, if you'll sign this NDA, I can tell you more. And it must've been the five longest minutes of his life. I actually read the NDA because I wasn't, you know, I was really not the kind of person used to this stuff. I read it, sign it, and then he says, we want to do this for eCommerce. And it blew my mind. I had never thought about search in general, let alone in the context of eCommerce. And bear in mind 99 just when Google is just really coming out, uh, and web search exists, but it's not that exciting a thing. And uh,

Grant Ingersoll: 12:33

Not really much for Amazon or you know, anything at the time, right? Much less.

D. Tunkelang: 12:37

Yeah, Amazon's a bookstore, but what's the big deal? So, uh, and, but I'm fascinated about the, uh, sort of the metaphor that he's putting forward and it doesn't seem like it's that hard, but it seems like a really cool idea. And I didn't. Yeah, I wasn't that excited about my job for that matter. I was pretty ready to leave New York. So, uh, the next thing that happens, it turns out that I'm not the only person that they've reached out to. They've reached out to several people and they put us all on a conference call. And this is really a virtual first meeting because a bunch of us had never met, never worked together. I was the most extreme as a complete cold call. They'd simply found my CMU webpage, which thankfully I kept up to date. And then, uh, I'm getting so excited about this that I'm sneaking out in the middle of my work day to jump on calls with folks. At that point I realized, I really want to do this. And even though the funding hadn't closed, I said, I'm in. And I gave my notice and luckily the funding did close and I only expected, by the way, that we would have enough seed funding to last six months, maybe a year. I said, look, if it all craters after that, it will still have been worth it. Yeah. And luckily it did pretty well.

Grant Ingersoll: 14:03

Yeah. I'll say, uh, yeah. So that's a talk about it. But you know, it's a classic example. I think of preparation meeting opportunity, right. And, and one of the things I, I think has been a real guiding principle in my career is, is I think exactly something you're hitting at is, is you, yeah, you have this bit of intentionality, but there's also this willingness to go explore, right? And, and, uh, as somebody who just put their son into college, I, I'd said to him, you know, have this mindset of guided exploration. And I think that really hits that a really a key aspect of your career. Because, you know, if we flash forward here a little bit Endeca is doing pretty incredibly well and yet you then take this a leap to go onto something else. And so kind of bring us up, you know, a little bit more. You, you, I know you, you go from Endeca to LinkedIn, you know, walk me through that process cause that's almost as big a leap in many ways as a founder to leave a company. Right? Or maybe I'm overplaying that.

D. Tunkelang: 15:05

No. So when I came into Endeca as part of this group of really seven folks, when when we, uh, started off in earnest and I was there for 10 years, actually, the first major, well, a major milestone anyway, was that four years in, I moved back to New York. I wanted to be close to my family and I thought that would be it. And thankfully, uh, my, uh, my colleague said, look, let's see if this works. And it turns out it did work for another six years. I did do my share of commuting, but uh, yeah, New York to Boston wasn't so bad. Um, well the reason that I left at the 10 year Mark of Endeca at this point, 2009 wasn't that Endeca wasn't doing well. Uh, despite the sort of ups and downs of the tech economy. Endeca Did quite well through both the end of the 2002 and 2008 slow downs. What happened is that I in that time had really fallen in love with search as a problem and I had gone from somebody who never even thought about it too. Basically, someone who had now was, was, uh, kind of a world renowned expert in e-commerce search as if the such a thing could exist. It's easier to get there when, when, uh, and you know, everybody starting at the same time, and I didn't, in fact, if anything, I probably had a head start, but it turned out that the market was changing a bit and where Indeca had been able to, uh, sell fairly expensive software in the space. It was being commodified, in fact, particularly commodified by this open source software called Solar that I believe you know, something about.

Grant Ingersoll: 16:50

Yeah, you're welcome.

D. Tunkelang: 16:54

To, to my colleagues, credit, uh, Endeca was still keeping its presence in, that's in the space, but was partially pivoting into business analytics and business intelligence where there was more a green field, uh, to use the same technology really, but into a slightly different business. Unfortunately for me, I wasn't excited about it and I wanted to work on search and I kept pushing in that direction with the encouragement of my colleagues. But increasingly I could see that the work I was doing wasn't really getting, uh, getting onto the product roadmap the way I'd hoped. And so I gave notice. I said, look, I get it. What I want to do, my personal interests are no longer aligned with those of the company and I don't want to be trying to pull the company in the wrong direction. So it was a very amicable party, had actually given six months of notice. And, uh, the, but I didn't quite know what I was going to do next. And I talked to a bunch of folks in 2009, uh, I talked to folks at Google, even though I felt quite confident, it wasn't a good place for me that the type of person they wanted with somebody a bit, uh, in a bit of a profile, I didn't fit. Uh, it turns out that my engineering director, friends there were quite determined to persuade me otherwise, and they succeeded in persuading me to apply, which I did, went through the interview and got a job there. So there was a, you know, between Endeca and LinkedIn, I'd spent this one year at Google. I learned a lot. It was a job where I was working on basically on machine learning, uh, to relate their local search index and their web index. I also learned that I was right, that culturally I was not a great fit at Google. Somebody who had been part of a startup where I knew everyone and I could wander and try to do whatever it was that could be helpful for the company. I was suddenly in a very tiny box and I wasn't used to the sort of organizational navigation skills that you needed in my, in my view, to be effective at a place like Google and I was in a bit of a shock. I, at one point, I felt so inadequate to the job that I wonder whether my experience at Endeca was just a one hit wonder. And that was a[inaudible] pretty big blow from spending, you know, 10 years in a roller coaster for sure. I've been a roller coaster with really great highs to suddenly thinking, wow, right, do I just suck at what I do? And also I at that point had just, uh, had a baby girl and I was, so I was dealing with, uh, the struggles of being a father, which I also wasn't so sure I was so great at along with not being sure if I was good at my job. So that was a 2009 was a bit of a rough year that way. And by mid 2010, I felt quite sure that that wasn't the role for me and I was trying to think of what I should do about it. I couldn't just quit my job with nothing lined up, not, not with a child and a mortgage. And frankly, my, my ego is a bit wounded as well. And that's when LinkedIn reached out to me.

Grant Ingersoll: 20:32

If I recall correctly too. Like you moved from the East coast to the West coast for that job as well, right?

D. Tunkelang: 20:39

No, no, that was in New York when LinkedIn reached out to me. I was at Google in New York.

Grant Ingersoll: 20:47

Uh, okay. I mean, those, I'd love to delve into the too a little bit more because I think, you know, a lot of times in careers, people often just think that there's just this linear progression or even, you know, a steeper than that of, you know, rock climbing the ladder and, and you know, people don't always, hear so much about the hard parts sometimes, you know, is your new dad new job, you know, you left a place where you're, if not the guy, one of the guys, and then all of a sudden you're in this big, at this, by this time Google's big, right. This is not the early days. I mean, so how'd you, how'd you kind of get yourself out of that?

D. Tunkelang: 21:28

So, uh, the, so while I was, you know, yeah, you're right. Google is tens of thousands of people. I was, you know, so not, not in a good place as thinking about, uh, what I could do. In fact, uh, my wife and I talked about the idea of my becoming an independent consultant. That was a bit early on for me to do that, but it was something that I imagined as possible. I didn't, uh, I had, I wasn't pursuing outside opportunities. It was also just too soon. I wanted to[inaudible] try to make it through a full year at Google. LinkedIn reached out to me at roughly the head month point. So they know those folks. I mean, as I find out later, LinkedIn is actually quite good at modeling when people would be open to changing careers. And in my instance they nailed it. Uh, except for the fact that it was in California. No, I didn't imagine moving from New York to California. But again, they found me in the right state of mind at the right time. So I did respond to them. Uh, initially there. Then head of data science, a DJ Patil, uh, met with me in New York. He was on a trip to, to see, uh, NYU and some other folks in New York. The next thing I knew, I was doing a couple of phone screens and flying on site to mountain view. And it was funny sneaking away from Google New York to spend a day of interviews and mountain view a few steps from their headquarters, literally. Yeah. And then I came back and I remember coming back into work cause I took a red eyes. I literally flew back from the interviews into a day of work at Google and I immediately just felt like I had just spent a day in a world full of color and I was back to this kind of very gray, a gray setting. And I guess I knew right then I was moving. Yeah.

Grant Ingersoll: 23:29

Yeah. But so then now, like the job is in California, so this is, I mean this is a big deal career-wise, family-wise. all of that. Not to mention, you know, not as much in tech as it used to be, but you know, leaving a place in around a year still is not always the best. Right. So, what was that like? I mean, I'm sure you're having some very real conversations with your wife. Your daughter's young enough still that you know, she doesn't know any different

D. Tunkelang: 23:58

Yeah, my daughter had not yet turned three, so she didn't really get to be a stakeholder in this particular decision. My wife and I talked about this very carefully. For one thing, I felt quite sure that the offer from LinkedIn would be effectively a pay cut a nd it wasn't. It was certainly a pay cut in cash and no one could really know what the equity would be worth. So I thought, Whoa, wait a second, move to California and take all the sorts of risks. And then on top of that, it's a, it's a paycut. As it turned out, I was, I was a little bit off in terms of the value of the offer. Bear in mind, this is late 2010, just before LinkedIn's IPO. U h, that was a very, very lucky coinflip, u h, for me. But moving to California was a big deal. Even though my wife was originally from there, she was from a humble County way up North. And you were moving to the Bay area, which six hours, uh, is, I mean it's night and day there. There is. She had no connections out here. I knew my share of people from working, from interacting with them online, but that wasn't quite the same either. Mmm. We luckily the cost of living wasn't a major factor because New York city, isn't really any cheaper than are these back then was about the same as the Bay area. Uh, but at some level we looked at this, it was clear that I was miserable in my job and she said, look, we have to do this.

Grant Ingersoll: 25:34

No, that's great.

D. Tunkelang: 25:36

The crazy thing about the move was that I gave my notice on a Monday at Google and then later in the day someone am I my soon to be manager at LinkedIn said, you know, it would be great if you could be here a bit sooner. How soon? And you said, how about next Monday? So my notice turned into the effect of four days of notice to Google. Luckily folks there were, they understood how this game worked far better than I did. And I ended up showing up on a Monday morning, uh, at LinkedIn. I know a week after that my wife and daughter couldn't join me for four months. My daughter was in preschool, we had to sell an apartment. Uh, but[inaudible] it all worked out. I mean that was, that was December of 2010 and that was, it turned out an extraordinarily good move for us as a family.

Grant Ingersoll: 26:36

Yeah, that's super fascinating. And I love how you broke down kind of just, you know, cause again, like, you know, if you just look at somebody's LinkedIn, Oh, you just changed jobs. You don't, you don't see all of the things that go into making that decision. Especially one as impactful as I'm going to move from one coast to the other. I don't, I don't really know anybody and dig in. So, so, you know, LinkedIn, again, working on the search problem, but obviously a different kind of search problem, although still, you know, well, probably related more to search when you were at Endeca, then search at Google. I'm going to imagine because now you're again back in this graph world and, and thinking about that problem, you know, what are perhaps some highlights, uh, or things that happen at career-wise at LinkedIn, uh, that you might be able to share?

D. Tunkelang: 27:30

Well, first I wasn't being asked to do anything with search. I was brought into run this data science team and I said, what's data science? I'm gonna assume it's something like search, because that's what I know. And turned out I wasn't as far off as I thought. And partly because data science was such an amorphous concept at the time that I was able to make it be a little bit more of what I wanted it to be. And I had a fantastic team that I took over when I, when I joined it and was able to grow. But the, you're right, the problems were much more like Endeca is then like, then the Google's typical problems. It turned out at Google. I was working with more structured data than most of the other teams there because I was looking at local business search. So somehow I was able to stay somewhat within that field. For example, uh, one of the problems at LinkedIn was recognizing that, uh, software engineers, software developers, software architects are all very similar, but they're quite different. For example, for then mechanical engineers or uh, for that matter, product managers aren't the same as managers in general, right? This whole space of job titles, figuring out which ones were practically synonyms when you had a notion of seniority when you would have the same job title, but it might mean something in different industries turned out to be extremely important when you're producing job recommendations, when you're thinking about targeting marketing materials that people in a particular job, family and so forth. So being at LinkedIn made me think very hard about data representations and the way that those could power my experiences in search and discovery and recommendations. That is that that world where LinkedIn perhaps had the best structured data collection of any major internet property and yet it wasn't good enough.

Grant Ingersoll: 29:37

I can imagine just, I mean because again, you still have just people entering at whatever various, you know, levels of quality that they enter things at. Right? And so you still have this mixture of uh, structured and unstructured content, right?

D. Tunkelang: 29:54

Yeah. I mean LinkedIn had to figure out in some cases that you know, that different ways of describing the same company or in fact the same company, there's no single source of data. Uh, for those things, you know, job titles where it was and describe where it could be fun. And then eventually LinkedIn, while I was there, started getting interested in people's skills and that it's its own, I think what is a skill, how are skills related to one another. And again, the purpose of this is to have a more structured representation of the various entities and what LinkedIn would ultimately call this economic graph. It people, companies, projects and so forth. And it's an ongoing effort for the company. I left a few years ago, but the, I could see some of the challenges that was able to overcome. I would say that that introducing skills has been a major success for the company. But, uh, you would ultimately like to be able to say how good is someone at a skill, which is as far as I can tell, a very much an unsolved problem for the company and for the industry as a whole. Right. Well, so let me, I want to tie it together. A few things. Cause after all this was a show on careers, not necessarily on the individual topics. I mean, I love the, you know, me as a, as a geek in this space, loves all of it too. But I think one of the underpinnings here, kind of the meta level, right, is, you know, I would imagine if maybe I'm projecting myself, here a little bit correct me, but you know, there's this fascination with language, you know, you started to hit out on, you know, talking about synonyms and things like that.

Grant Ingersoll: 31:41

Um, you know, the love that you developed of searches that come out of, you know, where did that come from such that it became this North star for you, if you will in your career?

D. Tunkelang: 31:53

It's funny. Search itself, I stumbled into, but when I look back at how I got there, it should almost seem obvious in retrospect. My mother is a professor of Latin American literature. My father, uh, was, was a polyglot by virtue of just all of the places that he lived and probably the first significant programming experience that I had on my Commodore 64, was working with my dad to translate a program called Eliza from English to French as written in basic. And for those not familiar Eliza, it was essentially a fake psychologist that used very, very primitive. I hesitate to call it natural language on understanding. Really just looking for patterns so that you'd say, I don't feel good today. And you'd say, did you come here because you don't feel good today? Sometimes we'd get the grammar right and other times, not so much now studying French in grade school and my dad is from Belgium. So, uh, he could help me with that. So I was excited about language and communication even then. And then of course, uh, you know, one of the things about growing up in the, you know, in the eighties and nineties, is that this notion of what artificial intelligence could be was very much on people's minds. I mean, I saw the movie war games and is very excited about Joshua computer. I studied, uh, artificial intelligence and some of my classes back then, by the way, uh, nobody actually calls it AI anymore. It was already one of these AI winter. So yeah, I, that meant they didn't work, but I was still excited about this idea of[inaudible] computers, learning of people interacting with them. When I looked at network visualization, it was really about getting better insight, better understanding of something represented in a digital form. If only we could leverage people's understanding in a better way through the algorithms that we use. So I had that in the background. I just never imagined that, you know, the big problem would be we have massive amounts of information and the only problem is searching it. Bear in mind until there was a web, until people index to this, until they, how'd the content in a way that was, it was accessible search, wouldn't it be in people's minds is a problem.[inaudible] those events unlock this. We might've thought, great, as soon as we put the information in a big enough box, suddenly we'll have intelligence, we'll have this ability to use it. But it turns out that the problem of accessing that information was much harder than people imagined. And I happened to stumble into that problem the right time. And all these things that I'd love before, uh, became exciting to me. And I think, you know, one of the things I didn't say is I had thought about majoring in psychology and there again, this idea of understanding what's in people's heads, how people react, how people express their intent, again fit into this. So really all of these streams came together and it's search.

Grant Ingersoll: 35:07

Yeah, yeah, no, that's a, I think that's a beautiful way of kind of summarizing. And so, you know, the underpinnings here, just to kind of reflect back, you know, so there's this love of language and of how humans use language to communicate. And then, you know, what people don't often realize about searches is a big math problem underneath. And so you've got this really well found, you know, really good foundation of math, you know, from your days at MIT and CMU and, and then, you know, and then that's your foundation. And then you know, right place, right time, the web is exploding. All of these opportunities are starting to come along. And so that's perhaps now the kind of the segue and then I'm sure everybody who's listening has been saying, Hey, when's grant going to figure it out? What's it ask them about what the hell a high class consultant is. So what is it? How did that come about and what are you doing now?

D. Tunkelang: 36:03

Sure. So after four and a half years at LinkedIn, uh, I left for very similar reasons to the ones I've left Endeca. I felt that, uh, their mission had moved on in certain ways and mine felt a little bit different. I know I was fortunate, uh, thanks to the um, uh, the successes of both Endeca, which had been acquired since end of LinkedIn, which had gone public that I could take some time to figure out you know, what I wanted to do. So after LinkedIn I took several months in which I made a commitment to myself that I would not join a company full time and not start anything full time and that would give me a chance to explore what I might want because I had really jumped from job to job both my decade at Google and my Google LinkedIn moves have literally been, and on a Friday started on a Monday, I take a no brainer. I thought, you know, I need to reset. I need to recalibrate. During that time, I had some interesting experiences. Uh, there's website, Quora, that is where people do a lot of questions and answers, had decided to have some paid questions and answers and I had a blast answering questions and actually being paid for my answers if my answers were considered the best ones. That was fun and surprisingly even a source of income. I had been in contact with folks at Flipkart, which for those unfamiliar now now acquired by Walmart, one of the largest, uh, eCommerce companies in the world in India, and I was doing a little bit of consulting for their head of search. I knew folks at Pinterest and when they reached out to me for something else, I said, Hey, by the way, I'm taking this semi sabbatical, whatever you want to call it, but I could show up and be there a day a week. I ended up having a similar conversation with Etsy and the next thing I knew I was doing a day here or day there or an hour here, and I said, okay, this is, this is fun. Obviously not going to last, you know, eventually my six months that I've committed to this will run out and then I'll probably, all of these people will get tired of me and it'll be time for me to look for a real job again. Well, the six months went and this was going pretty well. I said, well, let's keep doing this. And now it's been close to four years. Yeah, yeah. I didn't realize it's been that long. That's great. Yeah. Well it started with pretty much the people that reached out to me were trying to persuade me to take full time jobs and I'd say, Hey, I have a better deal for you. You could just have me one day a week like for, for you they didn't necessarily see this as better. But they were willing to play after all, it was very, uh, very low commitment for them. And after I did this long enough, I said, you know, this is, this isn't just a phase or if it is, it's a good, it's a long enough phase that I should own it. And in a a moment of either a brilliance or idiocy, I decided I'm a high class consultant. I put the title down, I bought the URL. And what changed after that is that instead of people just trying to hire me full time, I'm gonna have to persuade or that I was looking for consulting opportunities. I started getting inbounds where people knew what kind of work I could do for them. And that was great because some of those were different sorts of inbounds that I've had your for. Some of these folks were in private equity or in uh, venture capital who wanted a brief due diligence project. And my being independent made me, uh, a better vet than someone who would be a full time employee somewhere. And they would have conflicts of interest or even not be allowed by their employer to do this kind of work. I had law firms ask me, uh, to help with analyzing patent portfolios, that sort of thing. And aye at this point I, instead of worrying, well, will I get any, any sorts of gigs coming up? Now I have to learn how to say no because the, the flywheel of this has been going. And one of the things that helps is that I had enough time that I was able to double down on my writing. I had written a book when I was in my last year at Endeca, but now I had more time to write posts on medium, on Quora. Even those who know me know that I can be pretty snarky on social media, particularly on Twitter and on LinkedIn. And even that was a way of just people noticing what I had to say and for the people who liked it, potentially a a way of generating leads. So I never explicitly invested in lead generation for the sort of consulting, but by being very much in the public eye, I seem to stay on people's radar.

Grant Ingersoll: 41:02

Yeah. Very cool. So many things in there that are super interesting, I mean maybe just kind of hit some of the highlights. What are some of the pros and cons of this? I mean, I think you hit on a lot of the good things. Like you get to work on all these fun things that you care about. It's very aligned with what you like to do, you know, but, but you're effectively at the end of the day, you're self employed, right? And so that means everything's riding on you. So kind of walk me through a little bit of what those pros and cons are. Of living this kind of, doing this kind of job at this point.

D. Tunkelang: 41:36

So the, the obvious pro of it for me. Who was that? I no longer felt, uh, well, it's a really, it's both a pro and a con. In the past, my identity was very strongly tied to my employer. If somebody asked me, who are you? I would say I'm the director of data science or engineering at LinkedIn. I'm the tech lead for local search at Google, chief scientist of Endeca. That was, that was really who I was. And suddenly it's like, hi, I'm Daniel. Right? I really, it became, I am and I love that now, but it was frightening because there's, it's so much easier to lean on an identity that feels much larger than you are, but now it's, now it feels like liberation. At the time, I felt a real sense of vertigo, uh, from that lack of a foundation. The other thing is that I obviously I, especially at the beginning, I had no idea if anyone would show up. It was like field of dreams. I built it, but will anybody come? And initially there wasn't that much work. I think I was working one day, two days a week at first. Now I was lucky to be in a position where I could afford that, but it wasn't clear that I should do that indefinitely and what would I do with the other days. It turned out, I found things to do with the other days. Uh, I got to spend more time with my family, but also I got to read, uh, read books and catch up on materials that I hadn't spending enough time on. I'd more time for writing and so forth. But again, yeah, I wasn't sure what my week to week would look like for a while. And I would say the first six months to a year felt a bit touch and go that way, which is why I also wasn't sure this would be something long term.

Grant Ingersoll: 43:32

So, uh, that was, you know, that uncertainty comes with the territory. And I would say to anybody considering this path, don't do it unless you are financially and psychologically in a place where you can handle that uncertainty because it's, you know, it is starting something up. The other thing is that I had been used to leading a team and being a strong voice in a company and suddenly I was coming in as an outsider. Now, the good thing about that was that I could detach myself a lot from the organizational politics of companies. I could say, look, I'm here to solve a problem. Do I agree with all the decisions that the leadership of the company is making? Yeah, maybe. Probably not. Do I have to care? I'm here to do a job. I'm here to do the job as well as I can.

D. Tunkelang: 44:26

I could sleep a lot better at night not feeling the weight of those concerns on my shoulders, but it also meant that I had less ability to effect changes. At least I had less authority and I would have to do a lot more leading from behind, essentially affecting change through persuasion rather than through authority. Thankfully, that was something I had some experience with and I was able to fall back into those patterns and then better could do that at multiple companies at the same time. But that was a big change. I guess the last point I'd make is that it's worked out for me financially, but most people don't go into consulting because it's uh, because it's lucrative for them. If anything, I would say that if you're in a position where you'd be a highly sought after consultant, you would also be a highly sought after executive. And so you could probably do at least as well financially sticking with one company in a large enough role. So I didn't do this for the financial upside. I did this for more of the, the psychological, the, the upside of, of be happier.

Grant Ingersoll: 45:33

That's the lifestyle side of it. There's so much we could delve into on just that. We'll have to have you back as a, as a second time to talk about happiness and careers. But what I want to do, just in the interest of time, this has been fantastic. Kind of looking backwards. I want to spend a few minutes, uh, talking about, you know, okay. So yesterday, this high class consultant and you do most of your work in search and machine learning and let's wrap it all up into, into data science. Kind of how do you see that field evolving? You walk through a little bit the here and now and some of the main risks and challenges going forward.

D. Tunkelang: 46:16

Sure. So I would say that, uh, in search and in data science, the machine learning, the good news is that the toolkit of open source tools, APIs, things that let people get off the ground is great in a sense anybody can do it now. Anybody can, uh, can go and download an open source search engine like solar or use Python and get these machine learning libraries or use of a variety of other tools for manipulating data. And they work as advertised, which I'll qualify in a moment. Yeah. And you can get started. So compared to the way things were when I came into this space, that's a wealth of things that, that make life better. And there been some of the technical innovations, uh, particularly an understanding of language have been amazing. I now take for granted that you can take words and sentences, convert them into vectors into these digital representations, compare them and kind of sort of be able to save two sentences mean the same thing by looking at them in a geometric space and measuring the distance between them. That's something that people talked about a bit back in the, in the 90s, early two thousands but today that technology is much more mature and available off the shelf. So in that respect you'd think, wow, there must not be much work left to do if people have solved all these problems. However, the expertise around search has not advanced that much, particularly around search experience. So what I often find is that I go into companies that are using these state of the art machine learning AI tools better are using best of breed open source packages in search. And yet they don't understand that, for example, when someone searches for a specific product that's different then when somebody searches for just a brand or for a category of product and that one of those experiences calls for directing the person into a highly targeted experience. And the other one is more about discovery and exploration. And funny enough that is the kind of thing we were talking about at Endeca back in 1999 and things aren't that much better there. And I believe the problem is that[inaudible] this isn't a technology issue, this is more a matter of building products and understanding users, the kind of thing that library and information scientists worked on. But those aren't the folks who dictated the way that technology would evolve. So now we have great tools, but there's a bit of a vacuum in the best ways to use those tools. And it doesn't seem like the kind of thing a company could solve well, because technology, right? Product services, you could sell those things or solutions and they have great margins. You sell them, you sell more of them, you make more money. The kind of expertise tends to be something delivered in a more high touch way, one-on-one with customers and get through it through consultancy. And that's great for someone like me who is a consultant, but it doesn't have the same scaling properties. I would love to see that. I would love to see technology put me out of business in the sector. I would love to see that what I'm doing scale better and I try and do my own writing to propagate this. But it kills me that very often companies are, they have really smart technology but they don't always do the smartest things with it.

Grant Ingersoll: 50:14

Yeah. Well that's summarizes the entire industry of technology period. And I think that's part of what I'm trying to highlight on this show of like all the different people. It's not just about writing software, right? You need, you need product managers who get the business, you need QA, you need data science. Uh, so, so many interesting aspects there. You know, I kind of would then want to, uh, wrap up with one of the question or two that I like to ask all my guests. And the first one is like, you know, for somebody getting started, you know, what are some resources that have been really helpful for you? Maybe it's books, maybe it's websites, maybe they're, you know, a mentor or two that really helped you on, on your path. You know, I'm kind of figuring out all of this stuff,

D. Tunkelang: 50:57

well this may seem a bit old school, but how would you recommend reading, I think it says Steven Kobe's book, the seven habits of highly effective being and uh, from maybe a little bit later, I think it's called getting to yes, by, uh, folks from the Harvard program on negotiation. Not usually a fan of these, of self-help businessy type books. But I think both of those helped me understand is that one you have to know your own objective function. Know your own utility function to, uh, to decide how to prioritize things. You need to understand that when you, you know, assuming that you are just procrastinating when you're good, it could do something else. It's always about prioritization. So in a way this is, you know, you could argue, this goes back to the Oracle at Delphi telling you that the know yourself. I think that that, that uh, the seven habits book, it still stands is one of the best frameworks for thinking about that.

Speaker 2: 52:01

Now, the negotiation book, okay. Takes the other aspect of this as well. How do you, how do you resolve disputes? And you know, people talk about a zero sum games versus win-win negotiations. But I felt there again,, that book gave me tools two in a principled way, deal with negotiating with other people. I would say that if you can understand what you want and you can negotiate with other people too, uh, find the best cooperative outcomes, uh, you're doing pretty well. I'd also say that the books about behavioral economics and juristics and biases, are a critical read for anybody still under the illusion that human beings are rational. I happened to be a big fan of the series by Cambridge university press a called a judgment under uncertainty, heuristics and biases by[inaudible] Turski and Slavic. Those are fairly academic. You could probably do pretty well reading the more popular treatments like, uh, Daniel Kahneman's thinking fast and slow or Dan Ariely's predictably irrational.

Grant Ingersoll: 53:13

Very cool. Yeah, no, that's awesome. And we'll be sure to link those up in the show notes. I will add to seven habits, uh, for me early on in my career to have kind of, well I shouldn't say early on, but like at this point where I started wondering, you know, how do I get better? You know, those first couple of jobs, I think you just kinda new to the working world, or at least I was, and I didn't really know how to navigate my own career. I kinda fell into a couple of things they worked out. Uh, but then seven habits, I was like, you know, there's gotta be more, I gotta, I gotta be able to figure out how do we, how do we get better at this stuff? And that book did a really nice job of, of allowing, you know, like you said, giving you that framework that you could operate on.

D. Tunkelang: 53:58

Yeah. I'd also say I read a lot of fiction lately. I recently read the, the three body problem series by Tisha. Leo is a Chinese science fiction author.

Grant Ingersoll: 54:08

I've heard thats fantastic. It's on my list, but I haven't, I haven't read it yet.

D. Tunkelang: 54:13

It's amazing. And I would say the thing about reading fiction, particularly science fiction is that it expands your mind. But let's not forget that a lot of technology today is built by people inspired by yesterday's science fiction. So I highly recommend finding things that will expand your world.

Grant Ingersoll: 54:33

Oh, that's perfect. And so that's probably a good segway into my final question and one I, well I actually have two more, but the last one is just will be a quick one. You know, the, the final, if you know, you've, you've got some time under your belt and this, this career of technology, you know, kind of sum it up. Like what, what's the best career advice you could give somebody who's maybe wants to go on the same path or you know, in a similar realm. Maybe it's around search, maybe it's just general, you know, Hey, go, go read science fiction. But kinda, how would you sum up that advice?

D. Tunkelang: 55:10

I guess a few things. One, find things that you're good at and love, right? It's not just about passion. I mean if you're passionate about cooking but can't, even fry an egg, that's not going to help you much. But when you're, you should know as you're doing things, if not only are you producing reasonable results, but it's giving you energy. Because if what you do doesn't give you energy, it's always going to be hard. It's unlikely that you will, you will get better at it. So be aware of that source that that's something I did early on. Uh, develop a taste for problems. I think this is advice maybe for someone like George polio, but or even Einstein, but have problems that are, that will sit with you for a very, very long time. Like search is a great one because it's always going to be a problem. It's never going to be fully solved. And that way it keeps replenishing the uh, and that way your mind is naturally open as because these problems are around. Because I don't, I mean to tell you, always have your mind open. Sure. But I think it's easier to have your mind open when you're listening because there are particular things that are on your mind that excite you. You only need a of them, but you know, make sure they're big. Yeah. I guess the firm specifically, if you have any aspiration to be either a specialist or a consultant, then uh, well I already said it be a specialist. That is to say you don't want to be a little fish in a really big pond. Yeah. You could say, well, am I telling you to be a big fish in a small pond? Kind of. I would say find the biggest pond, a which you could still be a big fish and think and then think about how you can expand that pond over time. This is very similar, by the way, the Coveys advice about being aware of your circle of, uh, influence versus your circle of concern. If your circle of concern is huge, but your circle of influence is tiny, you're not going to get much done. So try to grow your circle of influence so that your circle of concern is always a bit bigger, but be aware that, uh, it's only your circle of influence that matters. So aspire, but try to, uh, uh, to stay aware of where you can have impact.

Grant Ingersoll: 57:29

Yeah, that's awesome. And what a great way to, to wrap up the show. Uh, you know, I did say two last questions and the last one is simply is, you know, Daniel, this is a lot of great information in there and I know folks will want to hear more from you. Where can they go follow you? We'll say on Twitter or LinkedIn and learn more. Maybe maybe read up on your writing on this search space problem that you're so passionate about.

D. Tunkelang: 57:54

Well, luckily I have a name that's impossible to miss online. Tunkelang. So DTunkelang is both my Twitter account and my, my medium account. I write, uh, I post on, on Twitter, on medium, on LinkedIn, on Quora. And if you're specifically interested in search, I recommend queryunderstanding.com which is part of my broader medium, but it's specifically focused on thinking about search as this communication problem and dove language on your standing at the, and if you want snarky commentary on today's technology, politics and everything else, follow me on Twitter and I promise you an endless stream.

Grant Ingersoll: 58:38

That's perfect. And for the folks listening, we'll be sure to link up all of those in the show notes. Daniel, thank you so very much for joining us. So much rich content in there and thanks so much, especially for me as a surgeon or adhering the founding story of Endeca. Uh, I had never heard that, so it was fantastic to hear about that as well as somebody who's also been in the search space for for a long time. So yeah, thank you again.

Speaker 2: 59:04

Oh, thank you for hosting me. This is a wonderful conversation

Grant Ingersoll: 59:07

and as always for our listeners, if you'd like to show, we'd love to have you subscribe to the podcast on your favorite podcast app like iTunes or Stitcher or Spotify. And of course you can always visit us at developmentor.com to hear this episode if you want to replay it or older episodes as well as find other content on careers in technology. Thank you so much. Thank you, Greg.

music: 59:36

[inaudible].

Grant Ingersoll: 59:36

Thank you as always to our listeners for taking the time to listen. If you’d like to show, we’d love for you to subscribe on Apple podcasts or whatever your favorite podcast app is. You can also visit us at develomentor.com to hear older episodes as well as find other content on careers in technology. Most importantly, if you like the show, please tell your friends. Referrals are the lifeblood of any podcast. If you have any feedback on this episode or any episode where you’d like to be a guest, drop us an email at podcast@develomentor.com Finally, we here at Develomentor hope that each and each and every episode helps you move that one step closer to finding your path.

Outro: 1:00:34

[inaudible].