Welcome to Futureview and part two of the interview with the brilliant Avinash Kaushik . In this 30 minutes , Avinash explains more of what he's doing at Crowd , what he's learned about leadership in multiple organizations and the work he pioneered at Google , and that's where we pick up the conversation . Avinash explains how he developed systems to refactor marketing spend in-flight and then moved on to building creative testing programs to win before you spend . I hope you enjoy the rest of the interview .
Avinash KaushikSo in-flight optimization became a very big deal . So you use machine learning , sophisticated analysis , automation , to make decisions in-flight , figure out while we are spending money that some of it is not going to work out and stop it or change it . So we became so good at it that about 30 , 40 percent of the budget we could actually refactor while we were spending it . Now remember , for most companies they would have only realized after all the money spent that this didn't work , or we could figure out while we were spending money . When I presented that to our CFO she said well , win while you spend is great , but win before you spend is better .
Henry PineyWell , this is what I was just about to ask , actually , in terms of how something like creative pretesting plays into that process .
Avinash KaushikThat's exactly right . So we did two things in parallel . One is we started to build a lot of predictive models that could analyze media plans and predict likelihood to succeed before we spend money . So , for example , we would take the media plans recommended by our offline and online agencies There were two different ones . We could put it into a predictive algorithm and he would say 30 percent likely he had success . Okay , go back . We did this thing for 60 percent , whatever , But very quickly .
Avinash KaushikOne of the early lessons I learned and this is a lesson in humility I had spent so much time and energy in my career focusing on all other things that I'm trying to be embarrassed to admit that I had personally underestimated , until like seven years ago , the powerful impact of creative on driving overall success . So when I tried to figure out what could we do to win before we spend , I sort of ended up identifying the team , ended up identifying creative as an opportunity . I'm like , okay , if creative is such a big opportunity , then what is win before ? you tend , Well , pretest it , Don't put it out there because the CMO likes it , or I like it , or the creative director likes it .
Avinash KaushikSo we started working with Kantar And then , soon enough . We ran into Al and Sabrina , who I adore , and we started a relationship with them And then sort of Kantar did some of our offline stuff . We shifted all the digital to HMM and had and HMM continues to be a key provider to Google . But through my tenure there , the fourish years , we became more and more reliant on HMM because the thing we were able to show is not that creative pretesting is a good signal to get before you spend money . But I instituted this practice of trust but verify So okay , the testing says yes , this will do great . How often does that also do great in the real world ?
Henry PineyAnd how do you track that ? Sorry to butt in , but because I think that's something that again is quite siloed , i think , kind of historically in terms of creative pretesting , which is very well embedded across lots of sectors , But I think research companies in particular have really really struggled to match that against in market performance . How do you go about doing that ?
Avinash KaushikSo our approach . So there's also a psychological challenge I had to overcome because , let's say , I create a test of five ads and I go back to the super powerful , influential creative director , who is a lot of power , and I tell her or him hey , four of your five creatives failed . The first thing I was told is to fuck off .
Henry PineyWell , yeah , I've been there .
Avinash KaushikI am empathetic to the point of view and this is okay . Fine , you quant person , how could you put a bunch of people in a room and test my thing and think that that's like , how dare you trust my genius ? So I had to put a trust but verify program
to solve both of these challenges the people challenge , but also , how good is our methodology problem , because basically it's me questioning my own effectiveness or , in this case , my colleagues who did the testing . So what we did is very simple , henry we would take a bunch of creatives that failed , we put them in market and we put most of our money , of course , behind past creatives and then we would measure real world results . What happened ? And it turns out and I'm going to give you fake directional numbers , because obviously that belongs to Google But one of the things we found is that , if we said yes , our colleagues at HMM said this piece of creative will has passed creative pretesting .
Avinash KaushikIt also delivered above average results for brand marketing in the real world about two thirds of the time . I'm making up a number . Yeah , that's great . What ? 70 percent of time It also succeeds . It's fantastic .
Avinash KaushikBut the other one was even more cool , henry , if the creative pretesting says the creative will fail in the real world . It actually also failed in the real world at roughly 80 percent of the time . Say it was like really good at finding crap . On the yes side ,
it wasn't perfect , but remember , before you had zero knowledge . Now you have 70 percent , and 80 percent to deal with much higher countries . Now does this mean that we only relied on CanPower and HMM for your pretesting ?
Avinash KaushikOh , what we ended up doing as a company is to say roughly 85 percent of the budget will be spent on the signals we get from the creative pretesting program , when , before you spend , because we have trust but verify , we'll take 15 percent of the budget and keep it loose so that our creative teams can still imagine amazing , crazy things . Doesn't matter what the creative pretesting says . Let's just try a bit of things . Let's try a bit of formats . Let's try . We've always used babies in the ads . You know what ? Buy , buy babies . Let's use trees , go crazy . I think the secret was both of these things trust but verify but also giving our creative colleagues that freedom to have 15 percent of the budget . Use it however you want to use it , and that innovation that fostered meant that the results that Google was able to demonstrate in the near term using pretesting , and long term , using that 15 percent innovation built a competitive advantage for our company .
Henry PineyYeah , I can very much see that . I think that's great advice in so many different levels around how you present it to creatives . I love the spend before you win idea . The other analogy I've sometimes been given by other people is if you were to jump out of a plane , you'd like to test the parachute before you do it .
Henry PineyBut just diving into some of the technicalities now , i'd also like to spend a bit of time on crowd at you as well , to understand what you're doing there . How would you actually match it up , though ? Because generally
, creative pretesting will focus on awareness , unedited awareness , intent , consideration , brand metrics , all those types of things . Are you matching against actual sales , or is it view through rates , or what are you measuring the effectiveness ?
Avinash KaushikYes , so this varies for different companies . Some companies I work at they will work with at Crowd or Google before , which is Google's clients . They would spend 80 percent of the budget on brands , 20 percent performance . Sometimes it flips . So there's different variations in how much how people spend their marketing budget . So my answer to that question depends a little bit on how much you're spending on brand , because brand is the harder one to accommodate for and tie back to sales in the short term . So we typically take two strategies . Again , at Crowd we'll build a custom thing for depending on the client's budget and media strategy . But what we do is we say , in the short term for brand marketing and short term will be 100 days , not every day because you need for purchase intent We'll say over a period of 100 days . We have models that can tie brands impact on performance And because we're also measuring brand's impact on brand , the influence of the creative flows through that And so we're able to isolate that . Ah , we spend 60 percent of the budget of past creative , 40 on failed creatives . Now let's measure impact on brand plus sales . Or we spent 10 percent on past , 90 on failed . So actually we were able to model these things really well over that 100-day period .
Avinash KaushikFor metrics like purchase intent , where the impact horizon is shorter . Now you go back to consideration . The impact is over nine months or so , in which case we rely , or you go to unedited brand awareness , my favorite . The impact is over nine to 18 months . So the impact horizon influences what tactic you want to measure .
Avinash KaushikFor the latter two , we use our mixed models to identify brand's impact on sales . So if you're working with Crowd , you're working with any good company out there . They will be able to use their mixed models , especially if they use machine learning , to identify brand's impact and performance actually tied back to sales . So if you , since we work closely with Al and Sabrina , we would go back to Al and Sabrina and say , ah , here's how the creative is influencing the outcomes on brand , unedited brand awareness three points of lift , two points of lift , whatever , and then , over a period of six to 12 months , we're able to then tie that back to the number of phones sold , number of new accounts opened or whatever , and we can take two steps back and say , ah , and , by the way , tv did this , newspapers did this , email did this .
Avinash KaushikAll you have to do is sort of stretch the duration of the analytics because you need to collect more signals . If we were starting ads on Google , the impact horizon is very short . You spend money the next day . It works or not . So you use that . What do you call that ? The accordion brand is fully accordion a little more so you can get a solid signal back .
Henry PineyYeah , thank you . So it kind of boils down in some ways to a couple of things . First of all . I mean , firstly , you need to be very clear on what you're trying to measure on a timeframe And secondly , having patience . That's it . If it's around brand marketing , i'm assuming in terms of the detailed around survey , though . So if it's things like unaided awareness , that then you're running tracking studies and that that's like a thing Again , it's another survey based methodology . Would you use social media or another data forms as a proxy ?
Avinash KaushikNot for campaign impact . There are two challenges
with using quote unquote social listening tools . The problem is that for most companies this might not be for all companies . So for kindly cosmetics , maybe what I'm saying doesn't apply , but for if you take outliers like kindly cosmetics out because the Kardashian brand runs on social . So let's just set that aside for one moment . For most companies , the social audience is skewed and unrepresentative of the company's actual consumer base . So if you're bringing the social listening as a proxy for campaign , the best you can say is for the subset of this biased audience . This is the results , but it is not a good way to understand the holistic impact . So that's one big problem . And then the other one is that it is very hard to see quick impacts for things in social that you're trying to do over the long term , because the reach , unless you're buying a lot of paid media on social , the reach of the social channels , is so tiny . So the organic reach on Facebook now I believe they said a study said is 0.7% . So almost nobody's seeing your campaign unless you pay Facebook . By the way , i don't begrudge them , it's their business . So that's the other thing that biases your results .
Avinash KaushikWhat we do use social listening for is for two interesting and wonderful use cases . The first one is you should use social listening , use some NLP algorithms to parse it all out at scale to identify leading issues . Social people love complaining about products and services and companies . So I'm like , oh , that's okay , people love complaining . Is there a way that we can scrape all this data on a daily or weekly basis so we can find leading edge problems with our company ? We can find patterns in the data .
Avinash KaushikOh , 100,000 people suddenly complained about our seats . What the hell ? We didn't know anything about what's going on . So it's a really good leading indicator of challenges the company's having . Big patterns don't respond to individuals . That . The second really good use case , henry , closer to our heart , is that sometimes in some of our campaigns all the creative fails . We have no idea how to come up with new things that would resonate with the research and all the way we really don't know . So I've had like the best experience using social as a way to seed creative . Give influencers the power to say , hey , you built creative for us instead of our creative team .
Henry PineyThat's interesting .
Avinash KaushikYeah , let's see what you come up with . Use their ideas and then their data to figure out like , oh , we hadn't thought of this creative concept or we hadn't thought of that . So social becomes a very rapid way to experiment with creative ideas , using an outside force for good rather than our own company to let new ideas come in . So both YouTube and Facebook and Instagram all three of them we've used to unlock creative imagination using people outside our companies . So those are two really great uses of social using it to see if what the unneeded brand awareness was is an incomplete signal at best .
Henry PineyYes , as you say . I mean , on social , you tend to get the lovers or the haters are very vocal , aren't they ? But I love that idea of using influencers or the social world as kind of like a vast pool of creatives . I guess TikTok have done that for a large extent . I mean , they put an enormous number of creatives And some of it's not great , but some of it is .
Avinash KaushikShe bites into bullying .
Avinash KaushikI agree . Honestly , I think that the strategy of using influence to parrot the company's words and terms and creative concepts is such a waste of money and time . If you're going to spend money on influencers , give them the freedom to reimagine concepts of creative . Let them come up with stuff . Tiktok has been particularly good at it , as you said , because the formats that work on Tiktok are so dramatically different than YouTube or television or Facebook . What 30-second videos don't work on Tiktok ? 30-second ads don't work on Tiktok . By the way , if you're good at it , you can pull off that on YouTube with 15 seconds or 6 seconds . Tiktok's a completely different landscape . So I found it a really good way to challenge the company's current creative lockbox into which they may be trapped , And Tiktok is a way to challenge ourselves to think even more imaginatively . So it's a really great point , Henry .
Henry PineySo , avinash , i'm conscious we're coming up on time
. There's all sorts of things I'd love to ask you about , particularly AI , but that may have to be a separate podcast , if you'd be up for doing that . So , crowd , you've just described in really compelling terms some of the things that you're doing , but could you answer a couple of things ? What does a chief strategy officer do Which I've , by the way ? I've asked some other chief strategy officers . it'd be interesting to compare notes And what does crowd do and where is crowd going next ?
Avinash KaushikYes , so crowd is a full service media agency for our clients in the Middle East , uk and the US . We manage their media spend to drive outsized marketing results . So everything from a Pratt here and how we can use billboards with local geo-targeted promotions and changing billboard ads in London to deliver higher-foot traffic to Pratt , to how do we help compare the market , build a sustainable competitive advantage on paid and organic search So in the spectrum is everything else in there . So we do that . For some of our clients we also do lead a strategic analytics practice . So for our one of our large clients in the US , they spend hundreds of millions of dollars in marketing and we're helping put together a strategic analytics platform end-to-end people process tools , kpis , dashboards but we don't manage their media . So it's just this sophisticated , bleeding edge analytics practice . So those are two big areas of focus In terms of my role .
Avinash KaushikI lead product analytics , m&a and strategy at Crowd . So with my title , my big role is around building amazing strategy for the company What market should be in , what products should be offered , what should be our investment in AI versus people . One of my internal mantras is I come to work every day to create unhealthy profits for our customers . That's the reason I'm showing up , so translate that into real things . So big part of that and , of course , working with our investors , which is Lloyd's Bank they are the investors in Crowd , so figure out how to manage the goals and expectations they have set for us , our internal leadership and clients . That's a big chunk of my role . The other one , of course , building a bleeding edge analytics practice . It's sort of my natural home . But also what I'm excited about Crowd is to bring the other side of my work at Google and build amazing products . So we're building a bunch of tools . We have a number of products already , but to basically turbocharge that . That's where I'm spending my time And I also clean conference room tables and vacuum the floors .
Henry PineyI was just about to ask how do you fit all this in ? The first three things were probably enough that you mentioned . It must be very hard to remain hands-on though , especially when you're dealing with potential investor relations and helping to form strategy and looking at M&A and that type of thing .
Avinash KaushikIt definitely is . So not watching a lot of TV helps . No , i think that
one of the things that has been very instrumental in my career over the last , especially in the last decade and a half , is focus my strengths on the think and the strategy and how do you innovate ? Those are things that I spend most of my time And then figure out how I can identify incredible leaders who can go out and inspire and execute . And so I always say , like there's never a moment in my day , no matter where I am , what I'm doing , i'm not looking at a person and saying , is this the next leader ?
Avinash KaushikBecause I'm looking to figure out how we can elevate people , challenge them , make them uncomfortably excited And I'm lucky that we have so many of them at Crowd where I can just give them things , and they are so fantastic at doing them , so they are extraordinary in that they help share that burden . And then I can focus on these other things . But you know , henry , i wanna be honest . There's some days I'm cycling back to my apartment in Camden and I'm like , oh my God , i'm so tired .
Henry PineyI bet you are . I bet you are . I love some of the juxtapositions as well that you present . I like being uncomfortably excited . I think it's a great one . I am conscious of time , though , and so I'm just gonna do a couple of quick final questions to wrap up , possibly from all at personal level . So what have you changed your mind about recently , avinash ? Is there something that you used to believe was true but that
you now no longer believe is true ?
Avinash KaushikYeah , oh , i have a user manual I write for people to get to know me better when they join the team , and one of the item , number six , says I have strong opinions loosely held . So I have lots of things I've been wrong about because I have strong opinions on things , but they're loosely held And I could give you like so many examples . They apply to people , i'll be honest . Sometimes I'm like , oh , this person maybe isn't doing such a great job at this area And that sort of challenge them to do something . And I'm like , no , i was wrong . They sort of supply to that .
Henry PineyDo you feel that one is not Sorry , Srinivas . Do you feel that works , though , in terms of ? I've heard that mantra a lot , particularly within the tech industry . Do you believe it works in terms of managing people , though ? Because I think the danger of it is ? you know , I've heard senior people say strong opinions loosely held and challenge me , but people don't want to challenge the CEO or the CSO .
Avinash KaushikYou don't know . This is true . You have to figure out how to walk the talk . One of the leaders I worked with . They said to me oh , one of the things , avanash , i wanted to do when you join our team , this is in my previous roles . He said challenge me , give me feedback , because I'm constantly asking him feedback . Nobody gives me feedback . I said don't worry , i don't have a problem with that . And then I started to give her feedback And she was shocked . She was not used to getting it . I'm like well , people don't give you feedback because they don't perceive that you're open to hearing it . So you have to figure out how to create a culture where it is okay .
Avinash KaushikSo I'll give you one very simple example . I've created a new modern analytics maturity model . I wrote it on my flight to London the other day . I write on planes And the first thing I did is I sent it to the key leaders in the team and said criticize this . And they happily ripped it apart . So I'm modeling the thought that I am inviting and open . And , by the way , they gave me criticism . I used that to change my thinking and that became the final version . I'm giving you just one small talking about .
Henry PineyI love that . I love that Because the conventional way most people do it is what do you think ? and then people feel obliged to say , oh , this is really good , and then , with a couple of caveated maybes , but a two word instruction , criticize this . Yes , it's very clear , you've asked them to do it and they're going to do it .
Avinash KaushikThat's exactly right . So there's many , many things you can do to create an environment . Where I have one of my other practices is every company I've been at . We all gather the team together for a day , We put the best work on the screen and we collectively critique it , and I call them the naked sessions , in the sense that everything stays in the room . We're going to be very open and transparent And , henry , the first piece of work that goes up is mine . I stand up first . Usually I have the biggest title in the room , So my work goes up for us . Everybody sitting there critiques it , and then we go on to the other person And again , it's little things that you can do , where you can have strong opinions loosely held , but if you're not open as a leader to walking the talk , then all you hear are platitudes And underneath it , that's not what people think about you .
Henry PineyInteresting . It's also and again could be the subject of a different podcast . I think it's an area where it's been quite commonly observed that there are differences between US culture , particularly Silicon Valley culture , and European culture around this as well , but various reasons . I think the US arguably has a more individualistic approach and has a tendency to put senior people , often the CEO , on a pedestal , and whereas the European approach is perceived , to some extent , some of the Asian cultures has been more collectivist , of building from the ground up . But anyway , i digress .
Avinash KaushikI don't know . I agree with you . I have six months that I've been spending time in London and meeting the CEOs and folks here . There's definitely a different approach to Silicon Valley , and Silicon Valley titles don't matter . Honestly , it really doesn't . At Google , i was completely free to send Eric Schmidt an email saying hey , this thing he said in all hands , i disagree with it . And he would reply just extraordinary .
Henry PineyYeah , it's fantastic .
Avinash KaushikBut here and again in Japan you can't even say anything in Japan because it's been a long time working in Japan . So you do have to make cultural accommodations and adapt around it . So when I do a keynote in Tokyo I take a subtle different approach to language than when I'm in Sydney or Netherlands . I love doing tinos in the Netherlands . I can just say whatever I want .
Henry PineyYeah , that's the polar opposite . That's the European exception of renowned funding for non-Libreland .
Avinash KaushikSo we have to adapt . We have to adapt our language and things . But if you want to create a competitive advantage for your company , you have to figure out as a leader where are the new ideas going to come from ? Who's going to challenge convention ? How do you break down silos ? It doesn't matter if you're in Germany or in South Africa or you're in Brazil or America . You may execute that in different cultural ways . But if you don't have a spigot that opens up and lets in new ideas , then you become one more company that slowly dies over time .
Henry PineyYeah , very much so , aminesh . Again , i'm conscious
we are over time . You've been very , very generous . Final question for you . This is a personal one . Tell me , if it's too cheeky , what would your partner , what would your wife say ? your best and worst characteristics are No good .
Avinash KaushikI'm happy to answer it . I think she will say my worst characteristic is that I'm very impatient with her sometimes , and I think she's right . I think she's right . She's just an extraordinary woman , so competent and brilliant and smart and beautiful , and sometimes she'll do something and I'm like what ? This is it , and it's mostly because my standards for her are so high that I sometimes behave in impatient ways with her , so that that's actually a true criticism . I think the best thing she told me the other day is she's amazed at how completely self-motivated I am without any extrinsic requirements , in the sense that I can push myself super hard without needing somebody else to . Or she can't believe that I spend so much time writing a newsletter every week . It takes a lot of time every single week to put new ideas into the world , and then I do that for no personal benefits . So she remarks sometimes sometimes positively on the ability to be self-motivated , although she spends more of my time pointing out my flaws . Keeps me honest , for which I am grateful .
Henry PineyYes , it's very true , isn't it , that we often hold higher standards possibly unfairly high standards for those who we love the most or just reach the
most , who we have the highest opinion of . I said it was the final question , but just very quickly Any favorite recent books or pieces of media that you would recommend I know it's not TV shows , yeah .
Avinash KaushikYeah , i actually went back and read both of Yohal Harari's books recently all over again It's with all the things that I say Red Homo Deus and Sapiens all over again . And I think in context of the development of the last two years , 18 months , with AI , it was good to go back and read the two books again . They shot out the history of humanity and all the changes in all kinds of things over time and let it spark a set of new ideas about the moment that we are in again . So I recommend if people have not read them . Of course they are fantastic , fantastic books . Yohal has an amazing way of compressing an extraordinary amount of complexity into something very simple . But especially with an eye towards the present moment in AI , i found it sparked new reflections in my mind . So I was glad I sort of went back and read them all over again .
Avinash KaushikAnd one more , if I have fellow sci-fi fans I'm a huge sci-fi nerd And the best sci-fi books of all time I think are the Three Body Problem by Chik Singh Leo . It's a set of three books And I think they're the best ever written . I just adore them , even though I'm reading them in English and my wife was Chinese I can't get the depth and the nuance of the original Chinese . But I also recommend the Three Body Problem three books .
Henry PineyBoth sets of recommendations are fantastic . I have read them . The trouble is I then try to explain the Three Body Problem to a friend of mine who's a very good mathematician And I just totally butchered it . She was like but anyway , that's a different subject in itself . Avinash , thank you so much . It's been so much fun talking to you , incredibly illuminating , And hopefully I might be able to get you back on at some point and we can dive into all sorts of other issues .
Avinash KaushikNo , i'd love to Thank you so much for the opportunity , henry , it was wonderful speaking with you .
Henry PineyIt was wonderful , as always , to talk to Avinash . Out of the many great interviewees I've been lucky enough to have on so far , i think he's probably the king of the pithy phrase , with the unique ability to encapsulate big and sometimes quite complicated ideas . Part one he was very , very well received , so if he's kind enough to put up with me again , i'll aim to get Avinash back on soon . There's a lot we didn't get onto , in particular , the parameters by which he recommends you think about AI . Thanks again to Insight Platforms for their support and to HumanMadeMachine for sponsoring this episode . To explore how creative testing can directly impact your ROI , visit humanmademachinecom .