AI and the Future of Work: Artificial Intelligence in the Workplace, Business, Ethics, HR, and IT for AI Enthusiasts, Leaders and Academics

Pradeep Menon, CTO at Microsoft's digital natives division in APAC, demystifies ChatGPT and lets us in on a secret about AI and jobs

• Pradeep Menon • Season 3 • Episode 25

Share episode

Today’s guest is the author of a popular Medium blog where he has recently been dissecting generative AI for technologists. I read his introduction to the transformer architecture and immediately realized our audience needs to meet him. A bit like great recent guest Ken Wenger, Pradeep makes complicated technology accessible. 

By day, Pradeep Menon is a CTO at Microsoft's digital natives division in APAC. He has had one of the best ground floor views of generative AI since Microsoft first invested in OpenAI in 2019 and then again in March of this year.

Pradeep was previously in similar roles at Alibaba and IBM. He speaks frequently on topics related to emerging tech, data, and AI to global audiences and is a published author.

Listen and learn...

  1. What surprises Pradeep most about the capabilities of LLMs 
  2. What most people don't understand about how LLMs like GPT are trained 
  3. The difference between prompting and fine-tuning 
  4. Why ChatGPT performs so well as a coding co-pilot 
  5. How RLHF works 
  6. How Bing uses grounding to mitigate the impact of LLM hallucinations 
  7. How Pradeep uses ChatGPT to improve his own productivity 
  8. How we should regulate AI 
  9. What new careers AI is creating 

References in this episode...

Speaker 2:

Good morning, good afternoon or good evening, depending on where you're listening. Welcome back to AI and the Future of Work. Thanks again for making this one of the most downloaded podcasts about the Future of Work. If you enjoy what we do, please like, comment and share in your favorite podcast app, and we'll keep sharing great conversations like the one we have for today. I'm your host, dan Turchin, ceo of PeopleRain, the AI platform for IT and HR employee service. I'm also an investor in, an advisor to more than 30 AI-first companies and a firm believer in the power of technology to make humans better. If you're passionate about changing the world with AI, or maybe just looking for your next adventure, let's talk.

Speaker 2:

We learned from AI Thought Leaders weekly on this show and, of course, the added bonus is you get one AI Fun Fact each week. Today's Fun Fact Samsung has banned access to chat GPT for all of its nearly 300,000 employees. Why? Staff engineers leaked code to open AI when they were using chat GPT as a co-pilot. In a memo sent company-wide, samsung said interests in generative AI platforms such as chat GPT has been growing internally and externally. While this interest focuses on the usefulness and efficiency of these platforms, there are also growing concerns about security risks presented by generative AI, and we've been talking about a lot of those risks on this program. Recently quite interesting, from not only Samsung, but the entire country of Italy banned chat GPT as well. As always, we'll link to the full article in today's show notes.

Speaker 2:

But now shifting to this week's conversation, today's guest is the author of a popular media blog where he has recently been dissecting generative AI technology for technologists. I read his introduction to the transformer architecture and immediately realized I had to have him because our audience needs to meet Preet. But, like great recent guest Ken Wenger, preet Menon makes complicated technology accessible and he's been answering the questions we've all been asking By day. Preet is a CTO at Microsoft in the digital natives and corporate ventures division in Singapore. He has had one of the best ground floor views of generative AI since Microsoft first invested in open AI in 2019 and then again recently in March of this year. Preet was previously in similar roles at Alibaba and IBM. He speaks frequently on topics related to emerging tech, data and AI the global audiences, and he's a published author. Without further ado For deep, i've been looking forward to this one. Welcome to the podcast. Tell us a little bit more about your background and how you got into the space.

Speaker 1:

Preet and thanks for having me in this podcast. It's an honor, yeah, and we are living in very interesting times in the era of generative AI. I've been working in this space for around a couple of decades now, since 2004. My learnings in AI have all been mostly on the field right, so I've come from a field-based background.

Speaker 1:

I take my masters in data science as well, but I started way back in 2004, right in the space that we know now as data warehousing, and then, from there, worked on a few banks, moved on to work in the startup and then dabbled in AI in companies like IBM, microsoft and Alibaba. In 2020, i came back to Microsoft, wherein my role was mainly to help customers in Asia to use data and AI for their digital transformation. As such, now what I do is I work at the studio for the startups and digital data space, wherein the goal is essentially to help startups to adopt Microsoft technology, and generative AI has been the forefront in what they want to do and how they can disturb the market. I'm also the author of a book called Data Lake Housing Action, which is essentially an architectural overview of how organizations can frame the way they have data How the organization can frame the way they can turn their data into actionable assets. So that is in just my introduction.

Speaker 2:

It seems like every day our minds are blown by things that turns out generative AI can do that sometimes we didn't even realize. Give me one example of something you've seen recently that just amazed you.

Speaker 1:

I mean, i've been dabbling in this space since the introduction of Chadji to goody right And it has actually blown my mind many times. But one thing that I would I was really kind of surprised by what it can do is the remarkable abilities these models have to understand the context, produce contextually relevant responses and even demonstrate a degree of creativity in their output. So historically, these models that we kind of dabble with now have struggled to produce the kind of text that is coherent, contextually accurate and beyond a few sentences. But these ones are a whole new paradigm shift And let me give you a personal example as such.

Speaker 1:

So I always wanted very difficult to read scientific research papers. So they are very complicated, their languages are very tense. But then with ChadGPT or what we have now GPT4, integrated with Bing and Bing Search and all these things, i find it very easy to understand what these research papers say. I get their help to summarize it and then kind of assimilate the knowledge that they have. It has helped me tremendously in my conversation with customers on where the next space is And also understand these complex processes in a much more simpler manner. That is one thing that personally I have seen helped me a lot, and I'm sure there are many things that help others as well.

Speaker 2:

Great example. So I mentioned that I met you through your great blog post about the transformer architecture. I thought it was very articulate, very accessible And in fact it was much better than learning about it from ChadGPT. You do a better job than ChadGPT does, But thinking about some of the more popular blogs that you published recently on the topic, teach your audience something that you have explained in the blog post that maybe isn't obvious even to an audience of technologists.

Speaker 1:

Yeah, actually, i mean, obviously folks who are, i mean, different people would have a different level of understanding of what ChadGPT does. But one thing that I would say is that ChadGPT is actually a derivative of what GPT is, so the way ChadGPT is trained is kind of an adaptation of what GPT is. And one of the topics that I have found that folks misunderstand a lot is the aspect of the models of, i mean the aspect of prompting and fine tuning of these models. Because now these large models are kind of pre-trained with a lot of data, right From the internet, from the textbooks, many things, and they're scaled with billions of parameters. For example, we know that GPT3 has around 175 billion parameters And think of these parameters as nuggets of knowledge that they acquire as they train, right. But as these parameters increase, gpt4 is considered to have around one trillion plus. We don't know the numbers yet, but then as these parameters scale, they learn new scales. So it is amazing to see the skills that these models have learned by immense ability of the number of parameters right, and the appropriate way of communicating with these models is through the concept of prompting and fine tuning. I was amazed to see, given the right kind of prompts right. The model responds in a different manner And in my talks and all, i feel that many of them don't understand the difference between how do you prompt them and how do you fine tune them.

Speaker 1:

Right Now, for AI, example, right GPT model was fine tuned to learn more about the code And hence we have this model called les codecs, which is used by many of the programmers to code much more efficiently, which is the basis of GitHub code compiler right Now. The key takeaway here is that when you want a model to learn a new task not new knowledge, but new tasks then you fine tune them. Else. Many of the times, prompting is the right thing to do. It works very well. That is something that I find that many of them can't grab their mind around these concepts, thank you, thank you.

Speaker 2:

I've read a lot about how RLHF reinforcement learning based on human feedback works And I got to say it always feels unbelievable to me that when you take, let's say, a trillion parameters, whatever GPT-4 is, and all of the vast, you know, diverse use cases for GPT-4, it doesn't seem like any amount of human feedback through reinforcement learning could make a dent in terms of fine-tuning a model. Anything you can share about how RLHF works that might help our audience understand how it could possibly have an impact on the accuracy of these models.

Speaker 1:

From what I understand is that RLHF sorry, rlhf, reinforcement learning, human feedback is actually a process which is done to ensure that that GPT doesn't go beyond. I mean, it understands the context quite well, right, because what happens is that, if you look at the stages of how they are trained right, first they are trained with the transformer architecture plenty of data, huge supercomputing power to get to know what, where, where they are at. But then from there, you want to ensure that the expectations we have with a chatbot is that it is able to chat with us. And then from and the model output of the transformer architecture is not conducive for a conversation, right, and hence you go through this process, called as supervised fine-tuning, wherein I am giving responses to the chatbots such that it is able to, you know, chat with me. So human agent acts like a robot to chat with the chatbot and then grade it based on what their responses are. So that is a significant effort that you have to give in to ensure that you know the chatbot is able to chat with us.

Speaker 1:

But the challenge is that, even with that, if you ask it something which is beyond its what it is trained for, right, it will go bonkers. So, and hence this process of reinforcement, reinforcement learning through human feedback. Now, what happens here is that we would have a human being grading the responses from 0 to 5, right? So, depending on the response of the chatbot, if the response is not what we expect, then you kind of give it a penalty, and if it is better, then we give it a reward, right? So that is, that process itself is automated through a reward model based on reinforcement learning, but the human being is there in the loop to ensure that the responses given by the audience I mean by chatbot is conducive to what we think about, we think the right answer is And these kind of human feedback is important and to ensure that the right kind of guardrails are given to the chatbot and it doesn't go rogue and beyond the scope of what it needs to be outputting.

Speaker 2:

Two of the limitations major limitations, i should say of the transformer architecture are because it's just predicting the best next word, it can what we call hallucinate, or, you know, it can invent facts, but also it can replicate bias that's inherent in the data. As a technologist, you understand how the system works, but as a human being and you know, as someone who's interested in you know the ethical use of these technologies. How do you think about the trade-offs between the risks but also the capabilities of these models? I think I mean.

Speaker 1:

What I would say here is that this is like a platform shift, right? So, like any platform shift, you will have tremendous opportunities and with that you will have risks associated with the tech, right. So it is a balance that you have to do. First of all, i'm seeing that there are many risks, like there is a risk of misinformation and manipulation, and then a risk of security and privacy. There is a risk of who is accountable for the output that it gives. The fourth risk is the lack of explainability, and then you know how it impacts human creativity. That is, those are the few risks.

Speaker 1:

So I think that it's a balance, but I strongly believe that the opportunities that this deck has, right, completely overwears the risks that it has.

Speaker 1:

Right, because, at the end of the day, think of it like a co-pilot, right. So it is helping you to do your task much better, to be much more productive, much more efficient in what you do, and the output that it gives you is a draft. It is up to you to use it or to kind of validate it, to ensure that the output is quite validated and then take action accordingly. And hence I mean one of the companies like Microsoft are doing is. They are creating a concept called as grounding, which means that even when you should be in chat now, you get links that you know. The answer to the questions that you have asked is kind of annotated through this page, this page, this page and so on. So it is your responsibility to take that draft and then validate whether it makes sense or not, whether it is from the right source or not. But all it is time to do is to make you much more productive in the tasks that you want to do.

Speaker 2:

I love that concept of grounding and I know that being is way ahead of anyone else. Certainly open AI, despite the partnership. But when an LLM generates something, some content, that potentially harms or ends up creating some damages that impact people, who is responsible? Is it a user who trusts the output? or is it maybe the vendor of the LLM who is responsible for the training of the model that produced the content that created harm?

Speaker 1:

Listen, i mean to be honest, this is very open field now, right. But in my view, what I think is that LLMs, like any other tech, is only an enabler, so it will give you the information that you have asked for, based on its understanding and training and so on. So the ownership is up to you, right? It is a draft that you have to take in and then act. It is not acting on your BIPRA, it is just giving you the information that you have asked for and it is up to you how you want to act upon it, right? So it is the same thing like saying that, okay, i'm doing a Google search, but if I get a result which I don't want, whom should I blame? Should I blame Google for it or should I blame myself because I didn't think the right links? So I think the analogy is a bit similar. But just that, because this tech is quite new, it is very, i would say, human-like, right? So you chat with it. You have a notion of that. I'm chatting with not a bot, but with a real human. So we tend to take it much more personally, in my view, but the concept out there is. I mean, the fact is that it is just a tech, right. It is a tech that you need to use to make yourself much more productive.

Speaker 1:

Like to give you an example now, right, i mean when I should write blogs just a few months ago, it would take me at least one month because I need to do my research, i need to do my drafts and then kind of edit it.

Speaker 1:

So it was a time-thus-me process. But with chat capability, now I can turn on blogs in the week's time, because most of the work I mean the research work, all these things are kind of shortened because I use chat capability. All I have to do is to conceptualize what I want to write, conceptualize the prompts that I want to give to the chat capability and then ensure that the output that it gives me is what I want to recommend to my audience. So the entire process of writing a blog has come down to a week or so. That means that my productivity has improved by a few x times, right? So that is what I think chat capability will go. But the blogs which I write, the ownership of those blogs is mine, the content is mine, the source material is mine, the thought process is mine as well. All I am doing is that I am augmenting my productivity through a chatbot, which is cool.

Speaker 2:

I'm in Silicon Valley, you're in Singapore, and together we're surrounded by amazing innovation that often we see before it gets commercialized. In your role, what's a technology that's maybe emerging but that nobody else has seen yet? that has really impressed you. You think could help change the world.

Speaker 1:

I think, of course, i think the times are very dynamic now. Right, i think that application of generative AI is going to change the world and I think, in my view, that this is just a start. Right, the generative AI world now is mainly prompt, through text, or text to text, or text to images, and then we also have text to videos and so on. But the real thing which I see that is going to transform the way we live in. I mean, the world we live in is number one creating specialized, i would say, co-pilots which will help you to do the job much more efficiently than you can do now. And that technology, once it kind of emerges, right, when combined with the world of Metaverse AI I mean AR, vr and so on is going to transform a lot of industry.

Speaker 1:

I think that is still some time to come because of the fact that how AI tech works I mean Metaverse tech work. But imagine this right, you are a student and you have avatar of, say, your favorite teacher, like Socrates or whoever in front of you, and that avatar of your teacher is able to communicate with you in speech through the power of something like a GPT4, right? So that world, in my view. We have the tech in place. I'm sure we have the tech in place, but it has to be just ML. I mean, it has to be just ML committed together to form something that nobody else has seen. So we do have some startups working in this space and I think still early days, but it is possible.

Speaker 2:

I recently met a company called DID out of Israel that's letting you create avatars in real time, train them on LLMs, and they can be from an image, from, i think, as little as one image, and they become interactive avatars with full LLM capabilities, and that blew my mind. So I understand what you're talking about These kinds of tutors or various other ways of making these text to text or text image etc. Models just come to life almost in real time. It's an amazing use of the technology, exactly, and I think that I mean also.

Speaker 1:

I think that the notion of I mean, in my view, the way things would change is by adopting the notion of a co-co-pilot. So if you think of Iron Man, he had this guy called as Jarvis who would help him to do things. Of course it is not as good as Jarvis, but then it is somewhere there. So I think, in the world to come, you would have co-co pilots with you, helping you to do each of the jobs. So, be it scheduling, be it writing an email, writing a blog, interacting with someone, so all these things, i think, will help to improve productivity a lot.

Speaker 2:

Speaking of Jarvis, i feel like a lot of the popular press makes people scared about the potential for AI to eliminate jobs, but you and I are AI optimists. What do you think are some of the new types of careers that will be created by AI?

Speaker 1:

I think. I mean, i think the biggest challenge we have at this point in time is how do you regulate this? That is challenge number one. Challenge number two is what are the kind of ethics and policies that we need to have as a framework to ensure that it is used in the right manner? The third challenge we have is that these models we have huge and there is a challenge of explainability. How do they come out with what they come out? So those are the three key challenges that I see And I think that, apart from the jobs that we have now, like content developer, content generation generators and AI and education and all, there are two, three or four key new careers which I see coming up.

Speaker 1:

Number one is more on AI trainers and curators, so those who specialize in preparing, curating data and AI models, ensuring the quality of these models are good, the biases are minimal and the performance is optimal.

Speaker 1:

So that kind of things would come up.

Speaker 1:

Second thing which I'm seeing would be a career choice is AI ethics and policy advice, because that the responsible AI framework is extremely important now And I think all the big firms out there are doing that, but then it needs a third party to frame that, the guardrails.

Speaker 1:

So I think that would be a new field that is going to emerge, where there are experts who develop ethical guidelines, practices, policies surrounding the development and the use of generative air tech, ensuring that they're aligned with the social values, legal requirements and ethical principles. That, i think, is a new career which will emerge. And the third thing which I think is that the role of an AI explainability specialist right, whereas these are folks who are deep in NLMs, who strive to make these tools and texts more transparent, interpretable and explainable, because that is very important in understanding and trusting the AI generated. So I think these are the new fields which would come in, apart from content generators who use AI, which is still out there. So I think these are a few opportunities that folks can look out for as in when this becomes more pervasive.

Speaker 2:

Pardip, we're about out of time, but I'm not letting you off the hot seat without answering one very important last question for me. So you talked about how you use AI as a co-pilot in your writing and etc. Will there ever be a time, even a decade out, when an LLM or when an AI can do Pardip's job?

Speaker 1:

I think that is a very important question.

Speaker 1:

In my view, the technology, like LLMs, they are only there to improve human productivity. I don't think that it is going to replace my job, to be honest, but what I think is that it is so in a large organization, right, if I could do, if I needed 10 folks to do the same job. So I don't need 10 folks now, so I need to repurpose some of them because the human productivity can be improved a lot. So, instead of 10 folks doing a specific job, that would come down to maybe two or three. So I would have to ensure that the remaining seven folks are repurposed to do something else.

Speaker 1:

So, in my view, i don't think that chat, gpt or AI will replace my job, but what I think is that it is going to improve the productivity such that you I mean you could do more with less right And and and jobs, which are like critical thinking, creative, all, and also, you know, domain specific expertise. They won't be going on as soon as soon, but I think all we, all I can say at this point in time is is that you know it is a function of improving human productivity.

Speaker 2:

Well, the good news for Pradeep is that, at the pace that this technology is evolving, you're going to have a full time job explaining it to people for a long time. Hopefully you see that as positive.

Speaker 1:

Hopefully, yeah.

Speaker 2:

Yeah, pretty, this has been a lot of fun. The conversation just to sped by and I know we're just getting started. So much more we can discuss, but I do want to make sure the audience knows where they can learn more about you. Maybe read your blog and and learn more about your work.

Speaker 1:

And again, you could follow me on medium I mean, at the rate are positive men and also on LinkedIn, the same same name. So I write blog quite often and and and I love the process of having a co pilot to help me write the blogs.

Speaker 2:

I encourage everyone to go read more about Pradeep. You'll get excited about his work, just like I did. Pretty thanks again. This has been been a lot of fun hanging out. Thank you, dad. Well, that's, that's all the time we have for this week on AI and the future of work. As always, i'm your host, dan Turchin, from people rain, and we'll be back next week with another fascinating guest.

People on this episode

Head Shot

Dan Turchin

Host