John Yue (CEO & Co-Founder @ inference.ai) discusses AI workload sizing, matching GPUs to workloads, availability of GPUs vs. costs, and more.
SHOW: 815
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
Topic 1 - Our topic for today is sizing and IaaS hosting for AI/ML. We’ve covered a lot of basics lately, today we’re going to dig deeper. There is a surprising amount of depth to AI sizing, and it isn’t just speeds and feeds of GPUs. We’d like to welcome John Yue (CEO & Co-Founder @ inference.ai) for this discussion. John, welcome to the show
Topic 2 - Let’s start with sizing, I’ve talked to a lot of customers recently with my day job, and it is amazing how deep AI/ML sizing can go. First, you have to size for training/fine-tuning differently than you would for the inference stage. Second, some just think, pick the biggest GPUs you can afford and go. How should your customers approach this? (GPU’s, software dependencies, etc.)
Topic 2a - Follow-up question what are the business side, what are the business parameters that need to be considered? (budget, cost efficiency, latency/response time, timeline, etc.)
Topic 3 - The whole process can be overwhelming and as we mentioned, some organizations may not think of everything. You recently announced a chatbot to help with this exact process, ChatGPU. Tell everyone a bit about that and how it came to be.
Topic 4 - This is almost like a match-making service, correct? Everyone wants an H100, but not everyone needs or can afford an H100.
Topic 5 - How does GPU availability play into all of this? NVIDIA is sold out for something like 2 years at this point; how is that sustainable? Does everything need to run on a “Ferrari class” NVIDIA GPU?
Topic 6 - What’s next in the IaaS for AI/ML space? What does a next-generation data center for AI/ML look like? Will the Industry move away from GPUs to reduce dependence on NVIDIA?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
For some strange reason, “maintenance” has been in the news quite a bit lately. Is there ever a time when maintenance is enjoyable, or appreciated?
SHOW: 814
SHOW TRANSCRIPT: The Cloudcast #814
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
- AWS increased the price of longer-running EKS clusters by 6x
- Broadcom changes VMware licensing from perpetual to subscription
- Broadcom offers security patches to perpetual license customers
- Increasing the Kubernetes support window to 1 year
- Discovering the XZ backdoor (Oxide and Friends, podcast)
IS MAINTENANCE EVER APPRECIATED OR ENJOYABLE?
- Spent the day surrounded by maintenance activities (oil, AC, power-wash)
- The costs of maintenance are real and opportunity
- Maintenance often goes unappreciated and unseen
- Naming: Release Notes, Technical Debt, Chaos Engineering
TECHNICAL DEBT VS. MAINTENANCE
- Should we encourage a lack of maintenance vs. innovation as a priority?
- Should we encourage active maintenance with lower hard costs?
- Is there a way to put respect on maintenance? (e.g. OSS maintainers)
- Do we undervalue maintenance (e.g. Backup/Recovery, DisasterRecovery, etc.)?
- What maintenance best practices do you use? What are the good and bad of them?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Kalyan Veeramachaneni (@kveeramac, CEO/Founder @DataCebo) discusses the generation and value proposition of synthetic data for GenAI.
SHOW: 813
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
Topic 2 - First, for those not familiar, what is synthetic data? What is the use case and need? What problem is it solving today?
Topic 2a - Hopefully, listeners out there are making the connection to the advantages of GenAI for synthetic data, but take us through your original concept at MIT and the history of Synthetic Data Vault (SDV).
Topic 3 - We recently did a show on the security and privacy of training LLMs where we covered the need to mask PII for the training of models for compliance. I can also see bias issues coming into play or maybe training data that doesn’t exist in the real world (weather models example). What are some of the use cases that you’ve seen require synthetic data sets. Are there certain industries (healthcare, financials, etc.) that benefit?
Topic 4 - You were designing this based on GenAI before GenAI was “cool”. How has the rise of LLMs impacted this space?
Topic 5 - If I understand this correctly, organizations would put generative AI on a problem to describe a need for a data set, the model would then evaluate the available data and create a quality synthetic or “fake” dataset. How would the organization verify the quality of the dataset? How would they validate that a synthetic data set is as good as the original data?
Topic 6 - Let’s talk about resources for a bit. When I think of GenAI and training, I think of large amounts of hardware and in particular GPU’s that might have limited availability. Is that true here? Also, is this on-prem or in the cloud, or both?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
With the AI Era upon us, the challenge of trying to learn and make sense of the technologies, the business opportunities and the pitfalls is both exciting and equally terrifying.
SHOW: 812
SHOW TRANSCRIPT: The Cloudcast #812
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
WHEN WAS THE LAST TIME YOU REALLY LEARNED SOMETHING NEW IN TECH?
- Cloud was all new, but not completely
- The pace of cloud innovation seemed fast, but not in comparison to AI
- The focus was around one cloud vs. a new source every week
PICK A TOPIC, READ, BE CONFUSED, LOOK FOR CONFIRMATION, RINSE, REPEAT
- Where to start the learning process?
- Where to pick a source of learning?(Coursera, MIT online, blogs/videos/search, etc.)
- How to determine legitimacy of the sources?
- How far to learn before stopping to make sense?
- Trying to relate it to something you already know?
- How and when to ask questions?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Brad Winett (President/Co-founder @TrackItCloud) talks about platforms for entertainment and media. Topics include use cases, partnering with AWS, and creation and consulting services. We even dig into AR and VR a bit at the end.
SHOW: 811
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
Topic 1 - Our topic for today is media and entertainment in the cloud. I don’t believe we have ever done a show specifically on this topic, and there are some considerations worth talking about. For today, we have Brad Winett, President and Co-founder at TrackIt. Brad, welcome to the show. Let’s jump right in. The media industry as a whole has undergone major change, just like many others. Most of us see it from the consumer end as a cord-cutter. What made you jump into this market and this industry specifically?
Topic 2 - Platforms and content distribution in the early days of cloud was a differentiator. I think back to Netflix, they initially had a market advantage because they were able to scale better and to more devices than anyone and even open sourced a number of internally developed items and were the AWS poster child. Over time, these user experiences have become the norm. How should people out there think about media platforms? Are we past the days of build your own?
Topic 3 - What about use cases? Media streaming is pretty broad. What does a normal customer look like? Is this big streaming services, smaller companies, etc?
Topic 4 - How much of the tech stack is AWS products and how much of the stack is custom typically? Walk us through what a media streaming stack looks like. How is this different from a SaaS provider providing a turnkey service?
Topic 5 - I know TrackIt is a big AWS partner. Give everyone an overview of the landscape of AWS Partnership these days. Do you provide mainly professional services and consulting?
Topic 6 - Where does open-source software fit into this?
Topic 7 - I feel the standard last question these days is how AI will potentially enhance or impact this is some way.
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
What will be the adoption patterns for AI within the Enterprise? Will it follow the early days of Cloud Computing, or will new and different patterns emerge?
SHOW: 810
SHOW TRANSCRIPT: Cloudcast #810
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
WHAT WERE THE PATTERNS FOR ENTERPRISE IT AND CLOUD?
- Shadow IT
- High-scalability or Short-term Projects (and experimentation)
- Migration via “Cloud First” initiatives
- Difficult stuff came last
WHAT’S DIFFERENT ABOUT AI vs. CLOUD?
- CPU to CPU was easier to calculate vs. CPU + GPU
- Have we learned any lessons about how to value people's productivity?
- Does Enterprise AI need a Crawl, Walk, Run scenario? Do they need to be sequential and linked?
- Are Enterprise AI use-cases well defined?
- How long is the Enterprise willing to fail at experiments?
- What’s the Enterprise tolerance for GenAI “flaws” (e.g. hallucinations, lack of citations, etc.)
- Will GenAI rejuvenate Predictive AI projects in the Enterprise?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Aaron (@aarondelp) and Brian (@bgracely) discuss the biggest tech stories, announcements, and trends from March 2024.
SHOW: 809
SHOW TRANSCRIPT: https://bit.ly/cloudcast-809-transcript
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW NOTES:
Segments Covered in the Show:
- Good Old Fashioned Cloud News
- The AI Innovation Continues - Speed Round
- Trend 1 - KubeCon EU 2024
- Trend 2 - Microsoft continues to branch out from OpenAI as a partner
- Trend 3 - NVIDIA held a pretty massive GTC event
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Let’s dig into the mindset behind the VMware price increases that have been happening since Broadcom acquired the company in 2023.
SHOW: 808
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Want to win a Tesla Cybertruck or $100,000? Enter the WS02 Choreo Code Challenge (before April 30th)
- WSO2 Choreo - Why build a platform? Just add developers instead
SHOW NOTES:
- What VMware might look like under Broadcom (Eps.629, June 2022)
- What to do about VMware (Day Two Cloud, Feb 2024)
- VMware 10X Price Increase Leaves Companies Looking for Alternatives
- VMware pricing in the Broadcom Era
BROADCOM IS FITTING VMWARE INTO THEIR BUSINESS MODEL
- At least with acquisitions, Broadcom has a well-defined set of business metrics they expect from their companies
- Broadcom acts somewhere like private equity in terms of investment, innovation, revenue generation
IT'S A BOLD STRATEGY BROADCOM, LET’S SEE IF IT PAYS OFF FOR THEM
- In essence, the bet is that there is no replacement for VMware in the Enterprise
- The timing is interesting with the shifting of budgets for AI projects
- It puts customers in a position to pay more for limited upside, but having to distinctly cut other areas of their technology budget (risk the business)
- Customers have some options, but again they risk the business (e.g. hold off on security patches)
- Once a company accepts the new pricing, what guarantees are there about no additional big increases in the future?
- How much will this impact the longer-term vendor-customer relationship?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Sean Falconer (@seanfalconer, Head of Dev Relations @SkyflowAPI, Host @software_daily) talks about security and privacy of LLMs and how to prevent PII (personally identifiable information) from leaking out
SHOW: 807
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Want to win a Tesla Cybertruck or $100,000? Enter the WS02 Choreo Code Challenge (before August 30th)
- WSO2 Choreo - Why build a platform? Just add developers instead
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
Topic 1 - Our topic for today is the security and privacy LLMs. What’s Sean’s origin story?
Topic 2 - Let’s dig into LLM security and privacy. We see this concern a lot on the podcast and we’ve touched on it with various past shows, but we haven’t dug in deep. First, let’s frame the problem. What are we talking about when we talk about LLM security and privacy?
Topic 3 - First, there is a fear that customer PII information might leak out. Second, company IP or confidential into might leak out related to products or offerings. We’ve seen examples of both to date. This could be exposed in the form of integration into a model (query it for the answer) or in the fine-tuning or RAG stage. Either one could lead to compliance issues, lost rev etc. But, that same data at risk is the potential differentiation of the models. How do you both mask the data but take advantage of the data?
Topic 4 - One thing I’ve noticed is many orgs only think about privacy in relation to the fine-tuning stage where they are taking a broad model and making it company specific. It is about much more than that though. Just like standard software development, we have different stages. How is the data collected and stored, how is it used for training and fine-tuning, how is it used after deployment and during interaction stage, etc. How should security and privacy be handled across all phases?
Topic 5 - Let’s talk beyond LLMs for a bit. What about Data Lakes and Data Warehousing? I see this as a problem across all big data, correct?
Topic 6 - How does API security fit into this? Much of what we are talking about is at the storage and retrieval level. But, increasingly we see API issues exposing data. How does that fit in here?
Topic 7 - Let’s talk podcasts, we had Jeff, the previous host of Software Engineering Daily on a few times. How are things over at Software Engineering Daily? Tell everyone a bit about the show.
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
If you’re planning to deploy AI for your business, here’s 5 important capabilities your business needs from the cloud era in order to be successful.
SHOW: 806
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Want to win a Tesla Cybertruck or $100,000? Enter the WS02 Choreo Code Challenge (before April 30th)
- WSO2 Choreo - Why build a platform? Just add developers instead
SHOW NOTES:
- The Year Ahead in Cloud and Expectations for AI in 2024 (Altitude podcast)
IF YOU WANT TO DO AI WELL, YOU NEED TO HAVE DONE CLOUD WELL
- When we first started doing cloud, the smart people would say, “If you don’t do IT well today, you won’t do cloud well in the future”.
- The same pattern will repeat itself with AI
5 IMPORTANT CLOUD CAPABILITIES NEEDED TO SUCCEED WITH AI
- Automate everything - Make automation mission-critical
- Build the right abstractions and flexibilities (e.g. sharing GPUs, Devtools, etc.)
- Leverage platforms to bring together Data Science, MLOps and AppDev
- Know how much things cost (e.g. FinOps)
- Socialize success, and socialize learnings across teams
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Krish Ramineni (@krishramineni, CEO/Founder of @Firefliesai) talks about what it is like to build an AI product company in both the pre-LLM era as well as post-LLMs. We also discuss privacy and security concerns and AI behind the scenes.
SHOW: 805
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Want to win a Tesla Cybertruck or $100,000? Enter the WS02 Choreo Code Challenge (before April 30th)
- WSO2 Choreo - Why build a platform? Just add developers instead
SHOW NOTES:
Topic 1 - Welcome to the show. Before diving into today’s discussion, tell us a little about your background.
Topic 2 - Our show and listeners tend to be interested and employed in the Enterprise infrastructure and AI/ML space. Some may find it surprising that we are talking today, but we wanted to really dig into how an up-and-coming AI company provides value at scale from individuals all the way to large enterprises. What goes into both building the product as well as taking that product to market? So, let’s start there. You recently posted about “Free AI” on LinkedIn. What was the problem you were trying to solve, and how did that influence the product you built?
Topic 3 - As the foundational models in the industry keep improving and are going multi-modal, do you worry that the LLMs of the world might push out specialized models? How do you think about staying ahead of the curve? How does something like GPU shortages or big companies like Meta purchasing thousands at a time impact your decisions?
Topic 4 - Fireflies.ai is all about the abstraction of the technology away from the user. They have no idea (and shouldn’t) about the back end and everything “behind the curtain”. How do you think about this abstraction layer from a product standpoint?
Topic 5 - Now, let’s talk about PLG vs. traditional Enterprise software sales models. You did another post about that recently. We’ve worked in environments selling both (sometimes at the same time), and they are very different motions. Do you feel both are needed to build an AI company?
Topic 6 - How does Security and compliance with IT departments fit into all of this? I’ve spoken to customers that have a policy of no AI tools at the personal level for instance or maybe client, company and private data might be at risk and only certain tools are vetted and approved. I’ve seen other companies only allow tools licensed by their corp IT. How do you navigate this issue? How does something like GDPR play here?
Topic 7 - Last question, another AI specific concern we hear about is companies training models on user data. What is your thoughts here? How does a company fine tune and train new models and products but keep customer and company privacy from leaking out?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
For years, the CNCF has been the central governance body for cloud-native projects. But are there too many projects now? What if the CNCF was less governance and more like private equity?
SHOW: 804
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Want to win a Tesla Cybertruck or $100,000? Enter the WS02 Choreo Code Challenge (before April 30th)
- WSO2 Choreo - Why build a platform? Just add developers instead
SHOW NOTES:
WHY DOESN’T THE CNCF RECOMMEND A CLOUD-NATIVE STACK?
- Originally the CNCF was just trying to get projects to use them for governance.
- Many people wanted them to “define” a cloud-native stack.
- Defining a stack would have held back their business model - accepting projects and adding sponsors
HOW MANY PROJECTS WOULD GET “CNCF APPROVED” IF THEY TOOK A PRIVATE EQUITY APPROACH?
- CNCF currently has 184 projects, up 4x over the last 4 years.
- 14% graduates, 20% incubating, 62% sandbox
- Does the CNCF suffer from the “Big Tent” problem that caused so many issues with OpenStack?
- KubeCon keynotes are just a list of projects giving status updates - they could be an email.
- How many projects should the CNCF sponsor? How many categories should remain?
- How would a private equity group apply metrics to CNCF projects?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Eyal Solomon (@EyalSolomo44643, CEO/Founder of @lunardevapi) talks about integrating, controlling, and observability into 3rd party APIs and services. We discuss the trade-offs of integrating a 3rd party API and how it impacts simplicity and potential loss of insight.
SHOW: 803
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
Topic 1 - Welcome to the show. Before diving into today’s discussion, tell us a little about your background.
Topic 2 - Controlling 3rd party APIs is increasingly becoming an issue for many organizations. As the world gets built on APIs, consumption of another company's APIs to ingest services is critical. But, this leads to all sorts of control and potentially cost issues. Please give everyone an introduction to the problem.
Topic 3 - Does this mean 3rd Party APIs are a tradeoff? Yes, you might be able to integrate APIs for a payment system for instance quickly, or maybe the latest AI SaaS service, but in doing so won’t an organization potentially lose oversight? Where do most organizations run into problems first? This reminds me of the early days of cloud and people leaving instances running and then getting huge bills and not knowing until it was too late. Is this similar?
Topic 4 - We’ve seen a lot of products and companies tackling the issue of internal company APIs, but not 3rd party APIs. Once an organization determines they have a problem, maybe an observability problem, perhaps a cost problem, maybe a compliance issue, etc. how would they get started gaining API observability and control in their org?
Topic 5 - Is this a solution that sits in traffic flow? Does this potentially introduce latency? Is this almost like a WAF for 3rd party APIs? What kinds of policies or restrictions can be put in place?
Topic 6 - What are some of the most common use cases you’ve seen and how do you solve them? What business decisions have to be made if they decide to restrict access in some way?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
2024 is the year of business accountability across all aspects of the software industry. This means that all variations of the free tier are going away in one way or another.
SHOW: 802
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
WHY DID THE FREE TIER(S) EXIST IN THE FIRST PLACE?
- Why do free tiers exist? Marketing awareness, experimentation,
- Why is talking about making money, or profitability considered taboo?
- Why do we reconcile the love of employees and the hatred of paying for software?
WHAT ARE THE TRADE-OFFS WHEN THE FREE TIER GOES AWAY?
- Are free tiers and free open source software the same thing?
- Are “wants” and “value” the same thing?
- What should be free in the software world?
- Should end-users want the backers of a project or company to be successful?
- Will VCs continue to back companies that have free tiers, or companies that spend on marketing, or is the game needing new (undefined) rules?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Welcome to the "second monthly" Cloud News of the Month. Aaron and Brian discuss the biggest tech stories and five trends from February 2024.
SHOW: 801
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
Segments Covered in the Show:
- Good Old Fashioned Cloud News
- The AI Innovation Continues - Speed Round
- Trend 1 - Microsoft is branching out from just OpenAI as a partner, as they should
- Trend 2 - Selling Customer Data for AI training is Suddenly a Revenue Stream (and a source for lawsuits if it isn’t)
- Trend 3 - Chips are cool again - Everyone wants to go after NVIDIA, including NVIDIA
- Trend 4 - Google has more downs than ups - Will AI eat the search business?
- Trend 5 - Hugging Face leads OSS alternative AI but also has bad actors
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Lots of people want to turn their hobby or passion into a money-making activity, or as the kids call it “a side hustle”. Let’s explore how to do it, and some mistakes to avoid.
SHOW: 800
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
SO YOU WANT TO MAKE MORE MONEY?
- When do you decide to make more money?
- Is anyone willing to give you money? Is it profitable money?
HOW MUCH MORE ARE YOU WILLING TO WORK?
- Why are you doing this? What’s the goal? Is there a goal?
- Are you doing this with anyone else? Are you aligned? Always aligned?
- Are you willing to put in the hours? Does anyone else care about your hours?
- How much are you willing to invest in order to learn?
- Are you OK with failing?
- Do you understand how people make money in your niche?
- How will you address conflicts of interest? Where do those come from? Do they matter?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Ronald McCollum (@RonaldMcCollam, Solutions Engineering @GrafanaLabs) talks about updates in the observability space and learning more about Grafana and data visualization.
SHOW: 799
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and Spotify
- Keep up to date with Enterprise Tech with theCUBE
- Learn More About Azure Offerings : Learn more about Azure Migrate and Modernize & Azure Innovate!
- Azure Free Cloud Resource Kit : Step-by-step guidance, resources and expert advice, from
SHOW NOTES:
Topic 1 - Welcome to the show. Before diving into today’s discussion, tell us a little about your background.
Topic 2 - We last talked about Grafana back in 2019 and 2020. Observability continues to be a hot topic, how are you seeing the open-source community and open-source tools evolve in this space?
Topic 3 - We always hear about Grafana as a visualization tool. Grafana AND something (Grafana and Prometheus, Grafana and (insert logging/observability tool here). Is that still a fair assessment? Where does Grafana fit in a modern cloud-native observability stack these days?
Topic 4 - When you are speaking to folks out there, where does the data visualization story resonate the most in the organization, and does it become at times political and cultural (meaning cultural changes need to happen)? There can be an ROI/Business case to be made; developers integrations that will need to happen, SRE operations changes, etc. How do you get something that likely spans many different parts of the organization on board?
Topic 5 - Anytime I think about observability I think in two stages. Identification of the problem and resolution of the problem. Some tools address one or the other, and some attempt to do both. Where does Grafana fit on this continuum?
Topic 6 - I have to ask the AI question. How has AI changed or in your opinion will change observability and visualization in the near future?
Topic 7 - You’ve literally written the book on Grafana so this is a softball question. For those who are interested, how would you recommend they get started with Grafana
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
If you have to build a keynote presentation, or any presentation for a large audience, what are the key areas to focus on? Here’s some tips on story-telling and audience engagement.
SHOW: 798
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
BUILDING THE OVERALL STORY
- Think of your presentation as a story. It can be hero/villian, preacher approach (tell, tell, tell), an analogy, partially educational. Lots of different approaches work.
- Put yourself in the audience’s shoes. Who are they? Would they want to hear your story?
- What does the audience get out of your talk?
THE KEY AREAS TO FOCUS ON
- Are you bringing the right level of energy for the room? Are you educating, or motivating, or introducing new ideas?
- Are you bringing empathy for your audience, or are you trying to portray confidence/leadership?
- Are you at the right level for the audience? Technical vs. Business. Prior knowledge required? What about a mixed audience?
- How are you interacting with the audience? Do they provide any feedback (raising hands, laughing at jokes, applause, etc.)?
- Plan for 2-3 minutes per slide. Try not to waste any slides.
- If you’re introducing new ideas, don’t make people listen AND read at the same time. Keep the visual simple and easy to view.
- “Sell” the important parts. Emphasize them. Alter the pace to ensure people get it.
- Use data (stats, graphs, charts) that make concepts easier to understand, or better emphasized.
- Use comparisons - especially with new ideas.
- Demonstrations can be valuable, but make sure it’s at the right level for the audience - and that it works (pre-recorded is an option)
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Shreya Rajpal (@ShreyaR, CEO @guardrails_ai ) talks about the need to provide guardrails and validation of LLM’s, along with common use cases and Guardrail AI’s new Hub.
SHOW: 797
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Learn More About Azure Offerings : Learn more about Azure Migrate and Modernize & Azure Innovate!
- Azure Free Cloud Resource Kit : Step-by-step guidance, resources and expert advice, from migration to innovation.CloudZero – Cloud Cost Visibility and Savings
- Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and Spotify
- Keep up to date with Enterprise Tech with theCUBE
SHOW NOTES:
- Guardrails AI (homepage)
- Guardrails AI Hub
- Guardrails AI GitHub
- Guardrails AI Discord
- Shreya on TWIML podcast
- Guardrails AI on TechCrunch
Topic 1 - Welcome to the show. Before we dive into today’s discussion, tell us a little bit about your background.
Topic 2 - Our topic today is the validation and accuracy of AI with guardrails. Let’s start with the why… Why do we need guardrails for LLMs today?
Topic 3 - Where and how do you control (maybe validate is a better word) outputs from LLM’s today? What are your thoughts on the best way to validate outputs?
Topic 4 - Will this workflow work with both closed-source (ChatGPT) and opensource (Llama2) models? Would this process apply to training/fine-tuning or more for inference? Would this potentially replace humans in the loop that we see today or is this completely different?
Topic 5 - What are some of the most common early use cases and practical examples? PII detection comes to mind, violation of ethics or laws, off-topic/out of scope, or simply just something the model isn’t designed to provide?
Topic 6 - What happens if it fails? Does this create a loop scenario to try again?
Topic 7 - Let’s talk about Guardrails AI specifically. Today you offer an open-source marketplace of Validators in the Guardrails Hub, correct? As we mentioned earlier, almost everyone’s implementation and guardrails they want to implement will be different. Is the best way to think about this as building blocks using validators that are pieced together? Tell everyone a little bit about the offering
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Stu Miniman (@stu, Senior Director of Market Insights @RedHat) talks about the experience of The Sphere (@SphereVegas) in Las Vegas
SHOW: 796
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
- The Sphere (Las Vegas)
- What’s inside of The Sphere (video overview)
- Postcard from Earth (trailer)
- Stu’s “In the Clouds” interview series
Topic 1 - Welcome back to the show. I think it’s fair to say that you’re a veteran of Las Vegas. How many times do you think you’ve been out there for tech-event related things?
Topic 2 - With tech events coming back, and even non-tech events going live again, Las Vegas is going to be a destination for a lot of people around our community. What did you know about The Sphere prior to visiting this past week?
Topic 3 - Tell us about the basics of The Sphere. How do you get there? Where is it in Las Vegas? What’s the entrance like?
Topic 4 - Beyond the movies or concerts in the stadium, what else is going on in the building?
Topic 5 - Let’s talk about the actual stadium experience. What’s the seating like, what are you seeing, how’s the sound?
Topic 6 - Any tips or tricks for anyone considering a visit to The Sphere?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Marco Palladino (@subnetmarco, CTO/Co-Founder @thekonginc) talks about the evolution of APIs, the need for consolidation and integration of AI APIs, and introduces the AI Gateway.
SHOW: 795
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and Spotify
- Keep up to date with Enterprise Tech with theCUBE
SHOW NOTES:
- Kong (homepage)
- What is an AI Gateway?
- Kong launches AI Gateway (Marco's blog on Kong homepage)
- Kong introduces AI Gateway (Silicon Angle)
- Hugging Face (AI model community)
Topic 1 - Welcome to the show. Before we dive into today’s discussion, tell us a little bit about your background at Kong and prior to Kong.
Topic 2 - Now that the world is obsessed with AI and AI Models, what roles do APIs continue to play for applications?
Topic 3 - You’ve lived through the API marketplace days. Do you see a lot of differences between API marketplaces and model marketplaces (e.g. like HuggingFace)?
Topic 4 - You’ve talked recently about the concept of an AI Gateway, somewhat as an evolution of an API Gateway. Walk us through this new concept.
Topic 5 - API Gateways can provide a lot of protection for known things by looking into various headers, security keys, and packet data. Can you imagine AI Gateways being able to have insights into model data to do things like Prompt Control, Hallucination Control, etc.?
Topic 6 - We’ve seen the merging of API Gateways (North/South) and Service Mesh (East/West) traffic as new application patterns emerged. Do you think we’ll see new traffic patterns again with AI traffic and AI model interactions?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Big 3 cloud providers investing in AI with cloud credits. $7 trillion dollar big plans. VCs concerned about AI valuations. It feels like a new set of rules are being created.
SHOW: 794
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
- Learn More About Azure Offerings : Learn more about Azure Migrate and Modernize & Azure Innovate!
- Azure Free Cloud Resource Kit : Step-by-step guidance, resources and expert advice, from migration to innovation.CloudZero – Cloud Cost Visibility and Savings
SHOW NOTES:
- Microsoft invests $10B into OpenAI
- Amazon invests $b in Anthropic
- Google invests $2B in Anthropic
- Cloud GPUs
- Can you trust AI valuations, MANG investments (BG2 Podcast)
- Big Cloud deploys 1000s of GPU for AI - but revenues should be higher (El Reg)
- Tracking CAPEX by the big 3 cloud providers
- Sam Altman seeks (up to) $7 trillion for new chip building (and comps)
ARE CLOUD CREDITS THE NEW VENTURE CAPITAL?
- “Microsoft's Q2 2024 earnings report states that AI services contributed 6 percentage points of growth to Azure revenue. This is an increase from 3 percentage points in the previous quarter.”
KEEP AN EYE ON AI USAGE “TRENDS” FROM THE BIG CLOUD PROVIDERS
- Is this a problem for VCs?
- Is this “good” revenue for the cloud providers?
- Are they just smart for utilizing an asset that they can monopolize?
- How to track user-growth (e.g. OpenAI) and cloud-usage-growth (e.g. Big 3)
- How much will people invest to build an alternative to NVIDIA?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Welcome to the inaugural Cloud News of the Month. Aaron and Brian talk about the biggest tech stories from January 2024.
SHOW: 793
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and Spotify
- Keep up to data with Enterprise Tech with theCUBE
SHOW NOTES:
Topic 1 - Let’s begin with the state of work - employees vs. management.
Topic 2 - Lots of changes finally announced by Broadcom about the new VMware (here, here)
Topic 3 - We’re starting to see the end of the Unicorns of 2021. This feels like it’s going to be a quiet crash.
Topic 4 - The CNCF and Linux foundation released some project velocity numbers. Looks like Kubernetes, GitOps, Telemetry, Backstage are at the top, with some noise around Envoy/Istio/Cilium
Topic 5 - There was a lot of buzz around Basecamp’s numbers for repatriating back to their private cloud.
Topic 6 - AI Speed Round...
- Microsoft hits 3T company valuation
- Microsoft forms GenAI Team (less dependence on OpenAI)
- Meta’s Llama 3 and quadruple’s GPU’s in 2024
- GPUs might be under-utilized in the public cloud
- Google Cloud partners with Hugging Face
- OpenAI Launched GPT Store
- Apple is changing podcast terms and rules (downloads, transcripts, etc.)
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Some say the Internet has gone to sh*t. Between bundling and unbundling, how does “ensh*tification” happen, and are there ways to prevent or reduce it?
SHOW: 792
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- Learn More About Azure Offerings : Learn more about Azure Migrate and Modernize & Azure Innovate!
- Azure Free Cloud Resource Kit : Step-by-step guidance, resources and expert advice, from migration to innovation.CloudZero – Cloud Cost Visibility and Savings
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
SHOW NOTES:
- How does anyone use the Internet? (SDT eps.451)
- “Enshitification” - The 2023 Digital Word of the Year
- Enshitifcation is also known as platform decay (Wikipedia)
- How to success in business by bundling and unbundling (Jim Barksdale)
IN THE DIGITAL WORLD, IT’S EASY TO BUNDLE THINGS TOGETHER
- The only way to unlock valid is through bundling and unbundling
- There is a fine line between innovation, growth and platform creep.
- Nobody likes when free stuff goes away or gets complicated
CAPITALISM, FREE-ISM, AND ALIGNMENT OF INCENTIVES
- Expected growth rates are set by the companies.
- People in tech are always asking “what’s next?”
- There’s a difference between early adopters and mainstream
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Aneel Lakhani (@aneel, Investor, @crane_vc Crane Venture Partners) talks about the challenges of VC funding and building profitable companies based on open-source software.
SHOW: 791
CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"
SHOW SPONSORS:
- CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
- Learn More About Azure Offerings : Learn more about Azure Migrate and Modernize & Azure Innovate!
- Azure Free Cloud Resource Kit : Step-by-step guidance, resources and expert advice, from migration to innovation.CloudZero – Cloud Cost Visibility and Savings
SHOW NOTES:
- Crane VC homepage
- Aneel’s 2023 Look Ahead Show
- Aneel’s 2022 Look Ahead Show
- The Problem with Money in (Open Source) Software - Monktoberfest 2023
Topic 1 - Welcome back. You’re almost in the 5 timers club at this point, and a returning guest from our look-ahead shows in 2023. For our new listeners, tell everyone a little bit about yourself
Topic 2 - You recently gave a talk at Monktoberfest about the problem with money in software, specifically open-source software. The crux of it was that VCs require high returns on investments, and software has short windows to gain adoption.
Topic 3 - Could that talk have been given any time over the last 5+ years, or was it triggered by recent “events” with open source companies (e.g. Hashicorp)?
Topic 4 - Has VC thinking changed about software, given that open source provided essentially early marketing and awareness for “free,” but that comes with user expectations of free (cost) as well? Are there any new ideas about how to build cost-effective software moats?
Topic 5 - There are various “conspiracies” floating around that VCs negotiate “the rug pull” with companies at each funding round. Especially in the early stages (where you focus), how much is the VC focused on getting the idea off the ground vs. the validity of the business model? When does the business model discussion typically start?
Topic 6 - We’ve said before that there will never be another Enterprise Software company after VMware. Has the era of open-source enterprise startups ended, or does it just need to go through a phase of rediscovery?
FEEDBACK?
- Email: show at the cloudcast dot net
- Twitter: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Sizing AI Workloads
33:34
The Maintenance Episode
21:51
Synthetic Data for AI
25:58
The Fear and Excitement of Learning in a new era
31:53
Building Media and Streaming Platforms
31:34
Will Enterprise AI adoption patterns follow Enterprise Cloud adoption?
28:13
Cloud News of the Month - March 2024
33:24
The $69B bet against replacement
27:41
LLM Security and Privacy
26:09
Cloud Fundamentals needed for AI
21:55
Building an AI Product Company
26:16
What if the CNCF was private equity?
22:53
Integration and Observability of 3rd Party APIs
28:41
The End of the Free Tiers
24:34
Cloud News of the Month - February 2024
48:38
Anatomy of a Side Hustle
31:04
Observability and Visualizing Data with Grafana
27:52
Building a Keynote Presentation
31:28
Validation and Guardrails for LLMs
27:27
Experiencing The Sphere in Las Vegas
30:56
APIs and AI Gateways
33:29
The Curious Case of AI, Funding and Cloud Credits
24:14
Cloud News of the Month
48:08
Bundling, Unbundling and Ensh*tification
30:08
VCs and Open Source and Funding
33:29