{"version":"1.0.0","segments":[{"startTime":0.125,"endTime":3.095,"body":"I said last podcast,  that since"},{"startTime":0.125,"endTime":3.095,"body":"my background is in testing and"},{"startTime":3.095,"endTime":5.885,"body":"since we were talking about testing,"},{"startTime":3.095,"endTime":5.885,"body":"we should have an entire podcast"},{"startTime":6.145,"endTime":9.365,"body":"specifically on testing and agile."},{"startTime":9.545,"endTime":11.325,"body":"So that's what this podcast is."},{"startTime":11.635,"endTime":16.065,"body":"Testing in agile, testing and"},{"startTime":11.635,"endTime":16.065,"body":"agile, in agile, in agile and agile."},{"startTime":16.215,"endTime":20.599,"body":"So  just got my CSM certification"},{"startTime":16.215,"endTime":20.599,"body":"and in class they said you don't,"},{"startTime":20.609,"endTime":22.269,"body":"they don't have testers in scrum."},{"startTime":22.279,"endTime":23.199,"body":"They only have developers."},{"startTime":23.199,"endTime":25.069,"body":"That means we can fire"},{"startTime":23.199,"endTime":25.069,"body":"all our testers, right?"},{"startTime":25.169,"endTime":25.639,"body":"Absolutely."},{"startTime":25.639,"endTime":26.669,"body":"Who needs testers, right?"},{"startTime":26.739,"endTime":28.059,"body":"You only need developers."},{"startTime":28.139,"endTime":29.399,"body":"I didn't expect that answer that."},{"startTime":29.399,"endTime":29.619,"body":"No."},{"startTime":29.769,"endTime":29.969,"body":"Yeah."},{"startTime":30.119,"endTime":32.724,"body":"So if you're one of those people"},{"startTime":30.119,"endTime":32.724,"body":"then You really need to Think"},{"startTime":32.734,"endTime":35.794,"body":"through what the purpose of Scrum is."},{"startTime":35.824,"endTime":37.834,"body":"What are we delivering at the end of it?"},{"startTime":37.969,"endTime":41.769,"body":"Since a lot of people who listen"},{"startTime":37.969,"endTime":41.769,"body":"to this podcast probably use scrum,"},{"startTime":41.919,"endTime":43.809,"body":"let's start with the scrum guide."},{"startTime":43.819,"endTime":47.349,"body":"And I, and I think actually, the, the,"},{"startTime":43.819,"endTime":47.349,"body":"the thing that people get in their heads"},{"startTime":47.439,"endTime":50.994,"body":"right away this weird idea that you can"},{"startTime":47.439,"endTime":50.994,"body":"just get rid of all your testers because"},{"startTime":51.054,"endTime":54.874,"body":"whatever agile doesn't have testers"},{"startTime":51.054,"endTime":54.874,"body":"or, developers do testing and agile"},{"startTime":54.884,"endTime":56.324,"body":"so you don't need dedicated testers."},{"startTime":56.474,"endTime":59.514,"body":"If this comes from the Scrum Guide"},{"startTime":56.474,"endTime":59.514,"body":"in any way, shape, or form, let's go"},{"startTime":59.514,"endTime":62.134,"body":"ahead and read the Scrum Guide right"},{"startTime":59.514,"endTime":62.134,"body":"now and get this out of the way."},{"startTime":62.284,"endTime":63.604,"body":"First thing in the podcast."},{"startTime":63.744,"endTime":64.644,"body":"Yep, let's do that."},{"startTime":64.769,"endTime":68.439,"body":"There are exactly four references"},{"startTime":64.769,"endTime":68.439,"body":"to quality in the Scrum Guide."},{"startTime":68.759,"endTime":72.089,"body":"So, under the Developer section,"},{"startTime":68.759,"endTime":72.089,"body":"and by the way, Developers in the"},{"startTime":72.089,"endTime":76.529,"body":"Scrum Guide means anybody On the"},{"startTime":72.089,"endTime":76.529,"body":"scrum team that is committed to"},{"startTime":76.529,"endTime":81.199,"body":"creating any aspect of the usable"},{"startTime":76.529,"endTime":81.199,"body":"increment, which might include testers."},{"startTime":81.249,"endTime":82.789,"body":"It's anybody, exactly."},{"startTime":82.799,"endTime":88.694,"body":"So instilling quality by adhering to"},{"startTime":82.799,"endTime":88.694,"body":"definition of done is part of the job."},{"startTime":88.964,"endTime":89.724,"body":"That's number one."},{"startTime":89.734,"endTime":90.284,"body":"That's number one."},{"startTime":90.284,"endTime":91.644,"body":"You adhere to the definition of done."},{"startTime":91.654,"endTime":92.774,"body":"That's how you get quality in."},{"startTime":93.034,"endTime":94.034,"body":"Let's go to number two."},{"startTime":94.109,"endTime":95.999,"body":"This is the sprint section"},{"startTime":94.109,"endTime":95.999,"body":"of the Scrum Guide."},{"startTime":96.299,"endTime":98.889,"body":"During the sprint,"},{"startTime":96.299,"endTime":98.889,"body":"quality does not decrease."},{"startTime":99.189,"endTime":100.469,"body":"Well, that sounds pretty important."},{"startTime":100.619,"endTime":100.969,"body":"Yeah."},{"startTime":101.209,"endTime":101.549,"body":"Okay."},{"startTime":101.559,"endTime":102.379,"body":"Yeah, absolutely."},{"startTime":102.629,"endTime":105.909,"body":"And then, the retrospective"},{"startTime":102.629,"endTime":105.909,"body":"is a way to plan to increase"},{"startTime":105.909,"endTime":107.629,"body":"quality and effectiveness."},{"startTime":107.704,"endTime":109.494,"body":"It's giving the purpose"},{"startTime":107.704,"endTime":109.494,"body":"of the retrospective."},{"startTime":109.494,"endTime":110.774,"body":"So that's number three reference."},{"startTime":110.924,"endTime":115.294,"body":"And then number four is um, that in,"},{"startTime":110.924,"endTime":115.294,"body":"where it talks about a definition of done."},{"startTime":115.444,"endTime":119.614,"body":"It says it's a formal description of the"},{"startTime":115.444,"endTime":119.614,"body":"state of the increment when it meets the"},{"startTime":119.644,"endTime":122.114,"body":"quality measures required for the product."},{"startTime":122.264,"endTime":126.694,"body":"So, so we, we the team have decided"},{"startTime":122.264,"endTime":126.694,"body":"That there is a standard of quality."},{"startTime":126.694,"endTime":130.294,"body":"We've written it down in our definition"},{"startTime":126.694,"endTime":130.294,"body":"of done, and that is another way."},{"startTime":130.514,"endTime":135.039,"body":"So they just listed four, four, four"},{"startTime":130.514,"endTime":135.039,"body":"mentions of quality to get the idea"},{"startTime":135.049,"endTime":138.209,"body":"out of the way, to get the concept out"},{"startTime":135.049,"endTime":138.209,"body":"of the way real early in this podcast."},{"startTime":138.359,"endTime":140.919,"body":"The idea that quality is a core principle."},{"startTime":140.919,"endTime":145.379,"body":"Quality never decreases because you're"},{"startTime":140.919,"endTime":145.379,"body":"doing work in tiny vertical slices."},{"startTime":145.529,"endTime":150.049,"body":"Or because you're doing work in,"},{"startTime":145.529,"endTime":150.049,"body":"potentially shippable increments,"},{"startTime":150.199,"endTime":152.389,"body":"it's not that you're cutting"},{"startTime":150.199,"endTime":152.389,"body":"quality to get those increments."},{"startTime":152.439,"endTime":157.239,"body":"The idea is you never, part of your"},{"startTime":152.439,"endTime":157.239,"body":"definition of done, part of your agreement"},{"startTime":157.349,"endTime":161.724,"body":"together, your commitment is we're going"},{"startTime":157.349,"endTime":161.724,"body":"to do stuff to get stuff out quickly but"},{"startTime":161.874,"endTime":164.434,"body":"not at the expense of cutting quality."},{"startTime":164.584,"endTime":164.814,"body":"Yeah."},{"startTime":164.814,"endTime":168.824,"body":"Oftentimes, newly minted scrum masters"},{"startTime":164.814,"endTime":168.824,"body":"kind of tend to get this wrong a"},{"startTime":168.824,"endTime":170.484,"body":"little bit, cause they're focused on."},{"startTime":170.634,"endTime":173.784,"body":"The other thing, which is we"},{"startTime":170.634,"endTime":173.784,"body":"have to have a product increment"},{"startTime":173.784,"endTime":174.674,"body":"at the end of the sprint."},{"startTime":174.674,"endTime":175.344,"body":"We have to have it."},{"startTime":175.464,"endTime":176.434,"body":"There's only 10 days."},{"startTime":176.734,"endTime":178.204,"body":"So let's go ahead and do that."},{"startTime":178.254,"endTime":182.084,"body":"And they don't really put quality"},{"startTime":178.254,"endTime":182.084,"body":"at the forefront of their mind."},{"startTime":182.234,"endTime":187.214,"body":"Um, so what ends up happening is over"},{"startTime":182.234,"endTime":187.214,"body":"medium term, maybe even short term,"},{"startTime":187.234,"endTime":192.344,"body":"you end up accumulating some, technical"},{"startTime":187.234,"endTime":192.344,"body":"debt, which you never get back to."},{"startTime":192.494,"endTime":193.964,"body":"Think about that for a sec, right?"},{"startTime":193.964,"endTime":195.264,"body":"Because, that is important."},{"startTime":195.444,"endTime":199.084,"body":"When was the last time your"},{"startTime":195.444,"endTime":199.084,"body":"organization said, Take the next print"},{"startTime":199.084,"endTime":201.024,"body":"or two and just fix up tech debt."},{"startTime":201.204,"endTime":204.224,"body":"Unless the customer is screaming"},{"startTime":201.204,"endTime":204.224,"body":"in their face, this doesn't happen."},{"startTime":204.224,"endTime":207.519,"body":"Oh, I'll represent the product"},{"startTime":204.224,"endTime":207.519,"body":"side in this podcast today."},{"startTime":207.809,"endTime":210.799,"body":"It's like, although I was supposed"},{"startTime":207.809,"endTime":210.799,"body":"to represent the quality, like the QA"},{"startTime":210.799,"endTime":214.344,"body":"side, because that's my background,"},{"startTime":210.799,"endTime":214.344,"body":"on the product side, it only happens"},{"startTime":214.434,"endTime":217.964,"body":"if you've made some sort of agreement"},{"startTime":214.434,"endTime":217.964,"body":"with your product person to dedicate"},{"startTime":217.974,"endTime":221.909,"body":"some sort of percentage inside of your"},{"startTime":217.974,"endTime":221.909,"body":"capacity, whatever your capacity is."},{"startTime":221.919,"endTime":224.599,"body":"So like 25 percent of our"},{"startTime":221.919,"endTime":224.599,"body":"capacity, we're going to deal"},{"startTime":224.599,"endTime":225.719,"body":"with known problems or whatever."},{"startTime":225.804,"endTime":230.834,"body":"The reality of me saying that"},{"startTime":225.804,"endTime":230.834,"body":"number is sometimes you can do"},{"startTime":230.834,"endTime":233.234,"body":"that, but sometimes you just can't."},{"startTime":233.384,"endTime":235.024,"body":"It's a mandate at the"},{"startTime":233.384,"endTime":235.024,"body":"end of the day, right?"},{"startTime":235.044,"endTime":238.144,"body":"It's best avoided rather than"},{"startTime":235.044,"endTime":238.144,"body":"have to deal with it somehow."},{"startTime":238.294,"endTime":240.954,"body":"You know, allocating a fixed"},{"startTime":238.294,"endTime":240.954,"body":"capacity for every sprint."},{"startTime":241.104,"endTime":243.404,"body":"For me, that looks like"},{"startTime":241.104,"endTime":243.404,"body":"an opportunity cost."},{"startTime":243.624,"endTime":248.104,"body":"That 25%, whatever percentage you"},{"startTime":243.624,"endTime":248.104,"body":"use, is an opportunity cost, that"},{"startTime":248.124,"endTime":251.764,"body":"you're incurring by not delivering"},{"startTime":248.124,"endTime":251.764,"body":"something more useful, right?"},{"startTime":251.914,"endTime":254.854,"body":"Because you're basically doing"},{"startTime":251.914,"endTime":254.854,"body":"defect remediation at that point."},{"startTime":254.874,"endTime":255.124,"body":"Right."},{"startTime":255.199,"endTime":258.959,"body":"The other thing in the Scrum Guide,"},{"startTime":255.199,"endTime":258.959,"body":"before we move off of like what"},{"startTime":258.959,"endTime":263.329,"body":"the book says, it has a line that"},{"startTime":258.959,"endTime":263.329,"body":"the scrum team is cross functional."},{"startTime":263.329,"endTime":267.059,"body":"Scrum teams are cross functional,"},{"startTime":263.329,"endTime":267.059,"body":"meaning the members have all the skills"},{"startTime":267.059,"endTime":269.229,"body":"necessary to create value each sprint."},{"startTime":269.379,"endTime":273.534,"body":", so again even if you're an environment"},{"startTime":269.379,"endTime":273.534,"body":"where you say, well, my developers"},{"startTime":273.534,"endTime":276.334,"body":"do all the testing, the idea of"},{"startTime":273.534,"endTime":276.334,"body":"cross functionality is like, well,"},{"startTime":276.374,"endTime":278.324,"body":"I mean, who, who is your expert?"},{"startTime":278.474,"endTime":282.394,"body":"I mean, I understand anybody who says,"},{"startTime":278.474,"endTime":282.394,"body":"the, the, if I'm going to have any"},{"startTime":282.404,"endTime":287.014,"body":"empathy for the other side of it, like"},{"startTime":282.404,"endTime":287.014,"body":"the, the cash strapped startup scrappy"},{"startTime":287.154,"endTime":290.034,"body":"side of the world who might be listening"},{"startTime":287.154,"endTime":290.034,"body":"to this saying like, well, we just don't"},{"startTime":290.034,"endTime":291.774,"body":"have money, like we'd like to do that."},{"startTime":291.974,"endTime":295.804,"body":"We just don't have money to do it is,"},{"startTime":291.974,"endTime":295.804,"body":"I guess you're saying, I guess you're"},{"startTime":295.804,"endTime":298.824,"body":"saying at that point, like we, we,"},{"startTime":295.804,"endTime":298.824,"body":"we would like to be cross functional."},{"startTime":298.824,"endTime":299.794,"body":"We just can't afford to be."},{"startTime":299.944,"endTime":302.544,"body":"Yeah, what I'm hearing there"},{"startTime":299.944,"endTime":302.544,"body":"is we don't have money."},{"startTime":302.544,"endTime":305.354,"body":"So we're going to put something"},{"startTime":302.544,"endTime":305.354,"body":"out there that might be"},{"startTime":305.354,"endTime":307.074,"body":"crappy, but we'll fix it later."},{"startTime":307.224,"endTime":310.284,"body":"That's not going to work, especially"},{"startTime":307.224,"endTime":310.284,"body":"if you're in that startup environment"},{"startTime":310.284,"endTime":314.114,"body":"where it's very important to"},{"startTime":310.284,"endTime":314.114,"body":"land, with your customers, right?"},{"startTime":314.114,"endTime":317.169,"body":"With a product that they like,"},{"startTime":314.114,"endTime":317.169,"body":"not just simply that something"},{"startTime":317.169,"endTime":320.079,"body":"that satisfies the market, but"},{"startTime":317.169,"endTime":320.079,"body":"maybe even delights them, right?"},{"startTime":320.229,"endTime":322.689,"body":"In that environment, if"},{"startTime":320.229,"endTime":322.689,"body":"you're going to just simply..."},{"startTime":322.839,"endTime":325.899,"body":"Pay lip service to the quality out there."},{"startTime":325.989,"endTime":327.579,"body":"How's that gonna work out for you?"},{"startTime":327.679,"endTime":331.634,"body":"Well, when I was a people manager,"},{"startTime":327.679,"endTime":331.634,"body":"that era of my career, especially when"},{"startTime":331.634,"endTime":334.244,"body":"you, when you, when hiring for, Q."},{"startTime":334.244,"endTime":334.544,"body":"A."},{"startTime":334.734,"endTime":339.254,"body":"People and testers, I realized that there"},{"startTime":334.734,"endTime":339.254,"body":"is a skill set that you have to sort of"},{"startTime":339.264,"endTime":343.114,"body":"that out and find in the hiring process"},{"startTime":339.264,"endTime":343.114,"body":"and those are the people that are gonna be"},{"startTime":343.114,"endTime":347.694,"body":"really, really good in that position and"},{"startTime":343.114,"endTime":347.694,"body":"hiring for developers and hiring for Q."},{"startTime":347.704,"endTime":347.804,"body":"A."},{"startTime":347.804,"endTime":348.304,"body":"People."},{"startTime":348.304,"endTime":348.439,"body":"Yeah."},{"startTime":348.719,"endTime":351.629,"body":"Is, you're looking for very"},{"startTime":348.719,"endTime":351.629,"body":"different people, potentially."},{"startTime":351.779,"endTime":353.429,"body":"Yeah, that's a really good point."},{"startTime":353.449,"endTime":355.009,"body":"You are looking for different people."},{"startTime":355.039,"endTime":357.279,"body":"And this again is sometimes misunderstood."},{"startTime":357.279,"endTime":360.559,"body":"People just say, well, testers,"},{"startTime":357.279,"endTime":360.559,"body":"they can write code, sort of."},{"startTime":360.599,"endTime":361.559,"body":"They can write scripts."},{"startTime":361.709,"endTime":364.289,"body":"So they need, they need a"},{"startTime":361.709,"endTime":364.289,"body":"developer background, right?"},{"startTime":364.289,"endTime":367.939,"body":"They need to know some languages and have"},{"startTime":364.289,"endTime":367.939,"body":"some prowess and maybe an idea or two."},{"startTime":368.089,"endTime":369.149,"body":"I think you're right."},{"startTime":369.159,"endTime":370.469,"body":"It's a mindset issue."},{"startTime":370.479,"endTime":375.469,"body":"You need testers to have that testing"},{"startTime":370.479,"endTime":375.469,"body":"mindset, which says as we're building"},{"startTime":375.469,"endTime":380.729,"body":"it, Let's build quality in the other"},{"startTime":375.469,"endTime":380.729,"body":"side of it is we'll just get to it later"},{"startTime":380.919,"endTime":383.219,"body":"and we'll inspect for it at the end."},{"startTime":383.344,"endTime":384.764,"body":"See, that's, that's interesting."},{"startTime":384.774,"endTime":387.774,"body":"Cause again, the people management side"},{"startTime":384.774,"endTime":387.774,"body":"of it this is an interesting segue."},{"startTime":387.774,"endTime":392.214,"body":"I think we should take it now because,"},{"startTime":387.774,"endTime":392.214,"body":"I remember, when the company that I"},{"startTime":392.214,"endTime":395.824,"body":"was at decided to make a real effort"},{"startTime":392.214,"endTime":395.824,"body":"toward, integrating automation,"},{"startTime":395.894,"endTime":399.214,"body":"like we were strictly manual testing"},{"startTime":395.894,"endTime":399.214,"body":"until the point where we really"},{"startTime":399.214,"endTime":401.974,"body":"started coming into some realizations"},{"startTime":399.214,"endTime":401.974,"body":"that, if we really were going to."},{"startTime":402.124,"endTime":405.814,"body":"Elevate the level of quality we needed to"},{"startTime":402.124,"endTime":405.814,"body":"integrate automation into our practices"},{"startTime":405.814,"endTime":411.234,"body":"and then we did and we went about it and"},{"startTime":405.814,"endTime":411.234,"body":"I as a manager kind of pivoted and moved"},{"startTime":411.234,"endTime":418.484,"body":"into that, brave new world but You could"},{"startTime":411.234,"endTime":418.484,"body":"find somebody who has That that the skill"},{"startTime":418.484,"endTime":421.744,"body":"set we're talking about they did like"},{"startTime":418.484,"endTime":421.744,"body":"whatever's on the surface and whatever"},{"startTime":421.744,"endTime":425.044,"body":"they're being told is like not good"},{"startTime":421.744,"endTime":425.044,"body":"enough They need to dig a little deeper."},{"startTime":425.324,"endTime":428.904,"body":"You mean they need to chase"},{"startTime":425.324,"endTime":428.904,"body":"Those leads when they see them,"},{"startTime":429.054,"endTime":429.974,"body":"they can't help themselves."},{"startTime":429.974,"endTime":434.124,"body":"They really need to they need to know"},{"startTime":429.974,"endTime":434.124,"body":"why things don't work The way that they"},{"startTime":434.134,"endTime":437.624,"body":"expect them to work, like just being"},{"startTime":434.134,"endTime":437.624,"body":"told like just move on is just not good"},{"startTime":437.624,"endTime":439.794,"body":"enough Like there's a there's a mindset."},{"startTime":439.794,"endTime":443.704,"body":"So you find some way somebody with"},{"startTime":439.794,"endTime":443.704,"body":"that mindset, but also somebody who"},{"startTime":443.704,"endTime":449.399,"body":"has this this deep interest in solving"},{"startTime":443.704,"endTime":449.399,"body":"problems and, in, in seeing, menial tasks"},{"startTime":449.429,"endTime":453.069,"body":"automated, the kind of people that like,"},{"startTime":449.429,"endTime":453.069,"body":"they do it like once or twice and then it"},{"startTime":453.079,"endTime":457.229,"body":"bugs them that they, they can't just spend"},{"startTime":453.079,"endTime":457.229,"body":"a little more like, Hey, in the time that"},{"startTime":457.229,"endTime":458.989,"body":"it took, I've done this twice manually."},{"startTime":459.304,"endTime":463.634,"body":"In the time I've done it twice manually, I"},{"startTime":459.304,"endTime":463.634,"body":"could have figured out how to automate it."},{"startTime":463.644,"endTime":466.704,"body":"You know, finding someone"},{"startTime":463.644,"endTime":466.704,"body":"with both of those mindsets."},{"startTime":466.854,"endTime":470.044,"body":"The mindset of, I don't want to"},{"startTime":466.854,"endTime":470.044,"body":"repeat work and I'm willing to"},{"startTime":470.044,"endTime":473.124,"body":"go learn new things in order to"},{"startTime":470.044,"endTime":473.124,"body":"not have to repeat this again."},{"startTime":473.244,"endTime":478.094,"body":"And the mindset of, I'm really"},{"startTime":473.244,"endTime":478.094,"body":"bugged by not understanding why"},{"startTime":478.204,"endTime":479.634,"body":"things are the way they are."},{"startTime":479.834,"endTime":482.054,"body":"I need to dig into the"},{"startTime":479.834,"endTime":482.054,"body":"system and figure it out."},{"startTime":482.334,"endTime":483.374,"body":"Both of those mindsets together."},{"startTime":483.554,"endTime":484.704,"body":"You can find that and develop it."},{"startTime":484.854,"endTime":487.084,"body":"Um, I don't know if it's"},{"startTime":484.854,"endTime":487.084,"body":"more difficult or less."},{"startTime":487.209,"endTime":490.279,"body":"I probably could think about that"},{"startTime":487.209,"endTime":490.279,"body":"and have that as a separate session."},{"startTime":490.279,"endTime":494.439,"body":"You know, all your, basically,"},{"startTime":490.279,"endTime":494.439,"body":"are your, are your QA analysts the"},{"startTime":494.439,"endTime":496.669,"body":"same mindset of your QA engineers?"},{"startTime":496.819,"endTime":500.089,"body":"Because QA engineers have a different"},{"startTime":496.819,"endTime":500.089,"body":"career track of like software,"},{"startTime":500.139,"endTime":504.469,"body":"S D E T, the software developer"},{"startTime":500.139,"endTime":504.469,"body":"and test type of career track."},{"startTime":504.544,"endTime":506.964,"body":"Anyway, like we're getting kind"},{"startTime":504.544,"endTime":506.964,"body":"of off tangent, but, is, is"},{"startTime":506.964,"endTime":510.314,"body":"it, it was an interesting segue"},{"startTime":506.964,"endTime":510.314,"body":"because I've, I've specifically."},{"startTime":510.639,"endTime":517.029,"body":"In my role of, of, people leadership"},{"startTime":510.639,"endTime":517.029,"body":"in QA and testing, I, I can tell"},{"startTime":517.029,"endTime":520.219,"body":"you that those are different"},{"startTime":517.029,"endTime":520.219,"body":"mindsets that you have to find."},{"startTime":520.449,"endTime":520.929,"body":"Absolutely."},{"startTime":520.959,"endTime":521.589,"body":"For sure."},{"startTime":521.589,"endTime":521.949,"body":"They are."},{"startTime":521.949,"endTime":522.179,"body":"Right."},{"startTime":522.179,"endTime":526.099,"body":"So that, that kind of hound dog"},{"startTime":522.179,"endTime":526.099,"body":"mentality, go figure out what's wrong."},{"startTime":526.109,"endTime":528.679,"body":"Go keep sniffing right until"},{"startTime":526.109,"endTime":528.679,"body":"you figure out what's wrong."},{"startTime":528.754,"endTime":532.374,"body":"as opposed to kind of just say, Hey,"},{"startTime":528.754,"endTime":532.374,"body":"something's wrong, throw it back at"},{"startTime":532.374,"endTime":534.564,"body":"the developer that worked on it last."},{"startTime":534.714,"endTime":537.294,"body":"That's really that junior level mentality."},{"startTime":537.294,"endTime":541.714,"body":"So once people, mature beyond that,"},{"startTime":537.294,"endTime":541.714,"body":"they're looking at holistically why"},{"startTime":541.714,"endTime":544.884,"body":"this is happening, and then they,"},{"startTime":541.714,"endTime":544.884,"body":"to your point, they get frustrated"},{"startTime":544.894,"endTime":548.944,"body":"having to do things manually, and"},{"startTime":544.894,"endTime":548.944,"body":"they learn the skills to automate."},{"startTime":549.094,"endTime":554.194,"body":"I think it's a natural progression for"},{"startTime":549.094,"endTime":554.194,"body":"somebody who is a manual tester to want"},{"startTime":554.194,"endTime":557.844,"body":"to automate things, and that should"},{"startTime":554.194,"endTime":557.844,"body":"be encouraged as much as possible."},{"startTime":557.994,"endTime":562.974,"body":"It used to be the case where people"},{"startTime":557.994,"endTime":562.974,"body":"carried Both skill sets on a team"},{"startTime":562.974,"endTime":566.414,"body":"like we have an automation tester, but"},{"startTime":562.974,"endTime":566.414,"body":"we also have manual testers, right?"},{"startTime":566.564,"endTime":567.474,"body":"That's an overhead."},{"startTime":567.714,"endTime":569.654,"body":"I think it's also a function of maturity."},{"startTime":569.804,"endTime":575.714,"body":"I think lately what I'm seeing is you"},{"startTime":569.804,"endTime":575.714,"body":"have a QA person who is obviously very"},{"startTime":575.714,"endTime":579.134,"body":"well versed on how to test manually,"},{"startTime":575.714,"endTime":579.134,"body":"but they're also an automation person."},{"startTime":579.144,"endTime":584.734,"body":"They write automation scripts and they"},{"startTime":579.144,"endTime":584.734,"body":"get engaged early on with developers"},{"startTime":584.819,"endTime":588.399,"body":"to the point where they really"},{"startTime":584.819,"endTime":588.399,"body":"don't want to wait until something"},{"startTime":588.399,"endTime":589.769,"body":"is thrown over the wall at them."},{"startTime":589.919,"endTime":594.619,"body":"So right after the sprint starts,"},{"startTime":589.919,"endTime":594.619,"body":"they huddle up with the developers"},{"startTime":594.619,"endTime":596.244,"body":"and say, what are we doing right?"},{"startTime":596.274,"endTime":599.204,"body":"Let me take some of these"},{"startTime":596.274,"endTime":599.204,"body":"automation, pieces and start"},{"startTime":599.204,"endTime":600.604,"body":"writing that or even better."},{"startTime":600.754,"endTime":602.814,"body":"Let me write some of"},{"startTime":600.754,"endTime":602.814,"body":"these unit tests for you."},{"startTime":602.824,"endTime":605.834,"body":"You write others, and we can"},{"startTime":602.824,"endTime":605.834,"body":"kind of work together, aka."},{"startTime":605.984,"endTime":609.414,"body":"And once they do that for a little"},{"startTime":605.984,"endTime":609.414,"body":"while, you can start seeing more"},{"startTime":609.414,"endTime":613.174,"body":"developers and more, testing type"},{"startTime":609.414,"endTime":613.174,"body":"folks get engaged in that and move"},{"startTime":613.174,"endTime":616.904,"body":"away from pair to mob programming"},{"startTime":613.174,"endTime":616.904,"body":"where everybody owns everything."},{"startTime":617.054,"endTime":620.514,"body":"And I know this doesn't often sit well"},{"startTime":617.054,"endTime":620.514,"body":"with, like the PMOs of the world where"},{"startTime":620.514,"endTime":625.174,"body":"they want to assign things and have one,"},{"startTime":620.514,"endTime":625.174,"body":"one net to choke type of scenario, but"},{"startTime":625.324,"endTime":627.444,"body":"These teams are doing really, really well."},{"startTime":627.594,"endTime":629.214,"body":"So there's something going on there."},{"startTime":629.454,"endTime":632.824,"body":"To go straight into the idea"},{"startTime":629.454,"endTime":632.824,"body":"of this category, that the, the"},{"startTime":632.854,"endTime":637.429,"body":"testing, mindset is different than"},{"startTime":632.854,"endTime":637.429,"body":"the typical developer's mindset?"},{"startTime":637.554,"endTime":638.944,"body":"Is that like an overblown statement?"},{"startTime":638.964,"endTime":641.314,"body":"Is that like an overused statement"},{"startTime":638.964,"endTime":641.314,"body":"or is that actually kind of true?"},{"startTime":641.394,"endTime":643.074,"body":"No, I, I, I think it's true."},{"startTime":643.384,"endTime":644.794,"body":"And here's the reason why."},{"startTime":644.834,"endTime":647.604,"body":"I've seen the other side, which is..."},{"startTime":647.889,"endTime":649.959,"body":"Is to say, they get"},{"startTime":647.889,"endTime":649.959,"body":"something thrown at them."},{"startTime":650.049,"endTime":653.539,"body":"The developer says, I've assigned"},{"startTime":650.049,"endTime":653.539,"body":"this to you, Fred or Mary, right?"},{"startTime":653.539,"endTime":654.389,"body":"Go test this."},{"startTime":654.649,"endTime":657.599,"body":"And they test it and the first"},{"startTime":654.649,"endTime":657.599,"body":"thing that falls over, they say,"},{"startTime":657.599,"endTime":658.749,"body":"Oh, well, this doesn't work."},{"startTime":658.899,"endTime":660.549,"body":"And they assign it back at them, right?"},{"startTime":660.729,"endTime":661.539,"body":"I've seen that."},{"startTime":661.689,"endTime":663.409,"body":"I call it the ping pong game, right?"},{"startTime":663.449,"endTime":668.859,"body":"But the other side is the tester"},{"startTime":663.449,"endTime":668.859,"body":"isn't looking to ascribe blame."},{"startTime":668.939,"endTime":670.209,"body":"They're looking to see."},{"startTime":670.359,"endTime":674.309,"body":"What was missed and then they're"},{"startTime":670.359,"endTime":674.309,"body":"looking to educate the team as a"},{"startTime":674.309,"endTime":677.899,"body":"whole to say, if we work differently,"},{"startTime":674.309,"endTime":677.899,"body":"we can avoid this in the future."},{"startTime":678.049,"endTime":681.819,"body":"So then what happens is you"},{"startTime":678.049,"endTime":681.819,"body":"really evolve along that maturity"},{"startTime":681.819,"endTime":685.599,"body":"curve and they're helping the"},{"startTime":681.819,"endTime":685.599,"body":"process become better over time."},{"startTime":685.674,"endTime":690.979,"body":"This is part of how do QA people and"},{"startTime":685.674,"endTime":690.979,"body":"or testers, add quality to the product?"},{"startTime":690.979,"endTime":694.679,"body":", one of my gripes with, people who"},{"startTime":690.979,"endTime":694.679,"body":"think that they can operate with"},{"startTime":694.679,"endTime":697.259,"body":"like one test or for 10 developers"},{"startTime":694.679,"endTime":697.259,"body":"or whatever, you know what I mean,"},{"startTime":697.259,"endTime":699.009,"body":"like a weird ratios like that."},{"startTime":699.279,"endTime":702.439,"body":"Um, and I'm not going to get into"},{"startTime":699.279,"endTime":702.439,"body":"ratios in the podcast cause like every"},{"startTime":702.439,"endTime":704.889,"body":"company is a little bit different"},{"startTime":702.439,"endTime":704.889,"body":"on what your ratio can be, right?"},{"startTime":705.039,"endTime":708.529,"body":"If you have a tester, you already talked"},{"startTime":705.039,"endTime":708.529,"body":"about sitting down and peering with"},{"startTime":708.529,"endTime":711.939,"body":"them to write your, even if all your"},{"startTime":708.529,"endTime":711.939,"body":"tests are 100 percent of your tests are"},{"startTime":711.939,"endTime":716.189,"body":"automated, maybe, maybe, maybe not 100,"},{"startTime":711.939,"endTime":716.189,"body":"but if your company is dedicated to"},{"startTime":716.189,"endTime":721.544,"body":"automation, your testers can help unburden"},{"startTime":716.189,"endTime":721.544,"body":"the developer from having to think about"},{"startTime":721.574,"endTime":724.704,"body":"tests in addition to all the rest of"},{"startTime":721.574,"endTime":724.704,"body":"the things that they're thinking about."},{"startTime":724.854,"endTime":728.534,"body":"So the tester can be completely focused"},{"startTime":724.854,"endTime":728.534,"body":"on how are we going to test this?"},{"startTime":728.844,"endTime":731.984,"body":"And the developer is focused completely"},{"startTime":728.844,"endTime":731.984,"body":"on how are we going to code and create"},{"startTime":731.984,"endTime":736.194,"body":"this and where are all the edge cases"},{"startTime":731.984,"endTime":736.194,"body":"and where are all the paths, the little"},{"startTime":736.194,"endTime":739.574,"body":"perpendicular paths where that can"},{"startTime":736.194,"endTime":739.574,"body":"take us off course and then sitting"},{"startTime":739.574,"endTime":741.604,"body":"together, you're truly exploring."},{"startTime":741.754,"endTime":745.594,"body":"All of the options, and knocking"},{"startTime":741.754,"endTime":745.594,"body":"it out when, when it's first coded."},{"startTime":745.804,"endTime":748.774,"body":"Cause like, I think we, there really"},{"startTime":745.804,"endTime":748.774,"body":"shouldn't be any pushback on the"},{"startTime":748.784,"endTime":750.554,"body":"concept that fixing something."},{"startTime":750.704,"endTime":756.824,"body":"At the time of its first coding is"},{"startTime":750.704,"endTime":756.824,"body":"the least expensive method of solving"},{"startTime":756.824,"endTime":760.654,"body":"problems as opposed to catching defects"},{"startTime":756.824,"endTime":760.654,"body":"when they're out in production and"},{"startTime":760.654,"endTime":764.534,"body":"then bringing them back in, taking a"},{"startTime":760.654,"endTime":764.534,"body":"section of our velocity and dedicating"},{"startTime":764.544,"endTime":768.784,"body":"to solve that problem or go around that"},{"startTime":764.544,"endTime":768.784,"body":"technical debt or whatever you mean"},{"startTime":768.794,"endTime":772.664,"body":"that that's the most expensive after"},{"startTime":768.794,"endTime":772.664,"body":"it's already in production, already in"},{"startTime":772.664,"endTime":776.044,"body":"a branch, already in the release that"},{"startTime":772.664,"endTime":776.044,"body":"went to prod dealing with the change."},{"startTime":776.094,"endTime":777.444,"body":"That's the most expensive way."},{"startTime":777.444,"endTime":777.604,"body":"Okay."},{"startTime":777.604,"endTime":777.624,"body":"Thank you."},{"startTime":777.944,"endTime":780.324,"body":"Just like oh, we're gonna develop"},{"startTime":777.944,"endTime":780.324,"body":"this new website and put it out"},{"startTime":780.324,"endTime":781.314,"body":"and see if customers like it."},{"startTime":781.504,"endTime":785.948,"body":"That's the most expensive way of"},{"startTime":781.504,"endTime":785.948,"body":"testing if your idea is good, or crap."},{"startTime":785.948,"endTime":787.268,"body":"I mean, this has already been proven."},{"startTime":787.268,"endTime":789.618,"body":"So yeah, I think we can kind"},{"startTime":787.268,"endTime":789.618,"body":"of move beyond that, right?"},{"startTime":789.618,"endTime":792.108,"body":"It's an order of magnitude"},{"startTime":789.618,"endTime":792.108,"body":"cheaper to fix it."},{"startTime":792.148,"endTime":796.348,"body":"At the source, then, later what people"},{"startTime":792.148,"endTime":796.348,"body":"don't think about is the ancillary things."},{"startTime":796.628,"endTime":800.698,"body":"It's not just that the customers finding"},{"startTime":796.628,"endTime":800.698,"body":"this thing and you're fixing it later."},{"startTime":800.698,"endTime":802.808,"body":"And there's this opportunity"},{"startTime":800.698,"endTime":802.808,"body":"cost I mentioned earlier."},{"startTime":802.958,"endTime":803.978,"body":"It's the rep damage."},{"startTime":804.038,"endTime":808.158,"body":"Your company has already and your"},{"startTime":804.038,"endTime":808.158,"body":"product has already suffered rep"},{"startTime":808.158,"endTime":810.628,"body":"damage when reputation damage, right?"},{"startTime":810.798,"endTime":814.588,"body":"When the customer finds something,"},{"startTime":810.798,"endTime":814.588,"body":"they're already not happy about it."},{"startTime":814.738,"endTime":817.728,"body":"So you can't really put a value on that."},{"startTime":818.058,"endTime":820.618,"body":"So I think you need to think"},{"startTime":818.058,"endTime":820.618,"body":"about those things as well."},{"startTime":820.848,"endTime":824.988,"body":"But yeah, to your point about, have"},{"startTime":820.848,"endTime":824.988,"body":"freeing up the developers from the burden"},{"startTime":824.988,"endTime":829.148,"body":"of Thinking about testing and doing"},{"startTime":824.988,"endTime":829.148,"body":"testing, et cetera, in the bigger picture,"},{"startTime":829.158,"endTime":830.848,"body":"not just unit tests, stuff like that."},{"startTime":830.973,"endTime":832.683,"body":"There's some new things here now, right?"},{"startTime":832.683,"endTime":836.523,"body":"So lately, what we're seeing is"},{"startTime":832.683,"endTime":836.523,"body":"testers focusing on taking a piece of"},{"startTime":836.523,"endTime":839.933,"body":"functionality and then putting it through"},{"startTime":836.523,"endTime":839.933,"body":"load testing or performance testing"},{"startTime":840.113,"endTime":841.973,"body":"or scalability, those kinds of things."},{"startTime":842.013,"endTime":842.583,"body":"Or any NFR."},{"startTime":842.643,"endTime":844.223,"body":"Any NFR, yeah, exactly."},{"startTime":844.223,"endTime":848.453,"body":"So it's freeing up your development"},{"startTime":844.223,"endTime":848.453,"body":"folks to say, Just get it working"},{"startTime":848.483,"endTime":849.763,"body":"and then let us see how well."},{"startTime":849.793,"endTime":852.953,"body":"It's the elasticity in the product"},{"startTime":849.793,"endTime":852.953,"body":"that they're testing, right?"},{"startTime":853.103,"endTime":856.943,"body":"And when they reach a limit,"},{"startTime":853.103,"endTime":856.943,"body":"you've now learned something."},{"startTime":857.183,"endTime":857.983,"body":"Is it good enough?"},{"startTime":858.013,"endTime":858.853,"body":"Maybe it's good enough."},{"startTime":858.883,"endTime":861.283,"body":"Maybe you're only"},{"startTime":858.883,"endTime":861.283,"body":"expecting 100, 000 people."},{"startTime":861.433,"endTime":863.283,"body":"To hit this thing right at a time."},{"startTime":863.433,"endTime":866.253,"body":"Maybe it's not, maybe it's"},{"startTime":863.433,"endTime":866.253,"body":"a million plus, right?"},{"startTime":866.263,"endTime":870.733,"body":"So you've found this out ahead of"},{"startTime":866.263,"endTime":870.733,"body":"time before it's rolled out, but it's"},{"startTime":870.733,"endTime":873.423,"body":"not your developer doing this because"},{"startTime":870.733,"endTime":873.423,"body":"if they were doing that, it would"},{"startTime":873.423,"endTime":874.673,"body":"take them a certain amount of time."},{"startTime":874.963,"endTime":878.788,"body":"They may even have to learn new"},{"startTime":874.963,"endTime":878.788,"body":"tools, new skillsets perhaps to a"},{"startTime":878.788,"endTime":885.118,"body":"lesser degree, but still, so folks"},{"startTime":878.788,"endTime":885.118,"body":"to me, QA and developers jointly own."},{"startTime":885.268,"endTime":887.078,"body":"The delivery of the Sprint, right?"},{"startTime":887.228,"endTime":890.848,"body":"That Sprint goal is owned"},{"startTime":887.228,"endTime":890.848,"body":"jointly by both of them."},{"startTime":890.968,"endTime":894.698,"body":"I'm glad we're here because, this"},{"startTime":890.968,"endTime":894.698,"body":"is another one pulled, pulled"},{"startTime":894.698,"endTime":895.678,"body":"straight from the headlines."},{"startTime":895.708,"endTime":897.168,"body":"Like, oh, straight from the headlines."},{"startTime":897.168,"endTime":897.988,"body":"This plot was written."},{"startTime":898.283,"endTime":904.048,"body":"I, I worked on a mobile app one time,"},{"startTime":898.283,"endTime":904.048,"body":"that had, 100, 000 concurrent users."},{"startTime":904.248,"endTime":908.028,"body":"There was like 2 million registered"},{"startTime":904.248,"endTime":908.028,"body":"users in the system, but yet about 100."},{"startTime":908.028,"endTime":910.338,"body":"So like, that's like 5%, right?"},{"startTime":910.388,"endTime":910.578,"body":"Yeah."},{"startTime":910.578,"endTime":914.598,"body":"5 percent of the users in the system"},{"startTime":910.578,"endTime":914.598,"body":"were concurrent users, but still 100, 000"},{"startTime":914.598,"endTime":916.348,"body":"people using your mobile app at one time."},{"startTime":916.498,"endTime":917.008,"body":"That's a lot."},{"startTime":917.048,"endTime":917.938,"body":"It's a lot, yeah."},{"startTime":917.938,"endTime":918.148,"body":"It's not insignificant."},{"startTime":918.223,"endTime":919.713,"body":"It all went to one central database."},{"startTime":919.713,"endTime":923.003,"body":"And, a development, every single time the"},{"startTime":919.713,"endTime":923.003,"body":"development got to choose, they're like,"},{"startTime":923.153,"endTime":925.293,"body":"I'd rather test all my functional stuff."},{"startTime":925.443,"endTime":929.063,"body":"I want to make sure that my checkbox"},{"startTime":925.443,"endTime":929.063,"body":"is a checkbox, and it works, and"},{"startTime":929.063,"endTime":930.423,"body":"it saves a setting in the database."},{"startTime":930.423,"endTime":932.833,"body":"And, why would we ever test load tests?"},{"startTime":932.983,"endTime":933.913,"body":"You know, these things."},{"startTime":933.953,"endTime":935.033,"body":"That's what the users are for."},{"startTime":935.183,"endTime":938.113,"body":"Well, I mean, this is where,"},{"startTime":935.183,"endTime":938.113,"body":"this is where product comes in."},{"startTime":938.443,"endTime":942.503,"body":"This is where we, where having"},{"startTime":938.443,"endTime":942.503,"body":"dedicated QA people and testers can"},{"startTime":942.503,"endTime":945.593,"body":"help product, put out a quality product."},{"startTime":945.853,"endTime":950.303,"body":"Because, product probably has, especially"},{"startTime":945.853,"endTime":950.303,"body":"if you're product people, I, I often"},{"startTime":950.313,"endTime":954.133,"body":"have to At this stage of my career,"},{"startTime":950.313,"endTime":954.133,"body":"like I'm going into the, what is it?"},{"startTime":954.283,"endTime":954.803,"body":"September."},{"startTime":954.963,"endTime":955.153,"body":"Yeah."},{"startTime":955.153,"endTime":957.103,"body":"Going into the 19th year."},{"startTime":957.253,"endTime":959.893,"body":"Of working in, development,"},{"startTime":957.253,"endTime":959.893,"body":"software development."},{"startTime":960.043,"endTime":964.923,"body":"Uh, I often have to stop and remember"},{"startTime":960.043,"endTime":964.923,"body":"that, Connecting to the business is, the"},{"startTime":964.923,"endTime":966.363,"body":"most important thing to business people."},{"startTime":966.513,"endTime":966.843,"body":"You know what I mean?"},{"startTime":966.843,"endTime":970.723,"body":"Not, not, until it comes time to actually"},{"startTime":966.843,"endTime":970.723,"body":"code and create a feature, Then, then my"},{"startTime":970.723,"endTime":972.103,"body":"experience is the most important thing."},{"startTime":972.113,"endTime":974.863,"body":"Because they're like, Brian, go figure"},{"startTime":972.113,"endTime":974.863,"body":"out how to get this thing implemented."},{"startTime":974.883,"endTime":975.293,"body":"You know what I mean?"},{"startTime":975.443,"endTime":978.593,"body":"90 percent of the time, like"},{"startTime":975.443,"endTime":978.593,"body":"it's go figure out how to"},{"startTime":978.593,"endTime":979.963,"body":"get this thing implemented."},{"startTime":980.273,"endTime":983.703,"body":"But the other 10 percent of the time"},{"startTime":980.273,"endTime":983.703,"body":"is, Oh my goodness, everyone's fighting."},{"startTime":983.963,"endTime":988.093,"body":"Um, because, because they run into"},{"startTime":983.963,"endTime":988.093,"body":"something like this is, is, Oh, our"},{"startTime":988.093,"endTime":992.493,"body":"app is successful now we're in trouble"},{"startTime":988.093,"endTime":992.493,"body":"because no one ever thought that 100,"},{"startTime":992.613,"endTime":996.713,"body":"000 people would be clicking on our"},{"startTime":992.613,"endTime":996.713,"body":"mobile app at any one point together at"},{"startTime":996.713,"endTime":1000.983,"body":"the same point in time, 100, 000 people"},{"startTime":996.713,"endTime":1000.983,"body":"will be success is now our biggest enemy."},{"startTime":1001.133,"endTime":1004.858,"body":"You know, and now your testers are sitting"},{"startTime":1001.133,"endTime":1004.858,"body":"back kind of like, because they, you"},{"startTime":1004.858,"endTime":1008.158,"body":"probably have a work item in your system"},{"startTime":1004.858,"endTime":1008.158,"body":"that they've opened that says, hey, when"},{"startTime":1008.158,"endTime":1014.618,"body":"you scale over this amount, the servers"},{"startTime":1008.158,"endTime":1014.618,"body":"fall over and because your product and"},{"startTime":1014.618,"endTime":1018.473,"body":"your testers And your development are"},{"startTime":1014.618,"endTime":1018.473,"body":"not all speaking the same language."},{"startTime":1018.623,"endTime":1021.853,"body":"You don't have the, what the end"},{"startTime":1018.623,"endTime":1021.853,"body":"user needs directly in sight."},{"startTime":1021.883,"endTime":1022.473,"body":"You know what I mean?"},{"startTime":1022.483,"endTime":1024.973,"body":"You, Because you're not optimized,"},{"startTime":1022.483,"endTime":1024.973,"body":"a lot of companies are not"},{"startTime":1025.013,"endTime":1026.663,"body":"optimized for the end user."},{"startTime":1026.883,"endTime":1028.913,"body":"Again, this is , just from"},{"startTime":1026.883,"endTime":1028.913,"body":"what I've seen in my career."},{"startTime":1029.063,"endTime":1033.403,"body":"Development likes to optimize for getting"},{"startTime":1029.063,"endTime":1033.403,"body":"things out and, and off of their, off of"},{"startTime":1033.403,"endTime":1037.133,"body":"their plate as fast as possible, so they"},{"startTime":1033.403,"endTime":1037.133,"body":"can say, well, you're not waiting on us."},{"startTime":1037.283,"endTime":1039.723,"body":"They like to optimize a deadline"},{"startTime":1037.283,"endTime":1039.723,"body":"is what it boils down to."},{"startTime":1039.873,"endTime":1043.233,"body":"QA likes to optimize to say, we don't"},{"startTime":1039.873,"endTime":1043.233,"body":"have anything coming back on us to say"},{"startTime":1043.233,"endTime":1045.053,"body":"that, there's dings on the product."},{"startTime":1045.313,"endTime":1048.253,"body":"That the quality is bad because"},{"startTime":1045.313,"endTime":1048.253,"body":"that comes directly back on us."},{"startTime":1048.503,"endTime":1050.933,"body":"And then I don't even know"},{"startTime":1048.503,"endTime":1050.933,"body":"what product optimize is for."},{"startTime":1051.083,"endTime":1052.683,"body":"Buying yachts, I guess."},{"startTime":1052.963,"endTime":1053.403,"body":"I don't know."},{"startTime":1053.553,"endTime":1054.363,"body":"Yeah, I mean..."},{"startTime":1054.458,"endTime":1057.758,"body":"No, but you made a good point when you"},{"startTime":1054.458,"endTime":1057.758,"body":"started talking about this point, which"},{"startTime":1057.758,"endTime":1062.378,"body":"is really who represents the voice"},{"startTime":1057.758,"endTime":1062.378,"body":"of the customer when you're actually"},{"startTime":1062.378,"endTime":1066.168,"body":"working on the product at that point,"},{"startTime":1062.378,"endTime":1066.168,"body":"isn't it better to find it there?"},{"startTime":1066.308,"endTime":1069.738,"body":"So if, if you didn't know about the"},{"startTime":1066.308,"endTime":1069.738,"body":"product scope of this, a hundred thousand"},{"startTime":1069.738,"endTime":1071.538,"body":"people will be concurrently hitting this."},{"startTime":1071.848,"endTime":1073.298,"body":"That's one thing it's a miss."},{"startTime":1073.348,"endTime":1074.108,"body":"You didn't know about it."},{"startTime":1074.258,"endTime":1078.958,"body":"So there's a miss there, but if you"},{"startTime":1074.258,"endTime":1078.958,"body":"knew potentially there might be, then"},{"startTime":1078.978,"endTime":1083.718,"body":"that, that's where your, your testers"},{"startTime":1078.978,"endTime":1083.718,"body":"could say, let's write some automation"},{"startTime":1083.718,"endTime":1087.158,"body":"and let's, let's create an environment"},{"startTime":1083.718,"endTime":1087.158,"body":"where we can, we can hit this thing"},{"startTime":1087.158,"endTime":1092.178,"body":"really hard and see if it will even"},{"startTime":1087.158,"endTime":1092.178,"body":"stand upright when 100, 000 people do it."},{"startTime":1092.458,"endTime":1094.628,"body":"the alternative to that is muddle through."},{"startTime":1094.778,"endTime":1098.228,"body":"Go change a pool value somewhere,"},{"startTime":1094.778,"endTime":1098.228,"body":"that's not going to work for you."},{"startTime":1098.268,"endTime":1101.358,"body":"Well, I mean, the, the idea is that"},{"startTime":1098.268,"endTime":1101.358,"body":"your QA people, your testers in this"},{"startTime":1101.358,"endTime":1105.808,"body":"case, become an additional avenue"},{"startTime":1101.358,"endTime":1105.808,"body":"of feedback to your developers."},{"startTime":1105.808,"endTime":1107.728,"body":"Like the product manager may not know."},{"startTime":1107.908,"endTime":1109.628,"body":"Let's stay in the mobile"},{"startTime":1107.908,"endTime":1109.628,"body":"app world for a second."},{"startTime":1109.778,"endTime":1112.758,"body":"Typically mobile apps will"},{"startTime":1109.778,"endTime":1112.758,"body":"access the servers via an API."},{"startTime":1112.758,"endTime":1116.778,"body":"So an API, an endpoint has been"},{"startTime":1112.758,"endTime":1116.778,"body":"written for the mobile application"},{"startTime":1116.778,"endTime":1121.568,"body":"to do a specific operation, whatever"},{"startTime":1116.778,"endTime":1121.568,"body":"it is, load, load the widget for how"},{"startTime":1121.568,"endTime":1125.983,"body":"many items I have in my inventory"},{"startTime":1121.568,"endTime":1125.983,"body":"today or whatever, whatever."},{"startTime":1125.993,"endTime":1126.333,"body":"I don't know."},{"startTime":1126.353,"endTime":1127.103,"body":"I'm making up stuff."},{"startTime":1127.253,"endTime":1132.053,"body":"And that's fine, but if, if you're a"},{"startTime":1127.253,"endTime":1132.053,"body":"tester and you're using, Postman or"},{"startTime":1132.053,"endTime":1136.773,"body":"using JMeter or using some API testing"},{"startTime":1132.053,"endTime":1136.773,"body":"tool and you load up a test and you"},{"startTime":1136.773,"endTime":1140.863,"body":"just hammer the server with a hundred"},{"startTime":1136.773,"endTime":1140.863,"body":"thousand connections within three seconds"},{"startTime":1140.863,"endTime":1144.393,"body":"of each other, the, the developers"},{"startTime":1140.863,"endTime":1144.393,"body":"might say like, why would you do that?"},{"startTime":1144.543,"endTime":1146.013,"body":"Why would you ever run a test like that?"},{"startTime":1146.163,"endTime":1148.653,"body":"Said many developers to me in"},{"startTime":1146.163,"endTime":1148.653,"body":"my career many, many times."},{"startTime":1148.653,"endTime":1149.163,"body":"All the time."},{"startTime":1149.173,"endTime":1149.683,"body":"Many, many times."},{"startTime":1149.903,"endTime":1150.243,"body":"Yeah, yeah, I concur with that."},{"startTime":1150.243,"endTime":1151.093,"body":"Why would you do that?"},{"startTime":1151.393,"endTime":1154.743,"body":"I mean, the, the arbiter in that"},{"startTime":1151.393,"endTime":1154.743,"body":"case is now the product person"},{"startTime":1154.753,"endTime":1158.553,"body":"to say, Hey, listen, I'm running"},{"startTime":1154.753,"endTime":1158.553,"body":"these tests, but development is"},{"startTime":1158.553,"endTime":1160.043,"body":"saying, why would you do that?"},{"startTime":1160.043,"endTime":1163.368,"body":"It's not realistic, maybe that's not"},{"startTime":1160.043,"endTime":1163.368,"body":"written down in any of your requirements."},{"startTime":1163.388,"endTime":1167.213,"body":"Maybe there is no work item in any"},{"startTime":1163.388,"endTime":1167.213,"body":"system using any system It could be a"},{"startTime":1167.213,"endTime":1170.963,"body":"good system could be a bad system could"},{"startTime":1167.213,"endTime":1170.963,"body":"be Jira could be any system could be"},{"startTime":1170.963,"endTime":1174.653,"body":"any system There's no work item that"},{"startTime":1170.963,"endTime":1174.653,"body":"says make the system perform under"},{"startTime":1174.653,"endTime":1176.683,"body":"a hundred thousand concurrent users"},{"startTime":1177.07,"endTime":1180.89,"body":"I've been in this environment too where"},{"startTime":1177.07,"endTime":1180.89,"body":"you don't have access To go look at"},{"startTime":1180.89,"endTime":1184.52,"body":"production You don't have access to go"},{"startTime":1180.89,"endTime":1184.52,"body":"look at productions logs to go log in"},{"startTime":1184.52,"endTime":1186.22,"body":"with read only to productions database."},{"startTime":1186.37,"endTime":1190.12,"body":"Maybe you have like real secure"},{"startTime":1186.37,"endTime":1190.12,"body":"it policies or whatever, and you"},{"startTime":1190.12,"endTime":1191.16,"body":"can't go look at those systems."},{"startTime":1191.235,"endTime":1194.275,"body":"Like you built the system, but now"},{"startTime":1191.235,"endTime":1194.275,"body":"that it's a production, you don't"},{"startTime":1194.275,"endTime":1197.485,"body":"have, you're, you're, you're not a"},{"startTime":1194.275,"endTime":1197.485,"body":"big enough boy to go look at my head"},{"startTime":1197.485,"endTime":1199.005,"body":"sideways when I come across those."},{"startTime":1199.005,"endTime":1199.415,"body":"But you're right."},{"startTime":1199.655,"endTime":1200.375,"body":"They're out there, right?"},{"startTime":1200.415,"endTime":1200.995,"body":"These situations."},{"startTime":1200.995,"endTime":1203.685,"body":"It's like, why would you not have"},{"startTime":1200.995,"endTime":1203.685,"body":"your testers look at that stuff?"},{"startTime":1203.775,"endTime":1204.555,"body":"Because it's reality."},{"startTime":1204.555,"endTime":1205.195,"body":"They certainly are."},{"startTime":1205.195,"endTime":1207.845,"body":"But I mean, like the, if, if,"},{"startTime":1205.195,"endTime":1207.845,"body":"if the tester can't look at it"},{"startTime":1207.845,"endTime":1209.745,"body":"and when we're going to pretend"},{"startTime":1207.845,"endTime":1209.745,"body":"for a second in this, it's."},{"startTime":1209.9,"endTime":1213.87,"body":"Example, the tester is somewhat technical,"},{"startTime":1209.9,"endTime":1213.87,"body":"like what are the chances of the product"},{"startTime":1213.87,"endTime":1216.06,"body":"manager is going to know to go look?"},{"startTime":1216.08,"endTime":1217.06,"body":"Yeah, exactly."},{"startTime":1217.12,"endTime":1220.03,"body":"They don't go through coming"},{"startTime":1217.12,"endTime":1220.03,"body":"through log files, right?"},{"startTime":1220.21,"endTime":1223.28,"body":"Well, this is, I mean, the interesting"},{"startTime":1220.21,"endTime":1223.28,"body":"point about me in my career where"},{"startTime":1223.28,"endTime":1228.115,"body":"I am now is, I, will go push to"},{"startTime":1223.28,"endTime":1228.115,"body":"look at that stuff and I will get."},{"startTime":1228.385,"endTime":1231.775,"body":"Glances from the side, the side of"},{"startTime":1228.385,"endTime":1231.775,"body":"people's eyes have been like, What is this"},{"startTime":1231.785,"endTime":1234.345,"body":"product manager trying to get API access?"},{"startTime":1234.345,"endTime":1238.305,"body":"They're trying to get access to our"},{"startTime":1234.345,"endTime":1238.305,"body":"Splunk logs or, Elastic logs or whatever."},{"startTime":1238.305,"endTime":1239.985,"body":"Like, why, why do you need this?"},{"startTime":1240.085,"endTime":1240.795,"body":"You need data log access?"},{"startTime":1240.915,"endTime":1241.155,"body":"No."},{"startTime":1241.305,"endTime":1243.665,"body":"I'd like to know how users"},{"startTime":1241.305,"endTime":1243.665,"body":"are using the system."},{"startTime":1244.015,"endTime":1247.85,"body":"And, and you can make a, you make a"},{"startTime":1244.015,"endTime":1247.85,"body":"valid point by saying, Well, if you want"},{"startTime":1247.85,"endTime":1252.295,"body":"to know how users are using the system,"},{"startTime":1247.85,"endTime":1252.295,"body":"You should go ask the users and listen."},{"startTime":1252.355,"endTime":1253.225,"body":"That is valid."},{"startTime":1253.375,"endTime":1254.135,"body":"That is valid."},{"startTime":1254.285,"endTime":1256.665,"body":"But also you have data in the system."},{"startTime":1256.965,"endTime":1258.465,"body":"Why is that not exposed to me?"},{"startTime":1258.765,"endTime":1259.835,"body":"I should be able to see that."},{"startTime":1259.985,"endTime":1261.445,"body":"Listen, totally agree with that."},{"startTime":1261.455,"endTime":1265.215,"body":"So you can't really go and ask"},{"startTime":1261.455,"endTime":1265.215,"body":"100, 000 of your users to see if"},{"startTime":1265.215,"endTime":1268.375,"body":"they're all going to hit the end"},{"startTime":1265.215,"endTime":1268.375,"body":"point at exactly the same time."},{"startTime":1268.375,"endTime":1269.475,"body":"That's just not realistic."},{"startTime":1269.625,"endTime":1272.605,"body":"However, to your point that data's there."},{"startTime":1272.61,"endTime":1273.685,"body":"It's there in the log."},{"startTime":1273.685,"endTime":1275.725,"body":"It's there assuming you're"},{"startTime":1273.685,"endTime":1275.725,"body":"logging stuff, right?"},{"startTime":1275.725,"endTime":1276.935,"body":"So go look at your Splunk log."},{"startTime":1277.195,"endTime":1279.805,"body":"Go look at your Datadog or"},{"startTime":1277.195,"endTime":1279.805,"body":"whatever it is that you are using."},{"startTime":1279.805,"endTime":1280.225,"body":"Exactly."},{"startTime":1280.285,"endTime":1281.695,"body":"And figure that out."},{"startTime":1281.77,"endTime":1285.58,"body":"One thing I wanna talk about here is"},{"startTime":1281.77,"endTime":1285.58,"body":"the, is that little gap in between those"},{"startTime":1285.585,"endTime":1289.771,"body":"scenarios, which is people look at that"},{"startTime":1285.585,"endTime":1289.771,"body":"and then they say, Hey, listen, we had"},{"startTime":1289.772,"endTime":1294.571,"body":"20, 000 users get on at the same time,"},{"startTime":1289.772,"endTime":1294.571,"body":"and it's fine, everything worked great."},{"startTime":1294.831,"endTime":1297.841,"body":"So let's just say we may have 40."},{"startTime":1297.991,"endTime":1298.611,"body":"Double that."},{"startTime":1298.651,"endTime":1299.491,"body":"We're fine, right?"},{"startTime":1299.641,"endTime":1300.621,"body":"It's not validated."},{"startTime":1300.911,"endTime":1303.141,"body":"Because that 40, it just"},{"startTime":1300.911,"endTime":1303.141,"body":"comes out of the air."},{"startTime":1303.141,"endTime":1304.181,"body":"Somebody just makes that up."},{"startTime":1304.471,"endTime":1305.941,"body":"It was 20, we'll just double it."},{"startTime":1306.271,"endTime":1308.031,"body":"Because there's safety numbers, right?"},{"startTime":1308.041,"endTime":1312.071,"body":"So, even if twice as many users got"},{"startTime":1308.041,"endTime":1312.071,"body":"on on a Saturday, we'll be fine."},{"startTime":1312.221,"endTime":1314.281,"body":"And then, one day, it goes belly up."},{"startTime":1314.391,"endTime":1319.361,"body":"Because six times the number of users"},{"startTime":1314.391,"endTime":1319.361,"body":"went on, you didn't validate the actual"},{"startTime":1319.371,"endTime":1321.401,"body":"scope of your product ahead of time."},{"startTime":1321.526,"endTime":1324.851,"body":"What happened at a company I was"},{"startTime":1321.526,"endTime":1324.851,"body":"at one time is there actually was a"},{"startTime":1324.851,"endTime":1328.121,"body":"bug in the mobile app, like super,"},{"startTime":1324.851,"endTime":1328.121,"body":"super, super deep in the mobile app."},{"startTime":1328.271,"endTime":1331.831,"body":"And, the testers, would've had to,"},{"startTime":1328.271,"endTime":1331.831,"body":"you would've had to really, you"},{"startTime":1331.831,"endTime":1333.921,"body":"really would've had to have a very..."},{"startTime":1334.071,"endTime":1339.171,"body":"test, that happened at regular intervals,"},{"startTime":1334.071,"endTime":1339.171,"body":"like automated regular intervals to"},{"startTime":1339.181,"endTime":1343.311,"body":"expose this because, they made a change"},{"startTime":1339.181,"endTime":1343.311,"body":"in the mobile app that every time"},{"startTime":1343.311,"endTime":1346.311,"body":"someone loaded, a particular screen,"},{"startTime":1343.311,"endTime":1346.311,"body":"I don't remember what screen it was."},{"startTime":1346.436,"endTime":1348.996,"body":"it made too many requests to the server."},{"startTime":1349.186,"endTime":1353.176,"body":"So that 100, 000 concurrent"},{"startTime":1349.186,"endTime":1353.176,"body":"connections basically turned into"},{"startTime":1353.506,"endTime":1355.196,"body":"an infinite number of connections."},{"startTime":1355.406,"endTime":1358.896,"body":"And the database that was at the heart"},{"startTime":1355.406,"endTime":1358.896,"body":"of the system, it, it didn't crash."},{"startTime":1359.046,"endTime":1362.326,"body":"What it did was it queued, because it,"},{"startTime":1359.046,"endTime":1362.326,"body":"because of the nature of that database,"},{"startTime":1362.326,"endTime":1364.716,"body":"that particular technology, it queued up."},{"startTime":1364.866,"endTime":1366.586,"body":"All of those requests."},{"startTime":1366.596,"endTime":1368.306,"body":"So the users experience latency basically."},{"startTime":1368.316,"endTime":1371.646,"body":"So the users, they, extreme latency."},{"startTime":1371.646,"endTime":1374.666,"body":"Because the normal request for like,"},{"startTime":1371.646,"endTime":1374.666,"body":"whatever, 100, 000 actions to the"},{"startTime":1374.666,"endTime":1379.446,"body":"database, but basically the system"},{"startTime":1374.666,"endTime":1379.446,"body":"couldn't handle 10 times volume, but had"},{"startTime":1379.446,"endTime":1384.106,"body":"we load tested the system at 10 10x, We"},{"startTime":1379.446,"endTime":1384.106,"body":"would have we would have First of all, we"},{"startTime":1384.106,"endTime":1387.746,"body":"were caught by surprise in that situation"},{"startTime":1384.106,"endTime":1387.746,"body":"right because it was a bug sure we fix"},{"startTime":1387.746,"endTime":1391.866,"body":"a bug problem went away But for all the"},{"startTime":1387.746,"endTime":1391.866,"body":"users that that were not on the buggy"},{"startTime":1391.866,"endTime":1396.546,"body":"version or we're not on the platform that"},{"startTime":1391.866,"endTime":1396.546,"body":"had the bug They got impacted because"},{"startTime":1396.806,"endTime":1401.376,"body":"all this traffic came into the database"},{"startTime":1396.806,"endTime":1401.376,"body":"that all the applications went to."},{"startTime":1401.606,"endTime":1405.266,"body":"So, again, I bring this"},{"startTime":1401.606,"endTime":1405.266,"body":"up because of two reasons."},{"startTime":1405.266,"endTime":1408.966,"body":"Because, number one, NFRs are very"},{"startTime":1405.266,"endTime":1408.966,"body":"difficult to deal with without"},{"startTime":1409.236,"endTime":1411.076,"body":"testers kind of lobbying for them."},{"startTime":1411.226,"endTime":1414.636,"body":"Because they're usually the people with"},{"startTime":1411.226,"endTime":1414.636,"body":"that mindset that says, Oh, what happens"},{"startTime":1414.636,"endTime":1416.636,"body":"when we have twice as many normal users?"},{"startTime":1416.786,"endTime":1417.936,"body":"What happens if it does?"},{"startTime":1417.976,"endTime":1419.976,"body":"Oh, well, that's never happened before."},{"startTime":1419.976,"endTime":1420.956,"body":"So don't worry about it."},{"startTime":1420.986,"endTime":1424.156,"body":"You know, well, okay Well, at least"},{"startTime":1420.986,"endTime":1424.156,"body":"let's talk about it though, right?"},{"startTime":1424.186,"endTime":1425.686,"body":"Let's at least talk about it, right?"},{"startTime":1425.716,"endTime":1427.676,"body":"Oh, no, we don't have time to"},{"startTime":1425.716,"endTime":1427.676,"body":"talk about it We were busy."},{"startTime":1427.846,"endTime":1429.326,"body":"We got all these features"},{"startTime":1427.846,"endTime":1429.326,"body":"to implement, right?"},{"startTime":1429.476,"endTime":1433.306,"body":"The other side of this is it's in"},{"startTime":1429.476,"endTime":1433.306,"body":"an additional avenue for feedback"},{"startTime":1433.316,"endTime":1436.636,"body":"kind of like I said before it's an"},{"startTime":1433.316,"endTime":1436.636,"body":"additional avenue for feedback To"},{"startTime":1436.636,"endTime":1440.206,"body":"the developers that maybe they don't"},{"startTime":1436.636,"endTime":1440.206,"body":"get like maybe the developers get"},{"startTime":1440.356,"endTime":1443.646,"body":"feedback from executives Maybe they"},{"startTime":1440.356,"endTime":1443.646,"body":"get feedback from their development"},{"startTime":1443.656,"endTime":1445.576,"body":"managers and leads and stuff like that."},{"startTime":1445.796,"endTime":1448.656,"body":"Maybe they get feedback from the"},{"startTime":1445.796,"endTime":1448.656,"body":"product manager But maybe they"},{"startTime":1448.656,"endTime":1451.716,"body":"don't get this additional feedback"},{"startTime":1448.656,"endTime":1451.716,"body":"from other people that are highly"},{"startTime":1451.716,"endTime":1455.796,"body":"technical as well Saying hey XYZ Maybe"},{"startTime":1451.716,"endTime":1455.796,"body":"we should talk about this scenario,"},{"startTime":1455.946,"endTime":1457.036,"body":"yeah, yeah, definitely."},{"startTime":1457.036,"endTime":1461.936,"body":"I have something similar to share"},{"startTime":1457.036,"endTime":1461.936,"body":"in a situation where the connections"},{"startTime":1461.976,"endTime":1464.296,"body":"were being not terminated properly."},{"startTime":1464.636,"endTime":1468.696,"body":"So you would have, the pool was"},{"startTime":1464.636,"endTime":1468.696,"body":"being used up all the time, right?"},{"startTime":1468.846,"endTime":1471.476,"body":"And people would keep trying"},{"startTime":1468.846,"endTime":1471.476,"body":"because people are people, right?"},{"startTime":1471.476,"endTime":1472.826,"body":"They would just say, oh, it didn't work."},{"startTime":1472.856,"endTime":1473.556,"body":"Well, try it again."},{"startTime":1473.556,"endTime":1473.986,"body":"Refresh."},{"startTime":1474.006,"endTime":1475.486,"body":"Well, that just makes matters worse."},{"startTime":1475.636,"endTime":1477.266,"body":"And a developer."},{"startTime":1477.416,"endTime":1478.396,"body":"Doing their job."},{"startTime":1478.406,"endTime":1480.076,"body":"They're not going to catch that, right?"},{"startTime":1480.226,"endTime":1483.676,"body":"All they're going to do is make"},{"startTime":1480.226,"endTime":1483.676,"body":"sure that a singular test works."},{"startTime":1483.686,"endTime":1486.996,"body":"Well, they might not even have access"},{"startTime":1483.686,"endTime":1486.996,"body":"to know that they wouldn't, the pooling"},{"startTime":1486.996,"endTime":1489.956,"body":"connect, the memory on the pooling"},{"startTime":1486.996,"endTime":1489.956,"body":"connections is growing and growing."},{"startTime":1490.296,"endTime":1490.486,"body":"Right."},{"startTime":1490.506,"endTime":1493.806,"body":"They might not even have access to"},{"startTime":1490.506,"endTime":1493.806,"body":"see because they don't have, they"},{"startTime":1493.806,"endTime":1497.136,"body":"don't have access to those monitors"},{"startTime":1493.806,"endTime":1497.136,"body":"because of corporate permissions."},{"startTime":1497.316,"endTime":1497.656,"body":"That's right."},{"startTime":1497.656,"endTime":1498.056,"body":"Security."},{"startTime":1498.056,"endTime":1498.456,"body":"Yeah."},{"startTime":1498.776,"endTime":1500.096,"body":"And politics and all of that."},{"startTime":1500.096,"endTime":1500.906,"body":"Yeah, exactly."},{"startTime":1501.196,"endTime":1501.906,"body":"So there's that."},{"startTime":1502.196,"endTime":1506.156,"body":"You know, when we encountered a situation"},{"startTime":1502.196,"endTime":1506.156,"body":"similar to what you just highlighted, it"},{"startTime":1506.156,"endTime":1510.496,"body":"was a, it wasn't a case of the customer"},{"startTime":1506.156,"endTime":1510.496,"body":"experiencing latency, which is pretty bad"},{"startTime":1510.546,"endTime":1513.546,"body":"because then what follows after that is"},{"startTime":1510.546,"endTime":1513.546,"body":"timeouts and whatever else and they're"},{"startTime":1513.546,"endTime":1515.026,"body":"going to try again and again, right?"},{"startTime":1515.176,"endTime":1518.456,"body":"Uh, it was the database"},{"startTime":1515.176,"endTime":1518.456,"body":"went into a thrashing mode."},{"startTime":1518.606,"endTime":1521.266,"body":"And, that was terrible, right?"},{"startTime":1521.316,"endTime":1522.366,"body":"And they were logging."},{"startTime":1522.366,"endTime":1525.126,"body":"So the log file grew to the"},{"startTime":1522.366,"endTime":1525.126,"body":"point where it filled up."},{"startTime":1525.166,"endTime":1526.696,"body":"I mean, it was just"},{"startTime":1525.166,"endTime":1526.696,"body":"disaster after disaster."},{"startTime":1527.299,"endTime":1530.756,"body":"So a lot of these things that we're"},{"startTime":1527.299,"endTime":1530.756,"body":"highlighting born from experience."},{"startTime":1530.906,"endTime":1534.116,"body":"The developers aren't going"},{"startTime":1530.906,"endTime":1534.116,"body":"to catch for whatever reason."},{"startTime":1534.116,"endTime":1537.966,"body":"There are many reasons why, but"},{"startTime":1534.116,"endTime":1537.966,"body":"your testers could catch those"},{"startTime":1538.116,"endTime":1539.416,"body":"and probably should, right?"},{"startTime":1539.446,"endTime":1542.066,"body":"If they were given the right"},{"startTime":1539.446,"endTime":1542.066,"body":"permissions, et cetera."},{"startTime":1542.066,"endTime":1542.826,"body":"Yeah, exactly."},{"startTime":1542.976,"endTime":1546.866,"body":"So I think that alone tells you, if"},{"startTime":1542.976,"endTime":1546.866,"body":"you're not exposing these kinds of"},{"startTime":1547.156,"endTime":1551.436,"body":"bad experiences to your customers,"},{"startTime":1547.156,"endTime":1551.436,"body":"that's a good thing, surely, right?"},{"startTime":1551.436,"endTime":1554.926,"body":"I mean, because the flip side is you"},{"startTime":1551.436,"endTime":1554.926,"body":"do and the customers get really teed"},{"startTime":1554.926,"endTime":1556.846,"body":"off because they get timed out on."},{"startTime":1557.131,"endTime":1559.351,"body":"They try again, they get time down, right?"},{"startTime":1559.381,"endTime":1559.951,"body":"All of that."},{"startTime":1560.151,"endTime":1561.981,"body":"You know, the server"},{"startTime":1560.151,"endTime":1561.981,"body":"is not available today."},{"startTime":1562.001,"endTime":1563.071,"body":"Please try again later."},{"startTime":1563.131,"endTime":1564.991,"body":"It's like, no, I need this now."},{"startTime":1565.171,"endTime":1569.811,"body":"So a net net on all this is your"},{"startTime":1565.171,"endTime":1569.811,"body":"product folks aren't necessarily"},{"startTime":1569.811,"endTime":1574.321,"body":"going to be able to scope to that"},{"startTime":1569.811,"endTime":1574.321,"body":"level of technical, availability,"},{"startTime":1574.321,"endTime":1575.691,"body":"if you like, to the customer."},{"startTime":1575.691,"endTime":1575.931,"body":"Right."},{"startTime":1576.271,"endTime":1581.151,"body":"This is where a combination of"},{"startTime":1576.271,"endTime":1581.151,"body":"product and testers can really thrive."},{"startTime":1581.181,"endTime":1585.726,"body":"Yeah, well, the product people really"},{"startTime":1581.181,"endTime":1585.726,"body":"could benefit from having testers on"},{"startTime":1585.726,"endTime":1587.656,"body":"the team because they may, they may,"},{"startTime":1587.806,"endTime":1593.036,"body":"let me think about this under the guise of"},{"startTime":1587.806,"endTime":1593.036,"body":"building a quality system for customers."},{"startTime":1593.186,"endTime":1596.566,"body":"The testers may bring up things"},{"startTime":1593.186,"endTime":1596.566,"body":"that, that development may"},{"startTime":1596.566,"endTime":1597.736,"body":"not think about bringing up."},{"startTime":1597.861,"endTime":1600.041,"body":"Just from, from the perspective of."},{"startTime":1600.191,"endTime":1603.981,"body":"Like what I said earlier is like"},{"startTime":1600.191,"endTime":1603.981,"body":"development is concerned with just getting"},{"startTime":1603.981,"endTime":1606.601,"body":"things out the door as fast as possible"},{"startTime":1603.981,"endTime":1606.601,"body":"and moving on to the next thing because"},{"startTime":1606.601,"endTime":1611.161,"body":"they, especially in shops where, like a"},{"startTime":1606.601,"endTime":1611.161,"body":"development lead and a product manager"},{"startTime":1611.161,"endTime":1614.531,"body":"kind of go off in a corner and create"},{"startTime":1611.161,"endTime":1614.531,"body":"a series of like, Oh, we got to have"},{"startTime":1614.541,"endTime":1618.571,"body":"feature X and feature Y and feature Z"},{"startTime":1614.541,"endTime":1618.571,"body":"and just code these, code these team."},{"startTime":1618.681,"endTime":1621.441,"body":"We haven't even entertained the idea."},{"startTime":1621.786,"endTime":1625.826,"body":"In this podcast of what happens when"},{"startTime":1621.786,"endTime":1625.826,"body":"different segments are offshore."},{"startTime":1625.976,"endTime":1629.636,"body":"You know, well, we, we, we offshored"},{"startTime":1625.976,"endTime":1629.636,"body":"development of these features and, um."},{"startTime":1629.786,"endTime":1632.946,"body":"And, when we bring them back in"},{"startTime":1629.786,"endTime":1632.946,"body":"house, they need to go through."},{"startTime":1632.946,"endTime":1637.366,"body":"So, we haven't entertained the idea"},{"startTime":1632.946,"endTime":1637.366,"body":"that, the people writing the code and"},{"startTime":1637.366,"endTime":1641.966,"body":"writing the features and the people"},{"startTime":1637.366,"endTime":1641.966,"body":"testing them are, are separated by walls."},{"startTime":1642.276,"endTime":1644.086,"body":"Because that, that's a"},{"startTime":1642.276,"endTime":1644.086,"body":"whole different, like..."},{"startTime":1644.186,"endTime":1644.766,"body":"It is."},{"startTime":1644.786,"endTime":1647.386,"body":"You know, opportunity for issues."},{"startTime":1647.626,"endTime":1649.636,"body":"For sure, that's what"},{"startTime":1647.626,"endTime":1649.636,"body":"leads to this, right?"},{"startTime":1649.886,"endTime":1650.136,"body":"Oh, yeah."},{"startTime":1650.136,"endTime":1651.166,"body":"It's like a blame culture, you know."},{"startTime":1651.186,"endTime":1651.376,"body":"Yeah."},{"startTime":1651.376,"endTime":1653.186,"body":"Um, so, I agree with that."},{"startTime":1653.206,"endTime":1655.156,"body":"I, I think one of the"},{"startTime":1653.206,"endTime":1655.156,"body":"things I wanted to do..."},{"startTime":1655.306,"endTime":1660.276,"body":"kind of just finish up this topic with"},{"startTime":1655.306,"endTime":1660.276,"body":"is if you if you have these symptoms."},{"startTime":1660.426,"endTime":1662.426,"body":"In your company, on"},{"startTime":1660.426,"endTime":1662.426,"body":"your product, et cetera."},{"startTime":1662.576,"endTime":1667.996,"body":"And, you don't have that same"},{"startTime":1662.576,"endTime":1667.996,"body":"level of, respect, for testers."},{"startTime":1668.146,"endTime":1668.926,"body":"They're not there."},{"startTime":1668.926,"endTime":1671.386,"body":"They're not given the enough"},{"startTime":1668.926,"endTime":1671.386,"body":"room to work with, et cetera."},{"startTime":1671.536,"endTime":1672.246,"body":"Think about that."},{"startTime":1672.366,"endTime":1672.726,"body":"Right."},{"startTime":1672.756,"endTime":1676.31,"body":"See if you can connect some of the"},{"startTime":1672.756,"endTime":1676.31,"body":"dots because really your customers"},{"startTime":1676.31,"endTime":1679.9,"body":"don't care who's doing the work, all"},{"startTime":1676.31,"endTime":1679.9,"body":"they care about is getting the work of"},{"startTime":1679.9,"endTime":1681.77,"body":"the right quality at the right time."},{"startTime":1681.77,"endTime":1685.74,"body":"So a couple of the terms I want to"},{"startTime":1681.77,"endTime":1685.74,"body":"put out here, fitness for purpose."},{"startTime":1685.89,"endTime":1690.0,"body":"This doesn't mean that your product"},{"startTime":1685.89,"endTime":1690.0,"body":"has to be 100 percent bug free."},{"startTime":1690.15,"endTime":1692.62,"body":"It just has to be fit for purpose, right?"},{"startTime":1692.77,"endTime":1693.86,"body":"So the bugs are there."},{"startTime":1694.01,"endTime":1695.63,"body":"Maybe share them with your customers."},{"startTime":1695.63,"endTime":1698.4,"body":"If they're not severe enough"},{"startTime":1695.63,"endTime":1698.4,"body":"and say, here's what we got."},{"startTime":1698.55,"endTime":1700.43,"body":"Do we want to wait till these are fixed?"},{"startTime":1700.47,"endTime":1702.05,"body":"Which ones of these should be fixed?"},{"startTime":1702.2,"endTime":1703.56,"body":"What's next now, later."},{"startTime":1703.89,"endTime":1704.24,"body":"And."},{"startTime":1704.39,"endTime":1705.44,"body":"Have that discussion."},{"startTime":1705.54,"endTime":1707.26,"body":"Then they're not so surprised, right?"},{"startTime":1707.32,"endTime":1709.86,"body":"They're not going to write bad"},{"startTime":1707.32,"endTime":1709.86,"body":"reviews necessarily, because you've"},{"startTime":1709.86,"endTime":1711.37,"body":"engaged them in the discussion."},{"startTime":1711.7,"endTime":1714.17,"body":"So that's, that's one"},{"startTime":1711.7,"endTime":1714.17,"body":"term, fitness for purpose."},{"startTime":1714.32,"endTime":1717.52,"body":"And then the other one, for those of"},{"startTime":1714.32,"endTime":1717.52,"body":"you that are involved in providing"},{"startTime":1717.52,"endTime":1720.92,"body":"services, et cetera, is grade of service."},{"startTime":1721.07,"endTime":1723.67,"body":"I learned this a long, long time"},{"startTime":1721.07,"endTime":1723.67,"body":"ago when I was working in telecom."},{"startTime":1723.97,"endTime":1725.72,"body":"A service could be up, right?"},{"startTime":1725.76,"endTime":1728.29,"body":"But it doesn't perform as well."},{"startTime":1728.44,"endTime":1732.2,"body":"So, grade of service is really"},{"startTime":1728.44,"endTime":1732.2,"body":"talking about the quality."},{"startTime":1732.48,"endTime":1736.99,"body":"If you think about simple things like"},{"startTime":1732.48,"endTime":1736.99,"body":"you pick up a landline and you're talking"},{"startTime":1736.99,"endTime":1741.25,"body":"to somebody and the conversation is, the"},{"startTime":1736.99,"endTime":1741.25,"body":"connection is there, first of all, right?"},{"startTime":1741.54,"endTime":1742.81,"body":"It's a solid connection."},{"startTime":1742.96,"endTime":1745.01,"body":"However, you're hearing static sometimes."},{"startTime":1745.02,"endTime":1747.31,"body":"Sometimes there's the occasional dropout."},{"startTime":1747.46,"endTime":1749.89,"body":"The grade of service suffers right there."},{"startTime":1750.04,"endTime":1750.39,"body":"Right?"},{"startTime":1750.65,"endTime":1752.34,"body":"So that's really what I'm talking about."},{"startTime":1752.4,"endTime":1754.21,"body":"Um, yeah, the service is up."},{"startTime":1754.37,"endTime":1756.07,"body":"So you can check that box, right?"},{"startTime":1756.22,"endTime":1758.64,"body":"And so the developers can"},{"startTime":1756.22,"endTime":1758.64,"body":"say, look, it's working."},{"startTime":1758.7,"endTime":1760.97,"body":"The testers can say, yeah,"},{"startTime":1758.7,"endTime":1760.97,"body":"we tested it, it works."},{"startTime":1761.12,"endTime":1762.0,"body":"Ask the customer."},{"startTime":1762.15,"endTime":1765.3,"body":"If you're searching for this topic,"},{"startTime":1762.15,"endTime":1765.3,"body":"around the internet, I think you"},{"startTime":1765.31,"endTime":1769.79,"body":"like, you'll encounter shift left"},{"startTime":1765.31,"endTime":1769.79,"body":"testing and, and like, again, shift"},{"startTime":1769.79,"endTime":1773.27,"body":"left is like one of those terms"},{"startTime":1769.79,"endTime":1773.27,"body":"that is it's, it's, it's a buzzword."},{"startTime":1773.27,"endTime":1773.89,"body":"It's meaningless."},{"startTime":1773.89,"endTime":1774.67,"body":"It means nothing."},{"startTime":1774.89,"endTime":1778.39,"body":"But, my product manager hat to,"},{"startTime":1774.89,"endTime":1778.39,"body":"to this, I would not put a hat"},{"startTime":1778.39,"endTime":1779.94,"body":"on and mess up my wonderful hair."},{"startTime":1780.09,"endTime":1781.47,"body":"Uh, I don't have a good hair day."},{"startTime":1781.68,"endTime":1788.33,"body":"Um, it like buzzwords aside, If your, if"},{"startTime":1781.68,"endTime":1788.33,"body":"your testers, your QA people are helping"},{"startTime":1788.37,"endTime":1792.59,"body":"product represent the needs of the"},{"startTime":1788.37,"endTime":1792.59,"body":"users, like you're, you're shifting left."},{"startTime":1792.81,"endTime":1796.095,"body":"The, the idea is you're, you're"},{"startTime":1792.81,"endTime":1796.095,"body":"shifting to how can I make sure"},{"startTime":1796.095,"endTime":1798.31,"body":"this new feature benefits users?"},{"startTime":1798.46,"endTime":1802.46,"body":"And if that's what you're going into"},{"startTime":1798.46,"endTime":1802.46,"body":"your testing, and you know, before the"},{"startTime":1802.46,"endTime":1805.88,"body":"features even coded, you're talking to the"},{"startTime":1802.46,"endTime":1805.88,"body":"development team, you're talking to your"},{"startTime":1805.88,"endTime":1807.5,"body":"product manager, you know what I mean?"},{"startTime":1807.65,"endTime":1812.31,"body":"Like when you're at refinements or sprint"},{"startTime":1807.65,"endTime":1812.31,"body":"planning or whatever, that, that is to"},{"startTime":1812.31,"endTime":1816.295,"body":"me, when people say shift left, That's"},{"startTime":1812.31,"endTime":1816.295,"body":"what I think of when people say that."},{"startTime":1816.445,"endTime":1817.195,"body":"Totally agree."},{"startTime":1817.195,"endTime":1820.005,"body":"I, on the other hand, never have"},{"startTime":1817.195,"endTime":1820.005,"body":"a bad hair day, but, besides that."},{"startTime":1820.155,"endTime":1821.495,"body":"So I agree with that, right?"},{"startTime":1821.525,"endTime":1823.495,"body":"As early as possible shift left."},{"startTime":1823.645,"endTime":1829.035,"body":"Hey, listen, is there a reason why your"},{"startTime":1823.645,"endTime":1829.035,"body":"testers aren't involved in the very"},{"startTime":1829.065,"endTime":1831.635,"body":"early discussions with your UX folks?"},{"startTime":1831.765,"endTime":1832.645,"body":"Right, right."},{"startTime":1832.935,"endTime":1833.675,"body":"They should be there."},{"startTime":1833.895,"endTime":1836.065,"body":"I mean, oftentimes I hear"},{"startTime":1833.895,"endTime":1836.065,"body":"the opposite, though."},{"startTime":1836.065,"endTime":1838.115,"body":"I hear, well, there's nothing"},{"startTime":1836.065,"endTime":1838.115,"body":"for the testers to do yet."},{"startTime":1838.125,"endTime":1839.395,"body":"We haven't even written code yet."},{"startTime":1839.595,"endTime":1841.665,"body":"Like, that's the perfect"},{"startTime":1839.595,"endTime":1841.665,"body":"time to bring them in."},{"startTime":1841.75,"endTime":1844.03,"body":"When I do sprint reviews, for example."},{"startTime":1844.18,"endTime":1849.49,"body":"I'm, I'm demoing some features like"},{"startTime":1844.18,"endTime":1849.49,"body":"I think the, the, a lot of times, the"},{"startTime":1849.51,"endTime":1854.73,"body":"perfect people to review those items"},{"startTime":1849.51,"endTime":1854.73,"body":"and to demonstrate them to the users are"},{"startTime":1854.73,"endTime":1858.28,"body":"the testers because usually they're in"},{"startTime":1854.73,"endTime":1858.28,"body":"an environment that's not the dev, the"},{"startTime":1858.29,"endTime":1861.33,"body":"development environment where the, where"},{"startTime":1858.29,"endTime":1861.33,"body":"they were, all the data and everything"},{"startTime":1861.33,"endTime":1863.91,"body":"is showing is like, well, it's this"},{"startTime":1861.33,"endTime":1863.91,"body":"data is kind of fake and whatever,"},{"startTime":1864.06,"endTime":1867.98,"body":"because usually the, the testers are"},{"startTime":1864.06,"endTime":1867.98,"body":"operating in some kind of middle ground"},{"startTime":1867.98,"endTime":1872.59,"body":"stage environment or And if they're in"},{"startTime":1867.98,"endTime":1872.59,"body":"the environments where they've been in"},{"startTime":1872.59,"endTime":1876.94,"body":"production demonstrating the feature,"},{"startTime":1872.59,"endTime":1876.94,"body":"they have some sort of test user or test"},{"startTime":1876.94,"endTime":1879.09,"body":"data or test account that's in production."},{"startTime":1879.21,"endTime":1880.66,"body":"So it looks like real data."},{"startTime":1880.81,"endTime":1883.42,"body":"And, which is the best"},{"startTime":1880.81,"endTime":1883.42,"body":"case scenario, by the way."},{"startTime":1883.43,"endTime":1887.075,"body":"Look, I, I'm a big fan of testing in"},{"startTime":1883.43,"endTime":1887.075,"body":"production because  because I'm reckless."},{"startTime":1887.085,"endTime":1887.695,"body":"That's why."},{"startTime":1887.705,"endTime":1890.815,"body":"Cause I was like, no, I don't like,"},{"startTime":1887.705,"endTime":1890.815,"body":"but it's nothing like reality."},{"startTime":1890.965,"endTime":1894.645,"body":"No, I'm like, I, I, I was at a company"},{"startTime":1890.965,"endTime":1894.645,"body":"one time where I was like, where does"},{"startTime":1894.645,"endTime":1900.115,"body":"your sales people demo from, Oh, well"},{"startTime":1894.645,"endTime":1900.115,"body":"they have a whole sales account customer"},{"startTime":1900.115,"endTime":1901.295,"body":"and everything that has fake data."},{"startTime":1901.465,"endTime":1906.205,"body":"I was like, well, that seems like a"},{"startTime":1901.465,"endTime":1906.205,"body":"great place for QA to test in production."},{"startTime":1906.455,"endTime":1910.415,"body":"Like assuming that you have automation"},{"startTime":1906.455,"endTime":1910.415,"body":"that fires in every environment, why"},{"startTime":1910.415,"endTime":1912.22,"body":"can't you just fire the automation?"},{"startTime":1912.33,"endTime":1916.26,"body":"To look for certain things as criteria"},{"startTime":1912.33,"endTime":1916.26,"body":"for success in a different environment."},{"startTime":1916.26,"endTime":1919.77,"body":"So if you have a dev environment, you"},{"startTime":1916.26,"endTime":1919.77,"body":"probably have some, some data in there."},{"startTime":1919.96,"endTime":1922.5,"body":"If you have a middle ground stage"},{"startTime":1919.96,"endTime":1922.5,"body":"environment or something like that,"},{"startTime":1922.5,"endTime":1923.74,"body":"you probably have some data there."},{"startTime":1923.89,"endTime":1926.91,"body":"And if you have a production environment,"},{"startTime":1923.89,"endTime":1926.91,"body":"you'd certainly have data there."},{"startTime":1927.06,"endTime":1931.4,"body":"You can code the automation to look"},{"startTime":1927.06,"endTime":1931.4,"body":"for certain flags, certain triggers or"},{"startTime":1931.4,"endTime":1934.33,"body":"whatever to say, test successful or not."},{"startTime":1934.575,"endTime":1937.725,"body":"So it shouldn't be like testing,"},{"startTime":1934.575,"endTime":1937.725,"body":"quote, testing and production."},{"startTime":1937.975,"endTime":1941.055,"body":"Um, it shouldn't be this big thing,"},{"startTime":1937.975,"endTime":1941.055,"body":"if you're saying test, if you're"},{"startTime":1941.055,"endTime":1943.295,"body":"saying shift, you're testing left."},{"startTime":1943.445,"endTime":1945.985,"body":"First of all, that, that"},{"startTime":1943.445,"endTime":1945.985,"body":"terminology is meaningless to me."},{"startTime":1946.225,"endTime":1949.275,"body":"What I hear when I hear"},{"startTime":1946.225,"endTime":1949.275,"body":"that is your product people."},{"startTime":1949.425,"endTime":1955.215,"body":"Whoever's quote writing requirements"},{"startTime":1949.425,"endTime":1955.215,"body":"should be engaging the testers at the"},{"startTime":1955.215,"endTime":1957.265,"body":"time where the creation is happening."},{"startTime":1957.56,"endTime":1960.41,"body":"I think of people that, that, that"},{"startTime":1957.56,"endTime":1960.41,"body":"are like, oh, like all the people"},{"startTime":1960.41,"endTime":1963.49,"body":"that we like watch TikToks where"},{"startTime":1960.41,"endTime":1963.49,"body":"they're like, they meet with their"},{"startTime":1963.49,"endTime":1966.57,"body":"development manager and they cook up"},{"startTime":1963.49,"endTime":1966.57,"body":"requirements and nobody's in the room."},{"startTime":1966.71,"endTime":1967.58,"body":"It's just them two."},{"startTime":1967.835,"endTime":1968.105,"body":"Right?"},{"startTime":1968.255,"endTime":1972.125,"body":"Like, okay, you should at least have"},{"startTime":1968.255,"endTime":1972.125,"body":"somebody involved when the, when the"},{"startTime":1972.125,"endTime":1976.525,"body":"requirements are being vetted, if it's"},{"startTime":1972.125,"endTime":1976.525,"body":"in front of a customer or if it's in"},{"startTime":1976.525,"endTime":1980.465,"body":"front of your team, or if you have"},{"startTime":1976.525,"endTime":1980.465,"body":"both at the same time, which is ideal."},{"startTime":1980.485,"endTime":1983.955,"body":"You want to have your customer in the"},{"startTime":1980.485,"endTime":1983.955,"body":"room with your team and your tester"},{"startTime":1984.105,"endTime":1988.635,"body":"and your product manager saying,"},{"startTime":1984.105,"endTime":1988.635,"body":"Hey, you asked for this, you say,"},{"startTime":1988.635,"endTime":1990.355,"body":"this will relieve your pain point."},{"startTime":1990.665,"endTime":1991.585,"body":"Let's talk about it."},{"startTime":1991.735,"endTime":1994.015,"body":"Your tester can be there to ask questions."},{"startTime":1994.165,"endTime":1998.265,"body":"To listen to the back and forth"},{"startTime":1994.165,"endTime":1998.265,"body":"and can automatically be start."},{"startTime":1998.285,"endTime":2002.365,"body":"Like their brain is already"},{"startTime":1998.285,"endTime":2002.365,"body":"going into, how can I test that?"},{"startTime":2002.365,"endTime":2004.675,"body":"What we're about to create, fulfills."},{"startTime":2004.825,"endTime":2007.485,"body":"Every, every issue that"},{"startTime":2004.825,"endTime":2007.485,"body":"you're pointing out."},{"startTime":2007.56,"endTime":2010.46,"body":"It fixes every problem that"},{"startTime":2007.56,"endTime":2010.46,"body":"you're claiming to have."},{"startTime":2010.78,"endTime":2014.4,"body":"How can I, or how can I already"},{"startTime":2010.78,"endTime":2014.4,"body":"start to think about testing this?"},{"startTime":2014.55,"endTime":2017.99,"body":"And the earlier they start with that,"},{"startTime":2014.55,"endTime":2017.99,"body":"again, when we go into coding, they"},{"startTime":2017.99,"endTime":2019.31,"body":"can be sitting with your developer."},{"startTime":2019.31,"endTime":2020.09,"body":"They can be saying, Hey."},{"startTime":2020.345,"endTime":2022.095,"body":"These are the negative scenarios."},{"startTime":2022.425,"endTime":2025.945,"body":"These are the, if you're doing, yeah,"},{"startTime":2022.425,"endTime":2025.945,"body":"if you're doing test driven development"},{"startTime":2025.945,"endTime":2028.365,"body":"or whatever, like I don't know what"},{"startTime":2025.945,"endTime":2028.365,"body":"you're doing, but if you're doing test"},{"startTime":2028.365,"endTime":2032.165,"body":"driven development they were already"},{"startTime":2028.365,"endTime":2032.165,"body":"in all the conversations that brought"},{"startTime":2032.165,"endTime":2034.415,"body":"the work item in to development."},{"startTime":2034.78,"endTime":2037.0,"body":"You can start from the best scenario."},{"startTime":2037.075,"endTime":2037.845,"body":"But here's the thing."},{"startTime":2037.855,"endTime":2041.395,"body":"Those people that are watching this"},{"startTime":2037.855,"endTime":2041.395,"body":"or listening to this, that, just,"},{"startTime":2041.635,"endTime":2044.875,"body":"those, those testers that just stop"},{"startTime":2041.635,"endTime":2044.875,"body":"at testing with synthetic data."},{"startTime":2045.025,"endTime":2047.125,"body":"Yeah, you're missing"},{"startTime":2045.025,"endTime":2047.125,"body":"something there, right?"},{"startTime":2047.395,"endTime":2050.015,"body":"Because there's nothing"},{"startTime":2047.395,"endTime":2050.015,"body":"like testing in production."},{"startTime":2050.025,"endTime":2051.825,"body":"That is the acid test right there."},{"startTime":2051.975,"endTime":2055.485,"body":"So if you have various environments"},{"startTime":2051.975,"endTime":2055.485,"body":"that you're pushing your code across"},{"startTime":2055.495,"endTime":2060.685,"body":"dev test, QA, whatever stage, and then"},{"startTime":2055.495,"endTime":2060.685,"body":"finally production, there's no reason why"},{"startTime":2060.685,"endTime":2062.915,"body":"you can't avail yourself to the latest."},{"startTime":2063.065,"endTime":2066.505,"body":"techniques that you can trigger"},{"startTime":2063.065,"endTime":2066.505,"body":"all of these thousands of"},{"startTime":2066.505,"endTime":2068.705,"body":"automations every minute, right?"},{"startTime":2068.825,"endTime":2069.345,"body":"Literally."},{"startTime":2069.615,"endTime":2071.845,"body":"and use that to your advantage."},{"startTime":2071.865,"endTime":2075.315,"body":"The other thing is then look towards"},{"startTime":2071.865,"endTime":2075.315,"body":"progressing to the point where"},{"startTime":2075.315,"endTime":2076.575,"body":"you don't need those environments."},{"startTime":2076.895,"endTime":2079.205,"body":"Well, which is that blue green scenario."},{"startTime":2079.255,"endTime":2082.815,"body":"The last thing that I had that I"},{"startTime":2079.255,"endTime":2082.815,"body":"wanted to talk about was automation."},{"startTime":2083.015,"endTime":2083.125,"body":"Yeah."},{"startTime":2083.275,"endTime":2088.595,"body":"Uh, and like you started down this"},{"startTime":2083.275,"endTime":2088.595,"body":"road to talk about like, Oh, testing"},{"startTime":2088.595,"endTime":2093.105,"body":"in production, like is a dirty word"},{"startTime":2088.595,"endTime":2093.105,"body":"to a lot of people, but if you have,"},{"startTime":2093.215,"endTime":2095.645,"body":"if you have CICD practices in place."},{"startTime":2095.795,"endTime":2098.465,"body":"Like it shouldn't matter"},{"startTime":2095.795,"endTime":2098.465,"body":"what environment it is."},{"startTime":2098.605,"endTime":2103.725,"body":"Like you, if you have CICD In place and"},{"startTime":2098.605,"endTime":2103.725,"body":"you have tests that run as part of that"},{"startTime":2103.725,"endTime":2107.69,"body":"automation when your release process"},{"startTime":2103.725,"endTime":2107.69,"body":"goes off, you are testing in production,"},{"startTime":2107.81,"endTime":2112.98,"body":"you're testing in every environment"},{"startTime":2107.81,"endTime":2112.98,"body":"that you run those CI CD integrated"},{"startTime":2113.14,"endTime":2115.305,"body":"tests, so it shouldn't be a dirty word."},{"startTime":2115.595,"endTime":2120.255,"body":"To test in production, or dev, or stage,"},{"startTime":2115.595,"endTime":2120.255,"body":"or QA, or whatever environment you have."},{"startTime":2120.38,"endTime":2122.21,"body":"Those tests should be standardized."},{"startTime":2122.54,"endTime":2125.935,"body":"And the, and you should try to"},{"startTime":2122.54,"endTime":2125.935,"body":"get to a place where, I've been to"},{"startTime":2125.935,"endTime":2130.115,"body":"this place many, many times in my"},{"startTime":2125.935,"endTime":2130.115,"body":"career, in, in the maturity of test"},{"startTime":2130.145,"endTime":2131.785,"body":"automation at places I've been at."},{"startTime":2131.935,"endTime":2138.015,"body":"Where, they become so secure in their"},{"startTime":2131.935,"endTime":2138.015,"body":"test automation that they say, because"},{"startTime":2138.015,"endTime":2141.545,"body":"if, if, especially if you're doing"},{"startTime":2138.015,"endTime":2141.545,"body":"UI automation like frontend mm-hmm."},{"startTime":2141.625,"endTime":2146.445,"body":"automation, they, there's only so many"},{"startTime":2141.625,"endTime":2146.445,"body":"UI frontend tests that you can run."},{"startTime":2146.445,"endTime":2148.995,"body":"Like if you're running selenium"},{"startTime":2146.445,"endTime":2148.995,"body":"tests, for example, there's"},{"startTime":2148.995,"endTime":2150.375,"body":"only so many tests you can run."},{"startTime":2150.495,"endTime":2152.97,"body":"Every additional test,  it requires more."},{"startTime":2153.12,"endTime":2157.34,"body":"Time, like, seconds and minutes"},{"startTime":2153.12,"endTime":2157.34,"body":"to run more additional tests."},{"startTime":2157.56,"endTime":2160.71,"body":"So you eventually start looking through"},{"startTime":2157.56,"endTime":2160.71,"body":"your test library to say, like we"},{"startTime":2160.71,"endTime":2161.93,"body":"haven't changed this functionality."},{"startTime":2161.93,"endTime":2165.48,"body":"And so in so long or whatever, we're"},{"startTime":2161.93,"endTime":2165.48,"body":"just going to disable this test"},{"startTime":2165.48,"endTime":2168.94,"body":"because we're just going to take it"},{"startTime":2165.48,"endTime":2168.94,"body":"on faith that it runs successfully."},{"startTime":2169.065,"endTime":2172.535,"body":"and, we want our automation to"},{"startTime":2169.065,"endTime":2172.535,"body":"clear in a certain amount of time."},{"startTime":2172.715,"endTime":2174.645,"body":"Like we want to be able to run our."},{"startTime":2174.795,"endTime":2178.755,"body":"Automated battery of tests and"},{"startTime":2174.795,"endTime":2178.755,"body":"have it clear in two hours."},{"startTime":2178.83,"endTime":2182.35,"body":"I think about like a front end mobile"},{"startTime":2178.83,"endTime":2182.35,"body":"automation, like you have a mobile"},{"startTime":2182.35,"endTime":2184.27,"body":"app, you've changed some stuff."},{"startTime":2184.62,"endTime":2189.36,"body":"You want to release new application"},{"startTime":2184.62,"endTime":2189.36,"body":"into the, into the app store for users."},{"startTime":2189.65,"endTime":2193.81,"body":"And it runs, it runs in your backend"},{"startTime":2189.65,"endTime":2193.81,"body":"tests, grab the application, download"},{"startTime":2193.81,"endTime":2197.97,"body":"it from the app store, register a"},{"startTime":2193.81,"endTime":2197.97,"body":"new user, do some stuff, click some"},{"startTime":2197.97,"endTime":2199.03,"body":"things through the application."},{"startTime":2199.04,"endTime":2203.63,"body":"And like, front end testing is some of"},{"startTime":2199.04,"endTime":2203.63,"body":"the most expensive testing you can do."},{"startTime":2203.78,"endTime":2208.51,"body":"So, like in this, in this example that"},{"startTime":2203.78,"endTime":2208.51,"body":"I'm talking about now is, you really"},{"startTime":2208.51,"endTime":2214.195,"body":"need to put your money where the"},{"startTime":2208.51,"endTime":2214.195,"body":"most valuable are and figure it out,"},{"startTime":2214.205,"endTime":2217.615,"body":"figure out that the, we used to call"},{"startTime":2214.205,"endTime":2217.615,"body":"it , critical path back in the day."},{"startTime":2217.765,"endTime":2219.035,"body":"Yeah, that still exists."},{"startTime":2219.035,"endTime":2221.115,"body":"Critical path still exists in testing."},{"startTime":2221.265,"endTime":2224.285,"body":"Uh, I don't know how I got on this"},{"startTime":2221.265,"endTime":2224.285,"body":"topic, but, here we are on this"},{"startTime":2224.285,"endTime":2225.925,"body":"topic of critical path testing."},{"startTime":2226.075,"endTime":2226.705,"body":"Cool."},{"startTime":2226.925,"endTime":2227.945,"body":"Well, I learned something there."},{"startTime":2228.275,"endTime":2230.345,"body":"Critical path is still a thing."},{"startTime":2230.435,"endTime":2231.375,"body":"It's still relevant."},{"startTime":2231.525,"endTime":2232.815,"body":"It's still an elephant."},{"startTime":2232.965,"endTime":2233.565,"body":"Ha ha ha."},{"startTime":2233.565,"endTime":2233.835,"body":"Oh my god."},{"startTime":2234.045,"endTime":2234.945,"body":"I got nothing."},{"startTime":2235.095,"endTime":2237.895,"body":"Um, I think we should"},{"startTime":2235.095,"endTime":2237.895,"body":"wrap up by just saying..."},{"startTime":2238.235,"endTime":2242.285,"body":"The, if you are testing in production,"},{"startTime":2238.235,"endTime":2242.285,"body":"you either have a lot of faith,"},{"startTime":2242.285,"endTime":2244.085,"body":"you have good processes in place."},{"startTime":2244.16,"endTime":2245.415,"body":"Or, or you're No, no, no, no, no."},{"startTime":2245.415,"endTime":2246.915,"body":"Or, or you're, I stopped it either."},{"startTime":2246.915,"endTime":2249.255,"body":"So you're, I can't go to, or,"},{"startTime":2246.915,"endTime":2249.255,"body":"or you're a product manager."},{"startTime":2249.285,"endTime":2253.175,"body":"'cause I, I, I mess with my team"},{"startTime":2249.285,"endTime":2253.175,"body":"all the time and say I only test"},{"startTime":2253.445,"endTime":2256.925,"body":"in production, but also my team"},{"startTime":2253.445,"endTime":2256.925,"body":"only gives me access to production."},{"startTime":2256.925,"endTime":2256.926,"body":"So."},{"startTime":2257.406,"endTime":2260.076,"body":"I don't always test, but when"},{"startTime":2257.406,"endTime":2260.076,"body":"I do, I see a meme coming here."},{"startTime":2260.226,"endTime":2264.966,"body":"No, listen, the ultimate for me"},{"startTime":2260.226,"endTime":2264.966,"body":"is save your money and don't spit"},{"startTime":2264.966,"endTime":2268.396,"body":"up all these wicked environments"},{"startTime":2264.966,"endTime":2268.396,"body":"everywhere because they are a money"},{"startTime":2268.396,"endTime":2270.866,"body":"sap, just have dev and production."},{"startTime":2271.076,"endTime":2271.836,"body":"You don't need UAT."},{"startTime":2271.986,"endTime":2272.356,"body":"Yes."},{"startTime":2272.396,"endTime":2273.776,"body":"You don't need UAT, right?"},{"startTime":2273.816,"endTime":2277.616,"body":"In production, your new latest"},{"startTime":2273.816,"endTime":2277.616,"body":"code is in there, wrapped around"},{"startTime":2277.616,"endTime":2281.156,"body":"a feature toggle, and nobody's any"},{"startTime":2277.616,"endTime":2281.156,"body":"the wiser because it's turned off."},{"startTime":2281.306,"endTime":2286.206,"body":"Get your testers to test that"},{"startTime":2281.306,"endTime":2286.206,"body":"using automation, in combination"},{"startTime":2286.206,"endTime":2289.516,"body":"with all the regression that goes"},{"startTime":2286.206,"endTime":2289.516,"body":"there, there with, and if it passes,"},{"startTime":2289.516,"endTime":2291.996,"body":"guess what it's solid, right?"},{"startTime":2292.146,"endTime":2295.076,"body":"Flip that toggle and you know,"},{"startTime":2292.146,"endTime":2295.076,"body":"your customers are happy."},{"startTime":2295.226,"endTime":2296.916,"body":"If not keep that toggle."},{"startTime":2297.286,"endTime":2301.616,"body":"Unflipped, switched off, and work"},{"startTime":2297.286,"endTime":2301.616,"body":"on the feature until it works."},{"startTime":2301.766,"endTime":2302.506,"body":"So there you go."},{"startTime":2302.516,"endTime":2306.766,"body":"That's what I need to say about"},{"startTime":2302.516,"endTime":2306.766,"body":"blue green deployments rather than"},{"startTime":2306.916,"endTime":2309.236,"body":"multiple environments everywhere."},{"startTime":2309.386,"endTime":2312.656,"body":"I mean, at the point where you have"},{"startTime":2309.386,"endTime":2312.656,"body":"blue green deployments, like you, like"},{"startTime":2312.666,"endTime":2316.486,"body":"you're probably advanced enough where"},{"startTime":2312.666,"endTime":2316.486,"body":"you can, you can fly through a lot of"},{"startTime":2316.546,"endTime":2319.386,"body":"the concepts that we've talked about"},{"startTime":2316.546,"endTime":2319.386,"body":"at the point where you've, you've"},{"startTime":2319.426,"endTime":2321.466,"body":"implemented the most difficult part."},{"startTime":2321.496,"endTime":2322.366,"body":"You might as well."},{"startTime":2322.531,"endTime":2326.301,"body":"Roll your tests into it at that point,"},{"startTime":2322.531,"endTime":2326.301,"body":"like there's really not a, not a"},{"startTime":2326.301,"endTime":2330.801,"body":"reason not to, I mean, well, the flip"},{"startTime":2326.301,"endTime":2330.801,"body":"side of that, I agree, but the flip"},{"startTime":2330.801,"endTime":2335.931,"body":"side of that is how can, the likes"},{"startTime":2330.801,"endTime":2335.931,"body":"of Amazon roll out new functionality?"},{"startTime":2335.941,"endTime":2337.271,"body":"They're truly global, right?"},{"startTime":2337.421,"endTime":2339.571,"body":"You don't have the opportunity"},{"startTime":2337.421,"endTime":2339.571,"body":"to say, well, this is"},{"startTime":2339.591,"endTime":2341.461,"body":"production and this is whatever."},{"startTime":2341.611,"endTime":2345.541,"body":"And we're going to have something deployed"},{"startTime":2341.611,"endTime":2345.541,"body":"to production, which usually involves a"},{"startTime":2345.541,"endTime":2347.511,"body":"downtime because What are you going to do?"},{"startTime":2347.611,"endTime":2352.161,"body":"Penalize people in Japan that are using"},{"startTime":2347.611,"endTime":2352.161,"body":"Amazon or people in Hawaii, right?"},{"startTime":2352.191,"endTime":2356.401,"body":"No, it's it's there and it's through"},{"startTime":2352.191,"endTime":2356.401,"body":"blue green that they do that right,"},{"startTime":2356.541,"endTime":2360.551,"body":"right and they take You know, they"},{"startTime":2356.541,"endTime":2360.551,"body":"could they could take advantage of the"},{"startTime":2360.571,"endTime":2366.161,"body":"feature toggle functionality, too So they"},{"startTime":2360.571,"endTime":2366.161,"body":"release to production many times an hour."},{"startTime":2366.311,"endTime":2369.771,"body":"You heard that right an hour"},{"startTime":2366.311,"endTime":2369.771,"body":"and many teams I've worked with"},{"startTime":2369.921,"endTime":2373.951,"body":"struggle To release something every"},{"startTime":2369.921,"endTime":2373.951,"body":"sprint let alone an hour, right?"},{"startTime":2374.141,"endTime":2375.231,"body":"How is that possible?"},{"startTime":2375.231,"endTime":2376.281,"body":"Why can they do that?"},{"startTime":2376.431,"endTime":2377.681,"body":"What do they know that we don't know?"},{"startTime":2377.831,"endTime":2382.251,"body":"Well, I mean like the teams that"},{"startTime":2377.831,"endTime":2382.251,"body":"I work with Ever since I've been"},{"startTime":2382.376,"endTime":2387.946,"body":"exclusively in product, I insist"},{"startTime":2382.376,"endTime":2387.946,"body":"on, CICD, real time deployment."},{"startTime":2387.946,"endTime":2390.636,"body":"I, I insist on one, one feature."},{"startTime":2390.786,"endTime":2394.196,"body":"One, one change request, one"},{"startTime":2390.786,"endTime":2394.196,"body":"deployment to production."},{"startTime":2394.206,"endTime":2396.656,"body":"Like, I insist on real time"},{"startTime":2394.206,"endTime":2396.656,"body":"deployment to production."},{"startTime":2396.856,"endTime":2400.166,"body":"Like, I don't want to wait to"},{"startTime":2396.856,"endTime":2400.166,"body":"deploy to production like once every"},{"startTime":2400.316,"endTime":2401.756,"body":"Wednesday or something like that."},{"startTime":2401.806,"endTime":2403.496,"body":"I don't think that helps anyone."},{"startTime":2403.516,"endTime":2407.506,"body":"I don't think that's the best"},{"startTime":2403.516,"endTime":2407.506,"body":"thing for, for customer, delight."},{"startTime":2407.656,"endTime":2410.116,"body":"Yeah, I don't think that's the best"},{"startTime":2407.656,"endTime":2410.116,"body":"thing for, for customer delight."},{"startTime":2410.126,"endTime":2412.306,"body":"I don't think that's the best"},{"startTime":2410.126,"endTime":2412.306,"body":"thing, honestly, for the, like"},{"startTime":2412.306,"endTime":2413.606,"body":"the careers of the team members."},{"startTime":2413.756,"endTime":2416.506,"body":"To be honest, I don't think that's,"},{"startTime":2413.756,"endTime":2416.506,"body":"I think the company should figure"},{"startTime":2416.506,"endTime":2423.076,"body":"out how to say one change request,"},{"startTime":2416.506,"endTime":2423.076,"body":"whatever that is, story, bug, one single"},{"startTime":2423.076,"endTime":2428.896,"body":"story in Jira, or whatever ALM you're"},{"startTime":2423.076,"endTime":2428.896,"body":"using, whatever that is, it should be"},{"startTime":2428.896,"endTime":2431.006,"body":"able to go to production by itself."},{"startTime":2431.186,"endTime":2431.856,"body":"Absolutely."},{"startTime":2431.896,"endTime":2436.266,"body":"And also immediately go out in a"},{"startTime":2431.896,"endTime":2436.266,"body":"way where it doesn't impact anybody."},{"startTime":2436.496,"endTime":2440.796,"body":"Like we should be able to release in a way"},{"startTime":2436.496,"endTime":2440.796,"body":"where, the users don't really know that"},{"startTime":2440.796,"endTime":2442.386,"body":"the software is changing underneath them."},{"startTime":2442.476,"endTime":2443.146,"body":"Absolutely."},{"startTime":2443.146,"endTime":2445.931,"body":"This is, this is, this is"},{"startTime":2443.146,"endTime":2445.931,"body":"like, cost of doing business."},{"startTime":2445.931,"endTime":2448.491,"body":"Like minor, in my opinion anyway."},{"startTime":2448.501,"endTime":2449.571,"body":"You know, maybe all of you."},{"startTime":2449.761,"endTime":2453.071,"body":"Other people have other opinions,"},{"startTime":2449.761,"endTime":2453.071,"body":"figuring out how to do this in an"},{"startTime":2453.081,"endTime":2457.741,"body":"automatic manner, there's so many tools"},{"startTime":2453.081,"endTime":2457.741,"body":"and so many infrastructures that just"},{"startTime":2457.741,"endTime":2461.946,"body":"do this automatically without impacting"},{"startTime":2457.741,"endTime":2461.946,"body":"the user that, I remember the early,"},{"startTime":2462.086,"endTime":2465.356,"body":"early, early part of my career where"},{"startTime":2462.086,"endTime":2465.356,"body":"you had to take down time, had to bring"},{"startTime":2465.356,"endTime":2469.636,"body":"servers down, had to put up a splash"},{"startTime":2465.356,"endTime":2469.636,"body":"page that says, the site is currently"},{"startTime":2469.786,"endTime":2471.026,"body":"down for maintenance or whatever."},{"startTime":2471.036,"endTime":2471.056,"body":"Yeah."},{"startTime":2471.066,"endTime":2473.086,"body":"Yeah, there's so many tools are gone now."},{"startTime":2473.146,"endTime":2474.056,"body":"They should be gone."},{"startTime":2474.066,"endTime":2475.606,"body":"Theoretically, should be gone, yeah."},{"startTime":2475.826,"endTime":2478.636,"body":"There's no reason why they shouldn't"},{"startTime":2475.826,"endTime":2478.636,"body":"be gone, let me put it that way, right?"},{"startTime":2478.756,"endTime":2480.416,"body":"The tools are available to you now."},{"startTime":2480.566,"endTime":2482.126,"body":"I like that idea of..."},{"startTime":2482.276,"endTime":2486.536,"body":"Carving a piece of functionality so"},{"startTime":2482.276,"endTime":2486.536,"body":"small that it's just one thing, whether"},{"startTime":2486.536,"endTime":2490.656,"body":"it's something new, whether it's a"},{"startTime":2486.536,"endTime":2490.656,"body":"bug fix or whatever, because, one"},{"startTime":2490.656,"endTime":2493.146,"body":"times one times one always equals one."},{"startTime":2493.301,"endTime":2499.131,"body":"Again, since I'm, since I'm like that,"},{"startTime":2493.301,"endTime":2499.131,"body":"the technical person in product now, I"},{"startTime":2499.131,"endTime":2505.116,"body":"know that, I can just reject releases that"},{"startTime":2499.131,"endTime":2505.116,"body":"have only one, work item in them until"},{"startTime":2505.116,"endTime":2509.686,"body":"a point where I get to like a release"},{"startTime":2505.116,"endTime":2509.686,"body":"that has, Five, six work items or, or,"},{"startTime":2509.696,"endTime":2515.516,"body":"or 10 work items, seven of which are"},{"startTime":2509.696,"endTime":2515.516,"body":"related to the same web page or whatever."},{"startTime":2515.696,"endTime":2519.566,"body":"And I'll say, well, I'll take that"},{"startTime":2515.696,"endTime":2519.566,"body":"release and ignore the rest of them."},{"startTime":2519.776,"endTime":2523.686,"body":"But, but I, the business in"},{"startTime":2519.776,"endTime":2523.686,"body":"making that determination to say,"},{"startTime":2523.936,"endTime":2525.536,"body":"I want to go to production now."},{"startTime":2525.686,"endTime":2527.046,"body":"You just said when, right?"},{"startTime":2527.046,"endTime":2527.266,"body":"Yeah."},{"startTime":2527.266,"endTime":2528.726,"body":"And with what as well, right?"},{"startTime":2528.826,"endTime":2529.086,"body":"Right."},{"startTime":2529.086,"endTime":2529.276,"body":"Yeah."},{"startTime":2529.286,"endTime":2529.566,"body":"Yeah."},{"startTime":2529.566,"endTime":2530.126,"body":"I like that."},{"startTime":2530.156,"endTime":2530.466,"body":"Yeah."},{"startTime":2530.496,"endTime":2534.326,"body":"Well, well, it's, it's an opportunity"},{"startTime":2530.496,"endTime":2534.326,"body":"for me to communicate out to my"},{"startTime":2534.326,"endTime":2538.726,"body":"stakeholders of we made a build"},{"startTime":2534.326,"endTime":2538.726,"body":"now, and here's what you can see."},{"startTime":2538.926,"endTime":2541.026,"body":"I mean, I mean, it would be better."},{"startTime":2541.056,"endTime":2542.466,"body":"I certainly agree."},{"startTime":2542.616,"endTime":2543.476,"body":"It would be better."},{"startTime":2543.626,"endTime":2547.786,"body":"To just, as soon as my development team"},{"startTime":2543.626,"endTime":2547.786,"body":"is done with features, just send them out."},{"startTime":2547.936,"endTime":2549.436,"body":"Literally, as soon as they're done."},{"startTime":2549.586,"endTime":2552.316,"body":"And then the communication"},{"startTime":2549.586,"endTime":2552.316,"body":"becomes a bunch of little tiny,"},{"startTime":2552.616,"endTime":2554.406,"body":"little micro communications."},{"startTime":2554.716,"endTime":2558.156,"body":"As opposed to one release announcement"},{"startTime":2554.716,"endTime":2558.156,"body":"type of deal, you know what I mean?"},{"startTime":2558.521,"endTime":2563.701,"body":"At this point, we're on to another"},{"startTime":2558.521,"endTime":2563.701,"body":"topic of like, Announcing features and"},{"startTime":2563.731,"endTime":2565.571,"body":"stakeholder communication and whatnot."},{"startTime":2565.801,"endTime":2567.561,"body":"That's not really the"},{"startTime":2565.801,"endTime":2567.561,"body":"point of this podcast."},{"startTime":2567.561,"endTime":2568.831,"body":"This podcast is about testing."},{"startTime":2568.981,"endTime":2573.681,"body":"Although, although your testers can,"},{"startTime":2568.981,"endTime":2573.681,"body":"if they're enabled the right way, they"},{"startTime":2573.681,"endTime":2579.081,"body":"can have a big impact on product's"},{"startTime":2573.681,"endTime":2579.081,"body":"ability to communicate out to users."},{"startTime":2579.231,"endTime":2581.231,"body":"For sure, absolutely agree, yeah."},{"startTime":2581.356,"endTime":2583.146,"body":"I think that might be a wrap for us today."},{"startTime":2583.296,"endTime":2586.766,"body":"Alright, so let us know how your"},{"startTime":2583.296,"endTime":2586.766,"body":"teams are doing with your testing,"},{"startTime":2586.916,"endTime":2590.306,"body":"down in the comments below and,"},{"startTime":2586.916,"endTime":2590.306,"body":"like and subscribe that button."}]}