Runtime Arguments
Conversations about technology between two friends who disagree on plenty, and agree on plenty more.
Runtime Arguments
35: HTTP - The Unsung Hero of the Internet
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Jim leads a deep dive into HTTP — how it works, why it's still the backbone of the internet, and what's changed since Tim Berners-Lee built it at CERN in 1991. Wolf and Jim cover the protocol from request methods to modern performance upgrades, with detours into cookies, CORS, WebSockets, and the tools they reach for when debugging.
Topics covered
- What HTTP is: Hypertext Transfer Protocol, created by Tim Berners-Lee at CERN in 1991; a text-based request/response protocol that underlies the modern web and REST APIs.
- Request methods: GET, HEAD, POST, and friends — including the newer QUERY method (published mid-2026) — plus what idempotency means and why it matters.
- Headers: Host, User-Agent, Content-Type, Content-Length, cookies, and the Accept header's role in content negotiation.
- Status codes: the 2xx/3xx/4xx/5xx mental model, the difference between 301 and 302 redirects, and a couple of joke codes (418 I'm a Teapot, 451 Unavailable for Legal Reasons).
- HTTP versions: from 0.9 (1991, no headers) through 1.0, 1.1 (reusable connections), HTTP/2 (2015, header compression, ~71% adoption), and HTTP/3 (2022, QUIC over UDP instead of TCP — now on roughly 40% of sites).
- Cookies: how they enable state in a stateless protocol, session-based login, and the tracking side of things.
- CORS: what Cross-Origin Resource Sharing is, why it's painful, and Jim's approach of funneling everything through a single HAProxy front end to sidestep it.
- WebSockets: how Jim's team uses them for real-time resource locking and push notifications in their scheduling system.
- Tools: curl (including the handy `-L` and `-O` flags), Postman, VS Code's OpenAPI plugin, HTTPie, and TCPdump/Wireshark for packet-level debugging (including decrypting HTTPS traffic with browser-exported keys).
Also in this episode
- Catching up on Jim's early-morning bike rides at Kensington Metro Park, and a reminder to wave at fellow trail users.
- Listener feedback from Marlon (writing testable code) and DaveMQ (mocking in tests, and a callout for Wolf's missing Mastodon announcements).
- A recommendation of Jonathan Livingston Seagull as a favorite philosophy read.
- A shoutout to Julia Evans (Wizard Zines) for her HTTP zine.
Links:
- Julia Evans Wizard Zine: https://wizardzines.com/comics/status-codes/
- 418 - I'm a Teapot: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/418
Hosts:
Jim McQuillan can be reached at jam@RuntimeArguments.fm
Wolf can be reached at wolf@RuntimeArguments.fm
Follow us on Mastodon: @RuntimeArguments@hachyderm.io
If you have feedback for us, please send it to feedback@RuntimeArguments.fm
Checkout our webpage at http://RuntimeArguments.fm
Theme music:
Dawn by nuer self, from the album Digital Sky
Good morning, or afternoon, or evening, or night, uh, internet. It's another episode of Runtime Arguments. I'm Wolf, and here's my best friend, Jim.
Jim McQuillan:Hello Wolf, how are you?
Wolf:I'm tired, but good. I'm feeling pretty good. Um, this, of course, is episode number 35, and as always, I will point out that it's actually the 36th episode. We're going to talk about HTTP today. Jim has been putting together a bunch of facts. So he's going to lead the discussion. But why don't we start with Jim, how was your week?
Jim McQuillan:Oh, my week was good. Nice, uh, getting some work done. Um, you know, I mentioned last episode that I've been riding my bike. And I've been doing that a lot. Um, and… Uh, a lot for me is not much, uh, you know, to everybody else. Uh, we're gonna… we got some feedback, and maybe I'm prematurely talking about it, but, uh, Dave MQ, um, uh, listened to the episode, and he heard me talking about bike riding and Tour de France and stuff, and he suggested that I check out. Ragbrai, R-A-G-B-R-A-I, it's a 7-day ride across Iowa, and I think he did it, or he thought about doing it or something, and he thought maybe I should consider it. And one of the things I need to make clear is, uh, he's talking about a 7-day ride? I do like… Hour long rides. So, I am not… I am not a 7-day rider yet. I… I'm pushing a lot of weight around on the bike. Yet. Yeah. But, uh… but I… I am… I'm… I'm really enjoying it. In fact, I… I get out there at, uh…
Wolf:Yet.
Jim McQuillan:I leave my house about 6.30 in the morning, and it's still dark. I get out. We have a wonderful place not too far away called Kensington Metro Park. You know, we're here in Michigan. And it's a 30-minute drive for me to get there. So, I arrive about 7am, and I'm, like, one of the first people out there, and I'm riding around, and it's cold, and I'm just loving every second of it, but one of the things I wanted to say is, uh, some days I get out there. and I'm riding, and I see other riders, or runners, or walkers, or people walking their dogs, or whatever, and some days I'm out there, and the people all look at me and wave and say hi, and good morning. And other days, I'm there, and boy, the people just don't look at me. They won't… they won't look up. They'll avert their eyes as I come rolling past. And I just don't understand it.
Wolf:Is that because of your rifle?
Jim McQuillan:Yeah, the bazooka I carry on my back shouldn't be. I try to keep it concealed. We're joking here. Uh, I'm gonna tie this back to Wolf here, okay? Um, if you ever… you know, Wolf and I, we have lunch every Saturday at a nice sushi place, uh, just south of Ann Arbor. And one of the things about Wolf… ah, yes, you are! One of the things about Wolf is, we walk in the door.
Wolf:I'm wearing the shirt right now.
Jim McQuillan:And if he sees somebody, whether he knows them or not, he'll start talking to them. Within seconds, he will know. if they have a dog, what the dog's name is, what kind of dog it is, and he'll show pictures of his dogs, and he's… he's… it's a lot of fun. It's… that's not me. That's not the way I am. But that's the way Wolf is, and it's kind of refreshing to have somebody that… just opens up like that. It's fun to watch, and I think I mock him sometimes for it. I don't mean to do that, Wolf. It's really refreshing to watch you. But my point is, um… If if. If you are out there. riding on a trail or walking on a trail and somebody rides past. Look up and say hello. You know how much that means? You don't have to be friends with the person. You just wave, nod, say hello, do something, say have a good day, something like that. It means the difference to the other person. And uh… I get… it costs you nothing, and you feel good about yourself. And I… I have not been great at that, but I'm… I try now. When I'm out on the trail, I look at people as they go past, and if they…
Wolf:And costs you nothing.
Jim McQuillan:glance in my direction at all, I will say hi, I will wave, whatever. I will… sometimes it's just a nod, sometimes it's a wink, whatever. I think it means a lot, and I know I love it when I see people do that. And why people don't do that, I just don't know. And it's funny, because some days… yeah…
Wolf:Well, I can think of one reason that absolutely impacts me.
Jim McQuillan:Yeah.
Wolf:And I actively work to mitigate it. But, um, talking to… you know, I'm in my 60s. Um, talking to, um…
Jim McQuillan:Yes.
Wolf:an adult white male in their 60s, because that's what I am.
Jim McQuillan:Yeah.
Wolf:Uh, is easy.
Jim McQuillan:Yes.
Wolf:But, let's say the person on the other end of the conversation is a 22-year-old. female. That's a harder conversation because there's going to be barriers. The immediate concern from the other side is, oh, who is this? What do they want from me? Is this guy trying to pick me up? No, honestly, I'm not. I'm more interested in your dog.
Jim McQuillan:And he's creepy.
Wolf:But I… So…
Jim McQuillan:Yes.
Wolf:I am open and friendly, but I work hard not to accidentally intimidate.
Jim McQuillan:Yeah, it… But it's so easy just to say hi, right? And like we mentioned, within a few seconds, you will know the names of their dogs. And it led me to something else. I'm sorry, I'm taking so much time at the how was your week section.
Wolf:Yeah.
Jim McQuillan:But there's a podcast I listen to when I'm on that drive between home and in Kensington and back. It's… I get, like, an hour to listen. I listen to, uh, uh, Hank and John Green. Their internet. people, personalities. I just enjoy listening to them. They're fun. They've got a podcast where people send them questions and stuff like that. And I know Wolf and I have talked about this, the size of the companies we've worked for. And I was thinking about it. I don't believe I've ever worked for a company that had more than 15 people in it. And I know, Wolf, you've worked for all kinds of different companies, with way more than 15 people. I think the company you're working for now has a lot more than that. And we have friends like Jay, who joins us at lunch. He's working for Slack, and I don't know, what is there, 40,000 employees there?
Wolf:Which is owned by Salesforce.
Jim McQuillan:Salesforce. Yeah. So he's working for a company that's got thousands or tens of thousands of employees. And like I said, I've never worked for a company with more than 15. And, uh, anyway, John Green, uh, he, he, uh, him and his brother, they own, own a company and he made a statement the other day that really made me laugh. And I think Wolf will enjoy this. He said, I don't want to work for a company that has so many people that I don't remember the names of their dogs. That's what he said. He wants to work for a small company.
Wolf:I can't imagine a company that size.
Jim McQuillan:Well, my company is that size, right? I can remember the names of the dogs of all of the people that work at my company. It turns out it's one other person.
Wolf:I… I work with a, um… one of the members of my team is named Nathan.
Jim McQuillan:That has a dog. Umm.
Wolf:Um, I work with a couple of Nathans, but this is the one I work most closely with, Nathan L. Congratulations to him, by the way, he's getting married, um, I think?
Jim McQuillan:Oh.
Wolf:within a week, but he has a beautiful dog named Franklin, and he's marrying a lovely woman named Kaylee. And also, by the way, in case anybody needs to know, Nathan is awesome. This guy rocks. He's a great programmer. I love working with him.
Jim McQuillan:Nice! Nice. Well, that's great. I would bet that you know the names of all the dogs of the people you work with. Maybe not everybody in the company. But of the people you work with, I'm sure you know their dog's names.
Wolf:I know a lot of dogs.
Jim McQuillan:That's a good-sized company.
Wolf:I don't know if I know all, because my memory is not terrific.
Jim McQuillan:Anyway… Alright, so how… how was your week?
Wolf:Uh, well, there was lots of work stuff I could talk about, but I'm not going to. And there was the fact that I worked really hard this week on… Improving… the amount of work I do, not making it more, but right-sizing it, figuring out how to do the right amount of work to achieve what I want to achieve, but still have time left. what things I had to give up, what things I had to move, things like that. But I'm not gonna talk about that either. Um… I, uh… first of all, I want to say, um, some of the feedback we're going to talk about, and the thing that you just said comes from DaveMQ. we have gotten a pretty good amount of feedback from Dave MQ, who seems to enjoy our show. I think it might be time to elevate him to the pretend status of. Friend of the show.
Jim McQuillan:Oh, sure.
Wolf:Umm.
Jim McQuillan:Sure. He's… he's there.
Wolf:Yeah, absolutely. But here's what I do want to talk about. I have read a lot of books in my life. And some of them espouse philosophies. In fact, they are arguments convincing you or trying to convince you to pick a particular philosophy. For instance, when I was 12 years old. I read Atlas Shrugged. Umm. Atlas Shrugged is great if you're bipolar manic and think you are awesome. If you're a normal human being, Atlas Shrugged just tells you, you suck. Um, Atlas Shrugged is not a… here's my opinion. Objectivism, which is Ayn Rand's philosophy. is not a great philosophy. Um, and I've read books that… taught me things that influenced my philosophy, like… Godel, Escher, Bach, and almost anything written by Richard Feynman. Umm. But there is a book. That I think, um… Really distills taught me. Umm. Put a name to my philosophy. And it's not what you think. And I haven't read it in a long time. Uhm… It's Jonathan Livingston Seagull. It's a philosophy book. It's not about, um, being an Ayn Rand, uh, hero who charges each other 25 cents to borrow the car. It's not about that. It's about… Excellence and learning and always trying to be. The ultimate. thing that you are capable of. That's what it's about. Um… And I hadn't read it in a long time, and I didn't have a copy. So, um, it was on sale in paperback, the complete edition, with… uh, Afterward by the author Richard Bach, who's written a couple of books, but the only one I read was, um, Jonathan Livingston Siegel. I bought two copies, um. One for me, and I sent one to Jim.
Jim McQuillan:Yes.
Wolf:Oh.
Jim McQuillan:Yes.
Wolf:And, uh, I…
Jim McQuillan:And it's sitting right here on my desk.
Wolf:I… it's… there's a lot of photographs in it. It's a quick read. Um, I went through part one last night, which I did in well under half an hour. I think a part is like a chapter. And it's everything I remembered. I didn't remember the exact details of the story. But the moral, the lesson, the philosophy? Yeah, that was what I remembered. And reading it again reminded me how pointed it was.
Jim McQuillan:Yah.
Wolf:And that, in my opinion, and of course, anybody who has a philosophy thinks this same thing, regardless of what their philosophy is. Boy, did I make the right choice, I think, in this way. The Eternal Student. The, um… Forever learner. Always a student and never a master. You could definitely pick worse. Um, so.
Jim McQuillan:Yes.
Wolf:So.
Jim McQuillan:You know, it's funny, I've never read the book, and now I have it, and it was… Wolf sent it to me, and I opened it up, I thought, oh my gosh, because I had just thought about what's my next book that I'm going to read, like, the day before this arrived, and that was one of the books I was thinking of.
Wolf:I…
Jim McQuillan:Because you mentioned it at lunch last week. Or two weeks ago. So I actually was thinking about that, and then the book arrives. And here's me, not knowing anything about it, I always thought the writer was named Jonathan Livingston Ziegel.
Wolf:I don't think I know any human beings with the last name Seagull.
Jim McQuillan:Hey, it's just what I thought. So yeah, I'm learning too.
Wolf:Well, teh teh… To everybody who's listening, um, there are books I can wholeheartedly recommend, and this is very near the top of that list.
Jim McQuillan:Who's still listening.
Wolf:Um… But, let's talk about feedback. Um, we got some feedback, both from, uh, friend of the show Marlon, and from friend of the show… Um, Dave MQ. And… Uh, Marlon specifically mentioned, um… It's important to know that when you're writing the code that is your deliverable.
Jim McQuillan:Yeah, this was about tests.
Wolf:You. It is not tests.
Jim McQuillan:But the testing episode. Yeah.
Wolf:You want to write that code. in a way that makes it easily testable. Um, and that's absolutely true. In fact, there's several, um, side quests as you write code. things you'd like to be able to do with the code under test. Um, that are not part of the main task, but maybe they should be. Um, you want to write it so that it's testable, you want to write it so that it's scriptable. Um, I advocate making your application scriptable, which is great when you start wanting to do, um, more complicated testing, especially, um. Integration tests, and it's interesting when you want to do server stuff, and faceless stuff, and headless stuff. You want to write your code so that it can be easily measured, and so that it has the right logging points. And some of this. almost all of this is about the shape of the code. It doesn't… the algorithms don't matter as much. What matters is, how did you divide it up into pieces? Did you pick the right divisions? And some of it is about the promises. Did you very clearly use whatever type system is available to you. to define exactly what promises you are making. And in some languages, that's easy, and in some languages, it's hard, but I promise you that in every language, it's important. Um, so that was a thing that Marlin wanted to make sure everybody knew. And then, um, there were two… Very different things that Dave MQ surfaced. Um, one was, uh, I didn't mention mocking. Uh, and we did go pretty long in that episode, so, um, I might have gotten to it had we gone a little longer. What mocking is, is… remember I talked about that a unit test is small? And you tried to test just… The smallest thing reasonable? Well, sometimes, when you're trying to test something, it just won't work without some other thing. That's, uh, bigger and harder, but you don't really want to test that thing right here, because you want your test to be small. There's a concept in testing called mocking. And many, uh, languages and or testing packages provide, uh, tools to mock. certain things, like mock a database. You don't really have a database, but you have a thing that acts exactly like a database. And we'll tell you, it got called 6 times, and one of the calls… uh… was named this, and blah blah blah. A lot of this stuff is very dynamic, so you don't have to program anything. Making a mock is usually pretty easy. Mocking is a very important tool. witch. When you need it. is awesome and the only answer. And if you don't need it, absolutely do not use it. Because mocking makes things harder, more complicated, but sometimes it solves a problem that can't really be solved any other way. DaveMQ didn't specifically say what he wanted me to talk about with respect to mocking, he was just surprised it didn't come up because it's so important. And he's right, it is important, and it should have been mentioned. But the other thing Dave M. Key brought, surfaced, is that. uh, it's kind of my… you know, Jim does a lot of the hard work for the episode, like the editing and publishing and all that, um… But I have a job. I'm supposed to do the announcement. when the episode is released on Mastodon, and that is not a hard job, and in spite of the fact that it's not a hard job. I suck. I have not been doing it. And what, uh, DaveMQ pointed out was that, um. he had listened to the testing episode, and he thought, oh, I'll have a conversation in the replies, the thread. to the announcement on Mastodon. And there was no announcement on Mastodon because I suck! So I wanted to make sure you all knew I know I suck. Dave, did you have any feedback?
Jim McQuillan:Well, first of all, my name's not Dave.
Wolf:Oh! Crap! How'd that happen? Maybe you can edit that out. I'm sorry. Jim!
Jim McQuillan:Well, yeah, yes, that's fine. Well, the Marlin feedback, he sent that to me directly and you covered that, the making your software testable.
Wolf:Was there feedback that you wanted to address?
Jim McQuillan:is a big help. Unfortunately, I totally agree with what he says, but it doesn't help me with my legacy code. Because my legacy code was not written with testing in mind, and… certainly it should have been, but it wasn't. So I'm still stuck, uh, trying to… trying to figure out where to… where to put the tests, you know, how to… how to write them and stuff, but that's… You know, it's just work I have to figure out. Uh, but that's it. I, I had no other feedback.
Wolf:Oh! I… I do have one other piece of follow-up.
Jim McQuillan:Huh-huh.
Wolf:The three A's?
Jim McQuillan:Yes.
Wolf:That I, um… foolishly said the middle one was EXECUTE, which doesn't even start with A, by the way.
Jim McQuillan:Yeah, right.
Wolf:Yeah, the three A's are Arrange. Assert. Those are the three A's for what a unit test looks like. And there's several other.
Jim McQuillan:Yeah, I… I'm sorry, say that again. Say that again. It was arrange.
Wolf:Arrange? act.
Jim McQuillan:Yep act.
Wolf:assert.
Jim McQuillan:Assert. You got the first and last one, right?
Wolf:Yes.
Jim McQuillan:Eh, two out of three ain't bad.
Wolf:Um… Yeah, so why don't we get into the meat? It's all about HTTP. Take it away, Jim.
Jim McQuillan:Yeah, uh… I thought about this a couple of weeks ago, doing this episode, and I was certain, when I suggested it to Wolf. He was going to… Uh, react in a way like he does sometimes when I, when I, when I suggest topics that don't seem that interesting and really right off the bat, HTTP doesn't seem that interesting. It seems kind of boring, but it is fundamental.
Wolf:Negatively.
Jim McQuillan:to the internet, to the web, to the way we work. It is so important. So I thought it was worth talking about. And when I, when I told Wolf what the topic was, he said, interesting. I don't know if he was being sarcastic, or if he meant, yeah, that's interesting.
Wolf:Or both.
Jim McQuillan:But… or both. But as we talked a little about it, he didn't… I never saw the negative.
Wolf:Um, I don't want to steal your… I don't want to steal your thunder.
Jim McQuillan:You don't want to steal what?
Wolf:Um… I don't want to steal your thunder, but you know who makes HTTP interesting?
Jim McQuillan:Yep. Who?
Wolf:Julia Evans in wizard zines. Why don't you talk about that for a second before you go on?
Jim McQuillan:Oh, yes. She does, doesn't she? Yeah, she's got, uh… we've mentioned her before. She writes these little comic book-type things that contain. excellent knowledge, uh, information about all kinds of topics, and she talked about HTTP. And, and, uh…
Wolf:And she thoroughly researches these. She doesn't just write stuff down.
Jim McQuillan:She does! She's, she's, she's amazing. And, uh, we will include, as always, a link to her, uh, uh, web, uh, webzines? Is that what they're called?
Wolf:wizard zines.
Jim McQuillan:wizard zines. We will include a link to that in the show notes, uh, because there is one at HTTP. Uh, I think on the protocol itself, I know she talks about the status codes, uh, and we're gonna talk about the status in a little bit. But, yeah, so today we're gonna talk about HTTP. Uh, you know what it stands for, Wolf?
Wolf:I do. Of course I do. I wrote for, I worked for Netscape.
Jim McQuillan:Hypertext. I know, I know, I know.
Wolf:Hypertext Transfer Protocol.
Jim McQuillan:Hype… You know what? I always thought it was hypertext transport protocol. Which isn't far off. But it's Transfer Protocol. It was created by Tim Berners-Lee at CERN.
Wolf:On a next cube.
Jim McQuillan:Oh, was that what he was using?
Wolf:It was!
Jim McQuillan:In 1991, uh, he was working at CERN, and they needed a way to make documents easily available. So he created that. And, uh, the first word in HTTP is hypertext. So what is hypertext? That name gets thrown around a lot. Right?
Wolf:Are you asking? Do you want me to answer?
Jim McQuillan:Uh, yeah, I could answer, but why don't you?
Wolf:Hypertext is a word that has been thrown around for a while. That's not a Tim Berners-Lee thing. I can't remember the name of the actual author, but…
Jim McQuillan:Right. Uh, Ted… Ted Nelson.
Wolf:Ted Nelson, but this was, like, his life. And the idea is, a book is linear, hypertext is nonlinear. It is text.
Jim McQuillan:Yes. Right.
Wolf:but full of links so you can follow multiple paths. It is a formalized choose-your-own-adventure. Um, and boy, did that work out.
Jim McQuillan:Yeah, I've… I've done… I've been down the Wikipedia rat hole many, many times. Just clicking link after link, and that's, uh, that's, that's hypertext, right? Um… Anyway, HTCP is what enables the internet for us. The entire internet is built on top of this. More than just web pages, but… API's. You know, it seems like all API's these days are built, they're restful API's. You know, we used to have other things like soap, and all kinds of different API schemes. But it seems like restful HTTP API's. are so common now, nobody even thinks about them anymore. They're just there. Uh, so HTTP is a protocol. Uh, it's a standardized protocol. There's RFCs out there, uh, that have been approved about the entire protocol. And it's, it's basically a request response model. You send a request, you get a response back. That's not a surprise, right? So, so these texts, these, uh, requests are text-based. Uh, the response can be… all kinds of stuff. It starts off as a text response, but embedded in that can be video, or audio, or more text, or whatever. So, to do these requests, you send a method. In the request, and I'm, I'm gonna talk about a couple of 'em here. There's, um, I don't know, there's about 10.
Wolf:But when you say method, you don't mean a hunk of code. What you mean is a name, like put or get, that describes what kind of thing you are intending.
Jim McQuillan:Yep. A verb. Yeah, it's a verb, right? The most common one, the very first one, is GET. Get me this information. And in the request, you provide information about what you want it to get. Um, that's probably the most common. Uh, and it's… and it's something called… there's… there's a thing about it, it's called itempotent. You want to tell everybody what itempotent means?
Wolf:Idempotent is, uh, awesome. Idempotent means that, um, it can act, it can make a change, but if you call it more than once. twice, a hundred times, a thousand times, it only makes that one change that it made the first time. After that, it does not do any further damage. Um.
Jim McQuillan:Yeah, in the spec, it's not supposed to make a change, but people have… they make changes. One of the common things is tracking, right? You request a website, and there's a hidden. image on the page, and just the fact that you requested that image, uh, ticks a counter someplace. And maybe along with that counter is additional.
Wolf:So you said people, um, you kind of left out a word. I think what you meant to say was bad people.
Jim McQuillan:I'm sorry, what did I say? Okay. Yeah, um, so, it's not supposed to make a change to the, to the resource that you're fetching. Okay? It's just supposed to fetch the resource. There's… if there's side effects along the way, like a counter gets ticked, that happens all the time. There's another request that's very similar. It's called a head. Uh, what that does is it… makes the fetch, but it doesn't want to get the body of the request back. It just wants the headers. We'll talk in a couple of minutes about what the headers are. But head, uh, typically is used to get, um… uh, first off, to see if that resource exists that you're fetching, uh, and also to get maybe an expires time or a last updated time on it, so that you can compare that with the time you have to see if it's worth fetching the rest of the data. Maybe you've got it in cache already, and you just want to use what you have. So you issue a head request, uh, post, that's the big one. That's where you're sending data to the server. I do an awful lot of that. When I'm updating the database from the client, I will send a POST request with a payload of information. Now, um… Ah. Big difference between GET and POST… first off, GET is supposed to just fetch information. POST is where you're pushing data back. But GET doesn't have any payload in the request. It's just headers. Uh, POST has the headers that it sends, and also a payload. the body of the request, and that's where you would send maybe a JSON document, or a file upload, or… I don't know. form data. If you're updating a form, you would send form data. The headers are what describes what kind of data it is you're sending. That's the content type header. There's a bunch of other methods. I'm not going to get into great detail, but there's put, delete, connect, options. We'll talk about options just a little bit when we talk about CORS, C-O-R-S. That's always fun. There's trace patch.
Wolf:When he says always fun, what he means is. Uh, don't get involved with cores, it will ruin you and your life. It will make everything bad, and everything you touch will turn to garbage.
Jim McQuillan:It's, uh, it's… Yeah, Wolf is right. CORS sucks, but it's a security mechanism to help keep you a little bit safer. And if you understand CORS, that helps an awful lot. I've got ways of not having to worry so much about CORS. Uh, and I'll explain that, uh, in a little bit as well. Um, there's a… there's actually a brand new, uh, recently, uh, approved method called Query. Uh, it was actually proposed in 2021, but finally approved and published. Uh, two months ago, in June. Uh, query is like GET, only it has a payload. You send a body of information. So, maybe your request to fetch information has a lot of parameters. That you need to send, and you don't want to include those parameters in the headers. So you use a query. Uh, and hopefully the, the… the other end. Of course, you know, the other end has to understand your request. If the machine you're talking to isn't set up to handle a query method, then it's not gonna work. But anyway, that's there.
Wolf:And that's a thing to know, in general, is that, um, HTTP. Uh, involves, um, some things just have to agree, because you don't have a chance to negotiate them, but a lot of things get negotiated.
Jim McQuillan:Yeah.
Wolf:Like, for instance, one of the… uh… Um… possible header lines is… Accept… Accept something, I can't remember if it's…
Jim McQuillan:Like accept, yeah, it's just accept.
Wolf:Yeah, and that is a prioritized list of the formats you are willing to take back. Please give me whatever is the first thing on my list, if you can. Otherwise, the next thing, otherwise the next thing, and that's where you say, for instance, please give me a compressed version, I'd love that. Please give me the markdown version. Please give me the…
Jim McQuillan:Hmmm. Right. Yeah, please give me the JSON version of the data coming back, but if you can't do that, give me the XML version of data coming back. everybody knows JSON's so much easier to deal with than… than, uh, XML, right? I… I… I detest XML, but sometimes I… I have to deal with it. But that's… the request… the accepts header is, um… Uh, well, that exists both on the, uh, on the request and the response, I think.
Wolf:You know, it's weird, it's important to know that.
Jim McQuillan:Um…
Wolf:HTML is a special. XML.
Jim McQuillan:It is. It's a subset of XML. Subset? Superset?
Wolf:Well. Oh, it's not a superset. XML is like a family of languages in the same way that Lisp is.
Jim McQuillan:Yah. Yeah.
Wolf:And then scheme is a family of schemes. that is, every kind of scheme is really a LISP, but it's not… not all LISPs are scheme, and then there's even narrower than that. It's the same. HTML is a specific XML, um…
Jim McQuillan:Right, right, right. Right.
Wolf:XML is very general and every tag closes and you know, HTML, if you write it perfectly is great. But HTML is also, and this is, we're sort of traveling away from HTTP. Sorry, I'll stop in a second.
Jim McQuillan:Right. Right. It's fine. It's fine.
Wolf:But, um, most things that interpret HTML are very relaxed about what they, um… are willing to take. Like, you don't have to close a paragraph. Opening a brand new paragraph implicitly closes the previous one.
Jim McQuillan:Yeah. Yes. Right, right.
Wolf:In a strict XML, oh no, absolutely not. And a bunch of other special rules. But yeah, XML is hard. HTML is easy mode.
Jim McQuillan:Right, right, right, right. You, yeah, it'll, it'll croak if you, if you do that.
Wolf:But boy, Markdown, way better.
Jim McQuillan:Yeah.
Wolf:Anyway, back to the topic, sorry.
Jim McQuillan:Yeah, yeah. Okay, so when you make a request, um, there's a bunch of headers you send. Uh, and then a blank line. And then the body.
Wolf:Exactly like email.
Jim McQuillan:Uh, and on a GET request, there is nobody. Very, very much like SMTP, uh, for email. Um, if you… if you are well-versed in things like TCP dump, uh, or the graphical version of that, uh, Wireshark. Uh, you get to see that. Uh, and it's easy to see it on straight HTTP. We'll talk about HTTPS in a couple of minutes, and that's much more difficult, but not impossible to look at. But I… I always enjoy, well, enjoy maybe is a strong word, but I like the fact that I can understand HTTP when I look at it in something like TCP dump, because it's just kind of neat. You see the headers, you see the blank line, you see the body. It's kind of neat. Another way you can view that is browsers, like Chrome or Firefox or Safari, they have a developer. console mode, uh, the debugger, and you can look, you can, there's always a network tab where you can go see the requests and the response. I do that quite a bit, looking at the data coming through. You get to see all that, the requests broken down, the headers, the request headers, the response headers, the body. It's pretty neat.
Wolf:And some of the tools that you're about to mention, or sometime in the very near future, uh, will give you nice raw views like you're discussing.
Jim McQuillan:Yes. Yeah, I find that stuff interesting. And when I, it's sort of a skill that I unlocked several years ago, understanding the HTTP protocol so that I could do things. And understanding it has allowed me to program for it. I write server-side applications, server-side stuff that responds to API requests. And I've got a pretty good understanding of how to make it do what I want it to do, how to deal with redirects and authorization issues and stuff. It's pretty neat. So, back to the request headers. There's some important ones. There's the host. uh, header, which is important, uh, so that the server knows what host you're actually trying to hit. It's one thing to hit the IP address. uh, you know, for the host name to resolve to an IP address. But one of the things that came along, I think it was in version 1 or 1.1 of HTTP, is the host header, so that you can have lots of hosts served by the same server on the same IP address. So you have one IP address.
Wolf:Which is a major feature of Apache and many hosting providers. One machine, but lots of websites from different customers.
Jim McQuillan:Yeah. Yes. I, I remember, I remember back in the nineties when, if you wanted to serve more than one, um, uh, domain or one, one, one server name with, uh, uh, as a web server, you had to have multiple IP addresses. And that, you know, if you were paying for static IP addresses, that was a major problem. Now, the host header contains the name of the host that you're actually trying to get. You might have several hosts that resolve to the same IP address. And it's that host header that allows the web server to distinguish which server you're actually trying to hit, or which service you're trying to hit. So that's the host header. And the method, uh, header, that contains the GET, HEAD, PUT, POST, whatever. Uh, user agent. That's a good one. That is supposed to tell you, uh, the type of client that's making the request.
Wolf:But the truth is, it doesn't.
Jim McQuillan:But boy, if you try to make sense of that, they pretty much all say Mozilla 5.0 as a means of compatibility. We can spend an entire episode, although we won't, talking about the user agent string. Because it's just a mess.
Wolf:And developer mode of almost all of these tools let you send any host, any user agent string that you want.
Jim McQuillan:Yes. Yeah.
Wolf:Umm. But this is… the thing is, um… Websites, um, servers, um. do this bad thing where they use the user agent string to decide if they can do the thing you want. That is incorrect. You don't do that. You use capability testing. You actually figure out, can you do this thing not by the user agent string, but with tests. Um… And yeah, that's why the user agent string is almost invisible. Entirely useless.
Jim McQuillan:It's pretty well worthless. It used to be used a lot back in the days before people figured out how to do capability testing. Um, and it was a big issue because remember back in the day and then in the middle 2000s, uh, we had like internet. Explorer 6. that didn't do a lot of things, and it was really important for your application to know whether it was running on IE6 or not, because it was just so awful at… at… running your JavaScript code, so you needed to know. if you were on IE6, and the way to do that was with the user agent. Uh, string. Uh, now, we use the user agent more as a diagnostic thing, so that when, uh, like a patient hits our, hits our website, and they're having trouble for some reason. This hasn't happened in quite a while, but it used to. they hit our, uh, either through our mobile app or through our, uh, the patient portal. We can… Kinda guess what browser they're using. Uh, by the user agent string. So, uh, where it's really come in handy is somebody hits, hits us with a, like, an Android phone or a tablet, and something's not right. Uh, or they hit us with iOS. We can at least tell the difference between Android and iOS. Uh, so that helps. Anyway, user agent is mostly… pointless and ridiculous. There's lots of other headers. There's the cookie header. We can talk about that. The content type. If you're sending data to the server. You need to tell it what kind of what format the data is in. Uh, right? It could be, uh, uh, what, application slash JSON? Um, it could be, uh…
Wolf:Application slash text application. Yeah.
Jim McQuillan:Texts, um, uh, it could be, um, uh, uh, multi-part form, uh, application slash multi-part form, uh, if you're doing, like, a form submit.
Wolf:I… I… Um…
Jim McQuillan:Umm.
Wolf:Mime might be. In that string somewhere.
Jim McQuillan:Uh, yeah, there's, there's, yeah, there's some mimes in there. Um. Mime URL encoded, or… anyway. Uh, the content type is how you tell it what you're sending, so that the server knows how to unpack it. It might be… application slash JSON, it might be application slash XML, so when the server gets this data, it knows how to decode it and do something with it. Another really important one, if you're sending data to the server, you really want to send a content length header. to tell it how much data you're sending. It turns out to be pretty important, because servers sometimes puke if you tell it you're gonna send, uh, a thousand bytes, uh, but you only send 900? The server, uh, will wait for that extra 100 bytes for some amount of time, and then time out and get angry with you. And the same can happen if you send more than you tell it you're gonna send. It can truncate that data and not swallow up the whole thing. So you want to get the content length correct. Um… There's a bunch of other headers. You can search Wikipedia or your favorite place for that. There's just all kinds of headers you can use. So let's talk about the response for a second. Um… The response is what comes back from the server. The first thing you'll notice is there's a status code that comes back. Uh, it's a three digit number and you, you've all seen 200. Yeah, yeah, yeah. Pretty much. Uh, they, they, they all start with, uh, they're all three digits long and they start with a 1, 2, 3, or 4.
Wolf:And there's one good one. And then all the rest are bad. 200 is the good one.
Jim McQuillan:4 or 5. Uh, and, and the, the mental shortcut is, uh, 2XX, it means it could be 200, 201, 20-something, means it worked. Right? A 3XX means look elsewhere. That's a redirect.
Wolf:And those are okay.
Jim McQuillan:A 4XX… yeah, they are, they happen, they're useful. A 4XX means you, the client, did something the server can't process, so it's your fault. And a 5XX means the server messed up. It's, it's on the server's, uh, uh, uh, it's a server problem that they need to deal with. I don't know, have you ever gotten a 500 as a status code? Oh, I hate that, I…
Wolf:Oh yeah, I've gotten every single error code you can get except I'm a teapot.
Jim McQuillan:Yeah. Yeah, there, there, there is one. Uh, 418, uh, I'm a teapot. Uh, that was from an April Fool's joke back in 1998. Uh, somebody submitted a proposal for an RFC, uh, number 2324. Uh, it was a proposal for the Hypertext Coffee Pot Control Protocol. And the RFC states that if a teapot is asked to brew coffee, it should refuse and respond with a 418. I'm a teapot. It never made it into an actual approved RFC, but there are some servers out there that will respond if you ask it to brew coffee. I don't really know what the syntax would look like for that request. But you'll get back a 418. I'm a teapot.
Wolf:I will say, um, that if I were designing a program, that would be an over-specified error. I probably would shoot more for something like. type error. You know, it's incompatible with the thing that you asked, or value error, something like that.
Jim McQuillan:Right. Right, right. It was a joke, yeah. There's another error in the 4 series, 451.
Wolf:Yeah, I think so.
Jim McQuillan:unavailable for legal reasons. That's a deliberate reference to Ray Bradbury's Fahrenheit 451 book. Basically, that book is about censorship and book burning and that kind of stuff. So yeah, if you get back a 451, it means, uh… It's not available for legal reasons, which probably means the lawyers got involved.
Wolf:You know.
Jim McQuillan:Umm.
Wolf:I actually have a tiny aside, uh, I'm gonna make it quick because it's so unrelated, but I worked at Apple for a while.
Jim McQuillan:Yeah. Sure. No.
Wolf:And while I was working at Apple, we were working… Apple Hardware was working on… I was in software, but they were working on a specific machine. And the code name for that machine was Sagan. And Carl Sagan was alive. And somehow he found out about this code name. And he sued. He's like, nope, can't use my name.
Jim McQuillan:Wow.
Wolf:Yeah, it turns out, I don't want to speak ill of the dead, but he wasn't as nice as you might think. A code name was unhappy for him.
Jim McQuillan:Well…
Wolf:um…
Jim McQuillan:It was an infringement on his, uh, on his trademarked, uh, name, huh?
Wolf:I don't want to know his dog's name. Right. Anyway, sorry, didn't mean to digress.
Jim McQuillan:Anyway, yeah, yeah. Uh, so, uh, the 200s are good. That means, uh, a 200 means, uh, the response was accepted and processed, and it's okay. I like getting 200s. There's a few others, uh, 20… anything starts with a 2. It means you're good, keep going.
Wolf:You got what you wanted.
Jim McQuillan:Yeah, yeah. The 300 series, there's only a couple of them, and I learned this the hard way. There's a big difference between 301 and 302. If you get back a 301, that means that the site has moved permanently.
Wolf:Yes.
Jim McQuillan:And what that does is the browser will cache that information, and it'll never try to talk the original site. A redirect is you make a request to some server. and the server responds, uh, the data's not here anymore, go there instead. And it returns a location for where you should go instead to get what you're asking for. So a 301 means this is a permanent move, and don't ever go here again, just always go there, so the browser caches that information, and it never has to do that first lookup again. Whereas a 302 means it's a temporary move. A temporary redirect. Um. And that's what you do if, like, you're playing around with your servers, and you're moving things around, and maybe testing something out, and you want to try hitting a different server. Uh, the browser won't cache that. It'll just keep asking for the original server every time, and it'll get the redirect, and then it'll hit the new server. And then 10 minutes from now, if you make the same request, it'll go to the original server first, get the 302, and go to the other server.
Wolf:Yeah, the thing to know about these 300s is they come from how you've configured Squiddy or Apache or anything that is an HTTP server that you're using.
Jim McQuillan:Well. Yup.
Wolf:Like, the fact that you said, this name actually means this, you have to write that down in a configuration file somewhere, uh, or database, or however your server needs to see it.
Jim McQuillan:Yep.
Wolf:And it's the fact that you told the server this that makes the server respond with a 300.
Jim McQuillan:Yeah, it's… a redirect is common, like, if you hit a website, example.com, and you don't provide anything after the slash, it's just http colon slash example.com. What you really want it to do is fetch a certain page, like maybe example.com/login.cgi. a crude example.
Wolf:or index.
Jim McQuillan:Um… So, or, yeah, or index, where you hit, you hit the, the raw site with no, no trailing, uh, uh, name on it, uh, and then what you'll get is a, is a redirect coming back, saying, from now on, go to example.com slash login.cgi. That's a great place to have a redirect, so that you will actually get the page that you should get. It's a way to say, well, here's what the default page is. Early on in my web career, we were trying things with different homepage formats and different login screens and stuff, and. we decided that we wanted to change that whole naming scheme. But people had bookmarks. You know, to the old login page. So we had to configure Apache to serve up a redirect with the correct name. So even though they had an old, outdated, stale bookmark, it would still get to the right login page through a redirect. Um, and when we were testing that, we used a 301 instead of a 302, and then… Uh, it made it really hard to try other things. So, for that stuff, when you're playing around, use a 302. When you've really got something you know has moved permanently, use a 301. That's all. There's a bunch of other codes, we don't need to get into all those, I just wanted to point out that. Every request will receive a response, and in that response is the status code, and the status code's important. Right? A very important one is 404. That means the resource that you requested isn't there at that address. How many people have seen a 404?
Wolf:Yeah, 200 and 404 are like the numbers. Normal people know.
Jim McQuillan:Yeah. Yeah, I mean, 404 is kind of a cultural thing, right? People know about it, at least the people I know. Yeah, yeah, yeah.
Wolf:Farewell.
Jim McQuillan:What's that from? Isn't that like Cloudflare or something had that? Who had the fail whale?
Wolf:I think the fail whale was Twitter maybe?
Jim McQuillan:Okay.
Wolf:And then GitHub, maybe?
Jim McQuillan:Yeah, yeah. When you configure your server, you can configure what to return back with each of these status codes, too. So you can return back some content, like the fail whale. It'll be a graphic image on your screen that nicely tells you this resource is no longer there. Um, or you can just go with this, with the default Apache page that says, uh, not found. Um, it's all configurable. Uh, another important one, uh, is the 401, unauthorized. Um… I'll tell you where I use that. I do a lot of API work, and we use a service called Twilio. You familiar with that? Uh, they, they, they're.
Wolf:Oh yeah, I have friends who work there. They're great.
Jim McQuillan:Oh, do you? It is a great company. And boy, if you need to do SMS texting, like from your application, you want to send out texts. Like I'm in the medical field, right? I need to send out appointment reminders and requests for feedback. All kinds of stuff. And so we send out SMS messages.
Wolf:Yeah, Twilio is the glue between phone and email.
Jim McQuillan:between, yeah, between the phone network, uh, in fact, they do voice as well, but we use it for SMS. And we send out a… a request, uh, we build up this H… uh, um, this…
Wolf:Internet.
Jim McQuillan:HTTP request, uh, that contains a little JSON payload that includes the phone number and the message we want to send, and it also includes a callback, uh, uh, URL. So that we send the request out to… Twilio. And when the recipient gets that, there's like a life cycle to these SMS messages. There's a message… Twilio will call our webhook several times. One of them will say that the message has been delivered to the handset. Uh, one of them might say that the user has read the message, if they have that not blocked. Yeah, right. Right. Um…
Wolf:If read receipts are turned on for them.
Jim McQuillan:Uh, and if the patient responds to a text message. Uh, it's that webhook that gets called. Uh, so Twilio will, uh… I mean, this could happen sometime later. You made the request to send out the text message, and these things happen seconds, or minutes, or maybe hours later, maybe days later. Uh, you'll get back these webhook requests. Yeah, it's all asynchronous. Um, so anyway, anytime.
Wolf:Asynchronously.
Jim McQuillan:Twilio calls your webhook, I don't know why they do this. They have, in my configuration on Twilio, in my profile, I've configured the authentication information. It includes an API token and some other stuff. But every time they send a request to my webhook. Uh, they don't send any authentication information, so the first thing they get from my server is 401, not authorized. And then they say, oh, okay, and then they turn around and send another request with the authorization information. So, my API, it sends back the 401, telling Twilio that. they can't connect without authorization information. And I also send back how they can authenticate, and then they do, and it's… it's a double request when… I don't understand why they don't just. Do it right the first time. But anyway, that's a 401.
Wolf:Strong agree.
Jim McQuillan:Yeah, it's… you know… I mean…
Wolf:They're smart people. I don't understand why they would do that.
Jim McQuillan:Yeah, I don't know. We send out thousands of SMS messages every day. And it might sound like we're annoying if we're sending out these messages, but patients want these messages. They want to know that they've got an appointment coming up. They want to know that they can click this link and log in, or they can check in for their appointment ahead of time. So we do send out a lot of messages. So we get back, you know, for every message we send out, we get back thousands of webhook requests from Twilio. And every one of those, they send two requests. One that gets a 401 not authorized, and then another one that doesn't. Were they authorized correctly? It's annoying. Thought I'd throw that out there. But if you're ever in need of sending SMS messages from a program… from a system. Twilio is the way to do it. They're inexpensive, and they've got the best API, and it works really, really well. So…
Wolf:Absolutely. And everybody I know there, they're all good people. I like them. I think they're good.
Jim McQuillan:Um… I don't know anybody there, but I know from the very first interaction I had with them that they're a great company. Um, and, and I, uh, it's one of those success stories I think that I really like. Um, uh, so it's good. So, uh, we've mentioned a couple of times now you send a request, you get a response. There's a bunch of response headers that come, uh, followed by a blank line. Followed by the body of the response. And those response headers, you get a status, you get a content type that's gonna tell you what kind of data they're sending you back. Uh, you get a content link, they're gonna tell you how much data they're gonna send you. Um, there's, there's, uh, a set cookie header, where if the server wants to, uh, store a cookie on your, on your, uh, in your browser, and we'll talk about that in just a minute. There's all kinds of stuff coming back. Um. Let's talk for a minute about the versions. This'll just take a second. Uh, the first version that Tim Berners-Lee, uh, released to the public back in 1991 was 0.9. Um, and it only had one method. In fact, it didn't even use the method, I think, because there were no headers in that version. The only thing you could do was fetch documents. Uh, I think… I should go look at the… the syntax of that version of HTTP. I don't know how you told it what you wanted, probably on the URI as part of the request, but. Uh, the only thing it could do is respond with the document that you, that you requested. That's it. There were no headers.
Wolf:But that's a super important lesson. Ship as soon as you have something useful.
Jim McQuillan:They had something useful. Right? They were using it. And it was enough to get things started. It wasn't until 1996, a full 5 years later, that headers were added, version 1.0 of the protocol.
Wolf:Yep.
Jim McQuillan:Um, and they, uh, they used a separate TCP connection for every request. So, you know. When you, uh… let's say you hit CNN. uh, for a… for a page. You want to get the front page of CNN. You hit the CNN HTTP colon slash slash, uh, CNN.com. You will make so many requests to CNN. You will make…
Wolf:For every image, for every blah blah blah.
Jim McQuillan:For the JavaScript, for the chunks of JavaScript, for everything, you'll probably make a hundred, you'll probably make over a hundred requests. Your browser will make over a hundred requests just for you to look at the homepage.
Wolf:For the CSES for…
Jim McQuillan:Um… And version 1.0 of the protocol opened up a separate connection for every one of those requests. So, you can see how that could be a huge demand on the server, if every time somebody hits your website, they're opening up… Dozens or hundreds of connections. Uh, it was just awful. Uh, but, you know, we're talking 1996. How many web pages were there? There weren't that many.
Wolf:Well, there were eight.
Jim McQuillan:Um… There were what? Eight?
Wolf:No, I'm kidding.
Jim McQuillan:I think Pizza Hut had one. It was the first… the first e-commerce site was Pizza Hut. You could order a pizza in the Silicon Valley area. Um, that was… that was, like, mid…
Wolf:Netscape had the fish cam.
Jim McQuillan:The fish cam.
Wolf:Yeah, we had a giant tank of fish, uh, exotic, I think it was a saltwater tank.
Jim McQuillan:Yah. Yeah, yeah.
Wolf:Uh, right there in, like, our development area, and we had a webcam aimed at it, and, um, I think it updated the photograph every… minute, maybe, or something around there, but it was a URL, you could… you could fetch that page, and you'd see the fish, and it was live. You know, within a minute.
Jim McQuillan:Okay, yeah. See, you just… Yah. Yeah, yeah. And that was probably version 1.0. 1.1 came just a year later that reused TCP connections. That was the big thing that version 1.1 added, the ability to reuse connections so that it didn't open a connection for every request. It could, like, piggyback these requests on the same connection, which makes a lot of sense, right? It wasn't until 2015 that version 2 came out. That's a long time to get here. But version 2, what they offered was less latency and higher speeds. So it was a big deal. And also, version 2 added compression. of the, uh, headers. It would do a binary representation of those headers and compress them, so the… sometimes the headers can be pretty substantial, um, because you can include a lot in there. Uh, so it would compress that, and it would be a single connection per server domain. So if you hit a server, uh, everything's gonna cross that one connection. Uh, even for all the pages within that server, you're gonna, you're gonna… Pull it across a single connection to that server.
Wolf:But this is one of those things I mentioned that is negotiated between the client and the server.
Jim McQuillan:Umm. Yeah. Yeah, it's what version of the protocol can you accept, is one of the things. Uh, because, you know, they come out with new versions of the protocol, not every browser or every client can accept that version, so it'll always fall back to a previous version. Uh, now, right now, 0.9 and 1.0, they're, they're obsolete. Nobody's using those anymore. But 1.1 is still in, in, in a lot of use. Uh, version 2 is, is, um… I've got some numbers here. HTTP version 2 is supported by 71 of the websites out there in the world. That's… that's pretty good, and it's re… it's supported by all web browsers. Um…
Wolf:Seventy-one percent.
Jim McQuillan:Yes, 71% of the websites are supporting version 2. So, you know, if not that, then they're gonna use version 1, or 1.1, rather. So, here's the thing that got me interested in talking about HTTP, and that is version 3. I've heard about version 2 and version 3 for quite a while, and I didn't really know what they were. I thought they were… Like not that big a deal. And I didn't really know if they were widespread adopted or not. But version three came out in 2022. And that did away with TCP and uses QUIC over UDP. Umm. And that, what happens? Yeah.
Wolf:Yeah, spoiler alert, it is a big deal.
Jim McQuillan:It is a big deal, um, and it's surprisingly well supported. Um… the difference is, if you make all your requests over a single TCP connection, everything is ordered properly in a TCP. connection. And if you make a whole bunch of requests, guaranteed delivery, if you make a whole bunch of requests, if one packet fails, it'll stop all the other things from coming. You might have a whole bunch of things lined up to fetch.
Wolf:End guaranteed delivery.
Jim McQuillan:If there's one problem fetching one thing, it'll delay, it'll stall those other things from coming. Up. QUIC on HTTP3 sort of multiplexes a whole bunch of requests over a single connection, and it's all UDP, so the packets can arrive out of order, they can… it's, you know, the thing about UDP… yeah, the thing about UDP is it's not a reliable protocol, but adding QUIC on top of that is…
Wolf:Or not at all.
Jim McQuillan:Is that that's where the reliability happens. And it turns out that that HTTP three is quite a bit faster than than the previous versions. Because of that one failure along the way doesn't stall the rest of the the rest of the transfers. So I started thinking, you know, I do a lot of HTTP programming. I use HTTP quite a bit. I thought, well, how can I implement this on mine? What do I need to do? And first of all, HTTP2 is already supported. Like. I use HAProxy as my, basically, my front end. When you hit any of my sites, uh, you're gonna be hit in an HAProxy, uh. Server. And then that HAProxy server's gonna fan those requests to the backend to do its thing. Well, HAProxy fully supports HTTP2, and it does it just right off the bat, so it's really nice. So all my stuff is coming across HTTP2, so that's good. Uh, so then I thought, well, what can I do about implementing HTTP3? Uh, turns out HAProxy supports that, too. So I turned that on this afternoon, and I'm hitting my site now over HTTP3. Uh, I can't really tell if it's faster or not, but they tell me it's faster. In fact, when I talked to Marlin about the fact that I wanted to talk about HTTP. He's the one that said, yeah, HTTP3 is really important. 40% of the websites.
Wolf:When when the thing you're fetching.
Jim McQuillan:Yes.
Wolf:is actually. 4D resources because of the image files and the primary content and then the CSES on top of that or multiple CSS and embedded fonts. When you have something.
Jim McQuillan:Yes.
Wolf:with so many individual objects, HTTP3 is a gigantic win. If you are delivering something that is, um…
Jim McQuillan:It is.
Wolf:uh… much simpler than that, the winds are not as magnified. But they're there!
Jim McQuillan:Yeah, um… The overhead is less, so, uh, it takes a lot less to set up a UDP connection. Uh, in fact, uh… Opening a bag of chips, are we?
Wolf:Fine. I'd probably put it down.
Jim McQuillan:Go ahead. I'll wait.
Wolf:I'm just so hungry!
Jim McQuillan:We'll be done in a couple of minutes. In fact, we'll cut it short so you can eat your chips. Anyway, setting up a UDP connection is a win, because it's faster than a TCP connection. So just turning it on, it's like, you're better right off the bat. You may see… you'll see some kind of an improvement. You may see a lot of improvement, depending on what kind of… What kind of traffic you're… what kind of resources you're requesting, how often, and stuff. the servers I have, my test server that I have for my customer, and my production server, turns out the HA proxy is linked to a version of SSL that doesn't support QUIC, the QUIC HTTP, the things you need for QUIC over UDP. Uh, that doesn't support it. Uh, there's a way to, like, pretend to support it, or to partly support it, and it turns out Safari works just fine. It's perfectly happy to use the… the, uh, scaled-back version of HTTP3. Uh, Chrome and Firefox, not so much. They refuse to talk HTTP3, uh, when you turn it on, uh, in this scenario. I do have another machine where, uh, I'm running HAProxy, and it is using a better version of SSL, so that works just fine. So it's really easy to set up Chrome. uh, or, or, uh, to set up a site that Chrome, Firefox, Safari, Edge, they'll all talk, uh, uh, HTTP3. So, that's… That's the thing, that's the reason why I wanted to talk about HTTP, because I thought that was pretty neat. Anything I can do to make my stuff faster, I think it's a really good idea. So, that's kind of enough about the protocols. Although, did I mention that 40% of the websites out there support HTTP3? Oh.
Wolf:You did not and I think that's a super interesting number.
Jim McQuillan:It is. And if I open up the developer tools in my Chrome browser and I browse the web, I go to Google or I go to CNN or I go anywhere, you can see exactly what protocol every request is making. And I was surprised. I'm getting an awful lot of requests that are over HTTP3, and I think that's pretty cool. Um, so enough about that, because that's… that's interesting, but we've talked enough about it. So let's talk just a minute about cookies. Right? Not the kind… I'm sorry to make you hungrier. But, um, everybody's seen cookies, right? You see these… so do I. I'll take them anyway. Anyway you have them. Um, just as long as they don't have nuts.
Wolf:I love my chocolate chip cookies soft. and warm. Yeah, no nuts. No nuts. None.
Jim McQuillan:Yeah, yeah, um. Uh, cookies. Um, HTTP is a stateless protocol. Every request you make stands on its own. One request has nothing to do with a previous request. The server doesn't care. You're just making a request, and it serves it up.
Wolf:Right up until you need state.
Jim McQuillan:Yes, you make a request, it serves the response. You make another request, it serves the response. There's no connection over the protocol to link them together, right? So we have to do things like send cookies from the server. And the cookie is just this little payload of information. It's a header, and it's just a little bundle of information. It might be an ID number, it might be a Java web token, it could be a lot of different things. And the server sends it to the browser, and the browser gets it in the headers, and it stores it in the cookie jar in the browser. So then, every request from that point on to that server is going to include that cookie. on the request back to the server. So now the server knows who you are. So let's say you log in to a site. Alright? You go through the login, you put in your username, your password, you send the request to the server. And, uh, the server… authenticates you. Says, yeah, this guy's good. Uh, so it will… I don't know any other way to do this. The server will send back a cookie, uh, that identifies you as a user. It'll send back some information, usually a session ID.
Wolf:And, like, the existence of that cookie means you're logged in.
Jim McQuillan:Yeah, having that cookie, uh, means you're logged in. If you send a request to this server and you don't send that cookie along with it, they're gonna assume you're not logged in, and they're gonna offer you the login page, right? They'll probably send you a redirect 302 for the login, and…
Wolf:Right.
Jim McQuillan:And then they'll get you logged in. So… That's, that's, uh, that's one of the good uses of cookies. Um… We use them in our system. We have to. That's how we know that the user is logged in. That's how we know who they are. We know their session ID, uh, things like that. Now, there's a lot of people, a lot of businesses out there that use cookies in a more evil way. They use it for tracking. And that's just… that's just annoying, right? They… They know who you are from one site to the next.
Wolf:Yeah, I just looked at those boots. I don't actually want them.
Jim McQuillan:Right. In fact, in fact, I looked at those boots, and I bought those boots, and for the next 3 weeks, every time I visit a website, they're offering me ads about boots.
Wolf:But I don't want any more boots because I bought them!
Jim McQuillan:Because… because there's a tr… because I bought them, don't they know? It's… I just did this, I was looking for a new chair, and so I searched for a chair. And now every site I go to is telling me about chairs, and offering me deals on chairs. And that's cookies doing that. It's annoying, but cookies are important. So anything else we should say about cookies?
Wolf:But you should know. Your browser and your phone are not recording your voice and doing things to, uh…
Jim McQuillan:Right, right, they're not doing that at all, are they?
Wolf:That's not happening. What's happening is cookies.
Jim McQuillan:Yeah. I'm not convinced that they're not recording my voice. I understand how cookies work, and they make that happen, but…
Wolf:Huh.
Jim McQuillan:you know, there are times where I think it's reading my mind. It's not even listening to my voice, it's just reading my mind.
Wolf:Yeah.
Jim McQuillan:Uh, so, I don't know, is there anything else to say about cookies? I think we've…
Wolf:I don't think there's anything more to say about cookies, and I wish… there was a way to talk about this interesting thing that browsers do, all browsers, um, it's part of the JavaScript thing. which is that you can have a local private database, usually implemented with SQLite.
Jim McQuillan:Yes.
Wolf:I don't understand why the cookies don't go into that database, um, and it's not part of HTTP, and it's not part of HTML, it's just a thing, a service.
Jim McQuillan:Right. Well, why would they go in their database?
Wolf:What?
Jim McQuillan:Why would they go in that database?
Wolf:Well, I have… you know what a cookie is? It's an individual file. Why? Why couldn't it just be a row?
Jim McQuillan:Well, it depends on… it depends on how the browser does it. It doesn't have to be a file. It's just… it's just local storage someplace.
Wolf:It doesn't have to be, but it almost always is.
Jim McQuillan:Right. Okay. Have you been looking at source code for browsers lately? I know at one point you were intimately familiar with them. Anyway, um…
Wolf:I know a lot about browsers, let's just say that.
Jim McQuillan:Yeah. Yeah. Well, you know, a lot more than I do. Um, and I'd like to think I know a lot. Anyway, let's, let's…
Wolf:Yeah, it's weird that, like, I'm the browser guy, but this is your episode. I think that's interesting.
Jim McQuillan:I know, I know, I know. Yeah, I found it interesting. So that's why we're talking. So. So the next thing we want to talk about is cores. We.
Wolf:But go on, you had a couple more things to talk about.
Jim McQuillan:touched on it earlier. CORS, Cross Origin Resource Sharing. If you've done any web development, uh, especially if it's involving more than one site, uh, you've almost certainly run into CORS issues. And… uh… as you mentioned earlier, it's the bane of your existence. It's just a…
Wolf:I had to buy a book.
Jim McQuillan:It's… Did you?
Wolf:I did!
Jim McQuillan:It's… yeah. Um, I've run into it, and I've dealt with it, and I… I went down that rabbit hole, and I learned a little bit about it. I can't say I'm an expert at it.
Wolf:Mostly, you solve your problems by using CORE's rules.
Jim McQuillan:But I did learn enough how to make it work. Yes.
Wolf:Inside of your, um… uh, server configuration.
Jim McQuillan:The server. Right.
Wolf:So, for instance, it's Apache or, uh, HAProxy or, um, Squiddy that knows CORS and understands some CORS rules.
Jim McQuillan:Right. Sure. Yeah.
Wolf:But the core's rules are the same, no matter who's doing it.
Jim McQuillan:Right.
Wolf:And you know there's solving your problem the right way which is going to take experiments and time. And there's using the asterisk, and, you know, a lot of people are going to use the asterisk.
Jim McQuillan:Uh-huh. Stop using the asterisk. Yeah. Yeah. Yeah. Well, you, you, you'll bump into cores if you have a webpage that got served up by a server and you try to make a request, uh, on, you, you got JavaScript in your, in your webpage, right? And that JavaScript tries to make like an Ajax request or another request of some sort of fetch. to another site. Umm. And it can't, because it's cross-origin resource sharing.
Wolf:Like, for instance. You're part of your… here's just an old time example, but you rely on jQuery. Um, but you don't want to just include jQuery. You want the real, for sure, best jQuery. So, when you include jQuery as part of the set of things that you get, you can say, and get the one.
Jim McQuillan:Yep. Sure. Yes.
Wolf:from the actual jQuery website.
Jim McQuillan:Or from a CDN or something, right? Anyway, you're not getting it from your site.
Wolf:Right, from a CDN. But that is NOT! That is not the domain YOUR page comes from. So suddenly, it is a cross-origin.
Jim McQuillan:Right, right.
Wolf:Resource.
Jim McQuillan:Right. Right. Uh, browsers enforce this thing called Same Origin Policy. That's the browser that's requiring all, all the fetches to be from the same, um, the same host, the same, uh, protocol, whether it's HTTP or HTTPS. What is that, the scheme? Um, and the same port number. So you can't have, like, a server listing on several different ports, and you fetch some of the things from one port, and other things from another port. Those are different servers, uh, to the browser. And it will refuse to fetch that, unless you set up cores. And that involves, like Wolf said, you gotta… your server has to send back some headers. The browser will make a request of the server, it's an options request. And then in that response coming back, there will be things like allow… access control, allow origin. And that's where you can list the origins that are allowed. you're allowed to hit. So if you had two servers, you could list both of your servers there. So you make a request of one, and that page requests the other, and it's fine. But like Wolf said, most people solve it by putting an asterisk there, which basically. subverts the entire same-origin policy. And it's not secure, but people do that to get around it.
Wolf:Yeah, it's super convenient, but absolutely don't do it. In Python, there's an equivalent. You import the stuff that you need in a particular file.
Jim McQuillan:Yeah, yeah, yeah. Yep. Right.
Wolf:Um, and a thing that you can say is, you know, from foo, where foo is some package, import star, import asterisk.
Jim McQuillan:Yeah.
Wolf:And, um…
Jim McQuillan:And that, that, that gets everything and the kitchen sink and is a waste.
Wolf:Don't do that. Yeah, uh, and there is a reasonable place to use that. When you are writing a test for foo, a test file.
Jim McQuillan:Yep.
Wolf:then it's reasonable to from foo import star. But any real thing, no.
Jim McQuillan:Right, right. But in production code, yeah, yeah, yeah. So if you get stuck dealing with cores, there's a lot of information out there, a lot of confusing information out there. I've been down that path. Uh, it's no fun. I solve this, uh, uh, a different way. Like I mentioned earlier, I use HA proxy as my, as my front end. That's the front door to my server. And if I need to pull data or, or resources from multiple servers. Behind the, you know, back in the, in the data center. All of my requests just go to HAProxy.
Wolf:What is that? Ignore cars?
Jim McQuillan:And then I let it deal with it. No, because all of the requests are coming from the same host, scheme, and port. And if behind the scenes.
Wolf:No, no, but, like, what you're saying is, all the backend servers that are behind HAProxy.
Jim McQuillan:Huh-huh. No.
Wolf:They see what HAProxy gives them, so it looks to them like everything is the same host, scheme, and port.
Jim McQuillan:Mmhm. Well, the servers behind it don't care. This is a client issue. It looks to the client like everything came from that one server.
Wolf:But.
Jim McQuillan:And then HAProxy fans out those requests to multiple backend servers. So it's, uh, remember CORS is a client side issue.
Wolf:Yet for a second that left my mind. I apologize.
Jim McQuillan:Yeah, yeah. It's that same origin policy. Well, if everything is funneled through your HA proxy setup, it just works beautifully.
Wolf:I was wrong.
Jim McQuillan:I initially got into HAProxy, uh, because I, you know, we use HTTP, obviously, uh, but I needed some real-time communication between my client and my server, and I needed it to be bi-directional, and, uh, so this is a long time ago. This is, this must have been about 2010 or 11 or so. Uh, I started looking into WebSocket. And that seemed like an interesting thing to look at. And WebSocket is a protocol that runs… it runs over the top of… it… it… you initiate it by starting a HTTP request.
Wolf:Inez.
Jim McQuillan:And then you upgrade the protocol. You don't have to do this. The WebSocket library does this for you or the browser does this because it knows about WebSocket and it issues a protocol upgrade request and it switches the connection to WebSocket. And now you've got this beautiful message-passing channel between the client and the server, and you can send requests to the server, and the server can send messages back to you, and it's just message-passing. Uh, we did this because we needed to, uh, have… basically, our first use of it was a lock broker. You know, the thing about pulling up patients on a page, you want to make sure that you pull up that patient on your page, and let's say you're scheduling a new appointment for that patient. You don't want somebody else also scheduling a patient for the same set, scheduling an appointment for the same patient. So we need to lock that patient. Well, there's all kinds of crazy schemes for doing that, but we chose to do it with WebSockets and we have Node.js on the back end. that's serving up the WebSocket connection, and Node.js is sitting there, and it's running, and it has state, and you request a lock on that patient, um, the way we've written it. You request a lock on that patient, and Node.js. stores that information. It grants you that lock. It's just a little piece of memory, and you have that lock, and now you can schedule the appointment. If somebody else tries, they're going to request the lock. They don't get it. Node.js says, you can't have it because somebody else has locked it. So it'll return back a response saying. you can't have this lock, because Mary Jane has the lock already. So it works out really well for us. We have this locking thing, and we use it for a lot more now. But, it's all over WebSocket.
Wolf:A nice thing about WebSockets.
Jim McQuillan:Mmhm.
Wolf:Is that, um… Once you have a connection, that connection lasts until you tear it down.
Jim McQuillan:It does, it's a persistent connection.
Wolf:But it lets you do… The equivalent of… um, the server can push to you interesting things. So, for instance, if you have 13, um.
Jim McQuillan:Exactly. Exactly.
Wolf:individual… I don't want to say terminals, but people looking at the scheduling page, like your… your employees.
Jim McQuillan:Yep.
Wolf:A thing that you could have done is when, uh, one of those 13 users, um. who has the ability to lock, locks patient blah, blah, blah, or uses room whatever, or whatever it is.
Jim McQuillan:Mmhm. Huh-huh. Mmhm. Huh-huh.
Wolf:You could actually make it be that it doesn't have to be that any of the other 12 terminals, whatever they are, um. tries and gets refused, you could push to them immediately. Room Y is unavailable during time Z.
Jim McQuillan:This. this is exactly what… we do this, exactly. Uh, when a… when a client, uh, brings up the screen, um, the… the client end, uh, sends a message to the server over WebSocket saying, I'm interested. in this resource. The resource might be a patient, uh, it might be a to-do request, it might be all kinds of different things. Basically, I'm interested in this resource. And so now, if anybody locks that resource, you'll get a message on your screen showing that that's currently locked by Mary Jane. Uh, so, and our software also deals with, now if you try to click on that and do something with it, it doesn't even send a request to the server, because it knows that resource is locked. And it… and it's… it's instantaneous that the Node.js server on the back end has this thing in memory, knows everybody that's connected. And the neat thing about these connections is if the connection goes away. we detect that. Node.js detects that. The WebSocket. server end, detects that the connection went away, and it releases the lock. So there's no more of these hanging locks, right? I mean, what's another way to lock these resources? Go update a table? Go update a flag in a table someplace? Right? You know how much of a problem that is.
Wolf:Yep. I do. WebSockets, um… Elevates, uh, what, what is a. acceptable but not perfect match protocol to something that is way more natural for certain kinds of problems.
Jim McQuillan:Yep.
Wolf:And suddenly, you are using a tool that fits the task so well, it just feels right. If you have a place where WebSockets might help, absolutely look into it, because WebSockets rock.
Jim McQuillan:It's… Yeah, if… Yeah, it's good for things like dashboards and stuff, where you're monitoring the status of things. you don't have to do poll requests, you don't have to do long polling, and make requests to the server, and just sit there waiting for a response. You'll get a message when the status changes. Uh, you know, assuming you wrote the software to, to push that message out. But, uh, it's, it's been great for us.
Wolf:Yeah, so WebSockets are great when you have a hundred or a thousand clients, less great when you have 10 million.
Jim McQuillan:There. Well, it's a lot of traffic. Right? A lot of… it's a lot of open connections, right? But, uh, you know, chat programs use WebSockets. That's how, you know, when you're, uh, when you're chatting with somebody, uh, with a desktop thing, like Twitter or Mastodon or whatever. and somebody else is typing, and you see the little dots, animated dots, showing you that they're typing, that's all WebSocket stuff, sending you that information.
Wolf:Yeah, a push from WebSockets.
Jim McQuillan:Uh, yeah, and it's really nice. So if you, if you, if you find that you need, uh, that kind of interactiveness between your client and your server, I, I highly advise, uh, take a look at WebSockets. Uh, we, we've been using them for 15 years and they work. really, really well. So, let's talk a little bit about tools. There are a lot of tools you can use when you're playing around with HTTP. One of the very, very popular ones is Curl. Right?
Wolf:Well, old school, popular with a certain population, where that population is Jim and friends.
Jim McQuillan:I mean. Okay, it's… it's… hey, yeah, and friends, and all the cool kids, right? It's a command line… it's a command line tool that exists everywhere. My car has curl built into it. If you get in my car, and you go through the… the settings thing on the console, and there's a place for licensing. There's a mention of curl with Daniel Stenberg's name on it. It's hilarious. Curl is in use.
Wolf:Who, by the way, Daniel Stenberg is one of the pillars of the internet.
Jim McQuillan:He is. He really is. He's in Denmark, I think, and he is the leader… he's the founder of Curl, and he does a fantastic job. I follow him on Mastodon, and it's just a blast to see what he has to say. He's got all kinds of great things. And Curl… It's just a fantastic tool. The neat thing about curl is you can, remember we talked about headers and request bodies and all that kind of stuff. There's options to the curl command line that you can see all of those headers as they go in the server request and the, and the response. So if you want to learn something about HTTP, curl is a great way to do it. It'll show you what's going on.
Wolf:And. Let me tell you the two most important option flags for curl.
Jim McQuillan:Okay.
Wolf:Minus capital L and minus O. I think it might be minus capital O. Minus capital L means follow redirects, as many as are allowed.
Jim McQuillan:Yes. Yes.
Wolf:And minus capital O means when you fetch this file, automatically name it, because cURL will download the file. Automatically name it according to the leaf name of the actual resource.
Jim McQuillan:Right. Okay, I just read something about this the other day, that it does its best to come up with a meaningful name for that file.
Wolf:If you use minus capital O, yes.
Jim McQuillan:Uh, which… Yeah. Yeah. Which I think is what they think is pretty cool. So that's, that's one of the tools. It's a very, very useful tool. Uh, Postman. I don't have a lot of experience with Postman. You've used it, though, haven't you?
Wolf:I have. Postman is…
Jim McQuillan:It's like an API testing tool.
Wolf:it's a GUI tool, um, it's not unlike curl, um, but the nice thing about… Postman is for regular people. You know, you and I can use curl all we want, but a person who.
Jim McQuillan:Yep. Yep.
Wolf:is not CLI… um, aligned, Postman might be the right thing. It's an app, and you can establish, um.
Jim McQuillan:Mmhm.
Wolf:specific kinds… specific, um, preferences in it. Like, this is the connection, this is the thing to send, it's a thing.
Jim McQuillan:Mmhm, mmhm.
Wolf:you pick that thing, and then you click send, and it does it. Um, and you can save bunches of those, and you can arrange by server, and there's lots of things that you can do. Basically, same capabilities of curl, but in a GUI form. Well, not everything curl can do.
Jim McQuillan:Yes. Yeah, is it…
Wolf:Super useful.
Jim McQuillan:Yeah, but… But would you say it's for testing APIs, RESTful APIs, or is it more than that? I always think of it in terms of an API… making an API request.
Wolf:It's fundamental. Fundamentally, every way that I've used it is about testing RESTful APIs.
Jim McQuillan:Yeah.
Wolf:Umm. But, you know, I think underneath the covers, it basically is curl, I think.
Jim McQuillan:License. I suppose you could make any request. Yeah, you. Yeah, every time I've looked at it, it's like, uh, the Postman site, I think, wants you to have a subscription, have a… have an account, and I've just…
Wolf:Yeah, you don't need an account, um, I can't remember what an account gives you, but there is, like, you can use it for free, I just don't know what the difference is between free and paid.
Jim McQuillan:I didn't do that. I think it remembers your stuff. Yeah. Yeah. Well, the thing that I use that's kind of like Postman is, uh, for a lot of my work lately, I've been using VS Code. the stuff that I'm editing locally. Uh, but there's this, this thing called OpenAPI, uh, where you have an OpenAPI YAML file that describes your API. It describes all the endpoints and the authentication method and, and all, you know, all the parameters for the endpoints and stuff. And then VS Code with the OpenAPI plugin is a great way to test your APIs. I just love doing that, if you have an OpenAPI YAML specification file. most of the sites that I've been doing APIs with have that. Um, if you're doing, like, Square, or, uh, uh, uh, who's the other place? Uh, uh, Twilio.
Wolf:Like they've got the YAML file already.
Jim McQuillan:they've got the YAML file. You download that, and you open it with VS Code, with that plugin, and it just gives you a really nice API, a nice UI.
Wolf:Boy, is that nice.
Jim McQuillan:Oh, it's fantastic. It's like the most powerful thing about VS Code for me. I think it's so cool.
Wolf:So I'm not a VS Code user, and that's mostly about the fact that it's an Electron app, which is unacceptable to me.
Jim McQuillan:Right, right. Yeah, that's a… that's a turn-off for you.
Wolf:and secondly, it comes from Microsoft, but, um, I will say, uh, a thing about VS Code is. it's great at rendering Markdown, even, and especially this is important, if your Markdown has mermaid diagrams in it. Great! You see them! They look terrific!
Jim McQuillan:Yeah. Yep. Yeah.
Wolf:Um, so, um, I want to not like VS Code.
Jim McQuillan:It's a… it's a cool tool.
Wolf:But I can't. VS Code's good.
Jim McQuillan:It is. I like it.
Wolf:I don't use it as my editor, because of course I use Helix, but uh…
Jim McQuillan:Right, right. I'm using Vim 80% of my day, but I do some VS Code stuff when I'm doing TypeScript. It's a great thing. And of course, I use Xcode for doing my iOS Swift. work. But, uh, anyway, that's a great tool. So you mentioned a tool, uh, called HTTPIE. HTTPIE?
Wolf:HTTP, right.
Jim McQuillan:Yeah. Tell us about that.
Wolf:But spelled weird. So, it's a Python module, and um… Awesome. CLI, I think, I'm pretty sure. Uh, I haven't used it that recently. But, um, it's another way to do the curl thing. Um… Uh, and, you know, direct access into Python, so that's useful. I don't think there's too much to say about it. It's popular.
Jim McQuillan:Yeah, okay. It's out there. If you're Python aligned, that's a tool that you can use. Um, it's probably not the tool I would reach for, just because I'm not a Python developer. Uh, and I mentioned earlier TCP dump and Wireshark. Uh, TCP dump and Wireshark.
Wolf:But now talk about the Big Daddy. Yeah.
Jim McQuillan:Yeah. I, you know, I've got a, yeah.
Wolf:Like TCP dump, okay. Wireshark, oh my god. If you're not using wire… if this is your thing, and you care about network stuff, and you're not using Wireshark, um…
Jim McQuillan:We. We. Yeah, Wireshark is fantastic. We have a friend of the podcast, Scotty. He's been a friend of mine for a long, long time. And he always says any day he gets to get out the soldering iron to do work is a good day.
Wolf:What is wrong with you? You need Wireshark.
Jim McQuillan:I feel like that with Wireshark, and with TCBDump as well. Any day that I get to get Wireshark to diagnose a problem. That's a good day. I like that tool. I like using it. It makes me feel very, very powerful when I use that. And it's great for doing HTTP. It will decode the HTTP packets for you and lay them out really nicely. And I learned something a while ago, you know, if you're doing HTTPS, you know, we didn't even talk about that, secure HTTP. Everybody's doing HTTPS, right? Where you have a… it's HTTP over TLS, where you've got a certificate. And… and everything you send back and forth is encrypted and… and authenticated, and… and everything's… Uh… uh…
Wolf:Well, not everything. Because it has to know the destination.
Jim McQuillan:Well, the IP address. Everything else is encrypted.
Wolf:Well, it has to know the actual, um… hostname as well.
Jim McQuillan:Yes. Yeah, but I wonder what it does with the rest of the headers. Does it know… it doesn't know the rest of the headers. I don't know.
Wolf:Uh, for almost all of them, it does not get to see them, that is correct.
Jim McQuillan:I'm not sure. Yeah. Okay. So, um, and, and we didn't talk about how, uh, uh, you have to have a certificate for HTTPS and the, and the most popular way to do that right now is with let's encrypt, uh, or for free, you can get your, your, uh, HTTPS. uh, uh, certificates, uh, X509 certificates, and it's, and it's all great. Um…
Wolf:Yeah, RSA was making a ton of money, and that all went away instantly.
Jim McQuillan:Oh, lots of companies were making a ton of money. Mark Shuttleworth, the guy that brought us Ubuntu and a few other things, that's how he made his money was by selling certificates, generating certificates at his company in South Africa. And he sold it in like, I don't know, 2004 maybe for an awful lot of money. And that's, that's what fueled his, his other projects. But anyway, now I don't know if anybody's making money on that anymore because of Let's Encrypt. It's a great way to, to, uh, create, uh, certificates for doing HTTPS. Uh, but my point is.
Wolf:Although they don't last very long.
Jim McQuillan:No, they don't, but that's, that's a, that's the, that's not their fault. It's not let's encrypt's fault. That's a shortening of the validity length that I don't know who decides that the IETF maybe. It's gonna get down to the point where I think certificates are gonna be valid for no more than, like, 40 days. Um, and so Let's Encrypt has this entire mechanism for… for, uh… fetching new certificates, we're updating the certificates. Um… So it doesn't matter if they're good for 40 days or 15 months or whatever. But if you're ever trying to debug a problem with HTTPS and you're using TCP dump or Wireshark or something like that, it's all encrypted. You don't get to see it. You can tell that there was some HTTP traffic. You can see the IP address, but you can't see much else. Uh, but there is a way to, uh, get the browser to dump the negotiated keys that it used, and feed that into, uh, uh, Wireshark, so that, like, you can capture. a packet stream from a conversation, and then the browser can give you the keys that it used for that conversation, and you can, like I said, feed that into Wireshark, and now, after the fact, you can decode that. Or you can decrypt that session. I had to do that a while ago, and it kind of blew my mind that I was able to do that, and it's pretty neat.
Wolf:You do need, you know, a browser that can do this, and you will need to turn on debug mode.
Jim McQuillan:Yes. Um, yeah. Yeah, you have to, like, enable it in the browser. Um, but it's absolutely possible. Um…
Wolf:Safari on your iPhone is not going to help you.
Jim McQuillan:No. Not likely. But, uh, for the situation that I had, I needed to debug something, and I had to do that, and it was great. But anyway, there's some tools, they're all useful if you have to get in and figure out what's going on. Um, so, I don't… we covered a lot, it's late, I think we need to wrap this thing up. Uh, and in summary, I just wanted to say, you know, HTTP was originally created, uh, to fetch documents off a server at CERN. And boy, it's turned into so much more than that. Our world is built on it.
Wolf:Yeah, now it can do ads.
Jim McQuillan:And it can track you! Yeah, yeah, it's so much better!
Wolf:And they can sell you multiple pairs of boots.
Jim McQuillan:Yeah, alright. Uh, so why don't you take us out, Wolf?
Wolf:Um, well, uh, first, I should say, if you got this far, and even if you didn't, thank you so much for listening. The whole reason we do this. Uh, well, you know, we love to talk to each other, so that's, that's a thing. But the reason we turn it into something everybody can listen to is because. people seem to care and get something out of it, and if you listened this far, that is you. You do care, and thank you so much. caring. We want your feedback. Send us email feedback@runtimearguments.fm. Um, we have a website that lists the episodes. It is http colon slash slash runtime arguments dot fm. Yes, I know, it's not https that has to do with our hosting. provider. You can send us Mastodon messages or email all of that contact information. will be in the show notes. If you're listening to this show, you've already got the show notes, you can see them right in front of you, and there will be a transcription. Um… I had a good time on this episode. Not a topic I would have chosen. But absolutely a fun conversation. So that's goodbye for me. Do you have anything left to say, Jim?
Jim McQuillan:Yeah. Yeah, you know, way back at the beginning, seemed like a couple hours ago now, I mentioned about how if you're out riding your bike, or if you're walking a trail or something, um, say hi, right? Nod. Wave. That's a form of feedback. Uh, if you're listening to this episode, and, uh, you happen to send us a little bit of feedback, just, uh, hey, I enjoyed it. Or, hey, that episode sucked. That's the same kind of thing, right? It's easy to do. It doesn't cost you anything.
Wolf:Or this was wrong, or why didn't you talk about this, or…
Jim McQuillan:Yeah, right, are you, yeah, here, why are you, why are you guys such knuckleheads? Just, uh…
Wolf:Every bit of that is super valuable to us.
Jim McQuillan:feedback doesn't cost you anything, let us know that you're out there. It really means a lot to us. So, yeah. So, hey, as always, Wolf. Thanks for podcasting with me. It's always a fun time.
Wolf:Thank you.
Jim McQuillan:All right. Goodbye, everybody.
Wolf:Goodbye.
People on this episode
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
CoRecursive: Coding Stories
Adam Gordon Bell - Software Developer
Two's Complement
Ben Rady and Matt Godbolt
Accidental Tech Podcast
Marco Arment, Casey Liss, John Siracusa
Python Bytes
Michael Kennedy and Calvin Hendryx-Parker