{"version":"1.0.0","segments":[{"startTime":0.16,"endTime":3.4,"body":"The rise of AI crawlers and their dance with"},{"startTime":3.4,"endTime":7.2,"body":"WordPress sites. Welcome back, tech enthusiasts. I'm your host,"},{"startTime":7.2,"endTime":9.68,"body":"Greg Doig, and today we're going to talk about"},{"startTime":10.32,"endTime":14.08,"body":"WordPress and AI crawlers. In the"},{"startTime":14.08,"endTime":17.68,"body":"vast digital world, websites are like bustling cities filled"},{"startTime":17.68,"endTime":21.4,"body":"with information. For decades, search engines sent out"},{"startTime":21.4,"endTime":25.2,"body":"their crawlers, which are automated bots, to explore"},{"startTime":25.2,"endTime":28.94,"body":"these cities. Indexing pages so people could find what they needed."},{"speaker":"But now a new kind of visitor has arrived","startTime":29.26,"endTime":32.46,"body":"AI"},{"speaker":"But now a new kind of visitor has arrived","startTime":32.46,"endTime":36.06,"body":"crawlers. Unlike traditional bots that index pages for"},{"speaker":"But now a new kind of visitor has arrived","startTime":36.06,"endTime":39.62,"body":"search rankings, AI crawlers scan websites to gather"},{"speaker":"But now a new kind of visitor has arrived","startTime":39.62,"endTime":43.1,"body":"content that trains large language models or"},{"speaker":"But now a new kind of visitor has arrived","startTime":43.1,"endTime":46.82,"body":"provides fresh data for AI-powered answers. There"},{"speaker":"are two main types","startTime":46.82,"endTime":50.62,"body":"training crawlers like OpenAI's GPT"},{"speaker":"are two main types","startTime":50.7,"endTime":54.39,"body":"bot, which collect data to teach AI how to answer better,"},{"speaker":"are two main types","startTime":54.63,"endTime":58.39,"body":"and live retrieval crawlers, which fetch real-time information"},{"speaker":"are two main types","startTime":58.95,"endTime":62.59,"body":"when you ask AI tools questions. WordPress"},{"speaker":"are two main types","startTime":62.59,"endTime":66.15,"body":"sites are especially friendly to these crawlers. Because WordPress"},{"speaker":"are two main types","startTime":66.15,"endTime":69.43,"body":"generates fully rendered HTML pages on the server,"},{"speaker":"are two main types","startTime":69.59,"endTime":73.15,"body":"AI crawlers can easily read content, metadata,"},{"speaker":"are two main types","startTime":73.15,"endTime":76.47,"body":"and navigation without extra effort. This"},{"speaker":"are two main types","startTime":76.47,"endTime":80.22,"body":"interaction is changing how websites get discovered. AI"},{"speaker":"are two main types","startTime":80.22,"endTime":83.98,"body":"crawlers can drive new visitors by powering AI tools that"},{"speaker":"are two main types","startTime":83.98,"endTime":87.58,"body":"summarize and link back to your content. But it also raises"},{"speaker":"are two main types","startTime":87.74,"endTime":91.34,"body":"questions. Should you let AI bots access everything?"},{"speaker":"are two main types","startTime":91.98,"endTime":94.94,"body":"Or protect sensitive areas like login pages?"},{"speaker":"are two main types","startTime":95.18,"endTime":98.74,"body":"Fortunately, site owners have control. You can manage AI"},{"speaker":"are two main types","startTime":98.74,"endTime":101.74,"body":"crawler access by editing your robots.txt file,"},{"speaker":"are two main types","startTime":102.78,"endTime":106.5,"body":"using WordPress plugins designed to block or allow specific"},{"speaker":"are two main types","startTime":106.5,"endTime":110.04,"body":"bots, Or leveraging services like Cloudflare's"},{"speaker":"are two main types","startTime":110.04,"endTime":113.12,"body":"Bot Fight mode to keep unwanted crawlers at bay."},{"speaker":"are two main types","startTime":113.6,"endTime":117.44,"body":"AI crawlers are here to stay, shaping the future of online discovery."},{"speaker":"are two main types","startTime":117.68,"endTime":121.48,"body":"With smart strategies, WordPress sites can harness their power"},{"speaker":"are two main types","startTime":121.48,"endTime":125.08,"body":"while protecting what matters most. The future of the web is"},{"speaker":"are two main types","startTime":125.08,"endTime":128.92,"body":"intelligent, and it's crawling your way. Once again, thanks"},{"speaker":"are two main types","startTime":128.92,"endTime":131.44,"body":"for stopping by and listening, and we'll talk to you soon."}]}