Tank Talk - Alaska's Bulk Fuel Podcast

I used AI to write my SPCC

• Integrity Environmental • Season 3 • Episode 4

Share episode

In this episode of Tank Talk, Shannon and Cassie dive into the hot topic of AI and environmental compliance. Cassie Kuzis, part of the Integrity Environmental team, joins the podcast for the first time with Shannon to weigh in on AI. With AI tools becoming more common in operations, Shannon decided to run a little experiment: Can ChatGPT write a Spill Prevention, Control, and Countermeasure (SPCC) Plan? 

Spoiler alert - it didn’t go well. 

Shannon walks us through the results of three AI-generated SPCC plans, revealing the major compliance failures, regulatory inaccuracies, and risky oversights that came with each attempt. From missing signatures and misapplied containment definitions to false statements and oversimplified training requirements, the experiment highlights why environmental plans and permits should always be developed by qualified professionals. 

Tune in to hear: 

  • Why SPCCs are legal documents - and why that matters 
  • Where AI went wrong (and what it surprisingly got right) 
  • What you should do when considering tech support for environmental planning 

This is a must-listen for facility managers, compliance officers, and anyone curious about the limits of AI in regulated industries. 

Additional Resources & AI Prompts from the episode:

https://www.integrity-env.com/i-used-ai-to-write-my-spcc 

Support the show

intro/outro created with GarageBand

Haley Hall:

Welcome to Tank Talk with Integrity Environmental. Join us as we sit down with founder, principal consultant and bulk fuel storage expert, Shannon Oelkers, to explore regulations, safety and essential tips for navigating the bulk fuel storage industry. Join us as we explore the unique joys of work and life in Alaska with industry experts, including our team, vendors we work with and the companies we support.

Shannon Oelkers:

Well, hello, welcome back to Tank Talk. This is Shannon Oelkers, your host, and today I'm joined by Cassie Kuzis. She's a member of our sales team and her and I today are going to do something really interesting. I used AI, Cassie, to write an SPCC plan. Whoa, I know right. Like, should we do that? I don't even know. So before we begin, Cassie, I do want to take a minute to just highlight something about Integrity and AI. Before we continue any further, at Integrity, we want to make it clear that, due to the highly regulated and legally complex nature of environmental planning and permitting, we never use artificial intelligence to develop environmental plans or permits.

Shannon Oelkers:

All of our work in this area is carried out by experienced environmental professionals who understand the nuances of regulations and the unique requirements of each facility. I want to reassure everybody we're committed to ensuring that every project we undertake meets the highest legal and ethical standards, and we remain dedicated to providing expert, human-driven solutions to our clients. So there's all the legal stuff out of there, but I still used AI to do something fun. So I used ChatGPT that's the AI that I have access to for free to write an SPCC.

Cassie Kuzis:

What inspired you to do that?

Shannon Oelkers:

Well, it was part natural curiosity just to see what it would spit out. But, Cassie, I was also interested because of how we see clients beginning to use AI to support operations. You know, I had a feeling that someone somewhere was going to have the bright idea of having AI help them draft up an SPCC.

Cassie Kuzis:

Oh yeah, that seems very likely. So what happened? What did you learn?

Shannon Oelkers:

Well, I learned a whole lot, but before we dive into the findings, though, I wanted to provide some background information that I think is relevant to this, which is part of what I learned, and everybody who listens to this podcast knows how much I love me some context.

Shannon Oelkers:

So first I want everyone to remember and we've talked about this before in some of our other podcast topics but at the fundamental level, a spill prevention, control and countermeasure plan, or SPCC plan, is a legal document. It's part of your agreement with the federal government to store fuel in bulk and meet the regulations of that 40 CFR 112. So all of the warnings that apply to legal documents will also apply to environmental planning and permitting. If there is a legal compliance or enforcement arm to a regulatory agency, any plan or permit you have through that is going to be a legal document.

Cassie Kuzis:

Gosh, it seems like it might be a bad idea to use AI for legal documents.

Shannon Oelkers:

You would be correct. Yes, for one thing, the AI model is really dependent, Cassie, on the information that has already been given to it. I have noticed that, especially for things like environmental permitting, there's a lot of misinformation on the internet or overly simplified information. There's not a whole lot of information about our industry, operations and infrastructure that's available to, like most AI models, in a context that would make sense for a planner permit. I'm sure there's some very sophisticated companies out there that have paid for their own learning management systems and AI and they're training them on things like this.

Shannon Oelkers:

However, even if the AI had good inputs, you have to have well phrased prompts, and if you don't have well phrased prompts, your AI can return nonsense answers with a really high degree of confidence, based on the inaccurate data that the AI has access to.

Shannon Oelkers:

So the second half of this equation is, if you don't know exactly what you're asking for, your prompts give you these really murky answers. So most of the people we deal with you know they understand and know what an SPCC is, but they don't necessarily understand many other environmental permitting regulations that are out there, like NEPA or stormwater. And if you ask a pretty vague question like write a spill plan for my facility, which sounds a whole lot like. Write an SPCC for my facility. Your learning management system or AI will return a compliant plan for something completely different, like hazardous material spills instead of bulk fuel oil or something like that. It might ignore fuels entirely. We call it the garbage in, garbage out problem. If you're not asking the right questions in the right way, you're not going to get the right things coming out of it.

Cassie Kuzis:

So that makes sense. Don't use AI for complex legal documents.

Shannon Oelkers:

I feel like there's a lot of content on the internet already reinforcing that, but I did want to cover those basics because that's the first problem with using AI for environmental permitting is that it's legal documents in addition to environmental permitting. So there's some reasons why we shouldn't do that. But now that we've covered those basics, I still decided to do it anyways, and here's why I wanted to see what it put out and I also wanted to see what the quality of the output was and then kind of discuss it here on the podcast so we could all learn from it.

Cassie Kuzis:

Shannon, what did you start off with?

Shannon Oelkers:

Well, when I went on to AI, I originally used one prompt and we did use chat, gpt, but I felt like the one prompt wasn't robust enough, so I actually went ahead and did two more. So I have three prompts and I will include all of the nominees air quotes here for those of you who can't see me for the plans that these three prompts came out with. And then I added some super fun commentary in the Word documents, just so you can. You know, you can judge along with me as I went through the plans, and those will be included in our show notes if you want to open them up and see what came out yourself, just out of curiosity.

Shannon Oelkers:

So here's the three prompts I use Cassie, the first one, very basic write a spill prevention, control and countermeasure parentheses SPCC plan in accordance with 40 CFR 112. Then my second prompt I was like okay, that was very general, let me, I'm going to go a little deeper. So the second prompt I said write a spill prevention, control and countermeasures plan in accordance with 40 CFR 112 for a facility with one 4,000 gallon tank storing gasoline and one 6,000 gallon tank storing diesel, plus four drums, 55 gallons in size, storing motor oil, 5w30 motor oil. And then my last prompt I kind of wanted to see what happened when I took the tanks out, so I said write a spill prevention, control and countermeasures plan for a facility with 20 isotopes that are 330 gallons each stored in a warehouse.

Cassie Kuzis:

Okay, so we've covered a few different options. How did they show up?

Shannon Oelkers:

Overall for all three plans that the chat GPT generated. It wrote pretty much what I expected. It spit out this like bare bones skeleton report with a lot of direction to the writer to fill in the blanks. Right, but it did have some significant errors and assumptions and there were a few surprises where it did better than I thought.

Cassie Kuzis:

What were the significant errors you found?

Shannon Oelkers:

Well, I'll go through what all three versions missed. I think that's the key part. So some things were captured in some of the responses and some were missed in others, but all of the versions missed. They did not have a complete required commitments and signatures section and I'll go into these in more detail. But there are certain things that the regulation says have to be signed and statements that have to be made as part of the SPCC plan. None of these three plans had a complete and compliant section there.

Shannon Oelkers:

None of the plans captured the principal engineer review correctly for SPCCs. All three of them were missing required content. It wasn't all the same content that was missing, which I find really interesting, but all of them showed that there was a couple sections missing that were supposed to be there and it was independent of the containers, like they all should have had certain sections, regardless of what type of fuel in the containers they were storing it in. But it was still missed. The other thing all three of these missed was referenced regulations and engineering standards. So in a regulation it'll say things like inspect the tank to best engineering practices or recognized engineering standards or, in accordance with 40 CFR 63, ensure the tank venting is appropriate, all right. None of those cross-referenced regulations and engineering standards came across at all. They were invisible to the AI. If it was cross-referenced or referenced to somewhere else, it just didn't follow that trail at all or even make a statement about it. It's just invisible.

Shannon Oelkers:

There was also some false statements in the content and all three of the plans and you can go to my commentary in each of the plans to see. But they made a lot of sort of truthful sounding statements but not really. And then there was definitely some wrong context for certain words, like the word containment for spill prevention or sorry, spcc plans has a very specific definition in the plan but AI was applying the more general across the whole internet approach definition of containment and same with drainage and a few other words where you could just tell, by the way the AI was writing it that it was not using the specific definition within 40 CFR 112. It was looking at it from when people on the internet write the word drainage what they mean.

Shannon Oelkers:

Just a couple other things. None of them address substantial harm determination and that may be because it's actually included in a different set of regs, but every SPCC has to include it to show that they do or do not need a facility response plan to go with the SPCC. So that was a little funky. I was like it needs to be in here, but it's not. And then the last thing I noticed amongst all three of these was that many of the sections like employee training and figures they were so generally listed that you couldn't actually create a compliant or adequate figure or a training program out of what they spit out.

Cassie Kuzis:

Gosh. Well, that's quite a few problems.

Shannon Oelkers:

It really really is. Stormwater compliance on a bulk fuel site isn't just a box to check, it's critical. Integrity's CGP SWPPP services make it simple. They deliver fast, professional plans designed for real-world use so you can stay compliant without any headaches. Our team has decades of experience, works directly on job sites and knows exactly what it takes to keep your project running smoothly. Plus, every bid includes a free hour of consulting because when issues come up, you're going to need those answers and fast Stay ahead of compliance. Get a bid today. Send us an email at info at integrity-envcom. Integrity stormwater solutions you can trust. If you're game Cassie, I can dig a little deeper on these and just sort of go through my findings.

Cassie Kuzis:

Yeah let's dig in Okay.

Shannon Oelkers:

So the first one was like the inaccuracy in signing and commitments. They basically had an inconsistent listing of the responsible person and then they had varying levels of committing resources and attesting to the accuracy of the content. Prompt one that plan that was generated didn't have any place for the owner or representative to sign. Prompt two had the owner and rep signature but not the principal engineer. And then the prompt three version had it had a prepared by, reviewed by block, but it was missing the attestation for accuracy and commitment of resources altogether. So it was sort of this weird like parts of it came through but not all of it for any of them. For the principal engineering review requirement those of you familiar with SPCCs will remember that 40 CFR 112.3 has specific requirements for PE review and there's exemptions for facilities under 10,000 gallons where you can do self certification. But self-certification, you know, of course, only applies if you're fully compliant. If there's anything that's non-compliant and you have to have deviations for any reason, then the PE has got to get involved. So it's a little complicated but none of the plans even addressed any of that. It just kind of AI got it wrong every which way. So for prompt one, ai stated that the PE review was required, but it didn't touch on volume threshold exclusions and that prompt was pretty vague, it didn't include any volume information. So I would have expected it to say PE review, you know, applies if over 10,000 gallons or these conditions exist, because that's how the reg is writ. Didn't show up in the AI Prompt. Two did need a PE stamp. It exceeded the 10,000 gallon criteria but it wasn't addressed at all in that prompt. And then the third one it did not need a PE stamp but it didn't include language that would identify the facility as a tier one and self-certifying, which is the alternate to the P stamp. So all of these plans that were generated, they just didn't really get the PE part correct, which you know. Fair enough, that's one of the more complicated pieces For not including the required content. All three of these plans were just missing whole sections on like facility security, employee training, spill prevention. What exactly got skipped varied a little bit between the different prompts, but not one of them had all of the required sections.

Shannon Oelkers:

There's also reference regulations and engineering standards that were missed. The biggest one that we saw was that if you have a bulk fuel tank under an SPCC plan, there's required third-party inspections under STI SP001 or API 653 for the different kinds of tanks and at different volumes of tanks. None of the plans touched on that at all. It was invisible, not sure what else to say, but if you wrote it you would have no idea that you needed to have your tanks inspected in accordance with those two engineering standards.

Shannon Oelkers:

And this one's interesting, cassie they had some false statements in there, like AI was paraphrasing or generalizing so much that, like in the first, the prompt one response plan, it said that it required 110% of container volume for sized secondary containment. And that's just not true. It's a good general rule of thumb, but if a secondary containment is not protected from the weather, it's got to include an allowance for the 24 hour 25 year rainfall number and that comes from fire code. But it's also listed in some guidance that the EPA has published since the SPCC regulation came out, and so this kind of goes back to that failure of referencing regulations and engineering standards, like it's making false statements because it is unable to cross-reference all these things and make the accurate statement. It's just a little weird. You see it and you're like it's almost right but it's not. And when it comes to compliance. That's still a compliance issue. And then, definitely throughout all of these plans, we just saw the wrong context being applied to really critical words like containment.

Shannon Oelkers:

It does not appear to me that AI is actually pulling the definitions from 40 CFR, 112, the regulation itself it's using, or, if it is, it's such a small drop in the pool compared to the overwhelming evidence on the wider internet that it just can't bring it up. And there may be some prompting that I could do. I'm sure somebody listening to this who's an AI expert would like. If only you had done this prompt, it would have been better. But I was trying to put myself into like the average regular person that was trying to write an SPCC for their facility and then some of the storage containment volumes were just mixed up again.

Shannon Oelkers:

A good example of this is like when we prepare SPCC plans, there's secondary containment requirements for all these different kinds of containers.

Shannon Oelkers:

Some of those requirements are general, like to collect weeps, drips and spills, you know, like the spill bucket at the fill point of your tank, for example. But others are like sized requirements. So you have a 330 gallon isotope. You need to have containment for it that is equal to 330 gallons in size, right, plus 10% for precipitation if it's outside and exposed to stormwater. Well, depending on the container type and how it's used and where it's located, you can sort of have all these different containment applications and it can be really confusing. So, for example, let's say we've got a helicopter company and they want to store 10, 55 gallon drums in a remote location as like a refueling supply station. Right, if you're the helicopter company and you're building a containment area for these 55 gallon drums and there's 10 of them does your secondary containment need to be 550 gallons, as big as everything you've got, 55 gallons for the biggest container or just five gallons to capture like weeps, drips and leaks?

Cassie Kuzis:

I'm not sure off the top of my head.

Shannon Oelkers:

A lot of people, don't? We get this question a lot. Actually, under 40 CFR 112, the correct answer would be 55 gallons, the volume of the largest container within the containment area. But if you don't have that context or definition, using AI to create your plan, we'll absolutely spit out a wrong answer. In the prompt number two, where we had two tanks, it said that the tanks had to have containment that was equal to, you know, 4000 gallons plus 10%, right, 110%. But it also said the tanks were double walled. So if you were writing this SPCC plan and getting ready to, you know, buy a double walled tank, for example, you're doubling up on the containment and that's an expense you wouldn't have needed to do because you've got double walled tanks, which is your 110% containment already. So that kind of complexity their AI is nowhere near touching.

Shannon Oelkers:

There's also a couple of examples where they listed the broad line items for things like. It would say things like employee training will be provided for, spill response, or figures showing the facility layout will be included in an appendix, but that leaves out any specifics for what the content of those things needs to be and the SPCC is pretty specific about. You need to have the following things in your figures. You need to have the following pieces of your training program, and so I feel like the spill response actions were somewhat captured in the plans better, but spill prevention, like transfer operations and maintenance, and they were all left out of all three of them. They just didn't reference them at all. They just didn't reference them at all.

Shannon Oelkers:

I feel like, after going through multiple reviews with the EPA, with our SPCC plans in general, these overgeneralized language that AI is producing is just not going to be compliant in the eyes of the EPA because it doesn't address specific nuances of each of the regulatory requirements. And I guess a good example of this it's a difference between check the truck for leaks and check the lowermost drains and valves for leaks prior to fueling and prior to departure. One of those is based on a regulatory language, the other one is a overgeneralized statement and a lot of people are like, oh yeah, I did, I checked the truck for leaks, but the EPA, with 100% authority I can say, will assign a violation if you don't include the lowermost drains and valves in your checklist and can show that that's part of your training to the employee.

Cassie Kuzis:

Ah, so there's just a lot of nuance there, yeah, and that's just like one tiny part Did any big surprises come up?

Shannon Oelkers:

Yeah, there was a couple. For whatever reason, AI got portable drums and containers pretty well. I don't know if that's due to more content about drum storage and warehousing being available on the internet or what, but it got a lot of the portable drums and container stuff pretty accurate and pretty thoroughly. Labeling and signage made an appearance on the third prompt. It did not make an appearance in the first two, which both had portable containers and tanks. So I don't know why, but I did notice that labeling and signage finally made it on there on the third time around. It also got spill reporting really well. Again, I think likely because there's so much correct information on the internet about that specific task already, because lots of people have how to notify, how to do spill notifications posted, you know, to the internet.

Cassie Kuzis:

Wow, that is really interesting, but it still seems like for now I should probably leave AI out of SPCCs.

Shannon Oelkers:

I really agree with that. Honestly, I think if you wanted to create your own SPCC, just reading the regulation line by line and responding to each portion of that regulation would give you much better results than AI. But in reality, I think most commercial facilities need a professional plan writer, not just reading it line by line, because you need somebody who understands the regulations, the context, your industry and operations and that's really going to turn into the best outcome for your facility overall.

Cassie Kuzis:

Professionals, like I said, integrity.

Shannon Oelkers:

Of course you can't see me through the radio, but I'm blushing modestly. Yes, Professionals like Integrity or engineering groups, you know you know your operations best. The regulations and the engineering standards and the fire code all play into writing an SPCC plan. I don't think it's typical for most people who are focused on the main job task of storing fuel to also have a good grasp of the fire code and engineering standards and inspection standards, and so all of that stuff feeds into. If you have a professional that can help you and they're working with you, you'll get the best of both worlds. You get your land-based knowledge from being right there on the ground doing the job, combined with their professional knowledge about all of these different regulations and standards.

Cassie Kuzis:

Well, do you think we should be worried about AI in the future? Will it ever be able to write these plans?

Shannon Oelkers:

I think our jobs are safe from AI for now and I have a feeling maybe forever, because facilities are pretty straightforward when they are built, but facilities change as they age and I don't know that AI can overcome facility specific content, like it's got to overcome the as designed right and then linking that at through time to the as observed, and so the changes a facility goes through in 15 years, even if you fed the AI, your engineer design work work, the tank farm is going to be different and there's all these like operational things about how they transfer fuel and the ways that they do things and when and how they use drip pan. You know, like there's just a lot of things that I think that human connector is always going to need to do, because you know, we know this from writing plans what you write is never what you find in reality, and so you need to constantly connect that you know, connect between reality and ideal, and I don't think AI can do that without a human person.

Cassie Kuzis:

Well, that's a relief.

Shannon Oelkers:

Yes, that kind of wraps up what I found out about AI. We'll get those show notes posted.

Cassie Kuzis:

Yeah, thanks for doing that. I'm sure a lot of people will be interested in your results.

Shannon Oelkers:

Yeah, it should be fun to see what else people see. If you end up having comments or you have some ideas about this topic, we would love to hear them. Go ahead and email us at info at integrity dash envcom. Or if you have ideas for future podcasts, I would really enjoy hearing those. We've had quite a few listener ideas in the past, so if you've got one, let me have it.

Cassie Kuzis:

Awesome. Well, thanks for that.

Shannon Oelkers:

Hi there, this is Shannon Oelkers and, as the owner of Integrity Environmental, I wanted to take a minute here at the end of the podcast to make sure that you knew the following this podcast is for informational purposes only and should not be considered legal or regulatory. Thank you very much for listening, and the views expressed in this podcast may not be those of the host that would be me or Integrity Environmental. Thank you very much for listening and if you do need professional regulatory advice, we'd be happy to help you as part of our consulting services.

People on this episode

Head Shot

Shannon Oelkers

Host