The AI Investigative Framework Interview with Heather Barnhart

 AI is becoming more common in digital forensics, but the biggest danger is people trusting it too much. Heather Barnhart discusses a framework that helps examiners assess when AI is appropriate, where it can assist with tasks like triage, and where it should not be used, while keeping trained human experts responsible for decisions. 

Notes:

https://www.linkedin.com/posts/heather-barnhart-cellebrite_ai-dfir-digitalforensics-ugcPost-7463670252950847488-b7s-/

Transcript

Welcome And Heather’s Forensics Path

0:02

All right. So uh welcome everybody to Data First 6 Now podcast. We are recording from Techno Security East 2026 in Austin Middle Beach, South Carolina. Uh today I'm accompanied not by one, but by two headers, you know, double the trouble. Um just again, super, super awesome to have you here. We both agree that it's thank you for your kindness to take the interview with us. Absolutely. So we're gonna talk about a project that you have started. But first, if you don't mind, just tell uh people that are viewing a little bit about yourself and and like what you do. Sure. Um I'm Heather Barnhart. I have been a digital forensic examiner for 24 and a half years, which is crazy. I always said I would do 20 and then work in a flower shop. Here I am. Flower shop starts tomorrow. I have in the 24 years, I've done everything from child exploitation to even working at Strauss Friedberg for a little bit, a lot of counterterrorism. A huge chunk of my life was at DOD and worked at Celebrite for almost seven years, and now I can consult to them as needed, but I'm full of everything beyond digital office of operations, ICS, everything. So representing the people that are in the classroom, and my main goal is that we have the right people, the right content for the students. Yeah, yeah. So so I I used to call Heather my sensei because it's one of the first uh mobile for instance courses that I took, I took with her. And I was so highly impressed by, and I've been like, you know, developing a friendship throughout the years, and as a good colleague and mentor me in so many ways. So and vice versa. So thank you. Very nice. All right, so we are here to talk about a project that you've you've started related to AI. So can you just give like a brief background of what the project is and what prompted you to start the project? Absolutely. Um thinking I would be sitting here even speaking about AI blows my mind. Uh, my husband Jared and I would fight nonstop. So Jared was full in on AI and I was full out for a really long time, which I think makes things a little bit interesting. Um, at RSA every year, I take the stage with Sans. And I shouldn't say every year, the last eight years, I have been the lady on the stage at RSA. There aren't many of us. And we're supposed to define the top threats. And it's hard. In digital forensics, it's not the same. Like we're not breached constantly, and there's not like a power grid that's gonna go down. So I try to think, okay, what is a threat that's going to impact everyone? Um, last year it was Idaho murders and dark periods of data. So this year I decided irresponsible use of AI is going to be my threat. And it didn't land very well. Well, yeah, the market is

Why “Irresponsible AI” Became A Threat

2:56

really pushing together. Yeah, well, I and I was like, okay, it's the dark side of AI, whatever you want to call it, that is what it is. And it's people not having foundational knowledge and using AI and just going forward. So I never intended for these frameworks or guidelines to come about ever. But the rule is if you bash something and scare the audience, you have to solve it for them. I'm like, That's a good point. Well, I don't know how to do that. So I you have six minutes to state the threat, make it very scary, and then remediate. So my PR team said, Heather, you have to create something that just like step by step, look at digital forensics and where is it safe? Where is it not? Where are people insane? So my first iteration of this was not recommended. I love it. Don't use it just for what? Not recommended. And then people said, you need to break it out. Digital forensics is different than incident response. So then the incident response one became a little bit lighter and friendlier, and people still said I was being so restrictive. So then I realized I can't do this on my own. And I recruited experts, very similar to the six steps of mobile validation. Like ask other people to help you. Help the community with this is just a suggestion. We have to make it something. And we, and that's what I'm trying to recruit people that have vastly different opinions to make this something about people who care about our skill and our trade. Let's protect it from button pushers. And sorry if that offends anyone with button pushers. It shouldn't. I mean, but we're both in the field, and that's okay. But the AI in the button pushers' hands is where where the threat part comes in, potentially. Yes. Yeah. Um, all right. So related to the framework itself, can you like briefly explain just how it's organized, like what sections you have in those frameworks? Yeah, so we'll talk about the digital forensics one, only right? Definitely. Okay, so with digital forensics, what we decided, and when I say we, I mean myself, Ovi Carroll, and Josh Eckman were the primary ones on this. Um, Taz Wake, Steve Anson, David Bianco, and others at SANS have chimed in and really pitched in and helped. But we kind of decided who's doing more DF and who's IR. So with this one, we decided to go with the Swig D. And some people were upset, like, oh, you didn't pick this framework or you didn't pick this one. And what we wanted to do is take a framework that we all trust and map where AI could be applied. But then Josh's idea was let's assign some risk to it. Because again, I was like, AI never. AI never. And I had that red on every box. And they're like, okay, well, you and I talked about this. If you're doing consulting and it's one company suing another company, that's different than CCM or homicide on where AI can be trusted or not. So some of them it's like, this is your risk. And the way Josh put it out there is letting organizations know the risk if they choose to use it at all. So it's simply just suggestions on where to stay in line. But and some people have fought me on the AI never shouldn't be there. And I was like, well, it is, as long as my name's on it, AI never has to be there. Well, and and and and and to your point, because the the the framework, and correct me if I'm wrong, has to be wide enough not only to say, well, there's law enforcement use. Yes. It has to include also civilian or or private sector use, right? Because uh I I would, I mean, it's something I'm gonna jump ahead a little bit, but I I would assume that, for example, law enforcement will look at that risk, but that risk might be a little bit different. So maybe there will be some space in the frame where say you're doing law enforcement, risk gonna be considered differently, possibly than in other sectors. And so just so everyone knows, we want feedback, and we being the three of us, myself, on what these frameworks need to be to help you. So they're simply guides from people who care in the community, matching to a framework we all trust. But should there be, if you're law enforcement, if you are healthcare, if you are doing um a divorce case, everything is different. And there is going to be the way it's planned right now, and if everyone doesn't like this, we can change it, three phases. Phase one, we release it and say, hey, here's a guide. We hope you know what you're doing, which is dangerous. Yes. Phase two is break down what each of those things mean. Like if it's evidence collection, how could AI do harm? But how could it maybe help you? And then if it's helping you, where can it do harm? So take people down the paths, and then phase three from experts in the field who have good experiences, bad experiences, and lessons learned. But it can't be, it can't really become a static framework because it's going to, it could change tomorrow. It has to be dynamic. And if anyone's like, oh, I'm so static in my process, this will never work for you. And you shouldn't use it. That that actually leads into the next question that I had. Um, so some people may argue that publishing these posters or these frameworks, you're giving organizations a false sense of security. So are you worried that it'll let them lead to, oh, I, you know, I looked at Heather Barnhart's guidance on best practices in AI and run with that. I mean, you're such a big name in the community. And honestly, like if you're putting out a blog on an artifact, I'm looking at it, I'm gonna know that that research is solid. I'm gonna still do the research myself to make sure, but um, that that research is solid. And so if they don't truly understand the risks and they're just going with what these say, um how can we mitigate that? Okay, so two things. One, I will never have best practices on AI. This is not that, this is not best practices on AI because I don't I don't use AI in digital forensic investigations. So that is a truth of mine. I will, I just started using it on CTF data that I created. Yeah. So I understand what it is, but I don't use it to do forensics. I use it to fix a broken Python script or give me help here. Um, I use it to make me more efficient in my daily tasks so that I can do forensics. Um, where people need to understand is my entire platform, my entire career has been sure, trust me, but verify. Like I'm just giving you some guidance. I have no idea how you work your case day in and day out or how you do it day in and day out. So we're all different. You have to find what

Building The Framework With SWGDE

9:10

fits, but if you do not have foundational skills, it's the worst thing you can do. I agree with that 100%. Um, I do still believe that some people will take it as a best practice. They will. Yeah. It's just like I've had people take my 585 class and thank me at the end and say, I'm sorry, I'm going to dump the phone and press the report button and never look at it. Thank you for your time though. That makes me sad. It's sad. It's awful. We've had that said in our classes before. Like the question is always, well, that statement, but the question sometimes too is when will I ever need to use this? Because the cases are moving through so fast that Well, that makes me mad. I'm not sad. Yeah, it does. It does, but and maybe that needs to be the primary tagline underneath the title of what is it, digital forensics and AI application framework or whatever. Yeah. Like this is a suggestion for trained examiners who understand what they're doing. This is essentially anytime you introduce AI, it is your person that was me at 22 years old that didn't know what a hard drive was. And I went on a search warrant. Right. And Sean Howe, my boss at the time, said, Go get the hard drives. And I looked at him and he's like, You don't know what that is, do you? And I was like, Teach me, I can learn that. I didn't know. Yeah. I truly didn't know. Definitely. Well, and and and and and the fact that to your point that this needs to be discussed. And I say this, and let me put it this way: within the whole AI aspect of in the forensics, we can, from my perspective, and I want to hear your words on it, it should be, I believe, examiner or practitioner led as opposed to led from other angles. I don't know what your thoughts on that. Yes. And so I had some people fight me on human in the loop versus human on the loop. And I was like, human, like human, we just need human. And I really got a lot of pushback from if you say it too much, what's the point of doing it? And I was like, that is the point, though. Like it is not your easy button. Right. It is something that if you use AI, you're brilliant. You know what you're doing. It will make you more efficient. If you have me at 22 years old using AI, I'm dangerous and it's a terrible thing. So for me, a human initiates, a human approves, a human looks at it. If a human doesn't know what the thing is, they go to another tool or another human or they recreate that data set. And I saw your thing on verify versus validate, and I got some feedback on this too. Like you put capital validation, capital V, that seems really offensive. I'm like, it should be. It's AI never. Like it should be all capitals. And some people at SANS didn't like that. And they're like, it seems so, and the issue is other parts of cyber are different, offensive operations. It's different than digital forensic. So I was very protective and firm on things, but I'm also like you guys, and I want to validate my tools and I want to validate the output. And I verify an artifact and I recreate it over and over and over, and then I'm still paranoid. Well, the technology doesn't help in the at least at this stage, I believe, delay that paranoia. That's why your framework is important, I would say, to at least be able to break it down in a way that's usable with care. Yeah. And I've had vendors ask, can you put us in the framework? And no, it's not a place for vendors. Like if a vendor can identify, oh, we belong in triage, great, you can take that and reference back and say you identify as triage, but the point is to not say you're gonna use this here and you're gonna use this here and you're gonna use this here. Yeah. But people press buttons in the tools and trust it. And I'm not one of those people, and I worked for a vendor for years. Yeah, that's definitely I want to recap something real quick. So you mentioned the whole human in the loop, right? So and the and it's I think it's important, you're making humans in the loop in the context of experts. Yes. Because I mean, and you know, examiner, as you said today, not not investigator. And I know some people identify differently. I had analyst and investigator in my title before. I was never truly an investigator, I'm an examiner. But true foundational skills with the knowledge on how to expand those skills, how to verify what data means. Because I guess our fear is that folks that don't have the level of expertise to be able to apply this new technology in our field are gonna, you know, misuse possibly uh when to use what and how. They will, and they'll misuse the tools and they'll misuse all the things. And it's unfortunate. And as you stated earlier, training's great, but people are like, I don't have time, I don't have anything. And now I meet with the younger generation, and they're like, we have two days that we can be away for training. And I'm like, you're never going, it's crazy. Yeah. Like I remember when I went through IASUS, I spent hours every evening prepping and learning and doing all the things, and people just don't have time for that anymore. And I also don't think they have the attention span, and I think they're so like or just don't want to do it. I mean, I've run into that where people just I'm not going to training. Um, okay, well, then you're not gonna ever be good at your job. Yeah. Yeah. Um, so with the human in the loop, I'm glad you brought that up. So, how should a small agency with limited resources prioritize um the guidelines when they may not have the personnel to keep the human in the loop? So you're a one-man shop or something. I saw the one-man shop as a nightmare. Even with just like verification and peer review of each other and validating things, it's really hard because you're the sole person, you're the single point of failure, unfortunately. Um, I saw something, I don't know who posted it, but it was talking about an AI expert, should not be a C level person. It needs to be the top examiner. Like your person that says you're the AI security person, you're the AI examiner, you need to be the best in your shop. So if you're a single person, and this is me personally, maybe not the other authors on this or future contributors, because there are a lot of people who want to contribute. I would use AI to make me more expedient in my daily workflow and focus on true forensics.

Human In The Loop Or Else

15:15

Yeah, that's a so maybe keep it out of that until you have someone that can just give be a sounding board. Because once you start relying on it so much, it's like a drug. Like you you're useless without it. Yeah. And and the smaller agencies, they're pressured to come on, I need all these phones done, all these phones done. And I mean, that's gonna speed it up, not accurately, but it's going to speed it up. So I could see pressure from up top. Yeah. And that's a fear of mine with people thinking that AI can replace digital forensic examiners and make things faster. It could possibly make certain things faster, but it can also make so much more work if it's not done properly to begin with. So that's where you have to decide where it can make us efficient in our workflow. And if it's gonna slow you down and make more work for your senior person, it's not for you. Yeah, and that's another fear that I have. I'm and for the folks that know the background, we came back from a talk that me and Heather had on the same topic. And it's the moving on the bottleneck, right? Traditionally, we have the research and the work of the investigation up front, and then the examiner comes to our report at the end. But now we're putting the report up front, the AI does the report, and then the examiner comes in and tries to uh validate that. Um that change of world workflow, is that something that's kind of shown within the framework something you think uh think to add in the in that sense? So people doing triage reports first. So I personally in all of my labs, we've never done the report right out of the gate and then dove in other than something we called like a tip to analysts, like on locations and stuff with terrorism, like quick tips, and we would throw that out. I also have never used a tool to create a forensic report. I would take bits and pieces back in the day of NCACE version three. I remember Sean Howe, my boss at the time, he would just test me on equation or on occasion, and he's like, What's this column put here in this thing you sent to me? I'm like, I don't know, it's from NKC. He's like, never do that. Yeah. Oh. It's like we mentor. Yeah, it's like really lessons learned. It's lessons learned. And if people are doing the reports first, and this is the issue. Like, if you're you're the investigator, I'm the examiner, and like, here you go, and then you do whatever you want, and we don't communicate, that's a huge issue with where AI fits. And it's the same thing with like reader reports going. And I get calls all the time, like, I can't open this database. I'm like, you're in reader. No, I'm not. I'm like, yes, you are, yes, you are. And I talk to others and they're like, Oh, I'm in reader. What am I supposed to do? I'm like, you shouldn't be doing this. What are you doing? Who's who gave you this? And that's another thing, too. What some of them they are limited to what the examiner put into that report. Absolutely. So if they think they're seeing everything, they're already limited. Yeah, they don't have the full picture. And it'll then becomes another abstraction on top of that. So I'm gonna be really careful of that. Okay. Yes. So you talked about in the frameworks um the AI optional and AI never. How did you decide to draw the line between AI optional and AI never? And was there a set criteria that each must meet? So I started with AI never on everything. Right. Nothing was AI recommended, is what I think we ultimately called it. Everything was AI optional or AI never. And then I was told, this is insane. Like you're saying it never has a place here, right? And you will fall behind. And I was like, okay, okay, okay. So then I started listening. So AI recommended was, and it's not strongly recommended. It's not, this is going to save you, this is gonna speed up time. It could cause a lot of work for you if you don't have proper people with trained skills that can implement it. So something simple. I was running the Sharon CTF data um in Genesis, and I tried it for my tip Tuesday. And by try it, I spent 10 to 15 minutes on it. I it took me about 10 to open it because I didn't even understand what I was doing. But then I just did simple things like who did she communicate with the most? And I saw, okay. And then I forget what the second question I asked, something about her communicating with auto. And it said she has a Google Drive that she shared. Maybe you should investigate Google Drive. So that would be a good clue on it, may identify other evidence. Um, feeding it things you don't know now. Should it go and collect the evidence for you? Absolutely not. And that would be a bigger risk. And that's where just weighing these things like can it identify something that you didn't see? Maybe. Um, I've had someone say, hey, I used AI and it identified stuff in a case that I've never seen. I know Jared has run a case that we worked through some things, and he's like, it pulled up some screenshot that you and I never found. But bottom line of my two RSA things together, the dark periods and AI, AI can't find what doesn't exist. So if criminal if criminals are getting smarter and nothing's there, if I used AI to work the Idaho murders, I would maybe say he's not the guy. Right. Because nothing was there. It's all in the logs that AI cannot do that. Right. Um, so for the sections of the documents themselves, you just kind of touched on this, but for the identification, if the LLM is identifying the evidence, wouldn't the that risk evidence exclusion if the LM LLM hasn't been trained to identify something that needs to be taken into account? Yes. Yes, and it would be, I would take that part, the identification, as you have data and it sees a connection, like a USB has been plugged in as a computer. Did you think of a USB? And I'm like, oh, I didn't know a USB was plugged in, or this phone has touched Google Drive, or it's syncing here. So things that are already on your data set and it's just making recommendations of something else that has touched that thing. So not just dreaming up, like, oh, Briggs has a NAS in his house, and maybe you should go like if it's doing that, that's crazy. Like, and obviously it will make mistakes and there will be things that don't pertain or that you don't have legal authority to look at. So you have to almost take it with a grain of salt. Right. Well, I mean, and and I think this is easy, this conversation is really useful for me because it really, and correct me if I'm wrong, but I believe it's really predicated on the person being an expert. Yes. You can actually because I I think again the marketing seems to be from all sides to anybody, right? And at least for our field, the framework really makes sense to me now that we're having this conversation if it's an expert. Somebody I say expert from the basic forensically trained, I know what they're doing, to be able to have that conversation with the AI as an augmentation of what you're talking about. And I want to put their

Small Shops And The Pressure To Speed Up

21:50

names on this and do very much like the six steps of mobile validation, like a community effort to protect digital forensics for those that are just gonna go run wild. We say in these different phases, this is how to be safe, this is where it can go off the deep end, this is where to protect yourself, but also let organizations know this is the risk if it goes wrong. Because someone said this too, and it was so good. I was like, oh, where did I read that? You should not use AI where you can't undo something. So if AI is making the decision of something that can never be undone, it does not belong in digital forensics. And that's my opinion. People could disagree and say, no, that's okay. But if it can't be undone and you can't go back and redo that thing, it should not be used. Because it's not an expert. Well, that's a good rule of thumb. And we make mistakes. Like, what is AI gonna do? Well, in the account, at least when I make a mistake, we're accountable for it. And we can have a way of tracing it back. But when the AI makes a mistake, this black box not only is harder for trace, but it has no accountability, right? And you know what? That was my final statement I made at RSA. It's your name on the report, not AI's. Yeah. Like be responsible with it. Definitely. So for the collection and preservation phase, um, so we've been using deterministic automatic automation to handle hash verification. Why use LLMs in this context? Wouldn't that be introducing additional risk into a process that is already well established? So for that section, I believe it says automation and it meant our traditional ways. Like we're already using automation for it. So it was comparing anything you automate like that, you could potentially automate with an LLM if you have a trained organization that knows what they're doing. And I'm not in a situation like that. Mary DeGrazia wrote Forensics 563, which trains organizations how to remain offline to protect their data and create LLMs for automation, but primarily incident response, which is different than digital forensics. Yeah, it's very different. Yeah, I think in our in our context, they definitely would not apply. Yeah. How IR doesn't. They move, it seems I'm coming wrong, but they seem to move really fast because they're trying to stop a threat that's happening most likely on the spot. Yes, and you're not going backward looking. Kevin Repa at the DeFer summit last year, Kevin Repa took the stage and he said, he's so sick of people saying Deefer. It's digital forensics and incident response and threat hunting and all the other things now. But he said, if the building's on fire, incident responders come, they put out the fire, and they're like, it's out. We're out of here. The fire investigator comes and is like, is this arson? How did it start? What is damaged? That's the digital forensics. So we have really different skill sets, but we're all lumped together, and how we do things is very different. We're not trying to fight a fire, we're trying to figure out what happened to prevent it from happening again. I love that analogy. And I think the feel has matured enough because at the beginning, how many years ago, lumping them together makes sense. You said the person doing the DF was doing the AIR as well. Yes. But not anymore. So uh that makes absolute sense to me. So for the triage section, triage is flagged as highly recommended. If the AI is wrong on determining the investigative focus areas, it'll affect the whole case. So why not use a more conservative category for the triage? Okay, so it's just recommended. AI recommended. Nothing is highly recommended. Nothing. Um it says AI recommended and for triage, it was just essentially, I always compare it to finding a thread to pull. If you're so lost and you're like, I don't even know where to start, I don't know what to do, finding that. But again, it could have you spinning wheels. So if you think of Idaho murders, there wasn't the thread. There wasn't the connection to the victims. There weren't the locations. So just finding that thing, if you can, that's where I think it would be useful. Yeah. Or even just, oh my gosh, back in the day, I would work inappropriate use of government computers. And we would go into a foreign service lounge and there would be 40 laptops, and someone's like, he was in here. Oh, which one? So if you could quickly scan and be like, okay, this one has the hits to this website. And then you know where to go. Yeah. So just simple things like that, I think it can help, but only if the data exists that you're asking. Right. And you need an experienced person to know what to ask. Because someone's gonna be like, hey, did Heather commit this murder? What if it says yes? Like, you know, it's like there's a possibility that it says that's not that you did it, but that it says that. Seriously. Holy crap. Yeah, I I've been looking at some of the AI models and asking questions, and some of them could say yes. Yes, and that's where it goes into the attribution is AI never. AI never, ever,

Triage Help Versus Attribution Never

26:35

ever gets to say if someone is guilty or innocent, did or did not. It can simply make a recommendation that an experienced person has to. So uh this opinion, why give you opinion on this? Attribution never, but let's say the questions the summoner does and the answers point to the person, point to the person. Wouldn't the AI maybe make in the attribution? Like we say attribution, the not the AI, but the person, what do you mean by that? So let's say AI came back and said Brian Koberger did not commit these murders because he didn't stalk the victims, he didn't do all these things. No, it does not get to decide that for me. Just like your tool shouldn't get to decide that for you. However, AI, just like investigators and anyone else you talk to, can form a bias in your brain because it's constantly gonna be putting it in front of your face. Um, I helped with an investigation recently, and the ATF was not happy with me, but it was a fire and a creation date of a note, and they wanted me to say that this woman created this note. This was the narrative. I shouldn't say they wanted me to say this. Yeah. Heather, a woman created a note eight months before the fire. She wrote her kids' obituary in it because she pre-meditated burning her house down with her kids inside. And I was like, oof, this is awful. So I recreated the data. I created a note and I did it in 2025. So it would roll over to 2026 and looked at it and the creating changed. So I will go in and testify and say that could have been a grocery list. I can't say what it was because it wasn't, I think it was 32 days after the note was changed, is when they acquired the device. So I couldn't even go into the wall and look at all the changes. Right. But I was like, this is what I will say. AI would have said that note was created on this date. Yeah. That would have shown that content, which is true, but not true. Right. So putting like the truth behind the data requires a human. And when we understand your point, we we make hypotheses of what could have happened, right? So the ATF does, we all we do that. We put hypotheses. That's the big part of not getting married to the hypotheses, right? Because the investigative work needs to then actually confirm or if it doesn't confirm it, then we develop another uh hypothesis. And that's when I went back to them. I was like, my creation date is not going to work for you. But did you find Accelerant? Did you find something else? Because some other piece of evidence might support your hypotheses in a different way. Um but that but that's that's kind of my fear that that that just outsourcing that type of process to the LLM and then you clear point that bias might blind this. But it already does in the tools. And that's the thing, the creation date, if you looked at it in Axiom, physical analyzer, it showed the creation date with that content. Because the tools can't even put the person behind that keyboard or phone doing that actual thing. But what's sad to me is I find there are so few people and there's a million excuses. I don't have the time, I don't know how, I don't have the training that take it to that level. They're just like, here's the creation date, here's the note, yep, you did it. Yeah. And it's scary. And AI will make it worse, but so do the tools. They do. As the tools keep accelerating, and once you just stay in one, it makes it worse. Yeah, I still uh we talked about that previously. Deterministic tools, yeah, there's limitations like that, right? Yeah. But I'm afraid that some of those limitations um will be worse if we don't take into account like the framework that you're giving us, right? You could go up the rails really quickly. Whereas at least deterministic tools, the going off the rails is kind of more constrained because the tool is uh deterministic. You can only go so far, which AI, depending on even how you query it, it might change what the answer is. And maybe we have to change the title of it to forensic examiners, for experienced forensic examiners. Or something that reflects the level of expertise uh that's needed to actually make use within our field. But I also don't want people to think that you can't hire entry-level people or that entry-level people can never, if you use AI in your lab, belong there. They can. They just need guidance. Yes, a mentor, someone helping them along the way. Yeah. I mean, I I would I would propose, you know, again to a community conversation, right? Uh people are coming in. Okay, you can use the AI for like to your point, to specifically these things in this context. Yeah. As you get a higher level of experience, then you can now use AI for more things, right? You got folks that are just copying hard drives because you're a technician. And you got folks like ourselves that were doing complex timelines and looking at data structures, right? We can't expect that guy to do or that gal to do that from the beginning, right? So I guess seeing AI as a more tiered approach, but I think kind of you your model could also uh reflect that as well. You know, I'll go back to Sean Hal again. I hope, Sean, I hope you're watching this. I hope you're watching this. He made me use Disk Edit. No one used Disk Edit. And I was like, what am I doing? Why? And he's like, you have to earn your end case dongle. He made me earn my tool dongles. So I was like, Oh, I love it. I don't know, but I love it. He actually was a huge part of IESIS for a really, really long time. And at IESIS, when I went through my cert, I was like, Oh, you have to use disk edit for the first three questions. The tests were so different then. First three questions were disk edit, and then you were allowed to use tools. But you had to prove to them that you knew what you think that seriously, I swear that's why I am who I am today, is his torture. And I had Bill Harback as a coach who tortured me the whole way through, which was phenomenal torture because it formulated how my mind works with cases. Yeah, that's that's awesome. Um, so the examination and analysis part. Um, what does human must validate all AI conclusions mean in the context of a field that is used uh used to consider something as validated via sampling? So I wanted that whole section to be AI never. Okay. And I got disagree. Yeah, I got some pushback. Josh pushed back a little bit, as did Ovi, and they're like, some people don't do triage, and triage is examination or analysis. I'm like, and they're like, what if like today the person in the audience has said that they throw some ideas back to AI that they see? I I don't do that, but I understand I also have friends that I can ask. And some people don't have that. So maybe AI is their friend, which is risky. Yes. To have a friendship with AI, risky. You need a better friend. We'll be your friend. But reach out. How this was talked down to me was using things like the app genie or media categorization or things like that that can be completely wrong, but you're still using it a little bit. Um, maybe ways to soften your tone, or I don't believe in I don't even remember it or something. Uh human watching. Yeah, human watching. I'm watching AI output. If I see AI output, I'm done reading it. I am done, I know it, I can smell it, I can see it, I'm done with it. It's not for me. But if you have something that you just need to collectively put into a better report and it's all your things, I think Claude, and that's my own opinion, sorry, is the most honest. Claude will keep my typos where I'm like, come on, man. All the things to change. You need to type in. Oh, I realize that for me. Whereas there are some things where I'm like, I didn't ask you to do that at all. This is so crazy. Yeah. And it keeps getting worse and worse because I think AI keeps believing it's getting smarter and smarter and smarter. And unless you challenge it, and it says, Oh, I'm sorry, you're right. I shouldn't have done that. This is wrong, or this is incorrect. Don't be a lazy human if you use AI. You have to challenge it. Yes, I find myself arguing with it constantly. And it it doubles down, I'm telling you, it will say this is the answer. So you really have to keep pushing and then ultimately go find the answer yourself. Like in your framework where it says um, you know, that a human has to validate it. Yeah. So yeah. Uh I wanna, I mean, I I do like the framework and I give props to you to that effort that you're putting out for the community because I see a lot of people, it's me, I'm a little soapbox moment. A lot of people trying to say, okay, this is the way we do things traditionally with the other forensics. I'm gonna take this tool, I'm gonna put it in enough situations that I'm gonna say, okay, then I can trust it to be consistent in all situations that are similar, right? And um, I guess this question comes from us talking about it and not believing that's the case with LLM, right? So your framework

Validation Limits And The LLM Problem

34:55

kind of breaks a little bit from the traditional way of how we validate the tool. Yeah. Because how LLM works. Is that part of the thought process? Yes, and that's where the verify, the validate, my capital V that people were like, we don't like the capital V, it's so harsh, and your standards. But there are some things we even do that we can't reproduce. That is like you once and it was magical. Yep. But because you're trusted and you're experienced and people value the work that you've done and the effort you put into the community, it's believed. So I think we have to make sure that we have people that stand behind these guidelines and say, from our experience, this is where it worked and this is where it failed us. And then other people have to grow with that. But if you're just gonna trust something that worked yesterday for six months from now, it's probably not going to work. And sadly, I think AI to do harm keeps getting better and better and better. The AI to detect AI isn't even really working anymore. So there's so many things that AI is using to do harm that we can't let it harm our skill and our craft. And what we've done for the last few decades. I'm all for AI, there'll be artisanal detail forensics. Yes, handcrafted. Yes. Artisanal detalforensics. So the validation section. Um, why is validation labeled as AI never? I agree, by the way. But what would you say to validation coming out of a consensus from multiple LLMs? And why wouldn't that be acceptable in your opinion? And people fought me on this one too. And I said I wasn't going to do anything at all if I couldn't have AI never in this category. And this is if I die tomorrow, fine. They can remove AI never, whoever takes it over. And maybe one day I'll be proven wrong, and that's fine. Fine. But looking at and how incorrectly data can be misconstrued by a tool that is designed to do the thing, you need a human behind it. Yes. When someone's life is on the line, whether it's someone losing their job or a victim getting justice, it matters. Like you can have someone lose a job, you can have someone lose a life, you could have all these things that go wrong. AI should not be determining determining that. A tool should not be determining that. A seasoned expert should be determining that. Absolutely. And that's the hill I'm willing to die on right there. Well, I think you should. I mean, don't not die, but I'm not like I'm not gonna get off that hill, or my name's not gonna be on it. We'll be with you on it. Thank you. Right there and there, holding the flag. But I think it's a big thing. And some people are like, oh, but I have AI fight AI, or I have this LLM and this one both agreed. But I've had Magnet, Celebrite, and Oxygen all agree because the data says that, but it doesn't mean it's correct on how it was created. Right. There's no true context

Reporting Prompts Documentation And Disclosure

37:40

behind the data. Have to have that testing. Yes, yeah, agreed. So the reporting. Um, the framework mentions documentation of AI involvement, but not disclosure. Was disclosure considered? Yes, and this is where some people were talking about, and I won't say where I worked at the time, but I worked in a lab that we weren't even able to, they didn't want accidents to be tracked. So, like, think of when I first got celebrate in the trace window and all the buttons you push and it shows, someone was like, Oh, if you make mistakes in a keyword search, it's gonna show up there. I was like, What? Oh, this is so bad. But there are some tools with logging that it would track every click you did, and they were so afraid that the other side would get that and then make our firm look like they didn't know what they were doing. So nothing was supposed to be tracked. But I think the prompts that you it's okay if you fumble. If I'm like, oh crap, I wrote, I misspelled your last name and now it looks ridiculous, or which case am I working again? But if you're fumbling through it, the prompt for the response has to be provided. Right. I think so. Oh, me too. And that's my opinion. But all the other things in between, this is where each group has to decide what are the cases you work, what is right for your organization, what is right for your lab. But the prompt to get the output, that that's like I use this version of this tool, I use this version of this, I use this prompt and got this result. That's the only way anything could potentially even be reproducible. Yeah, no, I I like that because I don't think the implementations right now are keeping track of that. As a community, we're gonna make sure that we convey that because it needs to. And depending on even if you give you a search warrant and you have to look in these parameters and you're asking questions outside of that, that things need to be discoverable. Yes. I mean, so I I agree with that. I guess the question would be what other things can we also think about, not right here, sitting right now, but as a community, that we need to be able to those disclosures because I'm also afraid that people like politicians that don't understand the field as experts will then put some requirements that will not make sense for our field. Which I do think will come sooner rather than later. Yeah, unfortunately. But another fear I have is people thinking, oh, look at this AI guidance. I'm just gonna open Chat GPT or open AI, and I'm just gonna start feeding evidence into it. Oh, yes. You you can't do that. Like you have to make sure, like, think air gapped, you're working offline, everything has to be in a protected, secured state. And that was another point I made, and this is more on the IR side. But people rely so heavily on AI that now their AI is the number one target. Like, if you get access to someone's AI and that's that's all they use, you know everything about that person now on their organization. So I'm just attacked that. It's like like an AI poisoning type of thing. Can you imagine to talk to an AI poison that's giving you wrong information? That's insane. Nothing now, I'm afraid. It's your fault. You're frightening poor brain. Yeah, we should. You're right. So um per swing DE, we verify outputs and validate tools, procedures, and processes. So, can LLMs as a tool be validated when the output has a stochastic randomness core core? Don't know. Well, in all honesty, I don't know. There are some people that say, depending on how you built it, something I have fought hard on was this exact thing with um protocol SIFT at SANS. What's the L? And they're like, oh, it's built on Eric Zimmerman's tools and memory forensic tools and these tools. Which versions? Who's validating those? But who's also validating the output? Because my fear is exactly what we've been talking about is that someone will run something and get a response out of the leap or one of Eric Zimmerman's tools and say, yep, that's it. Because the data says it doesn't mean it's correct. So just I don't know that we can validate it. My best advice to anyone is again, treat AI as the lowest person in your organization that needs constant training and hand holding. They're the rogue examiner that if you're not watching them and making sure they're doing everything exactly as you said, things are gonna go off the rails. And when did it go off the rails? Maybe impossible to diffuse load and loop as frequently as possible. And some people may disagree with me and say I'm too strict and stringent on that, but I'm the one sitting here. So not that what you just said, and then it has to be a clip. Thank you for that. No argument. You're right with her on that one. No, no, I actually actually validate it right now by what you're saying. So but that's the thing. I think people see this and they're like, holy cow, Heather switched, and now she's pro AI for everything. And that's not the case. I just want smart people who care about digital forensics to give guidance to those that just run with it. And I don't remember someone said to me, they're like, you're like five percent of the digital forensic community. People don't want to do what you're doing. I'm like, but they need to do what I'm doing. How can I become do what I'm doing? And then I realized very quickly I attended one of the celebrate training classes and listened to the questions, and my mind was blown. Like, oh, I have to change my message a little bit and start because I was going straight to the deep divers. When I started, I'm like, this is what I'm doing. But I realized I was reaching such a tiny pool of people, yeah, versus those that are like you can keyword search different ways. I'm like, no, why? Like what happened. That's so tough for me. That's so tough. You have over the whole field, but I'm like you, I'm like, no, we should all be raising up our level, right? Yes. And uh yeah, I don't I don't think that level is as wide as we would want it to be, but we the idea is to get it, you know. And that's the point. Like when we create the CTF questions and people are like, Heather and Josh are so mean, that's the point. You shouldn't be able to solve everything. You you should have to charge. I the forensics 500 basic CTF. And he complained. He's like, I don't like this, I can't solve everything, I don't have all the answers. I'm like, Welcome to Forensics bud. That's that's that's the line I give everybody forensics. That's my cash brain. That's the line I give everybody. I like even on when we did the celebrate uh CTF a few years ago, I was bitching. Yes. Um, and I'm like, I've been doing forensics for a while, and you've been doing it even your team, and we're like, can't find this. But it really makes you think outside of the box. Yeah, and some are so ridiculous. And lately the things I have done have been really Idaho murder-based. The Crystal Rogers that Josh and I worked on. Like, I try to tie it back to the weird crap I'm seeing. I'm like, well, this computer's getting erased right here for just a little month. Yeah, definitely. Um, so the there's constant calls to human verification. Vendors seem to be marketing the LLM

Marketing To Non Experts And Defining “Expert”

44:45

products to non-expert users. Uh, is the framework presupposed on the human being an expert capable of technical verification? We already talked about that quite a bit, but um, I guess your thoughts too on should should be marketing it at all to the non-expert. Yeah. We want the we want the experts to be using it most. But anyway, and this opened this opened my eyes a lot to different ways that we can blow this framework out more. What is an expert? Right. And maybe we state that like this is what we expect of you and you are going to use AI. Your risk is a little bit lower because you can do X, Y, and Z. However, and then just break it out. Like if you're new and I do agree people can get certified. Got lucky on a test doesn't make you an expert. Maybe you actually have to show for the work and you can show things. So I do think that that needs to be written somewhere. However, someone will grab it. And maybe someone got hired into a senior level position that they shouldn't, and they're like, I'm an expert, but really they're they're lacking foundational skills. Right. So at SANS, we're doing the Cyber Studio now. And the training is going to be affordable compared to normal training, which so imagine if you got hired into a senior position and you're like, I don't really know what this is, but I'm not gonna tell my boss you could take $200 training and get caught up on something that's going to build foundational knowledge. There's going to be some courses in there too on protecting against AI, learning how to properly use it. But more so than that, digital forensic foundational skills, which I guarantee most Cat Headley is doing it and it's coming out, it's gonna be one of the first ones. I guarantee everyone could use that refresher. Oh, definitely. Unlike whatever everything means and how to validate and verify. So she's doing the whole thing on that, but I think ultimately someone can grab it and do whatever they want and think that they know, but ultimately people are still pushing buttons and getting answers, and it's ending up in the courtroom, which is dangerous. Yeah. So um digital forensics is presupposed on the need to present findings that are admissible at court. How does the framework relate to the DOB? The framework. When you testify in court as an expert, you the person, the person, and we talked about this a little bit before. You, the person, are the expert. The tools that you use are not the expert. AI is definitely the expert. Anymore is to use AI to do your job for you instead of you actually doing the investigation. Because disqualified in the future, yeah. And my closing statement at RSA, I will put this everywhere. It's your name on the report, not AI. AI is not gonna dig you out of the hand and be like, I'm gonna have this LLM come in here and testify and say yeah, Claude's not coming in to help you. So you have to be able to defend what you did. But I also think having resources and people who understand digital forensics and the skill coming together and giving guidance to the community is one of the best things we can do. Yeah. Because then also we can kind of guide the courtrooms into what is trusting and what is absolutely absurd and shouldn't be allowed. And I think working together, we yeah, no, we'll we'll see. I haven't seen any case example yet of any AI output being uh daubered or in a hearing. So we'll see. I'm not gonna be that first guy. Yeah, I think the main things that I have heard of are the AI logs being used to prove criminal intent. So not being used to do forensics, but the criminal behind the keyboard, like how to create a pipe bomb, like how to start a fire, that kind of thing. So not the guarantee AI reports have slid through the courtroom, even though Absolutely. I I agree. I I agree. I mean, not in my agency, we we are not using it yet, but I believe we just from the top this morning, some agencies are utilizing it in some manner, so it's definitely getting into reports into the guidelines of the framework on giving some form of semblance and sanity to who people, some labs are being forced. Like you are going to use this. We got funding for it, you must do this or else. Like, if they're in that situation, look at the guidance and say, okay, I'm definitely not doing it here. Right. I may try it a little bit here, but you need an expert validating and verifying every step. The risk analysis with the framework thing to me personally is the most important part. Yes. Because at least it gives you, hey, there's risk at these level levels. The heightened risk, then you need to be some heightened mitigation before you actually engage in whatever the activity is. Yes, and just in case anyone tuned in late, this is digital forensics. Incident response is a little bit more different, yeah. Crazy. Yeah, they're like they're a little dancy with it. So that'll be a whole different one. We need a whole other hour for that. Yeah, digital forensics is different. So

Courtroom Reality Daubert And The AI Judge Fear

50:20

did I miss anything that you had? I have one closing question, but No, no, no, no, you're awesome as always. No, don't put your closing question. All right, so the closing question is Is there a section or phase of digital forensics or incident response where you believe no matter the advances in AI? Even if it gets a lot better. I really hope it's attribution. Like tying the guilt or innocence. AI should never do. Yeah. Yeah. I mean I just don't want LLMs to drive cars or flight planes. Yeah, me either. Have you seen that movie? I think the movie actually came out and it's the judge. And you go in front of an AI judge and it determines if you're innocent or guilty. I haven't even heard of this. It's uh that we don't want that. Yeah, no. I don't know what it's called. Is it Chris Pratt? Yeah, mercy, mercy for mercy. That's on my two watch list. It's an AI judge. Yeah, AI judge. Is it it's like in theaters now or it's out? It was in theater, it's out, but I'm uh it's on my watch list. Yeah. So that's my fear, and that's where it does not belong. It should never prove attribution. Never make the final determination. Yeah, agreed. Agreed. Well, thank you so much for your time. And uh off uh offline, I didn't know it's uh coming back for doing the IR part and uh and thank you for putting all that content out. And for you and also the other uh collaborators, we need efforts driven by the community members, by actually community members to actually make good use of whatever new technology is coming out. And also thank you for folks watching for being part of the conversation. It's not only suffering to our to our labs, to our organizations, uh, to make sure that our thoughts and understanding is part of the conversation and how we implement these technologies. So, with that, again, thank you, Heather. Thank you guys. Yeah. Bye. Bye everybody.