It’s going to be interesting when the inflection point Anthropic worries about comes to pass. Right now human + AI leads to better outcomes than humans alone or AI alone
What happens to accountability and liability when an inflection point occurs, when human in the loop causes more negative outcomes, more errors and is statistically more dangerous/error prone.
This perspective only lasts as long as we are in the Goldilocks zone of human + AI collaboration. And their concerns are warranted, how will society be able to deal with notions like liability and responsibility when the day comes that human in the loop itself becomes a liability and the main source of error. Right now humans do need to error check AI, soon that process of error checking will be where the majority of errors are introduced due to the human element
Thank you, Adeodatus. This is a serious question, and I don't think anyone has a complete answer yet.
The inflection point may well come for certain tasks. But notice that recognizing it requires exactly the skill the article defends. To show that human review now adds more errors than it removes, someone has to judge which results were right and which were wrong. That judgment doesn't disappear when the human leaves the loop. It moves upstream, into validating the system itself.
Aerospace already has a version of this. Autopilots fly more precisely than pilots for most of a flight, yet pilots are still trained to fly by hand, and accountability sits in certification: the system is qualified against evidence, and humans own the definition of its operating envelope. I expect FEA to follow a similar path. Liability will shift from signing off each result to qualifying the tools and defining the problem correctly.
That last part matters most. In my experience, the majority of serious errors are not in the solution but in the problem definition: the wrong loads, the wrong boundary conditions, the wrong assumption about how the structure behaves. An AI can solve a wrongly posed problem flawlessly. Knowing what the answer should roughly be is what tells you the question was wrong.
What you said at the end still will be valuable for sure, there’s still value in experience and intuition to know what to expect. I’m not doing anything really exciting, I do some dfm for injection molding and then some mold flow simulations to optimize the design, nothing safety critical or really exciting like FEA can be.
I’m confident AI will be able to do what I do very soon, but before I went for my engineering degree I worked as a plastics processor for a decade. It’s quite often that I over ride the simulation on design decisions just because it’s a) only so good and b) I have experience and intuition to over ride it
That last part is where engineers will still add value, but I have no idea how employers will ever quantify or identify that in candidates
Thank you, Matt. It's reassuring to know these concerns are coming up in other conversations too. It suggests this isn't just one analyst's frustration but a pattern many teams are noticing. I'd be very interested to hear what prompted those discussions. Was it a specific error that slipped through, a hiring or onboarding question, or the growing role of AI in model setup?
It’s going to be interesting when the inflection point Anthropic worries about comes to pass. Right now human + AI leads to better outcomes than humans alone or AI alone
What happens to accountability and liability when an inflection point occurs, when human in the loop causes more negative outcomes, more errors and is statistically more dangerous/error prone.
This perspective only lasts as long as we are in the Goldilocks zone of human + AI collaboration. And their concerns are warranted, how will society be able to deal with notions like liability and responsibility when the day comes that human in the loop itself becomes a liability and the main source of error. Right now humans do need to error check AI, soon that process of error checking will be where the majority of errors are introduced due to the human element
Thank you, Adeodatus. This is a serious question, and I don't think anyone has a complete answer yet.
The inflection point may well come for certain tasks. But notice that recognizing it requires exactly the skill the article defends. To show that human review now adds more errors than it removes, someone has to judge which results were right and which were wrong. That judgment doesn't disappear when the human leaves the loop. It moves upstream, into validating the system itself.
Aerospace already has a version of this. Autopilots fly more precisely than pilots for most of a flight, yet pilots are still trained to fly by hand, and accountability sits in certification: the system is qualified against evidence, and humans own the definition of its operating envelope. I expect FEA to follow a similar path. Liability will shift from signing off each result to qualifying the tools and defining the problem correctly.
That last part matters most. In my experience, the majority of serious errors are not in the solution but in the problem definition: the wrong loads, the wrong boundary conditions, the wrong assumption about how the structure behaves. An AI can solve a wrongly posed problem flawlessly. Knowing what the answer should roughly be is what tells you the question was wrong.
What you said at the end still will be valuable for sure, there’s still value in experience and intuition to know what to expect. I’m not doing anything really exciting, I do some dfm for injection molding and then some mold flow simulations to optimize the design, nothing safety critical or really exciting like FEA can be.
I’m confident AI will be able to do what I do very soon, but before I went for my engineering degree I worked as a plastics processor for a decade. It’s quite often that I over ride the simulation on design decisions just because it’s a) only so good and b) I have experience and intuition to over ride it
That last part is where engineers will still add value, but I have no idea how employers will ever quantify or identify that in candidates
This echos concerns I have been discussing several times this week.
Thank you, Matt. It's reassuring to know these concerns are coming up in other conversations too. It suggests this isn't just one analyst's frustration but a pattern many teams are noticing. I'd be very interested to hear what prompted those discussions. Was it a specific error that slipped through, a hiring or onboarding question, or the growing role of AI in model setup?