LLMs Can't Touch Life-Critical Code: Developers Scrutinize AI's Danger in High-Stakes Software
Expert consensus points to LLMs failing when handling complex, safety-critical software where failure carries catastrophic risk. Skeptics repeatedly hammer home that superficial code correctness does not equal functional integrity, especially in areas like law where explainability is mandatory.
The debate pits those who see LLMs as nothing more than sophisticated copilot tools against those who view them as fundamentally devaluing the coder role. Users like 'CorrectAlias' argue the tools only help expert prompt-engineers, while others, including 'adespoton', see them as necessary productivity layers. Meanwhile, 'z3rOR0ne' takes a sharp shot, asserting that AI removes the necessary developmental struggle inherent in traditional debugging processes.
The overriding sentiment is deep distrust regarding deployment. The weight of opinion suggests LLMs are unfit for anything requiring robust, verifiably complex logic. The major fault lines are professional skill devaluation versus functional unreliability in critical systems.
Key Points
LLMs are inadequate for safety-critical code.
Multiple sources argue that in domains where failure costs lives or fortunes, LLM output is too unreliable (brynden_rivers_esq, owenfromcanada).
Specialized knowledge requires explainability, which AI lacks.
LLMs fail when specialized or non-public knowledge is needed, particularly in regulated fields like law (phutatorius).
AI removes the educational struggle of coding.
The difficulty of debugging, traditionally learned via platforms like Stack Overflow, is a crucial developmental step that AI bypasses (z3rOR0ne).
Tool utility is gated by user expertise.
Some claim LLMs are only useful for already expert users who know how to guide the tool ('CorrectAlias'), while others dismiss them as 'pretty demos' for simple tasks ('TehPers').
Productivity claims are dismissed as hype.
The idea of massive productivity jumps is countered by views that either the hype is premature or that it reflects a lack of foundational skills in preceding developers (jubilationtcornpone).
Source Discussions (6)
This report was synthesized from the following Lemmy discussions, ranked by community score.