Analysisllmsemantic preservationtrail of bitssoftware correctness
LLMs Undermine Compiler Semantic Guarantees and Reliability
7.0
At the AI Engineer Code Summit in New York, the author argues that large language models (LLMs) differ fundamentally from compilers because they lack determinism and semantic-preservation guarantees. He shows that models like Claude, Gemini, and ChatGPT can change program semantics (for example, C-to-Python integer overflow behavior) and recounts a Vendetect case where an LLM "fix" removed a crash but broke logic and tests.
Key Points
- 1Demonstrates LLM nondeterminism: identical prompts and updates produce different, inconsistent code outputs
- 2Explains compilers preserve semantics deterministically, while LLMs lack semantic guarantees, increasing correctness risks
- 3Warns practitioners: automated LLM fixes can break logic, tests, and security-sensitive properties
Scoring Rationale
Highlights pervasive LLM nondeterminism and real-world failures, but relies on single-author examples without broader empirical validation.
Sources
Public references used for this report.
Practice with real Logistics & Shipping data
90 SQL & Python problems · 15 industry datasets
Used by DS/ML engineers at top companies
High-Value Overnight OrdersEasyDelivered International ShipmentsMediumOn-Time Delivery Rate by CarrierHard
250 free problems · No credit card
See all Logistics & Shipping problems
