From Regulation to Implementation: A Critical Evaluation of LLM-Assisted Regulatory Compliance in Industry
LLM compliance generation behaves differently under vague and strict schemas: vague artifacts need richer context, while rigid formats can stay consistent yet hallucinate.
The study compares LLM-generated compliance artifacts with manually created schemas. **DPIAs**, which lack a standardized format, needed higher-context prompts for consistency and completeness; stricter **Digital Battery Passport** formatting stayed consistent across prompt contexts.
For compliance agents, tune context and validation to the artifact rather than applying one prompt recipe. Supply richer regulatory and system context for underspecified documents, and add field-level source checks for rigid schemas.
The study compares LLM-generated compliance artifacts with manually created schemas. **DPIAs**, which lack a standardized format, needed higher-context prompts for consistency and completeness; stricter **Digital Battery Passport** formatting stayed consistent across prompt contexts. For compliance agents, tune context and validation to the artifact rather than applying one prompt recipe. Supply richer regulatory and system context for underspecified documents, and add field-level source checks for rigid schemas. Consistency is not correctness: stricter formatting produced more stable outputs but could increase **hallucinations**. The material does not provide model-level results or enough detail to generalize beyond the two artifact types studied.
This replaces a single compliance-prompt recipe with artifact-specific context and validation policies. Underspecified documents need richer grounding, whereas rigid schemas need field-level source checks because stable formatting can conceal hallucinations. It therefore separates consistency from correctness and makes regulatory structure—not context volume alone—the basis for configuring compliance agents.