Generative artificial intelligence (GenAI) is shifting from conversational assistants toward agentic systems -- autonomous decision-making systems that sense, decide, and act within operational workflows. This shift creates an autonomy paradox: as GenAI systems are granted greater operational autonomy, they should, by design, embody more formal structure, more explicit constraints, and stronger tail-risk discipline. We argue stochastic generative models can be fragile in operational domains unless paired with mechanisms that provide verifiable feasibility, robustness to distribution shift, and stress testing under high-consequence scenarios. To address this challenge, we develop a conceptual framework for assured autonomy grounded in operations research (OR), built on two complementary approaches. First, flow-based generative models frame generation as deterministic transport characterized by an ordinary differential equation, enabling auditability, constraint-aware generation, and connections to optimal transport, robust optimization, and sequential decision control. Second, operational safety is formulated through an adversarial robustness lens: decision rules are evaluated against worst-case perturbations within uncertainty or ambiguity sets, making unmodeled risks part of the design. This framework clarifies how increasing autonomy shifts OR's role from solver to guardrail to system architect, with responsibility for control logic, incentive protocols, monitoring regimes, and safety boundaries. These elements define a research agenda for assured autonomy in safety-critical, reliability-sensitive operational domains.
翻译:生成式人工智能正从对话助手转向代理系统——即在操作工作流中感知、决策与行动的自主决策系统。这一转变引发了自主性悖论:随着生成式人工智能系统被赋予更高的操作自主权,其设计理应体现更规范的结构、更明确的约束以及更强的尾部风险管控。我们认为,随机生成模型在操作领域可能具有脆弱性,除非配备能提供可验证可行性、分布偏移鲁棒性及高后果场景压力测试的机制。为应对这一挑战,我们基于运筹学构建了一个保障性自主的概念框架,该框架包含两种互补方法。首先,基于流的生成模型将生成过程构建为以常微分方程表征的确定性传输,从而实现可审计性、约束感知生成,并与最优传输、鲁棒优化及序贯决策控制建立联系。其次,操作安全性通过对抗鲁棒性视角构建:决策规则在不确定性或模糊集内的最坏扰动下进行评估,使未建模风险成为设计的一部分。该框架阐明了自主性提升如何使运筹学的角色从求解器转变为护栏再升级为系统架构师,承担起控制逻辑、激励协议、监测机制与安全边界的责任。这些要素共同定义了在安全关键、可靠性敏感的操作领域中保障性自主的研究议程。