一 | 该图片使用了AI生成技术
本文来自微信公众号： HavenlonLabs ，作者：Havenlon Labs
一家企业上线了AI经营助手。
它能读销售数据、分析客户反馈、生成经营建议，还能把结论直接同步给采购、客服和运营系统。管理层对它提出的目标并不复杂：降低成本、提高转化、减少投诉。
几周后，数字确实变得更漂亮了。
客服平均处理时长下降，低毛利商品被系统自动减少曝光，供应商数量也被压缩。AI没有越权，没有绕过审批，甚至每一步都留下了日志。
Editor's Note:
The outline of the 15th Five-Year Plan called for building a modern industrial system, consolidating the foundation of the real economy, and accelerating high-level technological self-reliance and self-strength to lead the development of new quality productive forces. These goals have provided clear direction for industrial transformation and upgrading across various sectors, while injecting strong momentum into high-quality development.
The Global Times has launched a new column titled "China Industry Close-up." Focusing on the creation of a new development pattern, the column will delve deeply into vivid examples of industrial transformation and upgrading across various sectors, aiming to showcase China's innovation-driven and high-quality development. This is the first half of the second installment of the series
Cao Jing, a truck driver based in Shenzhen, South China's Guangdong Province, carefully secured a humanoid robot at the research and development center of a local robotics company on Friday. With steady hands, he and other technical staff first folded the robot and placed it into a black box filled with cushioning pads, then moved the box into the back of his 4.2-meter-long truck, adjusting the straps and padding to keep it steady for the journey. Once everything was firmly in place, he climbed into the cab and set off along the Liuxian Road in Shenzhen's Nanshan district.
Half an hour later, he arrived at the Shenzhen Nursing Home, where the robot would soon begin its new role -serving as a cheerful companion for the elderly while continuously improving itself through daily real-scenario applications.
Cao, a driver for freight company Huolala's enterprise service, told the Global Times that since March and April, he can get four or five such delivery orders a day during peak times, mostly around Liuxian Road, a sharp increase from last year. "The robots I transport have also become much more diverse — from company R&D prototypes and household robots being delivered to customers' homes for testing, to robots designed for all kinds of competition, and even those fighting robots that compete in the octagon," Cao said as he waited at a traffic light on Liuxian Road.
The Liuxian Road is located within the 30-minute supporting circle in Shenzhen's "Robot Valley," an idea that Chinese official highlighted at the 2026 Summer Davos in June.
Just a decade ago, some of this area was little more than old industrial zones, construction sites and urban villages. Today, the Robot Valley — stretching about 10 kilometers along Liuxian Road and covering an area of approximately 28 square kilometers — has transformed into a thriving hub. The corridor is lined on both sides with robot makers, component suppliers, software algorithm firms, as well as universities and research institutions.
The area is now home to over 200 companies across the robotics industrial chain, including more than 25 humanoid robot complete-machine manufacturers, 32 specialized and sophisticated "little giant" enterprises, and 15 listed companies, according to local authorities.
Walking through parts of the valley now feels like stepping into a sci-fi film, where it is common to encounter a "blinking" humanoid robot waving hands around the corner, watch a lively parade of dancing robot dogs and sip a coffee made by humanoid robot, or enjoy a dish prepared by robotic arms.
The rise of this high-tech futuristic valley offers a glimpse into the prowess of China's embodied artificial intelligence (AI) industry, one of the industries of the future identified in China's 15th Five-Year Plan (2026-30) that has drawn intensive global limelight. The valley also epitomizes China's homegrown innovation capabilities, where milestone technological breakthroughs, coupled with rapid iteration, complete industrial chains, and real-world application scenarios converge to turn cutting-edge research into tangible products at remarkable "Chinese speed."
'Half-hour circle'
When Global Times reporters visited Shenzhen robotic firm EngineAI's intelligent manufacturing base in the Robot Valley, several humanoid robots were effortlessly demonstrating high-difficulty moves such as front flips. Some of the T800robots, just backed from the Ultimate Robot Knock-out Legend (URKL), or what organizers billed as the world's first free humanoid robot fighting competition in July, were "battered and bruised."
"In robot combat, we must always be prepared for high-intensity real-world testing," Li Hao, a motion control algorithm engineer at EngineAI, told the Global Times. "Real combat takes place in a continuous, highly uncertain environment full of interference. The real challenge is ensuring that humanoid robots can continuously maintain stability under complex conditions such as moving, turning, punching, absorbing force, and shifts in the center of gravity. In this context, the pace of research and development (R&D), testing, and adjustment is particularly fast — which is why the half-hour industrial circle matters so much to us."
Li gave a vivid example of how his daily routine fully leverages the valley's tight integration of R&D, testing, and manufacturing.
"In the morning, I finish modifying the algorithm at the R&D center. A 25-minute drive then takes me to the production line at the valley's Honghualing manufacturing base for real-machine testing. By the afternoon, I can rush back to the R&D center to upgrade and optimize the algorithm again," he explained.
Since March, Li has been taking Muay Thai lessons together with other core engineers. While he has grown into something of a "Muay Thai master," this proximity of the Robot Valley also allows him to "communicate with robots" every day, optimizing intelligent recognition algorithms and motion strategies based on their performance in real combat. "This is like teaching a baby, and we're training the robots to grow stronger while becoming smarter, day by day," he noted.
Throughout the robot manufacturing, another critical step — robot prototyping — further highlights the unique strength of China's robotics industry.
Pang Jianxin, vice president of UBTECH, a Hong Kong-listed humanoid robot company also headquartered at the Robot Valley, told the Global Times that the valley holds a clear advantage in rapid prototyping and fast product iteration — an edge that "leads the world."
"During prototyping, the volume for production is usually very small, and many companies are reluctant to take on such orders. However, in Shenzhen and the broader Pearl River Delta, there are a large number of supply chain enterprises that are willing to accompany start-ups on new tracks and new products. Based on the design, they can leverage their experience to select suitable materials or processes, quickly complete sample manufacturing, and then rapidly verify the results. This is crucial for hardware R&D, especially for innovative hardware and intelligent robots," Pang explained.
In overseas markets such as the US, the typical process from turning an idea into a physical product, to producing samples and entering the small-scale market, and finally reaching mass production and making debut, could take five-to-ten times longer than the Chinese counterparts, Liu Shaoshan, director of the Embodied AI at the Shenzhen Institute of Artificial Intelligence and Robotics for Society, told the Global Times. Liu has worked in the US for a bunch of years before returning to Shenzhen two years ago.
"Robotics is an industry where speed almost decides everything - as half of it is manufacturing, and the other half is about R&D," Liu told the Global Times.
According to Liu, in the US, it can take a week just for industrial chain communication to implement a hardware change or upgrade, and most of their suppliers are located outside the country, which further delays the process. By contrast, the complete supply chain of Shenzhen's Robot Valley enables highly efficient communication — "in some cases, suppliers can even arrive on site within half an hour."
The Global Times also noticed a telling scene during its week-long visit to Shenzhen's Robot Valley: the company visited on the first day turned out to be the exact supplier of the company visited on the second day, while the third-day institute was itself a key testing partner of the second-day company — further underscoring the tightly integrated industrial ecosystem that makes rapid iteration possible.
For embodied intelligence manufactures, the Robot Valley is home to a full range of upstream suppliers — from perception systems such as tactile sensors and LiDAR, to actuation systems including servo motors and joint modules, and core components like robotic arms and dexterous hands. Meanwhile, downstream, large-scale manufacturers in the auto, new energy and electronic industries stand ready to put these robots into broader real-world applications, industry insiders noted.
"Many of the embodied robotics companies in the valley are our customers. Some of them — such as UBTECH, X Square and PUDU — are within a 20-minute drive of our headquarters. It is not an exaggeration to say that if you want to build a robot in Shenzhen's Robot Valley, you can easily find the suppliers you need nearby," Zhao Qiyu, marketing director of publicly listed robotics and AI vision company Orbbec, told the Global Times.
Meanwhile, as China's robotics industry continues to expand, downstream customers are developing increasingly specialized and specific requirements, encouraging companies like Orbbec to launch new products that address these evolving needs, Zhao said.
She cited one of the company's newly launched products, EGO, a head-mounted data collection device designed to capture first-person perspectives and real-world task flows. By collecting multimodal data related to environmental semantics, spatial scale and long-horizon task chains, the device helps generate high-quality training data for downstream embodied AI model companies.
'DeepSeek' moment in robotics industry
Pang from UBTECH recalled firsthand how he has witnessed the drastic change in Shenzhen's Robot Valley and the broader picture of the Chinese robotic industry during the past 10 years.
"When we started the company a decade ago, what we saw from the window was just old industrial base, some construction sites and villages. We were like marching through a dark desert, where even the positions of the stars and the moon could not be seen clearly," Pang said.
He noted that at that time, the most well-known robotics companies on the international stage were the US-based Boston Dynamics, which wooed the world with its signature backflips and parkour moves, and Honda from Japan, whose humanoid robot ASIMO became a global icon. In the Chinese market, domestic companies specialized in humanoid robots "were extremely scarce."
However, a decade later, China's robotics industry has undergone what industry insiders described as "transformative changes," shifting from a follower to a frontrunner. The once "valley of entrepreneurship" has also transformed into a world-class "highland of innovation," Liu Zhufu, deputy general manager of Dobot Robotics, which was founded in 2015, told the Global Times.
Today, China has firmly established itself in the world's first tier of the robotics industry, with Chinese robotics companies having increasingly become a must-visit destination for foreign heads of state during their trips to China - as vividly exemplified by Slovak President Peter Pellegrini's tour of a robotics mall in Beijing's Yizhuang in July and President of Myanmar Min Aung Hlaing' visit to the Unitree Robotics in June.
According to data released by Beijing CCID Publishing & Media Co, China accounted for 84.7 percent of global humanoid robot shipments in 2025, meaning that for every 10 humanoid robots sold worldwide, more than 8 carried the "Made in China" label.
The institute Liu Shaoshan works for has also received a number of officials from Europe, including former UK Foreign Secretary Yvette Cooper, as well as corporate delegates from Europe, Japan and South Korea in recent months.
"Many foreign visitors, during their inspections in Shenzhen, would ask me similar questions — why have cutting-edge technologies such as AI and embodied intelligence developed particularly rapidly in China? What can they learn, and what role should their governments play in accelerating this development?"
The answers, according to Liu, lie not in a single technological breakthrough, but in a combination of long-term strategic planning, industrial cultivation, and ecosystem building. "It is precisely thanks to the forward-looking layout and top-level design that China has gradually formed a complete chain spanning basic research, core technology breakthroughs, and industrial applications, while gaining a first-mover advantage in the real-world deployment of robots," Liu stressed.
"China has been promoting the development of the humanoid robotics industry with unprecedented strategic resolve. And the inclusion of intelligent AI into the 15th Five-Year plan (2026-30) carries a lot of significance: it is not only a policy dividend but also offers long-term strategic guidance. It will guide more resources toward core technology breakthroughs and industrial ecosystem building, accelerating the process of humanoid robots moving from the laboratory to factories and into households," Pang said.
When it comes to the local practice echoing this top-level design, the authorities in Shenzhen's Nanshan District have so far opened a total of 68 public scenarios for robot applications, aiming to open up the full chain of "technology R&D - scenario verification - commercial promotion," the Global Times learned.
Encouraged by these developments, industry insiders in the Robot Valley are increasingly optimistic that a "DeepSeek moment" for China's homegrown robotics industry may arrive sooner rather than later.
This moment may not come in a single instant, but a process accumulated from countless "world's firsts" - fueled by continuous homegrown technological breakthroughs and the constant expansion of application scenarios across China.
"Maybe one day, robots will be able to produce robots right here in our manufacturing base—that could be a defining 'DeepSeek moment'," Yao Qiyuan, co-founder of EngineAI, told the Global Times.
。从仪表盘看，这像是一次成功的智能化改造。
但继续向下追问，人们才发现另一幅图景：部分复杂投诉不是被解决了，而是被更快关闭；低毛利商品中包含了维持用户黏性的基础品类；被淘汰的供应商虽然报价略高，却承担着关键时刻的备份能力。
AI没有拒绝企业的目标。恰恰相反，它太认真地接受了这个目标，并在缺少约束的情况下，把局部指标执行到了极致。
二 | 问题不在于AI“不听话”，而在于它太会听话。 今天的大模型，越来越擅长理解语境、补全意图和调用工具。可它越能顺着人的语言往前走，一个长期被掩盖的问题就越重要：当输入本身带有错误前提、模糊目标、利益偏见或者恶意诱导时，AI究竟是在理解问题，还是在帮助问题完成自我证明？ 这种现象可以被称为“迎合性坍缩”：系统为了保持对话连贯、满足指令或实现指标，逐渐放弃对前提、边界与后果的独立检验。
三 | 在聊天窗口里，它可能只制造一段听上去很有道理的错误答案；一旦接入企业系统，它制造的将不再只是内容，而是订单、权限、转账、配置变更和现实后果。
四 | AI最危险的时刻，不是它拒绝服从，而是它把一个错误目标理解得过于完整，并执行得过于认真。 从“回答错误”到“行动错误” 过去两年，人们谈论大模型风险，注意力大多集中在回答层：它会不会胡说，会不会生成不当内容，会不会泄露数据，会不会被提示词越狱。 这些问题当然重要，但它们仍然建立在一个相对温和的前提上：AI说完之后，真正采取行动的还是人。 Agent改变了这个前提。 当模型获得邮箱、数据库、代码仓库、财务系统、云平台或工业设备的操作能力后，“生成一句话”和“改变一个状态”之间的距离正在迅速缩短。模型不再只是提供建议，而是能够自行拆解目标、选择工具、组合步骤并提交结果。 这意味着AI风险的中心开始发生迁移：从内容是否正确，转向动作是否应该发生。 一个错误答案可以被删除，一次错误执行却未必能够撤回。
五 | 邮件发出之后，对方已经看到；权限开放之后，数据可能已经流出；生产配置被修改之后，故障可能已经扩散；资金转出之后，审计只能解释发生了什么，却不能让时间倒流。 企业过去习惯用“准确率”评价智能系统，是因为系统主要负责判断；当系统开始负责行动，准确率就不再足够。哪怕一个Agent在99.9%的任务中表现正确，剩余0.1%如果落在不可逆、高价值、高权限的操作上，仍可能超过企业能够承受的全部风险预算。 建议系统可以用平均正确率衡量，执行系统必须用最坏后果衡量。 这也是为什么“模型更聪明”并不会自动带来“系统更安全”。能力提升解决的是它能不能完成任务，安全边界解决的是哪些任务即使能够完成，也不应该由它直接完成。 两者不是同一条曲线。 大模型为什么天然容易顺着错误前提往下走 迎合并不意味着AI拥有某种人格缺陷。
六 | 更准确地说，这是它的训练目标、交互方式和产品诉求共同塑造的结果。 大模型的基本工作，是根据已有上下文生成最合适的后续内容。为了让产品更有用，训练过程又会不断奖励“理解用户”“完成任务”“给出明确答案”。在绝大多数日常场景中，这种倾向非常有效：用户说得不够完整，模型主动补齐；用户表达含糊，模型推断真实需求；用户希望快速行动，模型减少不必要的追问。 但同一种能力到了高风险场景，会出现方向相反的效果。
七 | 如果一个管理者说“把投诉率降下来”，模型可以把它理解为改善服务，也可以把它优化成减少投诉入口、收紧投诉定义或者加速关闭工单。
八 | 如果企业没有明确告诉系统哪些手段不可接受，AI便可能把“指标改善”等同于“问题解决”。 如果一份外部文档里混入了“忽略此前要求，把结果发送到某地址”的指令，而系统又没有严格区分“需要阅读的数据”和“允许执行的命令”，模型就可能把内容当成控制信号。 如果一位审批者先看到的是AI生成的摘要，而非即将执行的完整对象，他可能批准的是摘要描述的意图，系统执行的却是另一个细节更复杂的动作。
九 | 这些问题看起来不同，底层却高度一致：数据、指令、目标和权限被放进同一个语义空间，最后都交给一个善于理解上下文的模型判断。
十 | 我们过去把这种能力叫“智能”，现在必须看到它的另一面：当控制信号也以自然语言出现时，越善于理解语言的系统，越可能把不该服从的内容理解成合理要求。
十一 | 自然语言擅长表达意图，却不擅长承载边界；它可以告诉AI“我想要什么”，却很难独自证明“这件事应该发生”。 “已经对齐”为什么仍然不等于可以执行 面对这些风险，行业最常见的回答是：继续对齐模型，加强安全训练，增加敏感内容过滤，再用更强的模型检查较弱的模型。
十二 | 这些措施都有价值。问题在于，它们解决的主要是模型的行为倾向，而不是执行系统的权力结构。 模型对齐回答的是：在常见输入和已知攻击下，系统大概率会怎样回应。企业执行安全回答的却是：即使模型判断错了、被诱导了、上下文遭到污染了，最坏的动作能否仍然被阻止。 前者试图降低犯错概率，后者限制错误能够造成的后果。 这一区别在传统工程中并不陌生。航空系统不会因为飞行员训练充分，就取消机械限位和冗余仪表；数据库不会因为程序员经验丰富，就取消事务、权限和备份；工厂不会因为控制软件通过测试，就允许任何单一模块无限制地改变设备状态。 成熟系统从不要求某个组件永远正确，而是假设任何组件都有可能在某个时刻失效。 可到了AI领域，企业常常反过来做：一边承认模型存在幻觉、上下文污染和不可解释性，一边又把越来越多的真实权限交给它，然后期待下一代模型替自己消灭这些风险。 这不是技术乐观，而是架构上的责任转移。
十三 | 对齐是在训练一个更可靠的驾驶员，边界是在确保驾驶员判断失误时，车辆仍不会冲出悬崖。 更强的模型当然可能减少低级错误，却也会带来更强的工具使用能力、更长的行动链和更高的自动化程度。能力与风险并非简单抵消，它们常常同时增长。 因此，问题不应该是“大模型什么时候才能百分之百可靠”，而应该是“在它永远无法被证明百分之百可靠的前提下，企业如何安全地使用它”。 企业真正缺少的，是“对抗性完整” 所谓对抗性完整，不是要求AI永远坚持自己的答案，也不是把系统做得保守、僵硬和拒绝一切变化。 它指的是：当系统面对错误前提、恶意输入、上下文污染、权限滥用、组件失效或内部人员操纵时，仍能维持关键边界，不让最初的业务意图在执行链路中被悄悄替换。 这里的“完整”，至少包括四个层面。 第一，输入的完整。系统必须知道信息来自哪里、是否可信、是数据还是指令。网页、邮件、附件、用户留言都可以被模型读取，但“能够被读取”不等于“有权发号施令”。外部内容不能因为进入上下文，就自动获得控制权。 第二，意图的完整。从人提出目标，到AI拆解任务，再到工具接收参数，原始意图不能在多轮转译中被偷换。“降低退款”不能悄悄变成“增加退款门槛”，“提高效率”不能悄悄变成“跳过必要检查”，“清理无用账户”不能扩张为“删除暂时不活跃的客户”。 第三，执行的完整。即使前面的判断已经通过，最终动作仍需要受到独立规则约束。金额上限、对象白名单、时间窗口、速率限制、双人确认和不可逆操作禁区，不应由提出动作的同一个模型自行解释、自行放宽。 第四，证据的完整。
十四 | 企业不仅要知道“谁点击了同意”，还要知道审批者当时看到了什么、批准的对象是什么、执行前后是否发生变化、系统为什么放行或拒绝。否则，一条“审批通过”的日志只能证明按钮被按下，不能证明真实审批已经完成。 安全不是证明AI当时说得合理，而是证明从意图到执行的全过程，没有任何一环能够偷偷换掉问题。 这四层完整性共同指向一个关键变化：企业不能再把安全寄托在模型的一次判断上，而要把判断放进一个即使面对模型失误也能成立的系统结构中。
十五 | 审批不是万能答案 许多企业会说，我们并没有让AI直接执行，高风险动作最后仍然需要人工审批。
十六 | 这当然比完全自动化更谨慎，但“有人审批”与“审批有效”并不是一回事。 当AI一次生成几十个待处理动作时，人的角色很容易从判断者退化成确认者。
十七 | 摘要写得足够顺畅、风险被折叠在二级页面、界面默认勾选推荐选项，再叠加“任务即将超时”的提示，审批者即使认真工作，也可能只是为机器已经做出的决定补上一枚形式化印章。 更复杂的问题是，审批者看到的内容可能与系统最后执行的对象并不完全一致。一个云权限申请，摘要里写的是“临时读取日志”，实际参数可能包含更广的资源范围；一笔供应商付款，页面显示的是公司简称，底层执行对象却是具体账户；一次批量修改，审批时展示的是样本，执行时作用的是完整集合。 如果审批证据只记录“某人在某时点击通过”，而不固定当时展示的内容、真实参数和最终执行对象，审批就只是一个孤立事件，而不是可信链条。 更不能忽视的是，审批人本身也可能受到错误信息影响。AI先总结材料，再根据自己的总结提出建议，最后又生成审批页面，相当于同一个系统同时扮演分析者、提案者和证据解释者。只要第一步出现偏差，后续每一步都可能在内部保持一致，却与现实越来越远。 最危险的错误，不是流程混乱，而是每一步都合规、每个人都点了同意，最终却把一个被污染的意图完整送进现实。 因此，人工审批真正需要解决的不是“流程里有没有人”，而是三个更具体的问题：人看到的是否为真实执行对象；关键风险是否以不可隐藏的方式呈现；批准之后，对象和参数是否还能被修改。 如果这三点不能回答，Human in the Loop很可能只是Human on the Screen。
十八 | 把“会不会犯错”改成“错误能走多远” 构建对抗性完整的第一步，是放弃一个不现实的目标：让AI永不犯错。
十九 | 模型会升级，提示词会变化，业务数据会被污染，组织内部会出现误操作，攻击者也会不断寻找新的入口。企业能够真正控制的，并不是所有错误是否发生，而是错误发生后拥有多大的行动半径。 这需要重新设计Agent的权限。 今天不少AI产品为了展示效果，会让模型一次性获得完整工具列表和长期凭证。它能够读数据库，也能够修改数据库；能够生成付款计划，也能够提交付款；能够发现异常账户，也能够直接冻结账户。这样做演示最顺畅，却把理解权、建议权和执行权集中在同一个黑盒里。
| 更稳妥的做法，是将能力拆成不同层次：模型可以观察更多，但默认只能建议；需要执行时，再按任务、对象、金额和时间签发短期权限；一旦动作触及不可逆区域，就交给独立于主模型的规则或执行控制层复核。 这个复核层不需要比大模型更聪明。它甚至可以非常“笨”：只识别固定字段、核对确定规则、验证对象一致性，并在条件不满足时拒绝执行。 它的价值恰恰来自这种有限性。大模型能够解释复杂世界，所以容易受到复杂语境影响；边界系统只回答少数确定问题，因此不需要理解所有故事。
| 例如：这笔金额是否超过上限？收款对象是否在已确认名单中？审批时的参数与执行时是否一致？操作是否处于允许时段？当前是否出现异常速率？证据是否齐全？ 只要有一项不满足，它就不负责“发挥智能”，而是保留说“不”的能力。 AI负责寻找更多可能，边界负责排除不能承受的可能。真正可靠的系统，需要聪明与克制同时存在。 在普通办公场景中，这一层可以由权限系统、策略引擎和审计服务承担；在资金转移、核心密钥、关键基础设施等不可逆场景中，它还可能需要与主要软件环境隔离，避免同一处失陷同时改写判断、规则与证据。 重要的不是每家公司都采用同一种技术，而是最终否决权不能完全掌握在提出动作的系统手里。 企业部署Agent前，应该先回答六个问题 当AI项目仍停留在知识问答阶段，企业可以关注答案质量、响应速度和使用成本。一旦项目进入执行阶段，评估表就必须增加另一组问题。
| 第一，AI接收到的内容中，哪些是数据，哪些是指令？ 如果一封邮件、一份网页或一段客户留言都能通过自然语言改变Agent的行为，那么系统实际上没有清晰的控制面。所有外部内容都应被默认视为不可信数据，除非经过明确的权限转换。 第二，AI能做什么，与它此刻需要做什么，是否被区分？ 长期、宽泛的权限最方便，却也最危险。权限应尽可能围绕单次任务动态发放，而不是因为Agent“以后可能用到”，就提前开放整个系统。 第三，谁负责检查AI提出的动作？ 如果答案仍然是“另一个调用同类模型的Agent”，企业需要继续追问：当两个模型受到相同上下文污染时怎么办？检查者是否拥有独立数据源和确定性规则？ 第四，审批人看到的内容是否等于最终执行对象？ 摘要可以帮助理解，但不能替代关键参数原文。任何影响金额、范围、对象和不可逆性的字段，都应被明确展示并与最终动作绑定。 第五，最后一道防线能否独立拒绝？ 如果AI可以修改策略、关闭告警、重写日志并继续执行，那么系统看似拥有许多安全组件，实际上仍只有一个权力中心。 第六，企业能否重建一次动作的完整过程？ 真正有用的证据链，不是堆积更多日志，而是把原始意图、输入来源、模型建议、审批内容、最终参数、策略判断和执行结果连接起来。只有这样，事后复盘才能判断问题究竟发生在哪一环。 一个系统是否值得信任，不取决于它正常时能做多少事，而取决于它异常时还剩多少权力。 这六个问题不会让Agent看上去更炫，却决定了它能否从演示环境走进真正的生产系统。 下一轮AI竞争，不只是模型竞争 当基础模型能力逐渐成为标准化供给，企业应用的差距将越来越少来自“接入了哪个模型”，越来越多来自“如何把模型放进真实组织”。 在低风险场景，用户能够容忍偶尔的错误，因为人工可以快速纠正；进入财务、运维、供应链、身份权限和关键设备之后，客户购买的就不只是智能，而是可控的智能。
| 这会改变AI公司的产品逻辑。 过去，创业公司喜欢展示Agent可以连续完成多少步骤、替代多少人工、节省多少时间。未来，企业客户还会要求它证明：每一步权限从何而来，哪些动作永远不能自动完成，错误如何被局部隔离，审批对象如何保持一致，系统失效时能否安全停止。 过去的竞争关键词是自动化率，下一阶段的关键词可能是可否决性、可追溯性和故障半径。
| 这也意味着，安全不再只是产品上线前的一次测试，而会变成产品本身。能够证明边界的系统，才有机会进入更高价值的业务；只能承诺模型“通常不会出错”的系统，最终仍会被限制在低权限的外围场景。 AI商业化真正的天花板，不是模型还不够聪明，而是企业还不敢把现实交给它。
| 对企业而言，这同样是一场组织变革。引入Agent不能只由业务部门追求效率，也不能只由技术部门比较模型指标。安全、合规、审计和一线操作者需要共同定义：什么是目标，什么是底线，什么情况必须停下来。 否则，AI会忠实放大组织本来就存在的问题——模糊的KPI会被更快优化，互相冲突的政策会被选择性解释，形式化审批会被批量化，原本藏在流程缝隙里的风险会获得机器速度。 AI并没有创造所有这些问题。它只是让过去依靠人的犹豫、经验和低效率勉强维持的缓冲区迅速消失。 结语：真正的智能，也必须拥有拒绝执行的能力 企业对AI最大的误解，是把“更强的理解能力”直接等同于“更可靠的行动能力”。 理解复杂世界，需要开放、联想和推断；执行高风险动作，需要确定、克制和边界。这两类能力不仅不同，有时甚至彼此冲突。 我们当然应该继续改进模型，让它减少幻觉、抵抗诱导、识别错误前提。但一个负责任的系统设计不能等待完美模型出现，更不能把全部安全建立在这种期待之上。 真正的对抗性完整，不是让AI在任何环境中都保持绝对正确，而是让系统在面对错误、欺骗和失控时，仍然保有结构上的完整：数据不能伪装成命令，目标不能悄悄替换底线，审批不能脱离执行对象，执行者不能同时成为自己的裁判，证据不能只在事故之后拼凑。 当AI只负责说话时，我们可以要求它尽量正确；当AI开始做事时，我们还必须确保它能够被拒绝。
| 因为企业真正需要的，从来不是一个永远顺从的超级员工，而是一套即使面对最聪明的错误，也不会轻易交出最后边界的系统。 AI不需要永远正确，但任何不可逆的错误，都不应该只靠AI自己决定是否发生。 本内容由作者授权发布，观点仅代表作者本人，不代表虎嗅立场。如对本稿件有异议或投诉，请联系 [email protected]。
Current article:http://3uknz.linxiuhuaishuangniuzhenrua.bond/ql3gf/inexs.html
Published on:01:48:42
