Apache Airflow 3.3 安全加固拦截dag_id/run_id中的..路径穿越并支持allow_double_dot_in_ids配置【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflowApache Airflow 自 3.3.0 起默认拒绝包含连续点号..的 DAG ID 与 Run ID从根源上封堵基于标识符的路径穿越攻击同时提供[core] allow_double_dot_in_ids配置项默认False供确实依赖..命名的存量环境显式放行。本文以 63296.significant.rst 变更说明为主线结合 Airflow 源码与单元测试梳理该安全行为的触发位置、配置方法、升级影响与验证方式。变更背景与核心结论标识符identifier是 Airflow 中最常用的定位手段之一DAG ID 用于区分不同工作流Run ID 用于标识某次 DAG 运行。当这些 ID 被写入文件路径、日志目录、序列化数据等位置时一旦其中出现..这样的路径穿越片段攻击者就可能借由精心构造的 ID 跳出预期目录、读取或覆盖非授权文件形成路径穿越path traversal漏洞。本次变更PR/Issue 63296给出的修复方案是默认拒绝在dag_id与run_id中出现..连续点号直接抛出校验异常新增配置项[core] allow_double_dot_in_ids默认False仅在存在无法重命名的旧 DAG / 旧运行记录时才建议显式设为True放行。从变更类型看这是一次同时触及配置Config、API 行为API与运行行为Behaviour的破坏性significant变更升级到 3.3.0 及以上版本时需要纳入兼容性评估。校验逻辑在源码中的落地位置该安全策略并非只加在单一入口而是在 Airflow 的多个关键层面重复校验形成纵深防御。通用 Key 校验validate_keyairflow-core/src/airflow/utils/helpers.py 中的validate_key()是所有“键值类”标识符的通用校验入口KEY_REGEX re.compile(r^[\w.-]$) def validate_key(k: str, max_length: int 250): Validate value used as a key. if not isinstance(k, str): raise TypeError(fThe key has to be a string and is {type(k)}:{k}) if (length : len(k)) max_length: raise ValueError(fThe key has to be less than {max_length} characters, not {length}) if not KEY_REGEX.match(k): raise ValueError( fThe key {k!r} has to be made of alphanumeric characters, dashes, fdots and underscores exclusively ) if .. in k and not conf.getboolean(core, allow_double_dot_in_ids, fallbackFalse): raise ValueError(fThe key {k!r} must not contain consecutive dots (..) to prevent path traversal)其校验逻辑分三层类型与长度必须是字符串且长度不超过max_length默认 250字符集匹配^[\w.-]$即仅允许字母数字、连字符、点号与下划线路径穿越防护本次新增只要包含..且配置项allow_double_dot_in_ids未开启即抛出ValueError错误信息明确提示 “must not contain consecutive dots (..) to prevent path traversal”。该函数被 CLI 层复用例如 airflow-core/src/airflow/cli/commands/connection_command.py 中helpers.validate_key(args.conn_id, max_length200)即连接 IDconn_id同样受到..拦截。Run ID 校验DagRun.validate_run_id与序列化层Run ID 的校验定义在 airflow-core/src/airflow/models/dagrun.py 的 SQLAlchemy 列校验器validate_run_id中validates(run_id) def validate_run_id(self, key: str, run_id: str) - str | None: if not run_id: return None if .. in run_id and not airflow_conf.getboolean(core, allow_double_dot_in_ids, fallbackFalse): raise ValueError(fThe run_id {run_id} must not contain .. to prevent path traversal) if re.match(RUN_ID_REGEX, run_id): return run_id regex airflow_conf.get(scheduler, allowed_run_id_pattern).strip() if regex and re.match(regex, run_id): return run_id raise ValueError( fThe run_id provided {run_id} does not match regex pattern {regex} or {RUN_ID_REGEX} )注意其顺序先做..的路径穿越拦截再匹配标准RUN_ID_REGEX最后才回退到[scheduler] allowed_run_id_pattern自定义正则。RUN_ID_REGEX定义在同文件 dagrun.pyRUN_ID_REGEX r^(?:manual|scheduled|asset_triggered)__(?:\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\00:00)$同时序列化数据模型层也保留了同样的拦截逻辑。airflow-core/src/airflow/serialization/definitions/dag.py 在构造序列化 DAG Run 时重复检查.. in run_id并注释说明“SQLAlchemy 列校验器在某些场景下不可靠”因此序列化层做了二次防御——这保证了即使绕过 ORM 校验经序列化通道进入的数据同样无法携带..。配置项说明[core] allow_double_dot_in_ids配置项在官方配置模板 airflow-core/src/airflow/config_templates/config.yml 中定义如下allow_double_dot_in_ids: description: | Allow .. (consecutive dots) in DAG IDs and run IDs. By default, .. is blocked to prevent path traversal attacks. Set to True only if you have existing DAGs or runs whose IDs contain .. and cannot be renamed. version_added: 3.3.0 type: boolean example: ~ default: False关键信息所属 Section[core]类型布尔值boolean默认值False保持默认即启用路径穿越防护引入版本3.3.0适用前提仅当环境中存在 ID 含..且无法重命名的存量 DAG 或运行记录时才应开启官方描述明确强调 “Set toTrueonly if ... cannot be renamed”。配置方式与其他[core]配置一致例如写入airflow.cfg[core] allow_double_dot_in_ids True或通过环境变量AIRFLOW__CORE__ALLOW_DOUBLE_DOT_IN_IDSTrue设置。由于校验逻辑统一通过conf.getboolean(core, allow_double_dot_in_ids, fallbackFalse)读取见 helpers.py、dagrun.py 与 dag.py因此只需修改这一处配置即可影响所有校验入口。单元测试与行为验证仓库测试用例完整覆盖了新行为可作为升级后的自测参考。validate_key对..的拒绝airflow-core/tests/unit/utils/test_helpers.py 以参数化方式验证输入 key预期结果3/NoneTypeError非字符串simple_key/simple-key/group.simple_key/root.group.simple-key通过合法key with space/key_with_!ValueError字符集不合规251 个空格ValueError超长my..keyValueError消息为 “The key my..key must not contain consecutive dots (..) to prevent path traversal”..ValueError同上DagRun对路径穿越 run_id 的拒绝airflow-core/tests/unit/models/test_dagrun.py 中的test_dag_run_id_rejects_path_traversal覆盖三类恶意输入pytest.mark.parametrize( run_id, [ manual__..%2F..%2Fetc%2Fpasswd, # URL 编码形态的路径穿越 my..run, .., ], ) def test_dag_run_id_rejects_path_traversal(session, dag_maker, run_id): run_id containing .. should be rejected to prevent path traversal. with dag_maker(): pass with pytest.raises(ValueError, matchrmust not contain \.\.): dag_maker.create_dagrun(run_idrun_id, run_typeDagRunType.MANUAL)测试覆盖了普通连续点号my..run、..以及 URL 编码形态manual__..%2F..%2Fetc%2Fpasswd即../的编码变体确认创建 MANUAL 类型的 DAG Run 时会抛出匹配must not contain ..的ValueError。这提示升级时不仅要检查字面..还应警惕经过编码变形的历史数据。升级影响与迁移建议作为significant级别的变更升级到 Airflow 3.3.0 时应关注以下几点存量 ID 盘点扫描现有数据库与 DAG 文件找出所有含..的dag_id、run_id以及 conn_id 等经validate_key校验的标识符优先重命名而非开配置Airflow 官方配置描述明确建议优先重命名allow_double_dot_in_ids True只是“无法重命名”时的逃生通道长期开启会持续暴露路径穿越风险API / 序列化通道同样受限由于校验同时存在于模型层dagrun.py与序列化层dag.py通过 REST API、Task SDK 或序列化反序列化提交的 run_id 同样会被拦截错误信息定位校验失败会以ValueError形式抛出消息中均包含to prevent path traversal字样便于在日志与错误追踪中快速定位是否由该策略触发测试回归可参照上述两个测试用例在升级后对自定义 DAG 与手动触发流程做一次冒烟验证确认合法 ID仅字母数字、连字符、点号、下划线且无连续点号不受影响。总结Airflow 3.3.0 通过在validate_key、DagRun.validate_run_id与序列化层三处统一拦截..将“默认安全”落实到了 DAG ID / Run ID 的创建与写入全链路并通过[core] allow_double_dot_in_ids为存量环境保留了显式迁移窗口。对运维与安全团队而言核心动作是默认保持关闭、升级前盘点存量 ID、优先重命名、必要时显式放行并持续跟踪。【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflow创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
Apache Airflow 3.3 安全加固拦截dag_id/run_id中的..路径穿越并支持allow_double_dot_in_ids配置【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflowApache Airflow 自 3.3.0 起默认拒绝包含连续点号..的 DAG ID 与 Run ID从根源上封堵基于标识符的路径穿越攻击同时提供[core] allow_double_dot_in_ids配置项默认False供确实依赖..命名的存量环境显式放行。本文以 63296.significant.rst 变更说明为主线结合 Airflow 源码与单元测试梳理该安全行为的触发位置、配置方法、升级影响与验证方式。变更背景与核心结论标识符identifier是 Airflow 中最常用的定位手段之一DAG ID 用于区分不同工作流Run ID 用于标识某次 DAG 运行。当这些 ID 被写入文件路径、日志目录、序列化数据等位置时一旦其中出现..这样的路径穿越片段攻击者就可能借由精心构造的 ID 跳出预期目录、读取或覆盖非授权文件形成路径穿越path traversal漏洞。本次变更PR/Issue 63296给出的修复方案是默认拒绝在dag_id与run_id中出现..连续点号直接抛出校验异常新增配置项[core] allow_double_dot_in_ids默认False仅在存在无法重命名的旧 DAG / 旧运行记录时才建议显式设为True放行。从变更类型看这是一次同时触及配置Config、API 行为API与运行行为Behaviour的破坏性significant变更升级到 3.3.0 及以上版本时需要纳入兼容性评估。校验逻辑在源码中的落地位置该安全策略并非只加在单一入口而是在 Airflow 的多个关键层面重复校验形成纵深防御。通用 Key 校验validate_keyairflow-core/src/airflow/utils/helpers.py 中的validate_key()是所有“键值类”标识符的通用校验入口KEY_REGEX re.compile(r^[\w.-]$) def validate_key(k: str, max_length: int 250): Validate value used as a key. if not isinstance(k, str): raise TypeError(fThe key has to be a string and is {type(k)}:{k}) if (length : len(k)) max_length: raise ValueError(fThe key has to be less than {max_length} characters, not {length}) if not KEY_REGEX.match(k): raise ValueError( fThe key {k!r} has to be made of alphanumeric characters, dashes, fdots and underscores exclusively ) if .. in k and not conf.getboolean(core, allow_double_dot_in_ids, fallbackFalse): raise ValueError(fThe key {k!r} must not contain consecutive dots (..) to prevent path traversal)其校验逻辑分三层类型与长度必须是字符串且长度不超过max_length默认 250字符集匹配^[\w.-]$即仅允许字母数字、连字符、点号与下划线路径穿越防护本次新增只要包含..且配置项allow_double_dot_in_ids未开启即抛出ValueError错误信息明确提示 “must not contain consecutive dots (..) to prevent path traversal”。该函数被 CLI 层复用例如 airflow-core/src/airflow/cli/commands/connection_command.py 中helpers.validate_key(args.conn_id, max_length200)即连接 IDconn_id同样受到..拦截。Run ID 校验DagRun.validate_run_id与序列化层Run ID 的校验定义在 airflow-core/src/airflow/models/dagrun.py 的 SQLAlchemy 列校验器validate_run_id中validates(run_id) def validate_run_id(self, key: str, run_id: str) - str | None: if not run_id: return None if .. in run_id and not airflow_conf.getboolean(core, allow_double_dot_in_ids, fallbackFalse): raise ValueError(fThe run_id {run_id} must not contain .. to prevent path traversal) if re.match(RUN_ID_REGEX, run_id): return run_id regex airflow_conf.get(scheduler, allowed_run_id_pattern).strip() if regex and re.match(regex, run_id): return run_id raise ValueError( fThe run_id provided {run_id} does not match regex pattern {regex} or {RUN_ID_REGEX} )注意其顺序先做..的路径穿越拦截再匹配标准RUN_ID_REGEX最后才回退到[scheduler] allowed_run_id_pattern自定义正则。RUN_ID_REGEX定义在同文件 dagrun.pyRUN_ID_REGEX r^(?:manual|scheduled|asset_triggered)__(?:\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\00:00)$同时序列化数据模型层也保留了同样的拦截逻辑。airflow-core/src/airflow/serialization/definitions/dag.py 在构造序列化 DAG Run 时重复检查.. in run_id并注释说明“SQLAlchemy 列校验器在某些场景下不可靠”因此序列化层做了二次防御——这保证了即使绕过 ORM 校验经序列化通道进入的数据同样无法携带..。配置项说明[core] allow_double_dot_in_ids配置项在官方配置模板 airflow-core/src/airflow/config_templates/config.yml 中定义如下allow_double_dot_in_ids: description: | Allow .. (consecutive dots) in DAG IDs and run IDs. By default, .. is blocked to prevent path traversal attacks. Set to True only if you have existing DAGs or runs whose IDs contain .. and cannot be renamed. version_added: 3.3.0 type: boolean example: ~ default: False关键信息所属 Section[core]类型布尔值boolean默认值False保持默认即启用路径穿越防护引入版本3.3.0适用前提仅当环境中存在 ID 含..且无法重命名的存量 DAG 或运行记录时才应开启官方描述明确强调 “Set toTrueonly if ... cannot be renamed”。配置方式与其他[core]配置一致例如写入airflow.cfg[core] allow_double_dot_in_ids True或通过环境变量AIRFLOW__CORE__ALLOW_DOUBLE_DOT_IN_IDSTrue设置。由于校验逻辑统一通过conf.getboolean(core, allow_double_dot_in_ids, fallbackFalse)读取见 helpers.py、dagrun.py 与 dag.py因此只需修改这一处配置即可影响所有校验入口。单元测试与行为验证仓库测试用例完整覆盖了新行为可作为升级后的自测参考。validate_key对..的拒绝airflow-core/tests/unit/utils/test_helpers.py 以参数化方式验证输入 key预期结果3/NoneTypeError非字符串simple_key/simple-key/group.simple_key/root.group.simple-key通过合法key with space/key_with_!ValueError字符集不合规251 个空格ValueError超长my..keyValueError消息为 “The key my..key must not contain consecutive dots (..) to prevent path traversal”..ValueError同上DagRun对路径穿越 run_id 的拒绝airflow-core/tests/unit/models/test_dagrun.py 中的test_dag_run_id_rejects_path_traversal覆盖三类恶意输入pytest.mark.parametrize( run_id, [ manual__..%2F..%2Fetc%2Fpasswd, # URL 编码形态的路径穿越 my..run, .., ], ) def test_dag_run_id_rejects_path_traversal(session, dag_maker, run_id): run_id containing .. should be rejected to prevent path traversal. with dag_maker(): pass with pytest.raises(ValueError, matchrmust not contain \.\.): dag_maker.create_dagrun(run_idrun_id, run_typeDagRunType.MANUAL)测试覆盖了普通连续点号my..run、..以及 URL 编码形态manual__..%2F..%2Fetc%2Fpasswd即../的编码变体确认创建 MANUAL 类型的 DAG Run 时会抛出匹配must not contain ..的ValueError。这提示升级时不仅要检查字面..还应警惕经过编码变形的历史数据。升级影响与迁移建议作为significant级别的变更升级到 Airflow 3.3.0 时应关注以下几点存量 ID 盘点扫描现有数据库与 DAG 文件找出所有含..的dag_id、run_id以及 conn_id 等经validate_key校验的标识符优先重命名而非开配置Airflow 官方配置描述明确建议优先重命名allow_double_dot_in_ids True只是“无法重命名”时的逃生通道长期开启会持续暴露路径穿越风险API / 序列化通道同样受限由于校验同时存在于模型层dagrun.py与序列化层dag.py通过 REST API、Task SDK 或序列化反序列化提交的 run_id 同样会被拦截错误信息定位校验失败会以ValueError形式抛出消息中均包含to prevent path traversal字样便于在日志与错误追踪中快速定位是否由该策略触发测试回归可参照上述两个测试用例在升级后对自定义 DAG 与手动触发流程做一次冒烟验证确认合法 ID仅字母数字、连字符、点号、下划线且无连续点号不受影响。总结Airflow 3.3.0 通过在validate_key、DagRun.validate_run_id与序列化层三处统一拦截..将“默认安全”落实到了 DAG ID / Run ID 的创建与写入全链路并通过[core] allow_double_dot_in_ids为存量环境保留了显式迁移窗口。对运维与安全团队而言核心动作是默认保持关闭、升级前盘点存量 ID、优先重命名、必要时显式放行并持续跟踪。【免费下载链接】airflowApache Airflow - A platform to programmatically author, schedule, and monitor workflows项目地址: https://gitcode.com/GitHub_Trending/ai/airflow创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
)
