Sample Usage Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge1. Function DescriptionThis sample demonstrates how to use HcomAllReduce collective communication operator for graph construction, aimed at helping graph developers quickly understand collective communication definition and usage of this type of operators in graph construction.2. Directory Structurecpp/ ├── src/ | ├── CMakeLists.txt // CMake build file | ├── es_showcase.h // Header file | └── make_pfa_hcom_graph.cpp // sample file ├── rank_table/ | ├── a2/ | | └── rank_table_2p.json // A2(d802) 2-card rank table configuration (v1.0) ├── CMakeLists.txt // CMake build file ├── main.cpp // Program main entry ├── README.md // README file ├── run_sample.sh // Execution script └── utils.h // Utility file3. Usage Instructions3.1. Prepare CANN PackageInstalltoolkitandopspackages correctly following Environment PreparationSet environment variables (assuming package is installed at /usr/local/Ascend/)source /usr/local/Ascend/cann/set_env.sh3.2. Build and Execute1.2.1 Generate ES Interfaces and Build Graph for DUMPSimply run the following command to clean, generate interfaces, construct graph and DUMP graph:bash run_sample.shCurrent run_sample.sh behavior: automatically clean old build, build sample and default execute sample dump. When you see the following message, it indicates successful execution:[Success] sample execution successful, pbtxt dump generated in current directory. The file starts with ge_onnx_ and can be opened in netron for display1.2.2 Output File DescriptionAfter successful execution, the following files will be generated in current directory:ge_onnx_*.pbtxt- protobuf text format of graph structure, can be viewed with netron1.2.3 Build Graph and ExecuteImportant Prerequisite: Ensure your system has at least 2 available NPU devicesPlatform Support Description:A2 Platform:lspci | grep d802has output, script automatically usesrank_table/a2/rank_table_2p.jsonA5 Platform:lspci | grep d806has output, script will exit with error (this form is not supported)Other platforms: Current version does not support, will exit with error directly in scriptBesides basic graph construction and dump functionality, this sample also supports actually executing TP graph on multiple cards.Usage:bash run_sample.sh -t sample_and_runThis command will:Automatically generate ES interfacesCompile sample programAutomatically configure rank table and environment variables (RANK_TABLE_FILE,RANK_ID,DEVICE_ID)Run graph in parallel on 2 NPU devices (device ID automatically read from rank_table, each process corresponds to one rank and one device)Use HcomAllReduce for inter-card data synchronizationNote:Script will automatically identify hardware throughlspciand select corresponding rank table (currently only A2 usesrank_table/a2/rank_table_2p.json; A5 does not support this form)If you need to use other devices (like 2,3 or 4,5), please modifydevice_idin rank table file under corresponding platform directoryrun_sample.shwill automatically set all required environment variables, no manual configuration neededAfter successful execution, you will see:[Success] sample_and_run execution successful, pbtxt and data output dump generated in current directoryYou can view computation results through data file3.3. Log PrintingIf you need log printing to assist debugging during executable program execution, you can set the following environment variables beforebash run_sample.shto print logs to screen:export ASCEND_SLOG_PRINT_TO_STDOUT1 # Print logs to screen export ASCEND_GLOBAL_LOG_LEVEL0 # Log level set to debug level3.4. DUMP Graph During Graph Compilation ProcessIf you need to DUMP graph to assist debugging graph compilation process during executable program execution, you can set the following environment variables beforebash run_sample.sh -t sample_and_runto DUMP graph to execution path:export DUMP_GE_GRAPH24. Core Concepts Introduction4.1. Graph Construction StepsCreate graph builder (provides context, workspace and construction-related methods needed for graph construction)Add starting nodes (starting nodes refer to nodes without input dependencies, usually including graph inputs (like Data nodes) and weight constants (like Const nodes))Add intermediate nodes (intermediate nodes are computation nodes with input dependencies, usually generated by user graph construction logic, and connected using existing nodes as inputs)Set graph output (explicitly specify graph output nodes as computation result endpoints)4.2. Multi-card Running Key ConceptsEnvironment Variable Description:When running multi-card sample, script will automatically set the following environment variables:RANK_ID: Logical process number (0 or 1 in this sample)DEVICE_ID: Physical device ID (0 or 1 in this sample)RANK_TABLE_FILE: Rank table configuration file path (currently only A2:rank_table/a2/rank_table_2p.json; A5 does not support this form)For detailed introduction ofRANK_TABLE_FILE,RANK_ID,DEVICE_ID, please refer to Example a2: rank table configuration resource informationGE Initialization Configuration:std::mapge::AscendString, ge::AscendString config { {ge.exec.deviceId, device_id}, // From environment variable DEVICE_ID {ge.graphRunMode, 0}, {ge.exec.rankTableFile, rank_table_file}, // From environment variable RANK_TABLE_FILE {ge.exec.rankId, rank_id} // From environment variable RANK_ID };4.3. TP Graph ConstructionConcept Explanation:TP (Tensor Parallel) graph refers to graph structure running on multiple cards through tensor parallel method. This sample demonstrates how to use ES operators to build TP graph containing collective communication operators, achieving inter-card data synchronization and parallel computation.Graph Construction API Features:Supports multi-operator combination graph construction, including Flash Attention, matrix multiplication, collective communication operators, etc.Uses HcomAllReduce operator to achieve inter-card data aggregation, requires configuring rank table fileSupports data type conversion, can perform FP32 to FP16 conversion inside graph to improve performanceFor example, HcomAllReduce operator prototype is shown below, ES graph construction generated API is HcomAllReduce (C) or EsHcomAllReduce (C)REG_OP(HcomAllReduce) .INPUT(x, TensorType({DT_FLOAT, DT_INT32, DT_INT8, DT_INT16, DT_FLOAT16, DT_INT64})) .OUTPUT(y, TensorType({DT_FLOAT, DT_INT32, DT_INT8, DT_INT16, DT_FLOAT16, DT_INT64})) .REQUIRED_ATTR(reduction, String) .REQUIRED_ATTR(group, String) .ATTR(fusion, Int, 1) .ATTR(fusion_id, Int, -1) .OP_END_FACTORY_REG(HcomAllReduce)Its corresponding function prototype is:Function name: HcomAllReduce (C) or EsHcomAllReduce (C)Parameters: 5 in total, in order: x, reduction, group, fusion (optional, default 1), fusion_id (optional, default -1)Return value: output yC API:EsCTensorHolder *EsHcomAllReduce(EsCTensorHolder *x, const char *reduction, const char *group, int64_t fusion, int64_t fusion_id);C API:EsTensorHolder HcomAllReduce(const EsTensorHolder x, const char *reduction, const char *group, int64_t fusion1, int64_t fusion_id-1);Note: fusion and fusion_id are optional parameters in C API with default values, usually can be omitted【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
Sample Usage Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge1. Function DescriptionThis sample demonstrates how to use HcomAllReduce collective communication operator for graph construction, aimed at helping graph developers quickly understand collective communication definition and usage of this type of operators in graph construction.2. Directory Structurecpp/ ├── src/ | ├── CMakeLists.txt // CMake build file | ├── es_showcase.h // Header file | └── make_pfa_hcom_graph.cpp // sample file ├── rank_table/ | ├── a2/ | | └── rank_table_2p.json // A2(d802) 2-card rank table configuration (v1.0) ├── CMakeLists.txt // CMake build file ├── main.cpp // Program main entry ├── README.md // README file ├── run_sample.sh // Execution script └── utils.h // Utility file3. Usage Instructions3.1. Prepare CANN PackageInstalltoolkitandopspackages correctly following Environment PreparationSet environment variables (assuming package is installed at /usr/local/Ascend/)source /usr/local/Ascend/cann/set_env.sh3.2. Build and Execute1.2.1 Generate ES Interfaces and Build Graph for DUMPSimply run the following command to clean, generate interfaces, construct graph and DUMP graph:bash run_sample.shCurrent run_sample.sh behavior: automatically clean old build, build sample and default execute sample dump. When you see the following message, it indicates successful execution:[Success] sample execution successful, pbtxt dump generated in current directory. The file starts with ge_onnx_ and can be opened in netron for display1.2.2 Output File DescriptionAfter successful execution, the following files will be generated in current directory:ge_onnx_*.pbtxt- protobuf text format of graph structure, can be viewed with netron1.2.3 Build Graph and ExecuteImportant Prerequisite: Ensure your system has at least 2 available NPU devicesPlatform Support Description:A2 Platform:lspci | grep d802has output, script automatically usesrank_table/a2/rank_table_2p.jsonA5 Platform:lspci | grep d806has output, script will exit with error (this form is not supported)Other platforms: Current version does not support, will exit with error directly in scriptBesides basic graph construction and dump functionality, this sample also supports actually executing TP graph on multiple cards.Usage:bash run_sample.sh -t sample_and_runThis command will:Automatically generate ES interfacesCompile sample programAutomatically configure rank table and environment variables (RANK_TABLE_FILE,RANK_ID,DEVICE_ID)Run graph in parallel on 2 NPU devices (device ID automatically read from rank_table, each process corresponds to one rank and one device)Use HcomAllReduce for inter-card data synchronizationNote:Script will automatically identify hardware throughlspciand select corresponding rank table (currently only A2 usesrank_table/a2/rank_table_2p.json; A5 does not support this form)If you need to use other devices (like 2,3 or 4,5), please modifydevice_idin rank table file under corresponding platform directoryrun_sample.shwill automatically set all required environment variables, no manual configuration neededAfter successful execution, you will see:[Success] sample_and_run execution successful, pbtxt and data output dump generated in current directoryYou can view computation results through data file3.3. Log PrintingIf you need log printing to assist debugging during executable program execution, you can set the following environment variables beforebash run_sample.shto print logs to screen:export ASCEND_SLOG_PRINT_TO_STDOUT1 # Print logs to screen export ASCEND_GLOBAL_LOG_LEVEL0 # Log level set to debug level3.4. DUMP Graph During Graph Compilation ProcessIf you need to DUMP graph to assist debugging graph compilation process during executable program execution, you can set the following environment variables beforebash run_sample.sh -t sample_and_runto DUMP graph to execution path:export DUMP_GE_GRAPH24. Core Concepts Introduction4.1. Graph Construction StepsCreate graph builder (provides context, workspace and construction-related methods needed for graph construction)Add starting nodes (starting nodes refer to nodes without input dependencies, usually including graph inputs (like Data nodes) and weight constants (like Const nodes))Add intermediate nodes (intermediate nodes are computation nodes with input dependencies, usually generated by user graph construction logic, and connected using existing nodes as inputs)Set graph output (explicitly specify graph output nodes as computation result endpoints)4.2. Multi-card Running Key ConceptsEnvironment Variable Description:When running multi-card sample, script will automatically set the following environment variables:RANK_ID: Logical process number (0 or 1 in this sample)DEVICE_ID: Physical device ID (0 or 1 in this sample)RANK_TABLE_FILE: Rank table configuration file path (currently only A2:rank_table/a2/rank_table_2p.json; A5 does not support this form)For detailed introduction ofRANK_TABLE_FILE,RANK_ID,DEVICE_ID, please refer to Example a2: rank table configuration resource informationGE Initialization Configuration:std::mapge::AscendString, ge::AscendString config { {ge.exec.deviceId, device_id}, // From environment variable DEVICE_ID {ge.graphRunMode, 0}, {ge.exec.rankTableFile, rank_table_file}, // From environment variable RANK_TABLE_FILE {ge.exec.rankId, rank_id} // From environment variable RANK_ID };4.3. TP Graph ConstructionConcept Explanation:TP (Tensor Parallel) graph refers to graph structure running on multiple cards through tensor parallel method. This sample demonstrates how to use ES operators to build TP graph containing collective communication operators, achieving inter-card data synchronization and parallel computation.Graph Construction API Features:Supports multi-operator combination graph construction, including Flash Attention, matrix multiplication, collective communication operators, etc.Uses HcomAllReduce operator to achieve inter-card data aggregation, requires configuring rank table fileSupports data type conversion, can perform FP32 to FP16 conversion inside graph to improve performanceFor example, HcomAllReduce operator prototype is shown below, ES graph construction generated API is HcomAllReduce (C) or EsHcomAllReduce (C)REG_OP(HcomAllReduce) .INPUT(x, TensorType({DT_FLOAT, DT_INT32, DT_INT8, DT_INT16, DT_FLOAT16, DT_INT64})) .OUTPUT(y, TensorType({DT_FLOAT, DT_INT32, DT_INT8, DT_INT16, DT_FLOAT16, DT_INT64})) .REQUIRED_ATTR(reduction, String) .REQUIRED_ATTR(group, String) .ATTR(fusion, Int, 1) .ATTR(fusion_id, Int, -1) .OP_END_FACTORY_REG(HcomAllReduce)Its corresponding function prototype is:Function name: HcomAllReduce (C) or EsHcomAllReduce (C)Parameters: 5 in total, in order: x, reduction, group, fusion (optional, default 1), fusion_id (optional, default -1)Return value: output yC API:EsCTensorHolder *EsHcomAllReduce(EsCTensorHolder *x, const char *reduction, const char *group, int64_t fusion, int64_t fusion_id);C API:EsTensorHolder HcomAllReduce(const EsTensorHolder x, const char *reduction, const char *group, int64_t fusion1, int64_t fusion_id-1);Note: fusion and fusion_id are optional parameters in C API with default values, usually can be omitted【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
)
