)
周六周日在家倒腾了两天总算稍微了解点pycaffe的一些接口和api设置了但是针对不同层还是有很多注意的细节的所以也准备写个博客专门总结pycaffe的常见层参数设置LZ这个蠢啊之前居然手写prototxt一把辛酸泪太傻了/(ㄒoㄒ)/~~之前已经写过用Anaconda直接安装caffe这样安装caffe不香嘛为啥非得源码再来安装一次呢1.项目里后端是用的tensorflow的接口前端是要用caffe的接口caffe的有些参数设置还是源码查询比较方便2.有一些层是比较特殊的这种定制化的层可能需要使用c写完之后再进行源码编译不过源码编译还是有很多坑的且听LZ一一道来首先下载源码https://github.com/weiliu89/caffe按照tutorial安装依赖项http://caffe.berkeleyvision.org/installation.htmlLZ的小小笔记本是Ubuntu16.04所以需要先安装对应的依赖库sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler sudo apt-get install --no-install-recommends libboost-all-dev然后LZ使用的是CMake编译的It requires CMake version 2.8.7. The basic steps are as follows:mkdir build cd build cmake .. make all make install make runtest按照教程好像简简单单10分钟就能搞定但是事实却不是如此。当然你得设置下CUDA#---[Optionscaffe_option(CPU_ONLYBuild Caffe without CUDA supportOFF)# TODO:rename to USE_CUDAcaffe_option(USE_CUDNNBuild Caffe with cuDNN library supportON IF NOT CPU_ONLY)caffe_option(USE_NCCLBuild Caffe with NCCL library supportOFF)caffe_option(BUILD_SHARED_LIBSBuild shared librariesON)caffe_option(BUILD_pythonBuild Python wrapperON)set(python_version2CACHE STRINGSpecify which Python version to use)caffe_option(BUILD_matlabBuild Matlab wrapperOFF IF UNIX OR APPLE)caffe_option(BUILD_docsBuild documentationON IF UNIX OR APPLE)caffe_option(BUILD_python_layerBuild the Caffe Python layerON)caffe_option(USE_OPENCVBuild with OpenCV supportON)caffe_option(USE_LEVELDBBuild with levelDBON)caffe_option(USE_LMDBBuild with lmdbON)caffe_option(ALLOW_LMDB_NOLOCKAllow MDB_NOLOCK when reading LMDB files (only if necessary)OFF)caffe_option(USE_OPENMPLink with OpenMP (when your BLAS wants OpenMP and you get linker errors)OFF)首先正常操作cmake ..注意前方高能ERROR来袭首先第一个坑就到来了LZ之前为了编译tensorflow源码已经源码安装过protobuf了也是用了-stdc11这个标准所以在cmake报了一些类似如下的错误error: #error This file requires compiler and library support for the ISO C 2011 standard. This support is currently experimental, and must be enabled with the -stdc11 or -stdgnu11 compiler optionserror: nullptr was not declared in this scope 这种一长串所以第一步要添加-stdc11 cmakelmakefile得这么改不过makefile中LZ没有尝试如果尝试过的小伙伴欢迎指导CXXFLAGS -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -stdc11 NVCCFLAGS -D_FORCE_INLINES -ccbin$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS) -stdc11 LINKFLAGS -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -stdc11第二个可能遇到的错误nvcc fatal : Unsupported gpu architecture compute_20这个是CUDA版本决定的LZ的版本是10.1 这个文件是在makefile.config里面的就顺带说一下#CUDA architecture setting:going with all of them.#For CUDA6.0,comment the*_50 through*_61 linesforcompatibility.#For CUDA8.0,comment the*_60 and*_61 linesforcompatibility.#For CUDA9.0,comment the*_20 and*_21 linesforcompatibility.CUDA_ARCH:-gencode archcompute_20,codesm_20 \-gencode archcompute_20,codesm_21 \-gencode archcompute_30,codesm_30 \-gencode archcompute_35,codesm_35 \-gencode archcompute_50,codesm_50 \-gencode archcompute_52,codesm_52 \-gencode archcompute_60,codesm_60 \-gencode archcompute_61,codesm_61 \-gencode archcompute_61,codecompute_61//注释掉这两句-gencode archcompute_20,codesm_20 \-gencode archcompute_20,codesm_21 \第三个可能遇到的错误fatal error:hdf5.h:No such file or directory compilation terminated.用locate定位你的hdf5.h的位置在makefile中添加对应的路径#Whateverelseyou find you need goes here.INCLUDE_DIRS:$(PYTHON_INCLUDE)/usr/local/include/usr/local/include/usr/include/hdf5/serial/LIBRARY_DIRS:$(PYTHON_LIB)/usr/local/lib/usr/lib/usr/lib/x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/hdf5/serial修改libraryLIBRARIESglog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial第四个可能遇到的错误这个问题就是OpenCV 4版本接口改变导致的这个改的就有点多了需要把caffe源码涉及到这个 CV_LOAD_IMAGE_COLOR都换掉。../caffe/src/caffe/layers/window_data_layer.cpp:293:42: error: ‘CV_LOAD_IMAGE_COLOR’ was not declared in this scope cv_img cv::imread(image.first, CV_LOAD_IMAGE_COLOR);src/caffe/layers/window_data_layer.cppimage_database_cache_[window[WindowDataLayerDtype::IMAGE_INDEX]];cv_imgDecodeDatumToCVMat(image_cached.second,true);}else{//cv_img cv::imread(image.first, CV_LOAD_IMAGE_COLOR);cv_imgcv::imread(image.first,cv::IMREAD_COLOR);if(!cv_img.data){LOG(ERROR)Could not open or find file image.first;return;src/caffe/test/test_io.cppboolReadImageToDatumReference(conststringfilename,constintlabel,constintheight,constintwidth,constbool is_color,Datum*datum){cv::Mat cv_img;//int cv_read_flag (is_color ? CV_LOAD_IMAGE_COLOR ://CV_LOAD_IMAGE_GRAYSCALE);intcv_read_flag(is_color?cv::IMREAD_COLOR:cv::IMREAD_GRAYSCALE);cv::Mat cv_img_origincv::imread(filename,cv_read_flag);if(!cv_img_origin.data){src/caffe/util/io.cppvoidWriteProtoToBinaryFile(constMessageproto,constchar*filename){cv::MatReadImageToCVMat(conststringfilename,constintheight,constintwidth,constbool is_color){cv::Mat cv_img;// int cv_read_flag (is_color ? CV_LOAD_IMAGE_COLOR :// CV_LOAD_IMAGE_GRAYSCALE);intcv_read_flag(is_color?cv::IMREAD_COLOR:cv::IMREAD_GRAYSCALE);cv::Mat cv_img_origincv::imread(filename,cv_read_flag);if(!cv_img_origin.data){LOG(ERROR)Could not open or find file filename;cv::MatDecodeDatumToCVMat(constDatumdatum,bool is_color){CHECK(datum.encoded())Datum not encoded;conststringdatadatum.data();std::vectorcharvec_data(data.c_str(),data.c_str()data.size());// int cv_read_flag (is_color ? CV_LOAD_IMAGE_COLOR :// CV_LOAD_IMAGE_GRAYSCALE);intcv_read_flag(is_color?cv::IMREAD_COLOR:cv::IMREAD_GRAYSCALE);cv_imgcv::imdecode(vec_data,cv_read_flag);if(!cv_img.data){LOG(ERROR)Could not decode datum ;第五个可能遇到的错误Anaconda的三方库和本地库冲突了那就把Anaconda的环境变量先注释掉编译好了再放出来就好了最后放张编译成功的图。。。自己踩坑千千万跳出一坑又进了一坑据说OpenCV4.1版本之前存在安全隐患问题所以现在LZ的公司都换成OpenCV4.2版本了。。。参考地址1.https://blog.csdn.net/m_buddy/article/details/880854252.https://blog.csdn.net/qq_18649781/article/details/890474373.https://github.com/BVLC/caffe/issues/66804.https://github.com/BVLC/caffe/pull/6625/files/7f503bd9a19758a173064e299ab9d4cac65ed60fPS:LZ已经在自己家隔离20多天了到工作的地方还得隔离14天大门不出二门不卖真希望能够肆无忌惮的在一个阳关明媚的午后压马路O(∩_∩)O现在是关键时刻还不能放松越到紧要关头越要能忍住武汉加油湖北加油中国加油↖(ω)↗
周六周日在家倒腾了两天总算稍微了解点pycaffe的一些接口和api设置了但是针对不同层还是有很多注意的细节的所以也准备写个博客专门总结pycaffe的常见层参数设置LZ这个蠢啊之前居然手写prototxt一把辛酸泪太傻了/(ㄒoㄒ)/~~之前已经写过用Anaconda直接安装caffe这样安装caffe不香嘛为啥非得源码再来安装一次呢1.项目里后端是用的tensorflow的接口前端是要用caffe的接口caffe的有些参数设置还是源码查询比较方便2.有一些层是比较特殊的这种定制化的层可能需要使用c写完之后再进行源码编译不过源码编译还是有很多坑的且听LZ一一道来首先下载源码https://github.com/weiliu89/caffe按照tutorial安装依赖项http://caffe.berkeleyvision.org/installation.htmlLZ的小小笔记本是Ubuntu16.04所以需要先安装对应的依赖库sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler sudo apt-get install --no-install-recommends libboost-all-dev然后LZ使用的是CMake编译的It requires CMake version 2.8.7. The basic steps are as follows:mkdir build cd build cmake .. make all make install make runtest按照教程好像简简单单10分钟就能搞定但是事实却不是如此。当然你得设置下CUDA#---[Optionscaffe_option(CPU_ONLYBuild Caffe without CUDA supportOFF)# TODO:rename to USE_CUDAcaffe_option(USE_CUDNNBuild Caffe with cuDNN library supportON IF NOT CPU_ONLY)caffe_option(USE_NCCLBuild Caffe with NCCL library supportOFF)caffe_option(BUILD_SHARED_LIBSBuild shared librariesON)caffe_option(BUILD_pythonBuild Python wrapperON)set(python_version2CACHE STRINGSpecify which Python version to use)caffe_option(BUILD_matlabBuild Matlab wrapperOFF IF UNIX OR APPLE)caffe_option(BUILD_docsBuild documentationON IF UNIX OR APPLE)caffe_option(BUILD_python_layerBuild the Caffe Python layerON)caffe_option(USE_OPENCVBuild with OpenCV supportON)caffe_option(USE_LEVELDBBuild with levelDBON)caffe_option(USE_LMDBBuild with lmdbON)caffe_option(ALLOW_LMDB_NOLOCKAllow MDB_NOLOCK when reading LMDB files (only if necessary)OFF)caffe_option(USE_OPENMPLink with OpenMP (when your BLAS wants OpenMP and you get linker errors)OFF)首先正常操作cmake ..注意前方高能ERROR来袭首先第一个坑就到来了LZ之前为了编译tensorflow源码已经源码安装过protobuf了也是用了-stdc11这个标准所以在cmake报了一些类似如下的错误error: #error This file requires compiler and library support for the ISO C 2011 standard. This support is currently experimental, and must be enabled with the -stdc11 or -stdgnu11 compiler optionserror: nullptr was not declared in this scope 这种一长串所以第一步要添加-stdc11 cmakelmakefile得这么改不过makefile中LZ没有尝试如果尝试过的小伙伴欢迎指导CXXFLAGS -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -stdc11 NVCCFLAGS -D_FORCE_INLINES -ccbin$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS) -stdc11 LINKFLAGS -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -stdc11第二个可能遇到的错误nvcc fatal : Unsupported gpu architecture compute_20这个是CUDA版本决定的LZ的版本是10.1 这个文件是在makefile.config里面的就顺带说一下#CUDA architecture setting:going with all of them.#For CUDA6.0,comment the*_50 through*_61 linesforcompatibility.#For CUDA8.0,comment the*_60 and*_61 linesforcompatibility.#For CUDA9.0,comment the*_20 and*_21 linesforcompatibility.CUDA_ARCH:-gencode archcompute_20,codesm_20 \-gencode archcompute_20,codesm_21 \-gencode archcompute_30,codesm_30 \-gencode archcompute_35,codesm_35 \-gencode archcompute_50,codesm_50 \-gencode archcompute_52,codesm_52 \-gencode archcompute_60,codesm_60 \-gencode archcompute_61,codesm_61 \-gencode archcompute_61,codecompute_61//注释掉这两句-gencode archcompute_20,codesm_20 \-gencode archcompute_20,codesm_21 \第三个可能遇到的错误fatal error:hdf5.h:No such file or directory compilation terminated.用locate定位你的hdf5.h的位置在makefile中添加对应的路径#Whateverelseyou find you need goes here.INCLUDE_DIRS:$(PYTHON_INCLUDE)/usr/local/include/usr/local/include/usr/include/hdf5/serial/LIBRARY_DIRS:$(PYTHON_LIB)/usr/local/lib/usr/lib/usr/lib/x86_64-linux-gnu/usr/lib/x86_64-linux-gnu/hdf5/serial修改libraryLIBRARIESglog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial第四个可能遇到的错误这个问题就是OpenCV 4版本接口改变导致的这个改的就有点多了需要把caffe源码涉及到这个 CV_LOAD_IMAGE_COLOR都换掉。../caffe/src/caffe/layers/window_data_layer.cpp:293:42: error: ‘CV_LOAD_IMAGE_COLOR’ was not declared in this scope cv_img cv::imread(image.first, CV_LOAD_IMAGE_COLOR);src/caffe/layers/window_data_layer.cppimage_database_cache_[window[WindowDataLayerDtype::IMAGE_INDEX]];cv_imgDecodeDatumToCVMat(image_cached.second,true);}else{//cv_img cv::imread(image.first, CV_LOAD_IMAGE_COLOR);cv_imgcv::imread(image.first,cv::IMREAD_COLOR);if(!cv_img.data){LOG(ERROR)Could not open or find file image.first;return;src/caffe/test/test_io.cppboolReadImageToDatumReference(conststringfilename,constintlabel,constintheight,constintwidth,constbool is_color,Datum*datum){cv::Mat cv_img;//int cv_read_flag (is_color ? CV_LOAD_IMAGE_COLOR ://CV_LOAD_IMAGE_GRAYSCALE);intcv_read_flag(is_color?cv::IMREAD_COLOR:cv::IMREAD_GRAYSCALE);cv::Mat cv_img_origincv::imread(filename,cv_read_flag);if(!cv_img_origin.data){src/caffe/util/io.cppvoidWriteProtoToBinaryFile(constMessageproto,constchar*filename){cv::MatReadImageToCVMat(conststringfilename,constintheight,constintwidth,constbool is_color){cv::Mat cv_img;// int cv_read_flag (is_color ? CV_LOAD_IMAGE_COLOR :// CV_LOAD_IMAGE_GRAYSCALE);intcv_read_flag(is_color?cv::IMREAD_COLOR:cv::IMREAD_GRAYSCALE);cv::Mat cv_img_origincv::imread(filename,cv_read_flag);if(!cv_img_origin.data){LOG(ERROR)Could not open or find file filename;cv::MatDecodeDatumToCVMat(constDatumdatum,bool is_color){CHECK(datum.encoded())Datum not encoded;conststringdatadatum.data();std::vectorcharvec_data(data.c_str(),data.c_str()data.size());// int cv_read_flag (is_color ? CV_LOAD_IMAGE_COLOR :// CV_LOAD_IMAGE_GRAYSCALE);intcv_read_flag(is_color?cv::IMREAD_COLOR:cv::IMREAD_GRAYSCALE);cv_imgcv::imdecode(vec_data,cv_read_flag);if(!cv_img.data){LOG(ERROR)Could not decode datum ;第五个可能遇到的错误Anaconda的三方库和本地库冲突了那就把Anaconda的环境变量先注释掉编译好了再放出来就好了最后放张编译成功的图。。。自己踩坑千千万跳出一坑又进了一坑据说OpenCV4.1版本之前存在安全隐患问题所以现在LZ的公司都换成OpenCV4.2版本了。。。参考地址1.https://blog.csdn.net/m_buddy/article/details/880854252.https://blog.csdn.net/qq_18649781/article/details/890474373.https://github.com/BVLC/caffe/issues/66804.https://github.com/BVLC/caffe/pull/6625/files/7f503bd9a19758a173064e299ab9d4cac65ed60fPS:LZ已经在自己家隔离20多天了到工作的地方还得隔离14天大门不出二门不卖真希望能够肆无忌惮的在一个阳关明媚的午后压马路O(∩_∩)O现在是关键时刻还不能放松越到紧要关头越要能忍住武汉加油湖北加油中国加油↖(ω)↗
 Classification （CPU/GPU设置, feature map可视化，层名，参数输出， Top-1、Top-5输出）)
 NVIDIA 2D Image And Signal Performance Primitives (NPP) 初探)
平方复杂度无法被常规优化消除｜TaoToken 统一 Key 通道实测)
