文章目录
macos 使用brew instll fass, 后python3.12执行引用faiss包功能出现的问题
安装时遇到问题如下
ModuleNotFoundError Traceback (most recent call last)
File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:55, in dependable_faiss_import(no_avx2)54 else:
---> 55 import faiss56 except ImportError:ModuleNotFoundError: No module named 'faiss'During handling of the above exception, another exception occurred:ImportError Traceback (most recent call last)
Cell In[1], line 1914 texts = text_splitter.create_documents(15 [page.page_content for page in pages[:4]]16 )18 embeddings = OpenAIEmbeddings(model="text-embedding-ada-002")
---> 19 db = FAISS.from_documents(texts, embeddings)21 retriever = db.as_retriever(search_kwargs={"k": 5})23 docs = retriever.invoke(query="llama2有多少参数?")File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_core/vectorstores.py:635, in VectorStore.from_documents(cls, documents, embedding, **kwargs)633 texts = [d.page_content for d in documents]634 metadatas = [d.metadata for d in documents]
--> 635 return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:931, in FAISS.from_texts(cls, texts, embedding, metadatas, ids, **kwargs)912 """Construct FAISS wrapper from raw documents.913 914 This is a user friendly interface that:(...)928 faiss = FAISS.from_texts(texts, embeddings)929 """930 embeddings = embedding.embed_documents(texts)
--> 931 return cls.__from(932 texts,933 embeddings,934 embedding,935 metadatas=metadatas,936 ids=ids,937 **kwargs,938 )File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:883, in FAISS.__from(cls, texts, embeddings, embedding, metadatas, ids, normalize_L2, distance_strategy, **kwargs)871 @classmethod872 def __from(873 cls,(...)881 **kwargs: Any,882 ) -> FAISS:
--> 883 faiss = dependable_faiss_import()884 if distance_strategy == DistanceStrategy.MAX_INNER_PRODUCT:885 index = faiss.IndexFlatIP(len(embeddings[0]))File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:57, in dependable_faiss_import(no_avx2)55 import faiss56 except ImportError:
---> 57 raise ImportError(58 "Could not import faiss python package. "59 "Please install it with `pip install faiss-gpu` (for CUDA supported GPU) "60 "or `pip install faiss-cpu` (depending on Python version)."61 )62 return faissImportError: Could not import faiss python package. Please install it with `pip install faiss-gpu` (for CUDA supported GPU) or `pip install faiss-cpu` (depending on Python version).
直接使用本地 pip安装fass-cpu会遇到如下问题
venwei@svenweis-MacBook-Pro langchain$ pip3 install faiss-cpu
error: externally-managed-environment× This environment is externally managed
╰─> To install Python packages system-wide, try brew installxyz, where xyz is the package you are trying toinstall.If you wish to install a Python library that isn't in Homebrew,use a virtual environment:python3 -m venv path/to/venvsource path/to/venv/bin/activatepython3 -m pip install xyzIf you wish to install a Python application that isn't in Homebrew,it may be easiest to use 'pipx install xyz', which will manage avirtual environment for you. You can install pipx withbrew install pipxYou may restore the old behavior of pip by passingthe '--break-system-packages' flag to pip, or by adding'break-system-packages = true' to your pip.conf file. The latterwill permanently disable this error.If you disable this error, we STRONGLY recommend that you additionallypass the '--user' flag to pip, or set 'user = true' in your pip.conffile. Failure to do this can result in a broken Homebrew installation.Read more about this behavior here: <https://peps.python.org/pep-0668/>```大概意思是让尝试使用brew来安装, 安装完,再使用python来执行文件就出现文章头的错误. 提示内容也说如果非要使用pip安装就使用python虚拟环境. 目前使用python版本为python3.12, 于是就创建了虚拟环境,并且再次使用pip install faiss-cpu, 又遇到如下列表问题,
* 缺少其它安装包
* swig 执行时出现不同的错误
* ...
中间考虑过是否为python版本问题,运行看pypi上看下说明,只是大于Requires: Python >=3.8, 这句直接导致我浪费大量时间,中间也切换过python3.11遇到是swig执行时遇到问题.
于是考虑是不是swig版本与faiss的版本有相关的,又尝试不同的swig版本,结果仍出问题.
最后想编译faiss,在打算着手做时,才发现python的版本要求是3.8~3.10..WTF
下载并使用python3.10安装faiss-cpu,没用几分钟成功安装