Chapter 8, page 232

[ChiEaton]

I am running the code below on Colab. But getting the error in index that ValueError: input not a numpy array
Anyone helps? thanks.

import faiss

dim = embeds.shape[1]
index = faiss.IndexFlatL2(dim)
index.add(np.float32(embeds))

---

ValueError Traceback (most recent call last)
in <cell line: 0>()
3 dim = embeds.shape[1]
4 index = faiss.IndexFlatL2(dim)
----> 5 index.add(np.float32(embeds))

1 frames
/usr/local/lib/python3.11/dist-packages/faiss/swigfaiss_avx512.py in swig_ptr(a)
11448
11449 def swig_ptr(a):

11450 return _swigfaiss_avx512.swig_ptr(a)
11451
11452 def rev_swig_ptr(*args):

ValueError: input not a numpy array

[MaartenGr]

Thank you for sharing this. Just to be sure, did you install the packages using the code provided in the notebook or using the requirements.txt file?

[ChiEaton]

Thanks for getting back. I am running on Colab. I uncomment the packages. I just tried it. Below code were fine until the last line index.add(np.float32(embeds)) returning error "ValueError: input not a numpy array". This is weird, since np is numpy.

%%capture
!pip install langchain==0.2.5 faiss-cpu==1.8.0 cohere==5.5.8 langchain-community==0.2.5 rank_bm25==0.2.2 sentence-transformers==3.0.1
!pip install llama-cpp-python==0.2.78 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124

import cohere

Paste your API key here. Remember to not share publicly

api_key = 'xxxxx'

Create and retrieve a Cohere API key from os.cohere.ai

co = cohere.Client(api_key)

text = """
Interstellar is a 2014 epic science fiction film co-written, directed, and produced by Christopher Nolan.
It stars Matthew McConaughey, Anne Hathaway, Jessica Chastain, Bill Irwin, Ellen Burstyn, Matt Damon, and Michael Caine.
Set in a dystopian future where humanity is struggling to survive, the film follows a group of astronauts who travel through a wormhole near Saturn in search of a new home for mankind.

Brothers Christopher and Jonathan Nolan wrote the screenplay, which had its origins in a script Jonathan developed in 2007.
Caltech theoretical physicist and 2017 Nobel laureate in Physics[4] Kip Thorne was an executive producer, acted as a scientific consultant, and wrote a tie-in book, The Science of Interstellar.
Cinematographer Hoyte van Hoytema shot it on 35 mm movie film in the Panavision anamorphic format and IMAX 70 mm.
Principal photography began in late 2013 and took place in Alberta, Iceland, and Los Angeles.
Interstellar uses extensive practical and miniature effects and the company Double Negative created additional digital effects.

Interstellar premiered on October 26, 2014, in Los Angeles.
In the United States, it was first released on film stock, expanding to venues using digital projectors.
The film had a worldwide gross over $677 million (and $773 million with subsequent re-releases), making it the tenth-highest grossing film of 2014.
It received acclaim for its performances, direction, screenplay, musical score, visual effects, ambition, themes, and emotional weight.
It has also received praise from many astronomers for its scientific accuracy and portrayal of theoretical astrophysics. Since its premiere, Interstellar gained a cult following,[5] and now is regarded by many sci-fi experts as one of the best science-fiction films of all time.
Interstellar was nominated for five awards at the 87th Academy Awards, winning Best Visual Effects, and received numerous other accolades"""

Split into a list of sentences

texts = text.split('.')

Clean up to remove empty spaces and new lines

texts = [t.strip(' \n') for t in texts]

import numpy as np

Get the embeddings

response = co.embed(
texts=texts,
input_type="search_document",
).embeddings

embeds = np.array(response)
print(embeds.shape)

import faiss

dim = embeds.shape[1]
index = faiss.IndexFlatL2(dim)
index.add(np.float32(embeds))

[MaartenGr]

That is indeed odd. Could you try it by using the packages from the requirements.txt? Perhaps there is something going on there.

Also, two more questions:

• Did you change any code from the examples?
• Could you share the full error? There are 5 frames missing there that might give us some insight.

[MaximusPrimus]

I am also facing the same issue, I have packages as per requirements.txt. Infact while running the notebook straight into colab similar issue comes. despite being a numpy array, embeds on being added to index throws up similar error.

[MaartenGr]

Hmmm, that's odd. Perhaps something changed with the cohere embeddings. @jalammar could you take a look?

[MaximusPrimus]

The error is stemming from index.add() where it is not able to add embeds to index despite it being a numpy array.

[ChiEaton]

The error is stemming from index.add() where it is not able to add embeds to index despite it being a numpy array

Did you resolve it? thanks

[ycymio]

It works for me if I uninstall faiss-gpu and then use

!pip install langchain==0.2.5 faiss-cpu==1.8.0 cohere==5.5.8 langchain-community==0.2.5 rank_bm25==0.2.2 sentence-transformers==3.0.1
!pip install llama-cpp-python==0.2.78 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124

[MaartenGr]

@ycymio I think I figured out the issue... and it's a fun one 😅

When I run the pip installs as instructed in the notebook and then immediately afterwards run the rest of the cells, I get this error:

ValueError                                Traceback (most recent call last)
[<ipython-input-5-d6ebd9013bcb>](https://localhost:8080/#) in <cell line: 0>()
      3 dim = embeds.shape[1]
      4 index = faiss.IndexFlatL2(dim)
----> 5 index.add(np.float32(embeds))

1 frames
[/usr/local/lib/python3.11/dist-packages/faiss/class_wrappers.py](https://localhost:8080/#) in replacement_add(self, x)
    228         assert d == self.d
    229         x = np.ascontiguousarray(x, dtype='float32')
--> 230         self.add_c(n, swig_ptr(x))
    231 
    232     def replacement_add_with_ids(self, x, ids):

/usr/local/lib/python3.11/dist-packages/faiss/swigfaiss_avx512.py in swig_ptr(a)
  11448 
  11449 def swig_ptr(a):
> 11450     return _swigfaiss_avx512.swig_ptr(a)
  11451 
  11452 def rev_swig_ptr(*args):

ValueError: input not a numpy array

However, when I run the pip installs as instructed in the notebook and first restart the notebook, then I get no errors!

Perhaps Colab changed this, but I remember Colab giving clear warnings that you have to restart the notebook as other packages might already be imported. Because I didn't see this warning anymore, I figured they "solved" this. This does not seem to be the case anymore...

SOLUTION: The only thing you need to do is restart your notebook after installing the packages.