# AttributeError: 'ChatPromptValue' object has no attribute 'size'

**URL:** <https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895>\
**Category:** 🤗Transformers\
**Created:** [February 5, 2024, 7:49am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895 "2024-02-05T07:49:41Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![DP13](https://avatars.discourse-cdn.com/v4/letter/d/9fc348/32.png) [@DP13](https://discuss.huggingface.co/u/DP13)\
**Post date:** [February 5, 2024, 7:49am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/1 "2024-02-05T07:49:41Z")

</div>

AttributeError Traceback (most recent call last)  
 in \<cell line: 8\>()  
6 chain = setup\_and\_retrieval | prompt | model | output\_parser  
7  
----\> 8 chain.invoke(“What is AR?”)

11 frames  
/usr/local/lib/python3.10/dist-packages/transformers/models/gpt2/modeling\_gpt2.py in forward(self, input\_ids, past\_key\_values, attention\_mask, token\_type\_ids, position\_ids, head\_mask, inputs\_embeds, encoder\_hidden\_states, encoder\_attention\_mask, use\_cache, output\_attentions, output\_hidden\_states, return\_dict)  
774 elif input\_ids is not None:  
775 self.warn\_if\_padding\_and\_no\_attention\_mask(input\_ids, attention\_mask)  
 → 776 input\_shape = input\_ids.size()  
777 input\_ids = input\_ids.view(-1, input\_shape[-1])  
778 batch\_size = input\_ids.shape[0]

AttributeError: ‘ChatPromptValue’ object has no attribute ‘size’

---

<div class="post-metadata">

**Author:** ![CKeibel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ckeibel/32/55416_2.png) [@CKeibel](https://discuss.huggingface.co/u/CKeibel)\
**Post date:** [February 5, 2024, 8:45am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/2 "2024-02-05T08:45:06Z")

</div>

Hello @DP13!  
I’m assuming you are using langchain?  
I never used langchain myself but when I look into the docs [ChatPromptValue](https://api.python.langchain.com/en/latest/prompt_values/langchain_core.prompt_values.ChatPromptValue.html#) it seems, that `ChatPromptValue` does not have an attribute `size`.

Maybe you could share some more code, to get a better understanding why it fails?

---

<div class="post-metadata">

**Author:** ![DP13](https://avatars.discourse-cdn.com/v4/letter/d/9fc348/32.png) [@DP13](https://discuss.huggingface.co/u/DP13)\
**Post date:** [February 6, 2024, 6:55am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/3 "2024-02-06T06:55:56Z")

</div>

Hello @CKeibel  
Yes i am using langchain and issue is my llm is not taking prompt as a perameter llm shows that is only wants string.

---

<div class="post-metadata">

**Author:** ![DP13](https://avatars.discourse-cdn.com/v4/letter/d/9fc348/32.png) [@DP13](https://discuss.huggingface.co/u/DP13)\
**Post date:** [February 6, 2024, 6:59am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/4 "2024-02-06T06:59:04Z")

</div>

!pip install ctransformers

from langchain\_core.runnables import RunnablePassthrough, RunnableParallel  
from langchain\_core.output\_parsers import StrOutputParser  
from langchain import hub  
from langchain\_core.prompts import ChatPromptTemplate

# from ctransformers.chat\_models import ChatPrompt

retriever = vector\_store.as\_retriever(search\_type=“similarity”, num\_docs=1)

# prompt = hub.pull(“rlm/rag-prompt”)

template = “”"Answer the question based only on the following context:  
{context}

Question: {question}  
“”"  
prompt = ChatPromptTemplate.from\_template(template)

# prompt = ChatPrompt(template)

rag\_chain = (  
{“context”: retriever | format\_docs, “question”: RunnablePassthrough()}  
| prompt  
)

result = rag\_chain.invoke(query)

print(result)

Output = messages=[HumanMessage(content=“Answer the question based only on the following context:\nAugmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory.[1] AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects.[2] The overlaid sensory information can be constructive (i.e. additive to the natural environment), or destructive (i.e. masking of the natural environment).[3] This experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment.[3] In this way, augmented reality alters one’s ongoing perception of a real-world environment, whereas virtual reality completely replaces the user’s real-world environment with a simulated one Domain IP Address\nDomain IP Address\n\nARP: ARP stands for Address Resolution Protocol. It is used to convert an IP address to its corresponding physical address(i.e., MAC Address). ARP is used by the Data Link Layer to identify the MAC address of the Receiver’s machine. \n\nRARP: RARP stands for Reverse Address Resolution Protocol. As the name suggests, it provides the IP address of the device given a physical address as input. But RARP has become obsolete since the time DHCP has come into the picture.\n\n\nUnlock the Power of Placement Preparation!\nFeeling lost in OS, DBMS, CN, SQL, and DSA chaos? Our Complete Interview Preparation Course is the ultimate guide to conquer placements. Trusted by over 100,000+ geeks, this course is your roadmap to interview triumph.\nReady to dive in? Explore our Free Demo Content and join our Complete Interview Preparation course. RARP: RARP stands for Reverse Address Resolution Protocol. As the name suggests, it provides the IP address of the device given a physical address as input. But RARP has become obsolete since the time DHCP has come into the picture.\n\n\nUnlock the Power of Placement Preparation!\nFeeling lost in OS, DBMS, CN, SQL, and DSA chaos? Our Complete Interview Preparation Course is the ultimate guide to conquer placements. Trusted by over 100,000+ geeks, this course is your roadmap to interview triumph.\nReady to dive in? Explore our Free Demo Content and join our Complete Interview Preparation course.\n\n\nThree 90 Challenge ending on 5th Feb! Last chance to get 90% refund by completing 90% course in 90 days. Explore offer now.\nLast Updated : 17 May, 2023\n\n1.23k\n\nNext\nTypes of Network Topology\nShare your thoughts in the comments RARP: RARP stands for Reverse Address Resolution Protocol. As the name suggests, it provides the IP address of the device given a physical address as input. But RARP has become obsolete since the time DHCP has come into the picture.\n\n\nUnlock the Power of Placement Preparation!\nFeeling lost in OS, DBMS, CN, SQL, and DSA chaos? Our Complete Interview Preparation Course is the ultimate guide to conquer placements. Trusted by over 100,000+ geeks, this course is your roadmap to interview triumph.\nReady to dive in? Explore our Free Demo Content and join our Complete Interview Preparation course.\n\n\nThree 90 Challenge ending on 5th Feb! Last chance to get 90% refund by completing 90% course in 90 days. Explore offer now.\nLast Updated : 17 May, 2023\n\n1.23k\n\nNext\nTypes of Network Topology\nShare your thoughts in the comments\n\nAdd Your Comment\nSimilar Reads\nAdvantages and Disadvantages of Computer Networking\nOSI Model Full Form in Computer Networking\nMAN Full Form in Computer Networking\nTCP/IP in Computer Networking\nImportance of Computer Networking\nDifference Between BOOTP and RARP in Computer Networking\nDifference Between MSS and MTU in Computer Networking\nBasics of Computer and its Operations\nIntroduction to basic Networking Terminology\nSoftware defined Networking(SDN)\n[https://media.geeksforgeeks.org/auth/avatar.png\nGeeksforGeeks\nArticle](https://media.geeksforgeeks.org/auth/avatar.png%5CnGeeksforGeeks%5CnArticle) Tags :\nComputer Networks \nMisc\nPractice Tags :\nMisc\nAdditional Information\nTrending in News\nView More\n50 Most Popular Female Characters in Naruto\nHow to Earn Money Online by Creating eBooks with ChatGPT\nTop 10 AI Tools for Disinformation Detection in 2024\nBest ChatGPT Plugin for Data Analysis\n30 OOPs Interview Questions and Answers (2024)\n\nQuestion: What is AR?\n”)]

print(llm(prompt))

Output

AttributeError Traceback (most recent call last)  
 in \<cell line: 1\>()  
----\> 1 print(llm(prompt))

2 frames  
/usr/local/lib/python3.10/dist-packages/ctransformers/llm.py in tokenize(self, text, add\_bos\_token)  
334 add\_bos\_token = self.model\_type == “llama”  
335 tokens = (c\_int \* (len(text) + 1))()  
 → 336 n\_tokens = self.ctransformers\_llm\_tokenize(text.encode(), add\_bos\_token, tokens)  
337 return tokens[:n\_tokens]  
338

AttributeError: ‘ChatPromptTemplate’ object has no attribute ‘encode’

---

<div class="post-metadata">

**Author:** ![DP13](https://avatars.discourse-cdn.com/v4/letter/d/9fc348/32.png) [@DP13](https://discuss.huggingface.co/u/DP13)\
**Post date:** [February 6, 2024, 6:59am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/5 "2024-02-06T06:59:35Z")

</div>

Thank you for your help and advance thanks for the solution

---

<div class="post-metadata">

**Author:** ![CKeibel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ckeibel/32/55416_2.png) [@CKeibel](https://discuss.huggingface.co/u/CKeibel)\
**Post date:** [February 6, 2024, 8:13am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/6 "2024-02-06T08:13:27Z")

</div>

After researching a bit here are my first attempts:

When you are creating a `ChatPromptTemplate` from a template string like the following:

```auto
template = """Answer the question based only on the following context:
{context}

Question: {question}
"""

template = ChatPromptTemplate.from_template(template)

```

you also need to pass in values for your prompt.  
The following should be noted, `template.format_messages(...)` seems to return a list but `template.format(...)` returns a string.

So:

```auto
messages = template.format(context="The sky is blue.", question="The sky is blue.")

```

As the result message should be a string which looks like:

```auto
'Human: Answer the question based only on the following context:\nThe sky is blue.\n\nQuestion: What color is the sky?\n'

```

Then you should be able to pass `message` into the llm.

If you want to get a similar behaviour with a chain, you need to pass the the llm to the chain.

Here are some code snippets from the [langchain cookbook](https://python.langchain.com/docs/expression_language/cookbook/retrieval) .

Initialize **VectorStore** and **PromptTemplate** :

```auto
retriever = vectorstore.as_retriever()

template = """Answer the question based only on the following context:
{context}

Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

```

Initialize your **LLM**

```auto
# your llm
llm = ...

```

create a **chain**

```auto
chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt # your prompt template
    | llm # your llm
    | StrOutputParser()
)

```

and at the end you can **invoke** the cain:

```auto
chain.invoke("where did harrison work?")

```

I hope that it helps you. If you have any further questions, please let me know and I’ll take a closer look.

---

<div class="post-metadata">

**Author:** ![DP13](https://avatars.discourse-cdn.com/v4/letter/d/9fc348/32.png) [@DP13](https://discuss.huggingface.co/u/DP13)\
**Post date:** [February 6, 2024, 10:18am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/7 "2024-02-06T10:18:07Z")

</div>

Your last suggested chain part was my first implementation and i got the attribute error and i created this thread.

you can find 3rd line

chain = setup\_and\_retrieval | prompt | model | output\_parser

please suggest me if any other options are available @CKeibel

---

<div class="post-metadata">

**Author:** ![CKeibel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ckeibel/32/55416_2.png) [@CKeibel](https://discuss.huggingface.co/u/CKeibel)\
**Post date:** [February 7, 2024, 7:09am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/8 "2024-02-07T07:09:04Z")

</div>

It looks as if it is not possible to use a model from `CTransformers` with the ` ChatPromptTemplate` and a RAG chain.

The only thing I could find on the internet is using it with the `PromptTemplate` from `langchain.prompts`.

```auto
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

template = """Question: {question}

Answer:"""

prompt = PromptTemplate(template=template, input_variables=["question"])

llm_chain = LLMChain(prompt=prompt, llm=llm)

response = llm_chain.run("What is AI?")

```

[Reference](https://python.langchain.com/docs/integrations/llms/ctransformers)

If you want to use a `CTransformer` in a RAG setup, you could use faiss index or chromadb as vector store and an sbert model for document/ text embeddings. Then you would search with the sbert model in your vector store and retrieve documents which you pass on to you llm.

**init embedding model**

```auto
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer, Pipeline

def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Inference pipeline for embedding model
class EmbeddingPipeline(Pipeline):
    def _sanitize_parameters(self, **kwargs):
        preprocess_kwargs = {}
        return preprocess_kwargs, {}, {}

    def preprocess(self, text):
        encoded_text = self.tokenizer(text, padding=True, truncation=True, return_tensors='pt').to(device)
        return encoded_text

    def _forward(self, model_inputs):
        outputs = self.model(**model_inputs)
        return {"outputs": outputs, "attention_mask": model_inputs["attention_mask"]}

    def postprocess(self, model_outputs):
        sentence_embeddings = mean_pooling(model_outputs["outputs"], model_outputs['attention_mask'])
        sentence_embeddings = F.normalize(sentence_embeddings, p=2, dim=1)
        return sentence_embeddings[0].numpy()

model_id = "sentence-transformers/all-MiniLM-L6-v2"
model = AutoModel.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
encoder = EmbeddingPipeline(model=model, tokenizer=tokenizer, device=device)

```

**setup chromadb index**

```auto
import chromadb

collection = chroma_client.create_collection(name="squad_v2", metadata={"hnsw:space": "ip"})

# embed documents 
embedding_vector = encoder(document).tolist()
collection.add(
    embeddings=[embedding_vector],
    documents=[document],
    ids="1" # string id
)

```

**query the vetor store**

```auto
question = "Some question..."

embedded_question = encoder(question ).tolist()
    result = collection.query(
    query_embeddings=question,
    n_results=5 # getting 5 best results
)

contexts = "\n".join(result["documents"])

```

**use llm to generate answer**

```auto
template = """Answer the question based only on the following context:
{context}

Question: {question}
"""

prompt = PromptTemplate(template=template, input_variables=["question"])

llm_chain = LLMChain(prompt=prompt, llm=llm)

response = llm_chain.run(contexts , "What is AI?")

```

If the `llm_chain.run(contexts , "What is AI?")` does not take 2 arguments you could use a simple function to create you prompts.

```auto
def get_prompt(question, contexts):
    return f"""Answer the question based only on the following context:
{context}

Question: {question}"""

llm(get_prompt(context, question))

```

Here are some notebooks I implemented when I learned about RAG. (definitely not best practices 😅)  
[hybrid search](https://github.com/CKeibel/hybrid-search-rag/blob/4f92ee6098b1009da14dae4628887e32ba60cd5f/encoder.ipynb) - just embedding model tests

[Notebooks with different tests](https://github.com/CKeibel/retrieval-augmented-generation)

---

<div class="post-metadata">

**Author:** ![DP13](https://avatars.discourse-cdn.com/v4/letter/d/9fc348/32.png) [@DP13](https://discuss.huggingface.co/u/DP13)\
**Post date:** [February 9, 2024, 4:50am UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/9 "2024-02-09T04:50:38Z")

</div>

Thank you for your help @CKeibel and i learned a lot from you practises, as a beginner it helps me lot to learn and my query just solved from 6/8 reply without llm chain.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [February 9, 2024, 4:51pm UTC](https://discuss.huggingface.co/t/attributeerror-chatpromptvalue-object-has-no-attribute-size/71895/10 "2024-02-09T16:51:35Z")

</div>

Closed on February 9, 2024, 4:51pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.