# Using tie\_weights() always? (intend to separate source/target embedding)

**URL:** <https://discuss.huggingface.co/t/using-tie-weights-always-intend-to-separate-source-target-embedding/12002>\
**Category:** 🤗Transformers\
**Created:** [November 17, 2021, 11:46pm UTC](https://discuss.huggingface.co/t/using-tie-weights-always-intend-to-separate-source-target-embedding/12002 "2021-11-17T23:46:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jucho2725](https://avatars.discourse-cdn.com/v4/letter/j/839c29/32.png) [@jucho2725](https://discuss.huggingface.co/u/jucho2725)\
**Post date:** [November 17, 2021, 11:46pm UTC](https://discuss.huggingface.co/t/using-tie-weights-always-intend-to-separate-source-target-embedding/12002/1 "2021-11-17T23:46:46Z")

</div>

Hello  
(sorry, this is a duplicate of the post in beginner category. I don’t know where to post it)

I’m developing a machine translation model(Seq2seq), which have different source and target tokenizer. (As a beginner, I even do not know if this is the correct way to solve the machine translation problem.)

The problem happens when I want to divide shared(input, output) embedding into source/target embedding (I’m using T5ConditionalGeneration). After that, there is a function self.init\_weights() and here they operate self.tie\_weights() method.

As far as I know, the self.tie\_weights() make output embeddings the same as input embeddings. But in my case, then target embedding will be changed and cause indexing error.

My question is:

1. Should I call self.tie\_weights() in this case?
2. Is there any better approach that I can attempt?

Here is the related post below:

> [@What is the \`tie\_word\_embeddings\` option exactly doing?](https://discuss.huggingface.co/t/what-is-the-tie-word-embeddings-option-exactly-doing/8483):
>
> Hi, for some models there is this tie\_word\_embeddings parameter. I think it is for the text 2 text models. Can someone please explain what exactly this parameter is doing? Many thanks Philip

Thanks in advance for any answers!