# Controlling bos, eos, etc in api-inference

**URL:** <https://discuss.huggingface.co/t/controlling-bos-eos-etc-in-api-inference/5701>\
**Category:** Beginners\
**Created:** [April 22, 2021, 7:59pm UTC](https://discuss.huggingface.co/t/controlling-bos-eos-etc-in-api-inference/5701 "2021-04-22T19:59:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![cristivlad](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cristivlad/32/1542_2.png) [@cristivlad](https://discuss.huggingface.co/u/cristivlad)\
**Post date:** [April 22, 2021, 7:59pm UTC](https://discuss.huggingface.co/t/controlling-bos-eos-etc-in-api-inference/5701/1 "2021-04-22T19:59:26Z")

</div>

Is there a way to control the beginning of sentence, end of sentence tokens through the inference api? I could not find it in the documentation.

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [May 4, 2021, 3:18pm UTC](https://discuss.huggingface.co/t/controlling-bos-eos-etc-in-api-inference/5701/2 "2021-05-04T15:18:19Z")

</div>

Hi @cristivlad ,

Currently there is no way to override those within the API.  
We are adding and `end_sequence` parameter to enable stopping the generation when using prompt-like generation (for GPT-Neo for instance).

For BOS, what did you want to do with it ?

Cheers,  
Nicolas

---

<div class="post-metadata">

**Author:** ![louis030195](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/louis030195/32/3591_2.png) [@louis030195](https://discuss.huggingface.co/u/louis030195)\
**Post date:** [January 12, 2022, 8:33pm UTC](https://discuss.huggingface.co/t/controlling-bos-eos-etc-in-api-inference/5701/3 "2022-01-12T20:33:13Z")

</div>

I tried **end\_sequence** and indeed works, I would suggest updating the [doc](https://api-inference.huggingface.co/docs/python/html/detailed_parameters.html#text-generation-task), thanks in any case! 🙂

```python
    data = json.dumps(
        {
            "inputs": "1+1=2\n2+2=",
            "parameters": {
                "max_length": 50,
                "num_return_sequences": 1,
                "return_text": False,
                "return_full_text": False,
                "do_sample": True,
                "top_k": 50,
                "top_p": 0.95,
                "end_sequence": "\n"
            },
            "options": {
                "wait_for_model": True,
            },
        }
    )

```