# Zero-Shot Text Classification on a low-end CPU-only machine?

**URL:** <https://discuss.huggingface.co/t/zero-shot-text-classification-on-a-low-end-cpu-only-machine/110450>\
**Category:** Beginners\
**Created:** [October 6, 2024, 1:10pm UTC](https://discuss.huggingface.co/t/zero-shot-text-classification-on-a-low-end-cpu-only-machine/110450 "2024-10-06T13:10:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![new-in-town](https://avatars.discourse-cdn.com/v4/letter/n/dfb087/32.png) [@new-in-town](https://discuss.huggingface.co/u/new-in-town)\
**Post date:** [October 6, 2024, 1:10pm UTC](https://discuss.huggingface.co/t/zero-shot-text-classification-on-a-low-end-cpu-only-machine/110450/1 "2024-10-06T13:10:41Z")

</div>

I want to do zero-shot text classification either this model [1] (711 MB) or something similar:

Want to achieve high throughput in classification requests per second.  
The classification will run on low-end hardware: some Hetzner [2] machine without GPU (Hetzner is great, reliable and cheap they just do not have GPU machines), something like this:  
CCX13: Dedicated vCPU, 2 VCPU, 8 GB RAM  
CX32: Shared vCPU, 4 VCPU, 8 GB RAM

Now there are multiple options for deploying and serving LLMs:  
lmdeploy  
text-generation-inference  
TensorRT-LLM  
vllm  
There are more and more new frameworks for this. I am a bit lost.

Would you suggest the best option for deploying the above-listed model (No-GPU hardware)?

[1] [MoritzLaurer/roberta-large-zeroshot-v2.0-c · Hugging Face](https://huggingface.co/MoritzLaurer/roberta-large-zeroshot-v2.0-c)  
[2] [https://www.hetzner.com/cloud/](https://www.hetzner.com/cloud/)

P.S. I know about this post:

> [@way to make inference Zero Shot pipeline faster?](https://discuss.huggingface.co/t/way-to-make-inference-zero-shot-pipeline-faster/1384):
>
> Hi Can you guys give me tips how to make Zero Shot pipeline inference faster? My current approach right now is reducing the model size/parameter (trying to train “base model” instead of "large model) Is there another approach?

These recomendations - they’re great. But, uh, it’s been four years

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 8, 2024, 10:04am UTC](https://discuss.huggingface.co/t/zero-shot-text-classification-on-a-low-end-cpu-only-machine/110450/2 "2024-10-08T10:04:48Z")

</div>

@joeddav @nielsr Any good updates in the last 4 years on zero-shot text classification, mainly in low-end environments without GPUs?

---

<div class="post-metadata">

**Author:** ![juhoinkinen](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/juhoinkinen/32/32436_2.png) [@juhoinkinen](https://discuss.huggingface.co/u/juhoinkinen)\
**Post date:** [October 8, 2024, 12:21pm UTC](https://discuss.huggingface.co/t/zero-shot-text-classification-on-a-low-end-cpu-only-machine/110450/3 "2024-10-08T12:21:43Z")

</div>

You could check if Yake would be enough for your case, at least its resource demands are negligible compared to newer methods: [GitHub - LIAAD/yake: Single-document unsupervised keyword extraction](https://github.com/LIAAD/yake)

---

<div class="post-metadata">

**Author:** ![new-in-town](https://avatars.discourse-cdn.com/v4/letter/n/dfb087/32.png) [@new-in-town](https://discuss.huggingface.co/u/new-in-town)\
**Post date:** [October 9, 2024, 9:09am UTC](https://discuss.huggingface.co/t/zero-shot-text-classification-on-a-low-end-cpu-only-machine/110450/4 "2024-10-09T09:09:03Z")

</div>

thanks @juhoinkinen !

as far as I understand: YAKE works for paragraph-long or document-long text.  
and this zero-shot classification model in original text: it works well even with few words/sentence. This is what I need: sentence classification.