# Converting web scrapped data to a semistructured json payload

**URL:** <https://discuss.huggingface.co/t/converting-web-scrapped-data-to-a-semistructured-json-payload/51522>\
**Category:** Models\
**Created:** [August 21, 2023, 12:55pm UTC](https://discuss.huggingface.co/t/converting-web-scrapped-data-to-a-semistructured-json-payload/51522 "2023-08-21T12:55:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![DUnisala](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@DUnisala](https://discuss.huggingface.co/u/DUnisala)\
**Post date:** [August 21, 2023, 12:55pm UTC](https://discuss.huggingface.co/t/converting-web-scrapped-data-to-a-semistructured-json-payload/51522/1 "2023-08-21T12:55:59Z")

</div>

I am trying to scrap multiple universities sites for scholarships , and I want to feed it to an llm , hoping that it converts it into a consistent json format , for eg i scrap a site and feed it to llm it generates a payload such as  
`{scholarship name: "Presidential", gpa: "3.0", sat : "1550"}` by scouring through the scrapped webpage.  
it should work for at least most of the sites.

How would I go about doing this what models work best for this , I tried open ai’s gpt but it is limited in size of tokens it allows , when fine tuning done. plus when few shot learning done the example tokens alone is way too many tokens.

---

<div class="post-metadata">

**Author:** ![0sparsh2](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/0sparsh2/32/25784_2.png) [@0sparsh2](https://discuss.huggingface.co/u/0sparsh2)\
**Post date:** [April 13, 2024, 4:23am UTC](https://discuss.huggingface.co/t/converting-web-scrapped-data-to-a-semistructured-json-payload/51522/2 "2024-04-13T04:23:55Z")

</div>

Hi,  
I am working on a similar problem. Did you happen to get the solution for this?