| --- |
| license: apache-2.0 |
| datasets: |
| - briefai/LongShort-Dataset |
| language: |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - pytorch |
| - falcon |
| - Gen-AI |
| - Finance |
| - KPI Extraction |
| --- |
| # LongShort-Falcon-7B |
|
|
| ### Model Description |
|
|
| LongShort-Falcon-7B is a large language model fine-tuned on earnings call documents to extract financial KPIs from the earnings call documents. It is based on the Falcon-7B Instruct Architecture. |
| - Model creator: [Brief AI](https://huggingface.co/briefai) |
| - Original model: [Falcon-7B-Instruct](https://huggingface.co/tiiuae/falcon-7b-instruct) |
| |
| ### Dataset Description |
| - Data Source: Factiva |
| - Data Description: 28K+ Earnings Call Documents |
| - Data Scope: 1K+ public companies |
| - Fine Tuning Data: Collection of 60K+ samples. |
|
|
| ## Prompt template: LongShort-Falcon-7B |
|
|
| ``` |
| [INST]Given the context, answer the question. |
| |
| ### Question: |
| Extract all the finance-based performance indicators and evaluation metrics. |
| |
| ### Context: |
| {context} |
| |
| ### Answer: |
| [/INST] |
| |
| ``` |
| |
| ## Basics |
| *This section provides information about the model type, version, license, funders, release date, developers, and contact information.* |
| *It is useful for anyone who wants to reference the model.* |
|
|
| |
| **Developed by:** [Brief AI Team](https://huggingface.co/briefai) |
| |
| **Model Type:** Transformer-based Large Language Model |
| |
| **Version:** 1.0.0 |
|
|
| **Languages:** English |
|
|
| **License:** Apache 2.0 |
|
|
| **Release Date Estimate:** Wednesday, 29.November.2023 |
|
|
| **Send Questions to:** vishalparameswaran96@gmail.com |
|
|
| **Cite as:** Brief AI LongShort Language Model |
|
|
| **Funded by:** UChicago Data Science Institute |
|
|
| **Mentored by:** Nick Kadochnikov |
|
|
| ## Technical Specifications |
| *This section includes details about the model objective and architecture, and the compute infrastructure.* |
| *It is useful for people interested in model development.* |
|
|
| Please see [the LongShort training README](https://github.com/brief-ai-uchicago/LongShort-Dataset) for full details on replicating training. |
|
|
| ### Model Architecture and Objective |
|
|
| * Modified from Falcon-7B-Instruct |
|
|
| **Objective:** Financial KPI extraction from earnings call documents. |
| |
| ### Hardware and Software - Compute Infrastructure |
| |
| * 4 NVIDIA L4 GPUs & 48 vCPUs |
|
|
| * Environment: PyTorch (pytorch-2.0 w/ CUDA-11.8; see [Github link](https://github.com/pytorch/pytorch)) |
|
|
| * CPU: GCP G2 Standard 48 (Platform: Intel Cascade Lake) (Accelerator Optimized) |
|
|
| * CPU memory: 192GB RAM |
|
|
| * GPU memory: 30GB per GPU |
|
|
| ## Training |
| *This section provides information about the training.* |
| *It is useful for people who want to learn more about the model inputs and training footprint.* |
|
|
| The following bits and bytes quantization config was used during training: |
|
|
| * quant_method: bitsandbytes |
| * load_in_8bit: False |
| * load_in_4bit: True |
| * llm_int8_threshold: 6.0 |
| * llm_int8_skip_modules: None |
| * llm_int8_enable_fp32_cpu_offload: False |
| * llm_int8_has_fp16_weight: False |
| * bnb_4bit_quant_type: nf4 |
| * bnb_4bit_use_double_quant: True |
| * bnb_4bit_compute_dtype: float16 |
| |
| Framework versions |
| * PEFT 0.4.0 |
| |
| |
| ### Training Data |
| *This section provides a high-level overview of the training data. It is relevant for anyone who wants to know the basics of what the model is learning.* |
| |
| Details for the dataset can be found in [LongShort Dataset](https://github.com/brief-ai-uchicago/LongShort-Dataset) |
| |
| Training data includes: |
| |
| - 5000 Earnings Call Documents |
| |
| ## How to use |
| |
| This model can be easily used and deployed using HuggingFace's ecosystem. This needs `transformers` and `accelerate` installed. The model can be downloaded as follows: |
| |
| [LongShort-Falcon-7B](https://huggingface.co/briefai/LongShort-Falcon-7B) |
| |
| ## Intended Use |
| |
| This model is being created in order to enable public research on large language models (LLMs). LLMs are intended to be used for language generation or as a pre-trained base model that can be further fine-tuned for specific tasks. The use cases below are not exhaustive. |
| |
| ### Direct Use |
| |
| - Text generation |
| |
| - Exploring characteristics of language generated by a language model |
| |
| - Examples: Cloze tests, counterfactuals, generations with reframings |
| |
| ### Downstream Use |
| |
| - Tasks that leverage language models include: Information Extraction, Question Answering, Summarization |
| |
| |
| #### Out-of-scope Uses |
| |
| Using the model in [high-stakes](#high-stakes) settings is out of scope for this model. The model is not designed for [critical decisions](#critical-decisions) nor uses with any material consequences on an individual's livelihood or wellbeing. The model outputs content that appears factual but may not be correct. |
| |
| Out-of-scope Uses Include: |
| |
| - Usage for evaluating or scoring individuals, such as for employment, education, or credit |
| |
| - Applying the model for critical automatic decisions, generating factual content, creating reliable summaries, or generating predictions that must be correct |
| |
| #### Misuse |
| |
| Intentionally using the model for harm, violating [human rights](#human-rights), or other kinds of malicious activities, is a misuse of this model. This includes: |
| |
| - Spam generation |
| |
| - Disinformation and influence operations |
| |
| - Disparagement and defamation |
| |
| - Harassment and abuse |
| |
| - [Deception](#deception) |
| |
| - Unconsented impersonation and imitation |
| |
| - Unconsented surveillance |
| |
| - Generating content without attribution to the model, as specified in the [RAIL License, Use Restrictions](https://huggingface.co/spaces/bigscience/license) |
| |
| ## Intended Users |
| |
| ### Direct Users |
| |
| - General Public |
| |
| - Researchers |
| |
| - Students |
| |
| - Educators |
| |
| - Engineers/developers |
| |
| - Non-commercial entities |
| |
| - Financial Industry |
| |
| # Risks and Limitations |
| *This section identifies foreseeable harms and misunderstandings.* |
| |
| Model may: |
| |
| - Overrepresent some viewpoints and underrepresent others |
| |
| - Contain stereotypes |
| |
| - Contain [personal information](#personal-data-and-information) |
| |
| - Generate: |
| |
| - Hateful, abusive, or violent language |
| |
| - Discriminatory or prejudicial language |
| |
| - Content that may not be appropriate for all settings, including sexual content |
| |
| - Make errors, including producing incorrect information as if it were factual |
| |
| - Generate irrelevant or repetitive outputs |
| |
| - Induce users into attributing human traits to it, such as sentience or consciousness |
| |
| |
| # Evaluation |
| *This section describes the evaluation protocols and provides the results.* |
| |
| Result: LongShort-Falcon-7B gives 45.4% accuracy on a validation set of 10% of the original training dataset. |
| |
| |
| |
| **Train-time Evaluation:** |
| |
| Final checkpoint after 700 epochs: |
| |
| - Training Loss: 1.479 |
| |
| # Recommendations |
| *This section provides information on warnings and potential mitigations.* |
| |
| - Indirect users should be made aware when the content they're working with is created by the LLM. |
| |
| - Users should be aware of [Risks and Limitations](#risks-and-limitations), and include an appropriate age disclaimer or blocking interface as necessary. |
| |
| - Users of the model should provide mechanisms for those affected to provide feedback, such as an email address for comments. |
| |
| # Model Card Authors |
| Vishal Parameshwaran, Garima Sohi, Jose Gerala, Sanchit Narayan Kumar |
| |
| |
| |
