Text Classification
Safetensors
GLiClass
text classification
nli
sentiment analysis

GLiClass base v3: supported scope for compound NLI and reference-runtime compatibility?

#3
by volcomfever - opened

We are evaluating knowledgator/gliclass-base-v3.0 as an offline classifier for
short player-specific recommendations. Before running more experiments, we want
to distinguish an unsuitable task formulation from an implementation issue.
All examples below are authored synthetic text with fictional names, not podcast
transcripts or user data. These are saved results, not a new reproduction.

Exact Setup

  • Model revision: 77a70e6cd52e602ed18184ef37d18bdd3741e3d5.
  • GLiClass source: 68132def761c2dccc20d1a71c04abdc5db35463f.
  • Python 3.13.3; Torch 2.13.0; Transformers 5.13.0; tokenizers 0.22.2;
    safetensors 0.8.0; huggingface-hub 1.23.0.
  • macOS ARM host, CPU float32, eval/inference mode, 2 Torch threads, 1 interop
    thread, seed 0, batch size 4. No quantization or fine-tuning.
  • One hypothesis per sequence: <<LABEL>>{hypothesis}<<SEP>>{source}.
    We verified rendering against the pinned upstream prepare_input method.
  • Fast tokenizer, no truncation, all sequences under the 512-token limit.
    model(**inputs, max_num_classes=1).logits, followed by sigmoid, not softmax.
    Checkpoint loading reported no missing/unexpected/mismatched keys or errors.

Example And Actual Scores

Source:

For this week's lineup, Avery Stone is a must start. I also recommend starting
Blake Reed this week. Casey Lane is a talented player and reminds me of Stone,
but I am not making a lineup recommendation for Lane.

Our start hypothesis is constructed as:

The speaker's final position is to start {player} in a weekly fantasy lineup,
not merely as a temporary streamer. {scope} It is not merely a comparison,
quotation, advertisement or rejected suggestion.

Plain scope:

This is a definite statement within the stated timeframe, without an essential
if-condition, price or league-format restriction, or uncertain availability.

Qualified scope:

This statement has an essential if-condition, price or league-format restriction,
or uncertain availability; removing that qualification would misrepresent the
discussion.

The sit action substitutes sit {player} out of a weekly fantasy lineup.

Player/action Plain sigmoid Qualified sigmoid
Avery Stone / start 0.937131 0.951355
Blake Reed / start 0.585968 0.797746
Blake Reed / sit 0.442702 0.606968

We selected the higher state only if its score was >= 0.5 and the pair's absolute
difference was >= 0.1. That withholds Avery's start and co-emits qualified start
and sit for Blake. This selector is our uncalibrated design, not your recipe.
We recognize the hypotheses combine multiple semantic demands and are not
established as comparable alternatives.

Across all three safely completed synthetic cases, qualified scores are higher
in 103/120 pairs; our selector emits 42 labels, all qualified. The collapsible diagnostic below
includes all 240 scores, including abstentions and cases where final corrections
are distinguished by the raw action scores. This is not a comprehensive benchmark
or a claim that the model cannot perform ordinary classification. The larger
attempt stopped on a separate process-supervision failure; incomplete/quarantined
outputs are excluded. No causal model/runtime defect has been established.

Two Questions

  1. Is comparing compound hypotheses about owner, action, final stance and
    essential conditions within this model's intended NLI contract? If not, is
    there a validated task formulation/reference example for that distinction,
    or should we regard it as outside this checkpoint's supported capability?
  2. For this exact DeBERTa-based checkpoint and GLiClass commit, are the listed
    dependency versions supported? Is there a known compatibility correction or
    a pinned reference environment/output we can compare against?

We saw issue #44 about
Transformers v5, but it concerns gliclass-multilang-ultra and missing/tied
embedding weights. Our model loads strictly, so we are not assuming the same
cause
. Rendering/loading checks also do not prove numerical parity.

Supporting material is included below in collapsible sections, containing only
synthetic examples, scores and method.
A concrete reference or compatibility explanation would help us decide whether
another controlled test is justified, rather than doing a threshold/prompt sweep.

Exact inference method

Exact Inference Method

This is an explanation of the consumed run, not a runnable authorized retry.
The model and tokenizer were already local, pinned and loaded without remote code.
No install or acquisition is required to inspect this document or packet.

Model revision: 77a70e6cd52e602ed18184ef37d18bdd3741e3d5.
GLiClass source revision: 68132def761c2dccc20d1a71c04abdc5db35463f.
Python 3.13.3; dependency versions appear in the attachment.

The worker set Torch threads to 2, interop threads to 1 and seed to 0. It used a
fast AutoTokenizer with local_files_only=True, trust_remote_code=False, and
loaded GLiClassModel with local_files_only=True, use_safetensors=True,
output_loading_info=True, dtype=torch.float32. Missing, unexpected, mismatched
keys and error messages were all empty. Parameters were checked for CPU/float32.
Config was uni-encoder, prompt-first, DeBERTa-v2, max position embeddings 512.

For each case independently, queries were ordered by player, action, then
plain/qualified. Each sequence had one hypothesis, not 30 labels together.
The following is the computation after loading; surrounding process/resource
supervision is intentionally omitted, not waived for any future local run:

# `case` is one entry in synthetic-diagnostic.json; model/tokenizer already loaded.
model.to("cpu").eval()
texts = ["<<LABEL>>" + q["hypothesis"] + "<<SEP>>" + case["source"]
         for q in case["queries"]]
ids = tokenizer(texts, truncation=False, padding=False,
                add_special_tokens=True)["input_ids"]
assert max(map(len, ids)) <= 512
scores = []
with torch.inference_mode():
    for offset in range(0, len(texts), 4):
        inputs = tokenizer(texts[offset:offset + 4], truncation=False,
                           padding=True, return_tensors="pt")
        logits = model(**inputs, max_num_classes=1).logits
        assert tuple(logits.shape) == (min(4, len(texts) - offset), 1)
        scores.extend(torch.sigmoid(logits).flatten().tolist())

The retained worker verified the rendered strings against upstream
UniEncoderZeroShotClassificationPipeline.prepare_input, the class/text marker
counts, token limits and token identity between preflight and inference. We did
not verify numerical parity with the publisher's original dependency environment.

For each action/player pair we selected the larger of its plain/qualified scores
only when that score was at least 0.5 and their absolute difference was at least
0.1. Ties preferred plain but abstained on the margin. There was no cross-action
conflict resolver. This is explicitly our uncalibrated selector; it may be an
unsuitable use of the model. It is not presented as the publisher's algorithm.

All three synthetic cases and 240 saved scores (compact JSON)

This lossless representation deduplicates hypotheses: replace {player} in the indexed template with the corresponding player name. Each query tuple is [playerID, hypothesisTemplateIndex, score, selected]; query order and every numeric score are unchanged. Round-trip reconstruction was checked against the original packet.

{
  "cases": [
    {
      "id": "case-02",
      "players": [
        {
          "id": "p00",
          "name": "Avery Stone"
        },
        {
          "id": "p01",
          "name": "Blake Reed"
        },
        {
          "id": "p02",
          "name": "Casey Lane"
        }
      ],
      "queries": [
        [
          "p00",
          0,
          0.9371311068534851,
          false
        ],
        [
          "p00",
          1,
          0.9513549208641052,
          false
        ],
        [
          "p00",
          2,
          0.026997186243534088,
          false
        ],
        [
          "p00",
          3,
          0.1319342851638794,
          false
        ],
        [
          "p00",
          4,
          0.7452974915504456,
          false
        ],
        [
          "p00",
          5,
          0.754568338394165,
          false
        ],
        [
          "p00",
          6,
          0.33289778232574463,
          false
        ],
        [
          "p00",
          7,
          0.5322201251983643,
          true
        ],
        [
          "p00",
          8,
          0.06787434965372086,
          false
        ],
        [
          "p00",
          9,
          0.2012796550989151,
          false
        ],
        [
          "p00",
          10,
          0.20218506455421448,
          false
        ],
        [
          "p00",
          11,
          0.5288707613945007,
          true
        ],
        [
          "p00",
          12,
          0.36686405539512634,
          false
        ],
        [
          "p00",
          13,
          0.41082051396369934,
          false
        ],
        [
          "p00",
          14,
          0.014759764075279236,
          false
        ],
        [
          "p00",
          15,
          0.021872583776712418,
          false
        ],
        [
          "p00",
          16,
          0.4679846465587616,
          false
        ],
        [
          "p00",
          17,
          0.5681167244911194,
          true
        ],
        [
          "p00",
          18,
          0.598477840423584,
          false
        ],
        [
          "p00",
          19,
          0.6581751704216003,
          false
        ],
        [
          "p00",
          20,
          0.24287091195583344,
          false
        ],
        [
          "p00",
          21,
          0.515523374080658,
          true
        ],
        [
          "p00",
          22,
          0.411721408367157,
          false
        ],
        [
          "p00",
          23,
          0.47954708337783813,
          false
        ],
        [
          "p00",
          24,
          0.34901872277259827,
          false
        ],
        [
          "p00",
          25,
          0.3907780051231384,
          false
        ],
        [
          "p00",
          26,
          0.3585173487663269,
          false
        ],
        [
          "p00",
          27,
          0.40946027636528015,
          false
        ],
        [
          "p00",
          28,
          0.7079123258590698,
          false
        ],
        [
          "p00",
          29,
          0.8130996227264404,
          true
        ],
        [
          "p01",
          0,
          0.5859684348106384,
          false
        ],
        [
          "p01",
          1,
          0.797745943069458,
          true
        ],
        [
          "p01",
          2,
          0.4427020847797394,
          false
        ],
        [
          "p01",
          3,
          0.6069682836532593,
          true
        ],
        [
          "p01",
          4,
          0.7348505258560181,
          false
        ],
        [
          "p01",
          5,
          0.8567580580711365,
          true
        ],
        [
          "p01",
          6,
          0.19473421573638916,
          false
        ],
        [
          "p01",
          7,
          0.5618863105773926,
          true
        ],
        [
          "p01",
          8,
          0.4486672580242157,
          false
        ],
        [
          "p01",
          9,
          0.6636315584182739,
          true
        ],
        [
          "p01",
          10,
          0.4410376250743866,
          false
        ],
        [
          "p01",
          11,
          0.7188965082168579,
          true
        ],
        [
          "p01",
          12,
          0.18597297370433807,
          false
        ],
        [
          "p01",
          13,
          0.31851840019226074,
          false
        ],
        [
          "p01",
          14,
          0.03813454136252403,
          false
        ],
        [
          "p01",
          15,
          0.04563823342323303,
          false
        ],
        [
          "p01",
          16,
          0.3101642429828644,
          false
        ],
        [
          "p01",
          17,
          0.5373814105987549,
          true
        ],
        [
          "p01",
          18,
          0.3501313328742981,
          false
        ],
        [
          "p01",
          19,
          0.5292899012565613,
          true
        ],
        [
          "p01",
          20,
          0.3481258153915405,
          false
        ],
        [
          "p01",
          21,
          0.5831665396690369,
          true
        ],
        [
          "p01",
          22,
          0.34154731035232544,
          false
        ],
        [
          "p01",
          23,
          0.4162229001522064,
          false
        ],
        [
          "p01",
          24,
          0.3422969877719879,
          false
        ],
        [
          "p01",
          25,
          0.41127756237983704,
          false
        ],
        [
          "p01",
          26,
          0.449821412563324,
          false
        ],
        [
          "p01",
          27,
          0.43580952286720276,
          false
        ],
        [
          "p01",
          28,
          0.6304150819778442,
          false
        ],
        [
          "p01",
          29,
          0.7555913925170898,
          true
        ],
        [
          "p02",
          0,
          0.0040450068190693855,
          false
        ],
        [
          "p02",
          1,
          0.027819799259305,
          false
        ],
        [
          "p02",
          2,
          0.8575088977813721,
          false
        ],
        [
          "p02",
          3,
          0.9079477190971375,
          false
        ],
        [
          "p02",
          4,
          0.035884302109479904,
          false
        ],
        [
          "p02",
          5,
          0.33131253719329834,
          false
        ],
        [
          "p02",
          6,
          0.005341208539903164,
          false
        ],
        [
          "p02",
          7,
          0.041777439415454865,
          false
        ],
        [
          "p02",
          8,
          0.30534476041793823,
          false
        ],
        [
          "p02",
          9,
          0.5698426961898804,
          true
        ],
        [
          "p02",
          10,
          0.6790277361869812,
          false
        ],
        [
          "p02",
          11,
          0.8734519481658936,
          true
        ],
        [
          "p02",
          12,
          0.04346684738993645,
          false
        ],
        [
          "p02",
          13,
          0.10318804532289505,
          false
        ],
        [
          "p02",
          14,
          0.09682269394397736,
          false
        ],
        [
          "p02",
          15,
          0.130489319562912,
          false
        ],
        [
          "p02",
          16,
          0.004841203801333904,
          false
        ],
        [
          "p02",
          17,
          0.011175490915775299,
          false
        ],
        [
          "p02",
          18,
          0.0030298205092549324,
          false
        ],
        [
          "p02",
          19,
          0.007143597584217787,
          false
        ],
        [
          "p02",
          20,
          0.6992344260215759,
          false
        ],
        [
          "p02",
          21,
          0.867495596408844,
          true
        ],
        [
          "p02",
          22,
          0.3713495433330536,
          false
        ],
        [
          "p02",
          23,
          0.48088958859443665,
          false
        ],
        [
          "p02",
          24,
          0.29117560386657715,
          false
        ],
        [
          "p02",
          25,
          0.4190319776535034,
          false
        ],
        [
          "p02",
          26,
          0.44230490922927856,
          false
        ],
        [
          "p02",
          27,
          0.4424695372581482,
          false
        ],
        [
          "p02",
          28,
          0.7256657481193542,
          false
        ],
        [
          "p02",
          29,
          0.8446155190467834,
          true
        ]
      ],
      "source": "For this week's lineup, Avery Stone is a must start. I also recommend starting Blake Reed this week. Casey Lane is a talented player and reminds me of Stone, but I am not making a lineup recommendation for Lane."
    },
    {
      "id": "case-04",
      "players": [
        {
          "id": "p00",
          "name": "Avery Stone"
        },
        {
          "id": "p01",
          "name": "Blake Reed"
        },
        {
          "id": "p02",
          "name": "Casey Lane"
        }
      ],
      "queries": [
        [
          "p00",
          0,
          0.5879987478256226,
          false
        ],
        [
          "p00",
          1,
          0.5405396819114685,
          false
        ],
        [
          "p00",
          2,
          0.7822626233100891,
          false
        ],
        [
          "p00",
          3,
          0.9374974370002747,
          true
        ],
        [
          "p00",
          4,
          0.8911152482032776,
          false
        ],
        [
          "p00",
          5,
          0.9763442277908325,
          false
        ],
        [
          "p00",
          6,
          0.5256531834602356,
          false
        ],
        [
          "p00",
          7,
          0.7114448547363281,
          true
        ],
        [
          "p00",
          8,
          0.8107963800430298,
          false
        ],
        [
          "p00",
          9,
          0.9425542950630188,
          true
        ],
        [
          "p00",
          10,
          0.48064538836479187,
          false
        ],
        [
          "p00",
          11,
          0.7122218012809753,
          true
        ],
        [
          "p00",
          12,
          0.29038116335868835,
          false
        ],
        [
          "p00",
          13,
          0.24583442509174347,
          false
        ],
        [
          "p00",
          14,
          0.10184080898761749,
          false
        ],
        [
          "p00",
          15,
          0.0807473435997963,
          false
        ],
        [
          "p00",
          16,
          0.5315402746200562,
          false
        ],
        [
          "p00",
          17,
          0.7885897755622864,
          true
        ],
        [
          "p00",
          18,
          0.27863165736198425,
          false
        ],
        [
          "p00",
          19,
          0.3130609691143036,
          false
        ],
        [
          "p00",
          20,
          0.47327306866645813,
          false
        ],
        [
          "p00",
          21,
          0.7654085159301758,
          true
        ],
        [
          "p00",
          22,
          0.4690287411212921,
          false
        ],
        [
          "p00",
          23,
          0.618183970451355,
          true
        ],
        [
          "p00",
          24,
          0.3619967997074127,
          false
        ],
        [
          "p00",
          25,
          0.4421236217021942,
          false
        ],
        [
          "p00",
          26,
          0.6844139099121094,
          false
        ],
        [
          "p00",
          27,
          0.6897464394569397,
          false
        ],
        [
          "p00",
          28,
          0.9220777750015259,
          false
        ],
        [
          "p00",
          29,
          0.9763837456703186,
          false
        ],
        [
          "p01",
          0,
          0.025320211425423622,
          false
        ],
        [
          "p01",
          1,
          0.031610313802957535,
          false
        ],
        [
          "p01",
          2,
          0.885113000869751,
          false
        ],
        [
          "p01",
          3,
          0.9641008973121643,
          false
        ],
        [
          "p01",
          4,
          0.11167987436056137,
          false
        ],
        [
          "p01",
          5,
          0.5260342359542847,
          true
        ],
        [
          "p01",
          6,
          0.0697009339928627,
          false
        ],
        [
          "p01",
          7,
          0.2553972899913788,
          false
        ],
        [
          "p01",
          8,
          0.45742055773735046,
          false
        ],
        [
          "p01",
          9,
          0.7553350925445557,
          true
        ],
        [
          "p01",
          10,
          0.418113648891449,
          false
        ],
        [
          "p01",
          11,
          0.6381796002388,
          true
        ],
        [
          "p01",
          12,
          0.2635710537433624,
          false
        ],
        [
          "p01",
          13,
          0.21470573544502258,
          false
        ],
        [
          "p01",
          14,
          0.137454554438591,
          false
        ],
        [
          "p01",
          15,
          0.07004953920841217,
          false
        ],
        [
          "p01",
          16,
          0.2981662154197693,
          false
        ],
        [
          "p01",
          17,
          0.6631234884262085,
          true
        ],
        [
          "p01",
          18,
          0.2335987091064453,
          false
        ],
        [
          "p01",
          19,
          0.4002855718135834,
          false
        ],
        [
          "p01",
          20,
          0.2990517318248749,
          false
        ],
        [
          "p01",
          21,
          0.597548246383667,
          true
        ],
        [
          "p01",
          22,
          0.36951932311058044,
          false
        ],
        [
          "p01",
          23,
          0.5637538433074951,
          true
        ],
        [
          "p01",
          24,
          0.4061785936355591,
          false
        ],
        [
          "p01",
          25,
          0.4137997031211853,
          false
        ],
        [
          "p01",
          26,
          0.594001829624176,
          false
        ],
        [
          "p01",
          27,
          0.5651403665542603,
          false
        ],
        [
          "p01",
          28,
          0.960507333278656,
          false
        ],
        [
          "p01",
          29,
          0.9891722202301025,
          false
        ],
        [
          "p02",
          0,
          0.003657284425571561,
          false
        ],
        [
          "p02",
          1,
          0.006737728603184223,
          false
        ],
        [
          "p02",
          2,
          0.8134366869926453,
          false
        ],
        [
          "p02",
          3,
          0.9248197674751282,
          true
        ],
        [
          "p02",
          4,
          0.029528329148888588,
          false
        ],
        [
          "p02",
          5,
          0.2794571816921234,
          false
        ],
        [
          "p02",
          6,
          0.005166545044630766,
          false
        ],
        [
          "p02",
          7,
          0.014667526818811893,
          false
        ],
        [
          "p02",
          8,
          0.24777822196483612,
          false
        ],
        [
          "p02",
          9,
          0.5833617448806763,
          true
        ],
        [
          "p02",
          10,
          0.83893883228302,
          false
        ],
        [
          "p02",
          11,
          0.9111390113830566,
          false
        ],
        [
          "p02",
          12,
          0.03696395829319954,
          false
        ],
        [
          "p02",
          13,
          0.040329139679670334,
          false
        ],
        [
          "p02",
          14,
          0.13171125948429108,
          false
        ],
        [
          "p02",
          15,
          0.08744460344314575,
          false
        ],
        [
          "p02",
          16,
          0.007588236127048731,
          false
        ],
        [
          "p02",
          17,
          0.012706056237220764,
          false
        ],
        [
          "p02",
          18,
          0.007444404065608978,
          false
        ],
        [
          "p02",
          19,
          0.009921110235154629,
          false
        ],
        [
          "p02",
          20,
          0.6478537917137146,
          false
        ],
        [
          "p02",
          21,
          0.8603132367134094,
          true
        ],
        [
          "p02",
          22,
          0.2413395345211029,
          false
        ],
        [
          "p02",
          23,
          0.4026266038417816,
          false
        ],
        [
          "p02",
          24,
          0.1895246058702469,
          false
        ],
        [
          "p02",
          25,
          0.31641674041748047,
          false
        ],
        [
          "p02",
          26,
          0.556407630443573,
          false
        ],
        [
          "p02",
          27,
          0.5544778108596802,
          false
        ],
        [
          "p02",
          28,
          0.4291600286960602,
          false
        ],
        [
          "p02",
          29,
          0.5937700271606445,
          true
        ]
      ],
      "source": "Start Avery Stone this week only if Blake Reed is ruled out. If Reed plays, I have no recommendation for Stone. Do not add Casey Lane from waivers this week. I am not telling anyone who already rosters Lane to drop him."
    },
    {
      "id": "case-06",
      "players": [
        {
          "id": "p00",
          "name": "Devon Pike"
        },
        {
          "id": "p01",
          "name": "Ellis Hart"
        }
      ],
      "queries": [
        [
          "p00",
          0,
          0.022461654618382454,
          false
        ],
        [
          "p00",
          1,
          0.024007029831409454,
          false
        ],
        [
          "p00",
          2,
          0.7120764255523682,
          false
        ],
        [
          "p00",
          3,
          0.752071738243103,
          false
        ],
        [
          "p00",
          4,
          0.14374865591526031,
          false
        ],
        [
          "p00",
          5,
          0.3383628726005554,
          false
        ],
        [
          "p00",
          6,
          0.01756361499428749,
          false
        ],
        [
          "p00",
          7,
          0.024408314377069473,
          false
        ],
        [
          "p00",
          8,
          0.5723409652709961,
          false
        ],
        [
          "p00",
          9,
          0.7546119093894958,
          true
        ],
        [
          "p00",
          10,
          0.4535052478313446,
          false
        ],
        [
          "p00",
          11,
          0.674789309501648,
          true
        ],
        [
          "p00",
          12,
          0.08680153638124466,
          false
        ],
        [
          "p00",
          13,
          0.06746187806129456,
          false
        ],
        [
          "p00",
          14,
          0.11467324942350388,
          false
        ],
        [
          "p00",
          15,
          0.07960047572851181,
          false
        ],
        [
          "p00",
          16,
          0.02261444553732872,
          false
        ],
        [
          "p00",
          17,
          0.02914922498166561,
          false
        ],
        [
          "p00",
          18,
          0.0031140565406531096,
          false
        ],
        [
          "p00",
          19,
          0.003196791047230363,
          false
        ],
        [
          "p00",
          20,
          0.9194288849830627,
          false
        ],
        [
          "p00",
          21,
          0.9653127193450928,
          false
        ],
        [
          "p00",
          22,
          0.46560412645339966,
          false
        ],
        [
          "p00",
          23,
          0.5115017294883728,
          false
        ],
        [
          "p00",
          24,
          0.2512039542198181,
          false
        ],
        [
          "p00",
          25,
          0.3170682489871979,
          false
        ],
        [
          "p00",
          26,
          0.8547234535217285,
          false
        ],
        [
          "p00",
          27,
          0.8145004510879517,
          false
        ],
        [
          "p00",
          28,
          0.718561053276062,
          false
        ],
        [
          "p00",
          29,
          0.8672574162483215,
          true
        ],
        [
          "p01",
          0,
          0.00881687831133604,
          false
        ],
        [
          "p01",
          1,
          0.008704609237611294,
          false
        ],
        [
          "p01",
          2,
          0.9766358733177185,
          false
        ],
        [
          "p01",
          3,
          0.9958479404449463,
          false
        ],
        [
          "p01",
          4,
          0.053298596292734146,
          false
        ],
        [
          "p01",
          5,
          0.26474064588546753,
          false
        ],
        [
          "p01",
          6,
          0.010690297931432724,
          false
        ],
        [
          "p01",
          7,
          0.02062678150832653,
          false
        ],
        [
          "p01",
          8,
          0.8229706287384033,
          false
        ],
        [
          "p01",
          9,
          0.9250662326812744,
          true
        ],
        [
          "p01",
          10,
          0.6221590638160706,
          false
        ],
        [
          "p01",
          11,
          0.8185946345329285,
          true
        ],
        [
          "p01",
          12,
          0.07177187502384186,
          false
        ],
        [
          "p01",
          13,
          0.06094837188720703,
          false
        ],
        [
          "p01",
          14,
          0.047313615679740906,
          false
        ],
        [
          "p01",
          15,
          0.040780697017908096,
          false
        ],
        [
          "p01",
          16,
          0.011362308636307716,
          false
        ],
        [
          "p01",
          17,
          0.014027868397533894,
          false
        ],
        [
          "p01",
          18,
          0.006638678722083569,
          false
        ],
        [
          "p01",
          19,
          0.004879259504377842,
          false
        ],
        [
          "p01",
          20,
          0.8508890271186829,
          false
        ],
        [
          "p01",
          21,
          0.9422345757484436,
          false
        ],
        [
          "p01",
          22,
          0.4625106751918793,
          false
        ],
        [
          "p01",
          23,
          0.5417430400848389,
          false
        ],
        [
          "p01",
          24,
          0.3063877522945404,
          false
        ],
        [
          "p01",
          25,
          0.40620478987693787,
          false
        ],
        [
          "p01",
          26,
          0.7918057441711426,
          false
        ],
        [
          "p01",
          27,
          0.7869136929512024,
          false
        ],
        [
          "p01",
          28,
          0.7485273480415344,
          false
        ],
        [
          "p01",
          29,
          0.9005352854728699,
          true
        ]
      ],
      "source": "Devon Pike is a draft fade at his second-round price in redraft leagues; that is not weekly sit advice. For this week's lineup I was about to say start Ellis Hart, but that was a mistake. My final recommendation is to sit Ellis Hart this week."
    }
  ],
  "completeBenchmark": false,
  "environment": {
    "packages": {
      "huggingface-hub": "1.23.0",
      "numpy": "2.5.1",
      "packaging": "26.2",
      "safetensors": "0.8.0",
      "sentencepiece": "0.2.1",
      "tokenizers": "0.22.2",
      "torch": "2.13.0",
      "transformers": "5.13.0"
    },
    "python": "3.13.3"
  },
  "humanGold": false,
  "inference": {
    "activation": "sigmoid",
    "batchSize": 4,
    "device": "cpu",
    "dtype": "float32",
    "gradients": false,
    "inputTemplate": "<<LABEL>>{hypothesis}<<SEP>>{source}",
    "interopThreads": 1,
    "maxNumClasses": 1,
    "modelMode": "eval",
    "padding": true,
    "seed": 0,
    "threads": 2,
    "tokenLimit": 512,
    "truncation": false
  },
  "model": {
    "repository": "knowledgator/gliclass-base-v3.0",
    "revision": "77a70e6cd52e602ed18184ef37d18bdd3741e3d5",
    "sourceRevision": "68132def761c2dccc20d1a71c04abdc5db35463f",
    "weightsSHA256": "4f1043b82812a8f5ec7ff9f97c6744758f47836f9872ba0dbeb3a8e2b0b92a26"
  },
  "newInference": false,
  "provenance": "Saved D766 scores; all safely completed synthetic cases only",
  "schemaVersion": 1,
  "selection": {
    "calibrated": false,
    "crossActionConflictResolution": false,
    "stateMargin": 0.1,
    "threshold": 0.5
  },
  "queryTupleFields": [
    "playerID",
    "hypothesisTemplateIndex",
    "score",
    "selected"
  ],
  "hypothesisTemplates": [
    {
      "action": "start",
      "state": "plain",
      "hypothesis": "The speaker's final position is to start {player} in a weekly fantasy lineup, not merely as a temporary streamer. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "start",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to start {player} in a weekly fantasy lineup, not merely as a temporary streamer. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "sit",
      "state": "plain",
      "hypothesis": "The speaker's final position is to sit {player} out of a weekly fantasy lineup. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "sit",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to sit {player} out of a weekly fantasy lineup. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "streamer",
      "state": "plain",
      "hypothesis": "The speaker's final position is to use {player} as a temporary fantasy lineup replacement or streamer. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "streamer",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to use {player} as a temporary fantasy lineup replacement or streamer. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "waiver_add",
      "state": "plain",
      "hypothesis": "The speaker's final position is to add {player} from fantasy waivers or free agency. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "waiver_add",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to add {player} from fantasy waivers or free agency. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "stash",
      "state": "plain",
      "hypothesis": "The speaker's final position is to stash {player} on the fantasy bench for future value. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "stash",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to stash {player} on the fantasy bench for future value. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "avoid_add",
      "state": "plain",
      "hypothesis": "The speaker's final position is to avoid adding {player} from fantasy waivers or free agency. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "avoid_add",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to avoid adding {player} from fantasy waivers or free agency. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "trade_target",
      "state": "plain",
      "hypothesis": "The speaker's final position is to acquire {player} in a fantasy trade, not just prefer a multi-player package. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "trade_target",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to acquire {player} in a fantasy trade, not just prefer a multi-player package. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "trade_away",
      "state": "plain",
      "hypothesis": "The speaker's final position is to trade away {player} in fantasy, not just prefer a multi-player package. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "trade_away",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to trade away {player} in fantasy, not just prefer a multi-player package. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "draft_target",
      "state": "plain",
      "hypothesis": "The speaker's final position is to target {player} in a fantasy draft. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "draft_target",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to target {player} in a fantasy draft. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "draft_value",
      "state": "plain",
      "hypothesis": "The speaker's final position is to draft {player} as good value at the discussed draft price. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "draft_value",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to draft {player} as good value at the discussed draft price. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "draft_fade",
      "state": "plain",
      "hypothesis": "The speaker's final position is to avoid drafting {player} at the discussed draft price. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "draft_fade",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to avoid drafting {player} at the discussed draft price. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "adp_discussion",
      "state": "plain",
      "hypothesis": "The speaker's final position is to report {player}'s average fantasy draft position or ADP. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "adp_discussion",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to report {player}'s average fantasy draft position or ADP. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "riser",
      "state": "plain",
      "hypothesis": "The speaker's final position is to report that {player}'s fantasy workload, role or ranking has increased. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "riser",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to report that {player}'s fantasy workload, role or ranking has increased. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "faller",
      "state": "plain",
      "hypothesis": "The speaker's final position is to report that {player}'s fantasy workload, role or ranking has decreased. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "faller",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to report that {player}'s fantasy workload, role or ranking has decreased. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "injury_update",
      "state": "plain",
      "hypothesis": "The speaker's final position is to report an injury or injury-related availability update about {player}. This is a definite statement within the stated timeframe, without an essential if-condition, price or league-format restriction, or uncertain availability. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    },
    {
      "action": "injury_update",
      "state": "qualified",
      "hypothesis": "The speaker's final position is to report an injury or injury-related availability update about {player}. This statement has an essential if-condition, price or league-format restriction, or uncertain availability; removing that qualification would misrepresent the discussion. It is not merely a comparison, quotation, advertisement or rejected suggestion."
    }
  ]
}

Hi, following up on the two questions above. Could someone confirm whether these compound owner/action/condition hypotheses are within this checkpoint's intended scope, and point us to a supported reference environment or example? A candid indication that this formulation is outside the validated task would also help. If another maintainer is better placed to advise, a referral would be appreciated. Thank you.

Sign up or log in to comment