← Projects/AI · ML

TrustFin-SLM

Trustworthy Small Language Models for Financial Tasks

2026·In Progress·LLM · QLoRA · Fine-tuning · Financial AI · Evaluation
TrustFin-SLM

TrustFin-SLM

TrustFin-SLM is a research pipeline for building, validating, and evaluating trustworthy small language models (SLMs) on financial reasoning (FinQA), financial sentiment (FiQA), FOMC stance classification, and multi-dimensional trust benchmarks (FinTrust).

Datasets

  • FinQA - Financial numerical reasoning (train/dev/test)
  • FiQA Sentiment - Aspect-aware sentiment regression [-1,1] (train/valid/test)
  • FinBen-FOMC - FOMC stance classification (test only, eval-only)
  • FinTrust - 7-dimension trust benchmark (eval-only)

Training & Evaluation

  • Fine-tuned Qwen3-1.7B with balanced joint QLoRA across 16 baseline experiments
  • Selected backbone on validation data only (FinQA execution score 0.675, FiQA R² 0.730)
  • Built reproducible 4-benchmark evaluation pipeline with SHA-256 frozen data contracts and 68 validation gates
  • Kept protected test sets out of training and threshold tuning
  • Evaluation policy enforces strict separation between training/validation and protected test sets

EVG-CAS Abstention

Proposed the EVG-CAS abstention method: a frozen threshold cut protected-test answered-error by 31.0% at 80% coverage, validated with paired McNemar tests and bootstrap confidence intervals. Research paper draft in progress.

Project Structure

The project follows a phased approach (Phase 2 through Phase 4) with frozen artifacts, pre-registered evaluations, and rigorous statistical validation.

Status

Research paper draft in progress.