- LFM2.5-24B-A2B is a free model from Liquid AI that you can run on your own computer. In our tests it's decent, with trade-offs: 62 out of 100, #30 of 67.
- It solved 13 of 30 coding jobs and scored 80 on reading documents. On our hardest tasks it scored 33.
- Runs on a 24 GB graphics card or a Mac with 24 GB.
Coding
Our coding test is 30 programming jobs, from small ones like reading time durations or cleaning up messy data to harder ones like a config-file parser or a double-entry ledger. We run each answer against tests the model never sees, and a job only counts if everything passes. LFM2.5-24B-A2B got 13 of 30 right. The best local coders solved 29 of 30.
Reading documents
The second test hands the model things like an expense claim thread, a pay stub or an insurance statement, and asks for specific numbers and dates. Many questions need a bit of math, or noticing a correction further down the email. LFM2.5-24B-A2B scored 80; the best model scored 100.
| Test | Score | Public questions | Secret questions |
|---|---|---|---|
| Coding | 43 | 71 | 35 |
| Reading documents | 80 | 97 | 76 |
| Decisions | 92 | 96 | 91 |
On the 18 hardest tasks (included in the scores above) it scored 33. This number separates the top models.
This model did much better on our public questions than on the secret ones. Sometimes that's luck; it can also mean the model has seen the public questions before.
We tested the full-size version online. The smaller versions you download usually score a little lower.
Can your computer run it?
You can download LFM2.5-24B-A2B in several sizes. Smaller versions fit on more computers and lose a little quality, so pick the biggest one that fits.
| Your computer | Runs it? | Version to download |
|---|---|---|
| Graphics card, 8 GB | No | – |
| Graphics card, 12 GB | No | – |
| Graphics card, 16 GB | No | – |
| Graphics card, 24 GB | Yes | Q6 |
| Graphics card, 32 GB | Yes | Q8 |
| Mac, 16 GB | No | – |
| Mac, 24 GB | Yes | Q4 |
| Mac, 32 GB | Yes | Q6 |
| Mac, 48 GB | Yes | Q8 |
| Mac, 64 GB | Yes | Q8 |
| Mac, 96 GB | Yes | Q8 |
| Mac, 128 GB | Yes | Q8 |
For normal-length chats. Long documents need more memory. Q4 or Q8? · How we worked this out
How to try it
Search for "LFM2.5-24B-A2B GGUF" in LM Studio or on Hugging Face and pick the version that fits your computer. New to this? Start with our Ollama guide.
Want the strongest all-rounder instead? Qwen3.6 27B scored 97.
Its actual answers
What LFM2.5-24B-A2B wrote for our public questions, word for word.
Coding: INI parser (hard) Didn't work
Coding: TTL LRU (hard) Didn't work
Coding: Parse duration Worked
```python
import re
def parse_duration(s: str) -> int:
# Validate input is not empty or blank
if s is None:
raise ValueError("Empty or blank string")
# Remove all whitespace characters
cleaned = re.sub(r'\s+', '', s)
if not cleaned:
raise ValueError("Empty or blank string")
# Match number followed by a unit (case‑insensitive)
pattern = re.compile(r'(\d+)([hHmsS])')
matches = pattern.findall(cleaned)
# Ensure the entire string was consumed by the pattern
reconstructed = ''.join(num + unit for num, unit in matches)
if reconstructed != cleaned:
raise ValueError("Invalid format")
total_seconds = 0
seen_units = set()
last_order = -1
order_map = {'h': 0, 'm': 1, 's': 2}
for num_str, unit in matches:
unit_lower = unit.lower()
# Unknown unit
if unit_lower not in order_map:
raise ValueError(f"Unknown unit: {unit}")
# Repeated unit
if unit_lower in seen_units:
raise ValueError(f"Repeated unit: {unit}")
# Order check (h < m < s)
idx = order_map[unit_lower]
if idx < last_order:
raise ValueError("Units out of order")
seen_units.add(unit_lower)
last_order = idx
num = int(num_str)
if unit_lower == 'h':
total_seconds += num * 3600
elif unit_lower == 'm':
total_seconds += num * 60
else: # 's'
total_seconds += num
return total_seconds
``` Coding: Summarize ranges Worked
```python
def summarize_ranges(nums: list[int]) -> str:
# Remove duplicates and sort the numbers
uniq = sorted(set(nums))
if not uniq:
return ""
parts = []
i = 0
n = len(uniq)
while i < n:
start = uniq[i]
# Extend the run while numbers are consecutive
j = i + 1
while j < n and uniq[j] == uniq[j - 1] + 1:
j += 1
run_len = j - i
if run_len >= 3:
# Collapse a long run into a..b format
parts.append(f"{start}..{uniq[j - 1]}")
else:
# List each number individually for short runs
for num in range(start, uniq[j - 1] + 1):
parts.append(str(num))
i = j
return ",".join(parts)
``` Coding: Top customers Worked
```python
from typing import List, Dict, Tuple
def top_customers(rows: List[dict], n: int) -> List[Tuple[str, float]]:
"""
Aggregate customer spending from a list of transaction rows.
Parameters
----------
rows : list[dict]
Each dict must contain "customer" (str) and "amount" (str).
n : int
Number of top customers to return.
Returns
-------
list[tuple[str, float]]
Top ``n`` customers ordered by total spending descending,
then by name ascending. Totals are rounded to two decimal places.
"""
def parse_amount(amount_str: str) -> float | None:
"""Convert a monetary string like '$1,234.50' or '-$5.00' to a float."""
if not isinstance(amount_str, str):
return None
s = amount_str.strip()
if not s:
return None
# Detect sign
negative = False
if s.startswith('-'):
negative = True
s = s[1:]
# Remove currency symbol if present
if s.startswith('$'):
s = s[1:]
# Remove thousands separators
s = s.replace(',', '')
try:
value = float(s)
except ValueError:
return None
if negative:
value = -value
return value
# Map lower‑cased name -> {original_name, total}
customers: Dict[str, Dict[str, object]] = {}
for row in rows:
# Validate customer name
raw_customer = row.get("customer")
if raw_customer is None:
continue
cust = raw_customer.strip()
if not cust:
continue
# Validate and parse amount
amount_str = row.get("amount")
if amount_str is None:
continue
amount = parse_amount(amount_str)
if amount is None:
continue
# Group by case‑insensitive name, preserving the first trimmed spelling
key = cust.lower()
if key not in customers:
customers[key] = {"original": cust, "total": amount}
else:
customers[key]["total"] = customers[key]["total"] + amount
# Prepare result list with rounded totals
result: List[Tuple[str, float]] = []
for info in customers.values():
total = round(info["total"], 2)
result.append((info["original"], total))
# Sort: total descending, then name ascending (case‑insensitive)
result.sort(key=lambda x: (-x[1], x[0].lower()))
# Return top n entries
return result[:n]
``` Coding: Slugify Worked
```python
import re
import unicodedata
def slugify(title: str, max_len: int = 50) -> str:
"""
Convert a title into a URL-friendly slug.
1. Transliterate accented characters to ASCII and drop other non‑ASCII characters.
2. Lowercase.
3. Replace runs of characters other than a‑z or 0‑9 with a single hyphen,
then strip leading/trailing hyphens.
4. If the slug exceeds *max_len*, truncate without breaking a word:
keep the longest prefix of whole words whose total length is ≤ *max_len*.
If the first word itself is longer than *max_len*, hard‑cut it to *max_len*
characters.
5. The result never ends with a hyphen.
"""
# 1. Transliterate and drop non‑ASCII
normalized = unicodedata.normalize('NFKD', title)
ascii_str = normalized.encode('ascii', 'ignore').decode('ascii')
# 2. Lowercase
ascii_str = ascii_str.lower()
# 3. Replace non‑alphanumeric runs with a hyphen
slug = re.sub(r'[^a-z0-9]+', '-', ascii_str)
slug = slug.strip('-')
if not slug:
return ''
# 4. Truncate if necessary
if len(slug) <= max_len:
final_slug = slug
else:
words = slug.split('-')
# If the first word is longer than max_len, hard‑cut it
if len(words[0]) > max_len:
final_slug = words[0][:max_len]
else:
cur_words = [words[0]]
cur_len = len(words[0])
for w in words[1:]:
# Length including the hyphen that will separate the words
if cur_len + 1 + len(w) <= max_len:
cur_words.append(w)
cur_len += 1 + len(w)
else:
break
final_slug = '-'.join(cur_words)
# 5. Ensure no leading/trailing hyphen
final_slug = final_slug.strip('-')
return final_slug
``` Coding: Token bucket Worked
```python
from typing import Callable
class TokenBucket:
"""
A token bucket rate limiter.
The bucket starts full with `capacity` tokens. Tokens are refilled at a
constant rate of `refill_per_sec` tokens per second, never exceeding the
capacity.
"""
def __init__(
self,
capacity: float,
refill_per_sec: float,
clock: Callable[[], float],
) -> None:
self.capacity = capacity
self.refill_per_sec = refill_per_sec
self._clock = clock
# Bucket is initially full.
self._tokens = capacity
# Record the time of the last refill (initialized to the current time).
self._last_time = self._clock()
def _refill(self) -> None:
"""Refresh tokens based on elapsed time since the last refill."""
now = self._clock()
elapsed = now - self._last_time
if elapsed > 0:
# Tokens added during the elapsed interval.
added = elapsed * self.refill_per_sec
self._tokens = min(self._tokens + added, self.capacity)
# Update the timestamp for future refills.
self._last_time = now
def allow(self, cost: float = 1) -> bool:
"""
Attempt to consume `cost` tokens.
Returns True if enough tokens were available and were deducted,
otherwise False and no tokens are removed.
"""
self._refill()
if self._tokens >= cost:
self._tokens -= cost
return True
return False
@property
def tokens(self) -> float:
"""
Refill and return the current number of tokens.
"""
self._refill()
return self._tokens
``` Decisions: Refund window (hard) 100% right
{
"outcome": {
"full_refund": 0,
"partial_refund": 0,
"store_credit": 1,
"nothing": 0
},
"defective": {
"true": 0,
"false": 1
}
} Decisions: Incident matrix (hard) 100% right
{"severity": {"0": 0, "1": 1, "2": 0, "3": 0}, "page": {"true": 0, "false": 1}} Decisions: Tool followup (hard) 50% right
{
"tool": {
"web_search": 0.0,
"calculator": 0.0,
"calendar": 1.0,
"email": 0.0,
"none": 0.0
},
"confirm": {
"true": 0.0,
"false": 1.0
}
} Decisions: Legit security alert (hard) 100% right
{
"phishing": {"true": 0.01, "false": 0.99},
"action_needed": {"true": 0.02, "false": 0.98}
} Decisions: Meeting slot (hard) 100% right
{
"slot": {
"A": 0,
"B": 1,
"C": 0,
"D": 0
},
"raj_last": {
"true": 1,
"false": 0
}
} Decisions: Review mixed (hard) 100% right
{
"hardware": {"true": 0.98, "false": 0.02},
"support": {"true": 0.97, "false": 0.03}
} Decisions: Support checkout down 100% right
{
"department": {
"billing": 0.1,
"technical": 0.9,
"account": 0.0,
"shipping": 0.0,
"sales": 0.0
},
"urgency": {
"0": 0.0,
"1": 0.0,
"2": 0.0,
"3": 1.0
},
"outage": {
"true": 1.0,
"false": 0.0
}
} Decisions: Refund wrong plan 100% right
{
"department": {
"billing": 1.0,
"technical": 0.0,
"account": 0.0,
"shipping": 0.0,
"sales": 0.0
},
"refund": {
"true": 1.0,
"false": 0.0
},
"tone": {
"frustrated": 0.0,
"calm": 1.0
}
} Decisions: Moderation doxxing 100% right
{
"policy": {
"none": 0.0,
"harassment": 1.0,
"hate": 0.0,
"spam": 0.0,
"self_harm": 0.0
},
"personal_info": {
"true": 1.0,
"false": 0.0
}
} Decisions: Route calendar 100% right
{
"tool": {
"web_search": 0.02,
"calculator": 0.01,
"calendar": 0.95,
"email": 0.01,
"none": 0.01
},
"confirm": {
"true": 1.0,
"false": 0.0
}
} Decisions: Doc invoice missing due 100% right
{
"doc_type": {
"invoice": 1.0,
"resume": 0.0,
"contract": 0.0,
"bank_statement": 0.0,
"other": 0.0
},
"missing_due_date": {
"true": 1.0,
"false": 0.0
}
} Decisions: Phishing paypal 100% right
{
"phishing": {
"true": 0.99,
"false": 0.01
},
"risk": {
"0": 0.0,
"1": 0.0,
"2": 0.0,
"3": 1.0
}
} Decisions: Pii ssn email 100% right
{
"data_kind": {
"none": 0.0,
"contact": 0.0,
"financial": 0.0,
"government_id": 1.0,
"health": 0.0
},
"sensitive": {
"true": 1.0,
"false": 0.0
}
} Decisions: Review mixed 100% right
{
"sentiment": {
"positive": 0.1,
"neutral": 0.05,
"negative": 0.85
},
"defect": {
"true": 1.0,
"false": 0.0
},
"recommend": {
"true": 0.0,
"false": 1.0
}
} Documents: Saas escalator (hard) 100% right
{
"year2_price_per_seat_month": 47.25,
"year3_price_per_seat_month": 47.25,
"year1_invoice": 58320.00,
"year2_invoice": 61236.00,
"addon_months_billed": 6,
"addon_invoice": 38556.00,
"year3_invoice": 134946.00,
"year3_discount_percent": 15.0,
"total_contract_value": 293058.00,
"contract_end_date": "2027-02-28"
} Documents: Expense thread 92% right
{
"employee_id": "EMP-20417",
"destination_city": "Lisbon",
"trip_start": "2025-02-24",
"trip_end": "2025-02-27",
"approved_items": [
{
"date": "2025-02-24",
"category": "airfare",
"amount_usd": 1184.60
},
{
"date": "2025-02-24",
"category": "ground_transport",
"amount_usd": 38.88
},
{
"date": "2025-02-25",
"category": "meals",
"amount_usd": 229.31
},
{
"date": "2025-02-26",
"category": "lodging",
"amount_usd": 466.56
},
{
"date": "2025-02-27",
"category": "ground_transport",
"amount_usd": 44.82
}
],
"rejected_item_count": 1,
"per_diem_days": 3,
"per_diem_usd": 195.0,
"total_reimbursable_usd": 2159.17,
"approver_email": "priya.raman@corvane.com"
} Documents: Lease amendment 100% right
{
"tenants": ["Marcus Lin", "Sofia Lin"],
"landlord": "Ridgeline Property Group LLC",
"zip": "97205",
"lease_end": "2025-11-30",
"original_monthly_rent": 2150,
"monthly_rent_from_2025_06_01": 2236,
"late_fee_from_2025_06_01": 111.80,
"security_deposit": 2150,
"total_pet_deposits": 800,
"total_monthly_payment_july_2025": 2306,
"move_in_payment": 4700
} Documents: Ticket SLA 92% right
{
"ticket_id": "48213",
"account_id": "ACC-7731",
"open_issue": "inventory_sync",
"resolved_issues": ["billing_address", "invoice_pdf"],
"affected_orders": ["SO-99812", "SO-99820", "SO-99827"],
"priority": "P2",
"sla_due_local": "2025-09-15T15:30",
"sla_due_utc": "2025-09-15T20:30:00Z",
"reissued_invoice": "INV-2025-0812"
} Documents: Sales footnotes 100% right
{
"q3_total_usd": 15346000,
"q2_total_usd": 14464000,
"q2_central_originally_reported_usd": 3047000,
"q2_to_q3_change_pct": 6.1,
"top_region_q3": "East",
"fastest_growing_region_q1_to_q3": "International",
"regions_declining_q2_to_q3": ["East"],
"international_q3_organic_usd": 1731000,
"west_excluding_mountain_q3_usd": 4201000
} Size: 24B parameters. First tested OCT 14.
Comments
Sign in with GitHub to comment. Spam and abuse are hidden automatically.