Reviews
The short version
- Mistral Small 4 is a free model from Mistral AI that you can run on your own computer. In our tests it's not one we'd recommend right now: 51 out of 100, #35 of 63.
- It solved 16 of 30 coding jobs and scored 49 on reading documents. On our hardest tasks it scored 35.
- Runs on a Mac with 128 GB.
Coding
Our coding test is 30 programming jobs, from small ones like reading time durations or cleaning up messy data to harder ones like a config-file parser or a double-entry ledger. We run each answer against tests the model never sees, and a job only counts if everything passes. Mistral Small 4 got 16 of 30 right. The best local coders solved 29 of 30.
Reading documents
The second test hands the model things like an expense claim thread, a pay stub or an insurance statement, and asks for specific numbers and dates. Many questions need a bit of math, or noticing a correction further down the email. Mistral Small 4 scored 49; the best model scored 100.
| Test | Score | Public questions | Secret questions |
|---|---|---|---|
| Coding | 53 | 43 | 57 |
| Reading documents | 49 | 69 | 44 |
| Decisions | 82 | 89 | 80 |
On the 18 hardest tasks (included in the scores above) it scored 35. This number separates the top models.
This model did much better on our public questions than on the secret ones. Sometimes that's luck; it can also mean the model has seen the public questions before.
We tested the full-size version online. The smaller versions you download usually score a little lower.
Can your computer run it?
You can download Mistral Small 4 in several sizes. Smaller versions fit on more computers and lose a little quality, so pick the biggest one that fits.
| Your computer | Runs it? | Version to download |
|---|---|---|
| Graphics card, 8 GB | No | – |
| Graphics card, 12 GB | No | – |
| Graphics card, 16 GB | No | – |
| Graphics card, 24 GB | No | – |
| Graphics card, 32 GB | No | – |
| Mac, 16 GB | No | – |
| Mac, 24 GB | No | – |
| Mac, 32 GB | No | – |
| Mac, 48 GB | No | – |
| Mac, 64 GB | No | – |
| Mac, 96 GB | No | – |
| Mac, 128 GB | Yes | Q5 |
For normal-length chats. Long documents need more memory. Q4 or Q8? · How we worked this out
How to try it
Search for "Mistral Small 4 GGUF" in LM Studio or on Hugging Face and pick the version that fits your computer. New to this? Start with our Ollama guide.
Want the strongest all-rounder instead? Qwen3.6 27B scored 97.
Its actual answers
What Mistral Small 4 wrote for our public questions, word for word.
Coding: INI parser (hard) Didn't work
```python
import re
from collections import defaultdict
from typing import Dict, List, Tuple
def parse_ini(text: str) -> dict[str, dict[str, str]]:
sections: Dict[str, Dict[str, str]] = defaultdict(dict)
current_section: str = "DEFAULT"
lines = text.splitlines()
i = 0
default_section = {}
def process_line(line: str) -> None:
nonlocal current_section
stripped = line.strip()
if not stripped or stripped[0] in ';#':
return
if stripped.startswith('[') and stripped.endswith(']'):
section_name = stripped[1:-1].strip()
if not section_name:
raise ValueError("Empty section name")
current_section = section_name
if current_section not in sections:
sections[current_section] = {}
return
if '=' in line or ':' in line:
key_end = -1
for sep in ['=', ':']:
pos = line.find(sep)
if pos != -1:
if key_end == -1 or pos < key_end:
key_end = pos
if key_end == -1:
raise ValueError("Malformed line")
key = line[:key_end].strip().lower()
if not key:
raise ValueError("Empty key")
value_part = line[key_end + 1:]
comment_pos = -1
for pattern in [' ;', ' #']:
pos = value_part.find(pattern)
if pos != -1:
if comment_pos == -1 or pos < comment_pos:
comment_pos = pos
if comment_pos != -1:
value_part = value_part[:comment_pos]
if value_part.startswith('"') and value_part.endswith('"'):
value = value_part[1:-1]
else:
value = value_part.strip()
if current_section == "DEFAULT":
default_section[key] = value
else:
sections[current_section][key] = value
return
if line[0].isspace():
if not sections[current_section]:
raise ValueError("Continuation line without preceding key")
last_key = next(reversed(sections[current_section]))
sections[current_section][last_key] += '\n' + line.strip()
return
raise ValueError("Malformed line")
for line in lines:
process_line(line)
result = {}
for section, kvs in sections.items():
if section == "DEFAULT":
continue
inherited = default_section.copy()
inherited.update(kvs)
result[section] = inherited
def interpolate(value: str, section: str, visited: set) -> str:
def replace_match(match):
ref = match.group(1)
if ':' in ref:
s, k = ref.split(':', 1)
if s not in result:
raise KeyError(f"Section {s} not found")
if k not in result[s]:
raise KeyError(f"Key {k} not found in section {s}")
ref_value = result[s][k]
else:
k = ref
if k in result[section]:
ref_value = result[section][k]
elif k in default_section:
ref_value = default_section[k]
else:
raise KeyError(f"Key {k} not found")
if ref_value in visited:
raise ValueError("Interpolation cycle detected")
visited.add(ref_value)
return interpolate(ref_value, section, visited)
pattern = re.compile(r'\$\{([^}]+)\}')
while True:
new_value = pattern.sub(replace_match, value)
if new_value == value:
break
value = new_value
return value
final_result = {}
for section, kvs in result.items():
final_kvs = {}
for key, value in kvs.items():
final_kvs[key] = interpolate(value, section, set())
final_result[section] = final_kvs
return final_result
```Coding: TTL LRU (hard) Didn't work
```python
from collections import OrderedDict
from time import time
from typing import Any, Callable
class TTLCache:
def __init__(self, capacity: int, ttl: float, clock: Callable[[], float]):
self.capacity = capacity
self.ttl = ttl
self.clock = clock
self._cache: OrderedDict[Any, tuple[Any, float]] = OrderedDict()
self._stats = {
"hits": 0,
"misses": 0,
"evictions": 0,
"expirations": 0,
}
def put(self, key: Any, value: Any) -> None:
if self.capacity <= 0:
return
now = self.clock()
if key in self._cache:
expiry = now + self.ttl
self._cache.move_to_end(key)
self._cache[key] = (value, expiry)
return
self._remove_expired(now)
if len(self._cache) >= self.capacity:
self._cache.popitem(last=False)
self._stats["evictions"] += 1
expiry = now + self.ttl
self._cache[key] = (value, expiry)
def get(self, key: Any, default: Any = None) -> Any:
if self.capacity <= 0:
self._stats["misses"] += 1
return default
now = self.clock()
if key not in self._cache:
self._stats["misses"] += 1
return default
value, expiry = self._cache[key]
if now >= expiry:
del self._cache[key]
self._stats["expirations"] += 1
self._stats["misses"] += 1
return default
self._cache.move_to_end(key)
self._stats["hits"] += 1
return value
def __len__(self) -> int:
now = self.clock()
self._remove_expired(now)
return len(self._cache)
def stats(self) -> dict[str, int]:
return self._stats.copy()
def _remove_expired(self, now: float) -> None:
expired = [k for k, (_, expiry) in self._cache.items() if now >= expiry]
for k in expired:
del self._cache[k]
self._stats["expirations"] += 1
```Coding: Parse duration Worked
```python
import re
def parse_duration(s: str) -> int:
if not s.strip():
raise ValueError("Empty or blank string")
pattern = re.compile(
r'^\s*'
r'(?:(?P<hours>\d+)\s*h\s*)?'
r'(?:(?P<minutes>\d+)\s*m\s*)?'
r'(?:(?P<seconds>\d+)\s*s\s*)?'
r'\s*$',
flags=re.IGNORECASE
)
match = pattern.fullmatch(s)
if not match:
raise ValueError("Invalid duration format")
groups = match.groupdict()
if not any(groups.values()):
raise ValueError("No duration components found")
hours = int(groups['hours']) if groups['hours'] is not None else 0
minutes = int(groups['minutes']) if groups['minutes'] is not None else 0
seconds = int(groups['seconds']) if groups['seconds'] is not None else 0
if groups['hours'] is not None and groups['minutes'] is not None and groups['seconds'] is not None:
pass
elif groups['hours'] is not None and groups['minutes'] is not None:
pass
elif groups['hours'] is not None and groups['seconds'] is not None:
raise ValueError("Units out of order: minutes missing between hours and seconds")
elif groups['minutes'] is not None and groups['seconds'] is not None:
pass
elif groups['hours'] is not None:
pass
elif groups['minutes'] is not None:
pass
elif groups['seconds'] is not None:
pass
return hours * 3600 + minutes * 60 + seconds
```Coding: Summarize ranges Worked
```python
def summarize_ranges(nums: list[int]) -> str:
if not nums:
return ""
nums = sorted(set(nums))
result = []
start = nums[0]
prev = nums[0]
for num in nums[1:]:
if num == prev + 1:
prev = num
else:
if prev - start >= 2:
result.append(f"{start}..{prev}")
else:
result.extend(str(x) for x in range(start, prev + 1))
start = prev = num
if prev - start >= 2:
result.append(f"{start}..{prev}")
else:
result.extend(str(x) for x in range(start, prev + 1))
return ",".join(result)
```Coding: Top customers Didn't work
```python
import re
from collections import defaultdict
def top_customers(rows: list[dict], n: int) -> list[tuple[str, float]]:
customer_totals = defaultdict(float)
customer_names = {}
amount_pattern = re.compile(r'^[+-]?\$?[\d,]+\.?\d*$')
for row in rows:
customer = row.get("customer", "").strip()
amount_str = row.get("amount", "")
if not customer or not amount_str or not amount_pattern.match(amount_str):
continue
try:
amount = float(amount_str.replace("$", "").replace(",", ""))
except ValueError:
continue
if customer not in customer_names:
customer_names[customer] = customer
customer_totals[customer] += amount
sorted_customers = sorted(
customer_totals.items(),
key=lambda item: (-item[1], item[0])
)
return sorted_customers[:n]
```Coding: Slugify Didn't work
```python
import unicodedata
import re
def slugify(title: str, max_len: int = 50) -> str:
# Step 1: Transliterate accented characters to ASCII and drop non-ASCII
slug = unicodedata.normalize('NFKD', title)
slug = ''.join(c for c in slug if not unicodedata.combining(c))
slug = slug.encode('ascii', 'ignore').decode('ascii')
# Step 2: Lowercase
slug = slug.lower()
# Step 3: Replace non-alphanumeric runs with single "-", strip leading/trailing "-"
slug = re.sub(r'[^a-z0-9]+', '-', slug)
slug = slug.strip('-')
# Step 4: Shorten if longer than max_len without cutting words
if len(slug) > max_len:
words = slug.split('-')
result = []
current_len = 0
for word in words:
if current_len + len(word) + (1 if result else 0) <= max_len:
result.append(word)
current_len += len(word) + (1 if result else 0)
else:
break
slug = '-'.join(result)
# If even the first word is too long, hard-cut it
if len(slug) > max_len:
slug = slug[:max_len]
# Step 5: Ensure it doesn't end with "-"
slug = slug.rstrip('-')
return slug
```Coding: Token bucket Worked
```python
import time
from typing import Callable
class TokenBucket:
def __init__(self, capacity: float, refill_per_sec: float, clock: Callable[[], float]):
self._capacity = capacity
self._refill_per_sec = refill_per_sec
self._clock = clock
self._last_refill_time: float = clock()
self._tokens: float = capacity
def _refill(self) -> None:
now = self._clock()
elapsed = now - self._last_refill_time
self._last_refill_time = now
self._tokens = min(self._capacity, self._tokens + elapsed * self._refill_per_sec)
@property
def tokens(self) -> float:
self._refill()
return self._tokens
def allow(self, cost: float = 1) -> bool:
self._refill()
if self._tokens >= cost:
self._tokens -= cost
return True
return False
```Decisions: Refund window (hard) 100% right
{"outcome": {"full_refund": 0.0, "partial_refund": 0.0, "store_credit": 1.0, "nothing": 0.0}, "defective": {"true": 0.0, "false": 1.0}}Decisions: Incident matrix (hard) 50% right
{"severity": {"0": 0.0, "1": 0.9, "2": 0.1, "3": 0.0}, "page": {"true": 0.9, "false": 0.1}}Decisions: Tool followup (hard) 100% right
{"tool": {"calendar": 0.99, "email": 0.01, "none": 0.0, "web_search": 0.0, "calculator": 0.0}, "confirm": {"true": 0.95, "false": 0.05}}Decisions: Legit security alert (hard) 100% right
{"phishing": {"true": 0.01, "false": 0.99}, "action_needed": {"true": 0.05, "false": 0.95}}Decisions: Meeting slot (hard) 0% right
{"slot": {"A": 0.0, "B": 0.0, "C": 0.0, "D": 0.0}, "raj_last": {"true": 0.0, "false": 1.0}}Decisions: Review mixed (hard) 100% right
{"hardware": {"true": 0.99, "false": 0.01}, "support": {"true": 0.99, "false": 0.01}}Decisions: Support checkout down 100% right
{"department": {"billing": 0.05, "technical": 0.9, "account": 0.02, "shipping": 0.01, "sales": 0.02}, "urgency": {"0": 0.0, "1": 0.0, "2": 0.1, "3": 0.9}, "outage": {"true": 0.99, "false": 0.01}}Decisions: Refund wrong plan 100% right
{"department": {"billing": 0.95, "technical": 0.01, "account": 0.01, "shipping": 0.01, "sales": 0.02}, "refund": {"true": 0.99, "false": 0.01}, "tone": {"frustrated": 0.1, "calm": 0.9}}Decisions: Moderation doxxing 100% right
{"policy": {"none": 0.0, "harassment": 0.95, "hate": 0.0, "spam": 0.0, "self_harm": 0.05}, "personal_info": {"true": 1.0, "false": 0.0}}Decisions: Route calendar 100% right
{"tool": {"calendar": 1.0, "web_search": 0.0, "calculator": 0.0, "email": 0.0, "none": 0.0}, "confirm": {"true": 1.0, "false": 0.0}}Decisions: Doc invoice missing due 100% right
{"doc_type": {"invoice": 1.0, "resume": 0.0, "contract": 0.0, "bank_statement": 0.0, "other": 0.0}, "missing_due_date": {"true": 1.0, "false": 0.0}}Decisions: Phishing paypal 100% right
{"phishing": {"true": 0.99, "false": 0.01}, "risk": {"0": 0.0, "1": 0.01, "2": 0.05, "3": 0.94}}Decisions: Pii ssn email 100% right
{"data_kind": {"none": 0.0, "contact": 0.0, "financial": 0.0, "government_id": 1.0, "health": 0.0}, "sensitive": {"true": 1.0, "false": 0.0}}Decisions: Review mixed 100% right
{"sentiment": {"positive": 0.0, "neutral": 0.0, "negative": 1.0}, "defect": {"true": 1.0, "false": 0.0}, "recommend": {"true": 0.0, "false": 1.0}}Documents: Saas escalator (hard) 40% right
{"year2_price_per_seat_month": 47.25, "year3_price_per_seat_month": 47.25, "year1_invoice": 54000, "year2_invoice": 56700, "addon_months_billed": 18, "addon_invoice": 14220, "year3_invoice": 56700, "year3_discount_percent": 15, "total_contract_value": 181620, "contract_end_date": "2027-02-28"}Documents: Expense thread 88% right
{"employee_id": "EMP-20417", "destination_city": "Lisbon", "trip_start": "2025-02-24", "trip_end": "2025-02-27", "approved_items": [{"date": "2025-02-24", "category": "airfare", "amount_usd": 1184.6}, {"date": "2025-02-24", "category": "ground_transport", "amount_usd": 38.88}, {"date": "2025-02-25", "category": "meals", "amount_usd": 229.4}, {"date": "2025-02-26", "category": "lodging", "amount_usd": 466.56}, {"date": "2025-02-27", "category": "ground_transport", "amount_usd": 44.82}], "rejected_item_count": 1, "per_diem_days": 3, "per_diem_usd": 130, "total_reimbursable_usd": 2094.26, "approver_email": "priya.raman@corvane.com"}Documents: Lease amendment 92% right
{"tenants": ["Marcus Lin", "Sofia Lin"], "landlord": "Ridgeline Property Group LLC", "zip": "97205", "lease_end": "2025-11-30", "original_monthly_rent": 2150.0, "monthly_rent_from_2025_06_01": 2236.0, "late_fee_from_2025_06_01": 111.8, "security_deposit": 2150.0, "total_pet_deposits": 800.0, "total_monthly_payment_july_2025": 2271.0, "move_in_payment": 4700.0}Documents: Ticket SLA 92% right
{"ticket_id": "48213", "account_id": "ACC-7731", "open_issue": "inventory_sync", "resolved_issues": ["billing_address", "invoice_pdf"], "affected_orders": ["SO-99812", "SO-99820", "SO-99827"], "priority": "P2", "sla_due_local": "2025-09-15T15:30", "sla_due_utc": "2025-09-15T20:30:00Z", "reissued_invoice": "INV-2025-0812"}Documents: Sales footnotes 33% right
{"q3_total_usd": 15346, "q2_total_usd": 14464, "q2_central_originally_reported_usd": 3047, "q2_to_q3_change_pct": 7.5, "top_region_q3": "East", "fastest_growing_region_q1_to_q3": "International", "regions_declining_q2_to_q3": ["East"], "international_q3_organic_usd": 1731, "west_excluding_mountain_q3_usd": 4201}Size: 119B parameters. First tested OCT 11.
Comments
Sign in with GitHub to comment. Spam and abuse are hidden automatically.