- Ministral 3 8B 2512 is a free model from Mistral AI that you can run on your own computer. In our tests it's not one we'd recommend right now: 37 out of 100, #42 of 56.
- It solved 9 of 30 coding jobs and scored 43 on reading documents. On our hardest tasks it scored 18.
- Runs on an 8 GB graphics card or a Mac with 16 GB.
Coding
Our coding test is 30 programming jobs, from small ones like reading time durations or cleaning up messy data to harder ones like a config-file parser or a double-entry ledger. We run each answer against tests the model never sees, and a job only counts if everything passes. Ministral 3 8B 2512 got 9 of 30 right. The best local coders solved 29 of 30.
Reading documents
The second test hands the model things like an expense claim thread, a pay stub or an insurance statement, and asks for specific numbers and dates. Many questions need a bit of math, or noticing a correction further down the email. Ministral 3 8B 2512 scored 43; the best model scored 100.
| Test | Score | Public questions | Secret questions |
|---|---|---|---|
| Coding | 30 | 29 | 30 |
| Reading documents | 43 | 62 | 38 |
| Decisions | 84 | 89 | 82 |
On the 18 hardest tasks (included in the scores above) it scored 18. This number separates the top models.
This model did much better on our public questions than on the secret ones. Sometimes that's luck; it can also mean the model has seen the public questions before.
We tested the full-size version online. The smaller versions you download usually score a little lower.
Can your computer run it?
You can download Ministral 3 8B 2512 in several sizes. Smaller versions fit on more computers and lose a little quality, so pick the biggest one that fits.
| Your computer | Runs it? | Version to download |
|---|---|---|
| Graphics card, 8 GB | Yes | Q4 |
| Graphics card, 12 GB | Yes | Q8 |
| Graphics card, 16 GB | Yes | Q8 |
| Graphics card, 24 GB | Yes | Q8 |
| Graphics card, 32 GB | Yes | Q8 |
| Mac, 16 GB | Yes | Q6 |
| Mac, 24 GB | Yes | Q8 |
| Mac, 32 GB | Yes | Q8 |
| Mac, 48 GB | Yes | Q8 |
| Mac, 64 GB | Yes | Q8 |
| Mac, 96 GB | Yes | Q8 |
| Mac, 128 GB | Yes | Q8 |
For normal-length chats. Long documents need more memory. Q4 or Q8? · How we worked this out
How to try it
Search for "Ministral 3 8B 2512 GGUF" in LM Studio or on Hugging Face and pick the version that fits your computer. New to this? Start with our Ollama guide.
Want the strongest all-rounder instead? Qwen3.6 27B scored 97.
Its actual answers
What Ministral 3 8B 2512 wrote for our public questions, word for word.
Coding: INI parser (hard) Didn't work
```python
import re
from typing import Dict, Any
def parse_ini(text: str) -> Dict[str, Dict[str, str]]:
sections: Dict[str, Dict[str, str]] = {}
current_section = None
current_key = None
current_value = []
default_values = {}
lines = text.splitlines()
# Regex patterns
section_pattern = re.compile(r'^\s*\[([^\]]+)\]\s*$')
key_value_pattern = re.compile(r'^(\S+)\s*(=|:)\s*(.*)$')
comment_pattern = re.compile(r'^\s*(?:;|#).+$')
inline_comment_pattern = re.compile(r'^\s*(.*?)\s*(?:;|#)\s*(.*)$')
continuation_pattern = re.compile(r'^\s+(.*)$')
quoted_value_pattern = re.compile(r'^"(.*)"$')
for line in lines:
stripped = line.strip()
if not stripped or comment_pattern.match(line):
continue
# Check for section
section_match = section_pattern.match(line)
if section_match:
current_section = section_match.group(1)
if current_section not in sections:
sections[current_section] = {}
current_key = None
current_value = []
continue
# Check for key-value pair
if current_key is None:
if not current_section:
raise ValueError("Key line before any section header")
kv_match = key_value_pattern.match(line)
if not kv_match:
raise ValueError(f"Malformed line: {line}")
key, sep, value = kv_match.group(1), kv_match.group(2), kv_match.group(3)
current_key = key.lower()
current_value = []
# Handle inline comments
inline_match = inline_comment_pattern.match(line)
if inline_match:
value = inline_match.group(1)
inline_comment = inline_match.group(2)
if inline_comment.strip():
value = value.strip()
# Check for quoted value
if value.startswith('"') and value.endswith('"'):
value = value[1:-1]
current_value.append(value)
else:
# Split on first = or :
value = value.split(sep, 1)[0].strip()
current_value.append(value)
continue
# Handle continuation line
if continuation_pattern.match(line):
continuation = line.strip()
if continuation.startswith('"') and continuation.endswith('"'):
continuation = continuation[1:-1]
current_value.append(continuation)
else:
current_value.append(continuation)
continue
# Handle value assignment (not continuation)
if current_key is not None:
value = line
if value.startswith('"') and value.endswith('"'):
value = value[1:-1]
current_value.append(value)
else:
# Split on first = or :
sep_pos = min(value.find('='), value.find(':'))
if sep_pos != -1:
value = value[:sep_pos].strip()
current_value.append(value)
# Join all parts of the value
joined_value = '\n'.join(current_value).strip()
if current_section:
sections[current_section][current_key] = joined_value
else:
default_values[current_key] = joined_value
current_key = None
current_value = []
# Process DEFAULT section
if 'DEFAULT' in sections:
default_values.update(sections.pop('DEFAULT'))
# Merge DEFAULT values into all sections
for section in sections:
for key in default_values:
if key not in sections[section]:
sections[section][key] = default_values[key]
# Resolve interpolations
def resolve_interpolation(value: str, section: str, current_section: str) -> str:
def replace_match(match):
ref = match.group(1)
if ':' in ref:
target_section, key = ref.split(':', 1)
if target_section not in sections:
raise KeyError(f"Section '{target_section}' not found")
target_value = sections[target_section].get(key)
if target_value is None:
raise KeyError(f"Key '{key}' not found in section '{target_section}'")
return resolve_interpolation(target_value, target_section, current_section)
else:
key = ref
# Check current section first
if key in sections[current_section]:
current_value = sections[current_section][key]
else:
# Check DEFAULT
if key in default_values:
current_value = default_values[key]
else:
raise KeyError(f"Key '{key}' not found in section '{current_section}' or DEFAULT")
return resolve_interpolation(current_value, current_section, current_section)
interpolate_pattern = re.compile(r'\$\{([^}]+)\}')
try:
return interpolate_pattern.sub(replace_match, value)
except RecursionError:
raise ValueError("Interpolation cycle detected")
# Apply interpolation to all values
for section in sections:
for key in sections[section]:
sections[section][key] = resolve_interpolation(sections[section][key], section, section)
return sections
``` Coding: TTL LRU (hard) Didn't work
```python
import time
from collections import OrderedDict
class TTLCache:
def __init__(self, capacity: int, ttl: float, clock):
self.capacity = capacity
self.ttl = ttl
self.clock = clock
self.cache = OrderedDict()
self.stats = {
'hits': 0,
'misses': 0,
'evictions': 0,
'expirations': 0
}
def _clean_expired(self):
now = self.clock()
expired_keys = [k for k, (v, expiry) in self.cache.items() if now >= expiry]
for k in expired_keys:
del self.cache[k]
self.stats['expirations'] += 1
def put(self, key, value):
if self.capacity == 0:
return
now = self.clock()
expiry = now + self.ttl
self._clean_expired()
if key in self.cache:
# Update existing entry
self.cache.move_to_end(key)
self.cache[key] = (value, expiry)
else:
# Insert new entry
if len(self.cache) >= self.capacity:
# Evict LRU entry
self.cache.popitem(last=False)
self.stats['evictions'] += 1
self.cache[key] = (value, expiry)
def get(self, key, default=None):
if self.capacity == 0:
self.stats['misses'] += 1
return default
now = self.clock()
self._clean_expired()
if key not in self.cache:
self.stats['misses'] += 1
return default
value, expiry = self.cache[key]
if now >= expiry:
del self.cache[key]
self.stats['expirations'] += 1
self.stats['misses'] += 1
return default
# Update recency and expiry
self.cache.move_to_end(key)
self.cache[key] = (value, now + self.ttl)
self.stats['hits'] += 1
return value
def __len__(self):
now = self.clock()
count = 0
for v, expiry in self.cache.values():
if now < expiry:
count += 1
return count
def stats(self):
return self.stats.copy()
``` Coding: Parse duration Didn't work
```python
import re
from typing import List, Tuple
def parse_duration(s: str) -> int:
if not s.strip():
raise ValueError("Empty or blank string")
# Normalize whitespace and split into parts
parts = re.split(r'\s+', s.strip())
if not parts:
raise ValueError("Empty or blank string")
# Process each part to extract number and unit
processed_parts: List[Tuple[int, str]] = []
seen_units = set()
for part in parts:
# Match number and unit (case-insensitive)
match = re.fullmatch(r'^(\d+)([hms])$', part.lower(), re.IGNORECASE)
if not match:
raise ValueError(f"Invalid part: '{part}'")
num_str, unit = match.groups()
num = int(num_str)
unit_lower = unit.lower()
if num < 0:
raise ValueError(f"Negative number in part: '{part}'")
if unit_lower in seen_units:
raise ValueError(f"Repeated unit: '{unit}'")
seen_units.add(unit_lower)
processed_parts.append((num, unit_lower))
# Check order of units
expected_order = ['h', 'm', 's']
actual_units = [unit for (_, unit) in processed_parts]
for i, unit in enumerate(actual_units):
if unit != expected_order[i]:
raise ValueError(f"Units out of order. Expected {expected_order[i]} at position {i+1}")
# Calculate total seconds
total_seconds = 0
for num, unit in processed_parts:
if unit == 'h':
total_seconds += num * 3600
elif unit == 'm':
total_seconds += num * 60
elif unit == 's':
total_seconds += num
return total_seconds
``` Coding: Summarize ranges Worked
```python
def summarize_ranges(nums: list[int]) -> str:
if not nums:
return ""
nums = sorted(set(nums))
result = []
i = 0
n = len(nums)
while i < n:
start = nums[i]
j = i + 1
while j < n and nums[j] == nums[j - 1] + 1:
j += 1
if j - i >= 3:
result.append(f"{start}..{nums[j - 1]}")
else:
for k in range(i, j):
result.append(str(nums[k]))
i = j
return ",".join(result)
``` Coding: Top customers Worked
```python
import re
from collections import defaultdict
def top_customers(rows: list[dict], n: int) -> list[tuple[str, float]]:
customer_totals = defaultdict(float)
customer_names = {}
for row in rows:
customer = row.get("customer", "").strip()
amount_str = row.get("amount", "")
if not customer or not amount_str:
continue
# Parse amount
amount_str = amount_str.replace("$", "").replace(",", "")
if not re.fullmatch(r'^[+-]?\d*\.?\d+$', amount_str):
continue
try:
amount = float(amount_str)
except ValueError:
continue
# Normalize customer name
normalized_name = customer.lower().strip()
if normalized_name not in customer_names:
customer_names[normalized_name] = customer.strip()
customer_totals[normalized_name] += amount
# Prepare results
results = []
for name in customer_names:
total = round(customer_totals[name], 2)
results.append((customer_names[name], total))
# Sort by total descending, then name ascending
results.sort(key=lambda x: (-x[1], x[0]))
# Return top n
return results[:n]
``` Coding: Slugify Didn't work
```python
import unicodedata
import re
def slugify(title: str, max_len: int = 50) -> str:
# Step 1: Transliterate accented characters and drop non-ASCII
normalized = unicodedata.normalize('NFKD', title)
ascii_only = ''.join(c for c in normalized if unicodedata.category(c) != 'Mn')
transliterated = ''.join(
c if unicodedata.category(c) == 'Ll' or c.isdigit()
else c.lower() if c.isalpha() else ' '
for c in ascii_only
)
# Step 2: Lowercase (already done in transliteration)
# Step 3: Replace non-alphanumeric runs with single "-"
slug = re.sub(r'[^a-z0-9]+', '-', transliterated).strip('-')
# Step 4: Handle max_len
if len(slug) > max_len:
words = slug.split('-')
prefix = []
current_len = 0
for word in words:
if current_len + len(word) + (1 if prefix else 0) <= max_len:
prefix.append(word)
current_len += len(word) + (1 if prefix else 0)
else:
break
if not prefix:
return slug[:max_len]
slug = '-'.join(prefix)
if len(slug) > max_len:
# Hard-cut if even first word is too long
slug = slug[:max_len]
# Ensure no trailing "-"
return slug.rstrip('-')
``` Coding: Token bucket Didn't work
```python
import time
class TokenBucket:
def __init__(self, capacity: float, refill_per_sec: float, clock):
self.capacity = capacity
self.refill_per_sec = refill_per_sec
self.clock = clock
self.tokens = capacity
self.last_refill_time = self.clock()
def _refill(self):
now = self.clock()
elapsed = now - self.last_refill_time
self.last_refill_time = now
refill_amount = elapsed * self.refill_per_sec
self.tokens = min(self.capacity, self.tokens + refill_amount)
@property
def tokens(self):
self._refill()
return self.tokens
def allow(self, cost: float = 1) -> bool:
self._refill()
if self.tokens >= cost:
self.tokens -= cost
return True
return False
``` Decisions: Refund window (hard) 100% right
```json
{
"outcome": {
"full_refund": 0.0,
"partial_refund": 0.0,
"store_credit": 0.95,
"nothing": 0.05
},
"defective": {
"true": 0.05,
"false": 0.95
}
}
``` Decisions: Incident matrix (hard) 0% right
```json
{
"severity": {
"0": 0.0,
"1": 0.0,
"2": 0.95,
"3": 0.05
},
"page": {
"true": 0.95,
"false": 0.05
}
}
``` Decisions: Tool followup (hard) 100% right
```json
{
"tool": {
"calendar": 0.95,
"none": 0.05
},
"confirm": {
"true": 0.99,
"false": 0.01
}
}
``` Decisions: Legit security alert (hard) 100% right
```json
{
"phishing": {
"true": 0.05,
"false": 0.95
},
"action_needed": {
"true": 0.05,
"false": 0.95
}
}
``` Decisions: Meeting slot (hard) 50% right
```json
{
"slot": {
"A": 0.0,
"B": 0.0,
"C": 1.0,
"D": 0.0
},
"raj_last": {
"true": 0.95,
"false": 0.05
}
}
``` Decisions: Review mixed (hard) 100% right
```json
{
"hardware": {
"true": 0.95,
"false": 0.05
},
"support": {
"true": 0.99,
"false": 0.01
}
}
``` Decisions: Support checkout down 100% right
```json
{
"department": {
"billing": 0.05,
"technical": 0.9,
"account": 0.02,
"shipping": 0.01,
"sales": 0.02
},
"urgency": {
"0": 0.0,
"1": 0.0,
"2": 0.05,
"3": 0.95
},
"outage": {
"true": 0.99,
"false": 0.01
}
}
``` Decisions: Refund wrong plan 100% right
```json
{
"department": {
"billing": 0.95,
"account": 0.05,
"technical": 0,
"shipping": 0,
"sales": 0
},
"refund": {
"true": 0.99,
"false": 0.01
},
"tone": {
"frustrated": 0.1,
"calm": 0.9
}
}
``` Decisions: Moderation doxxing 100% right
```json
{
"policy": {
"harassment": 0.95,
"none": 0.05
},
"personal_info": {
"true": 0.99,
"false": 0.01
}
}
``` Decisions: Route calendar 100% right
```json
{
"tool": {
"web_search": 0.01,
"calculator": 0.01,
"calendar": 0.95,
"email": 0.02,
"none": 0.01
},
"confirm": {
"true": 0.99,
"false": 0.01
}
}
``` Decisions: Doc invoice missing due 100% right
```json
{
"doc_type": {
"invoice": 0.99,
"resume": 0.005,
"contract": 0.003,
"bank_statement": 0.001,
"other": 0.001
},
"missing_due_date": {
"true": 0.999,
"false": 0.001
}
}
``` Decisions: Phishing paypal 100% right
```json
{
"phishing": {
"true": 0.999,
"false": 0.001
},
"risk": {
"0": 0.0,
"1": 0.0,
"2": 0.001,
"3": 0.999
}
}
``` Decisions: Pii ssn email 100% right
```json
{
"data_kind": {
"none": 0.0,
"contact": 0.1,
"financial": 0.0,
"government_id": 0.85,
"health": 0.05
},
"sensitive": {
"true": 0.95,
"false": 0.05
}
}
``` Decisions: Review mixed 100% right
```json
{
"sentiment": {
"positive": 0.05,
"neutral": 0.05,
"negative": 0.9
},
"defect": {
"true": 0.95,
"false": 0.05
},
"recommend": {
"true": 0.05,
"false": 0.95
}
}
``` Documents: Saas escalator (hard) 20% right
```json
{
"year2_price_per_seat_month": 47.19,
"year3_price_per_seat_month": 47.19,
"year1_invoice": 64800.00,
"year2_invoice": 68628.00,
"addon_months_billed": 6,
"addon_invoice": 7542.80,
"year3_invoice": 74008.00,
"year3_discount_percent": 15.00,
"total_contract_value": 210838.80,
"contract_end_date": "2026-02-29"
}
``` Documents: Expense thread 67% right
```json
{
"employee_id": "EMP-20417",
"destination_city": "Lisbon",
"trip_start": "2025-02-24",
"trip_end": "2025-02-27",
"approved_items": [
{
"date": "2025-02-24",
"category": "airfare",
"amount_usd": 1184.6
},
{
"date": "2025-02-24",
"category": "ground_transport",
"amount_usd": 38.72
},
{
"date": "2025-02-25",
"category": "meals",
"amount_usd": 230.19
},
{
"date": "2025-02-26",
"category": "lodging",
"amount_usd": 468.6
},
{
"date": "2025-02-27",
"category": "ground_transport",
"amount_usd": 44.54
}
],
"rejected_item_count": 2,
"per_diem_days": 2,
"per_diem_usd": 65.0,
"total_reimbursable_usd": 2476.75,
"approver_email": "priya.raman@corvane.com"
}
``` Documents: Lease amendment 83% right
```json
{
"tenants": ["Marcus Lin", "Sofia Lin"],
"landlord": "Ridgeline Property Group LLC",
"zip": "97205",
"lease_end": "2025-11-30",
"original_monthly_rent": 2150.00,
"monthly_rent_from_2025_06_01": 2236.00,
"late_fee_from_2025_06_01": 111.80,
"security_deposit": 2150.00,
"total_pet_deposits": 800.00,
"total_monthly_payment_july_2025": 2376.00,
"move_in_payment": 2900.00
}
``` Documents: Ticket SLA 82% right
```json
{
"ticket_id": "48213",
"account_id": "ACC-7731",
"open_issue": "inventory_sync",
"resolved_issues": ["billing_address"],
"affected_orders": ["SO-99812", "SO-99820", "SO-99827"],
"priority": "P2",
"sla_due_local": "2025-09-15T16:00",
"sla_due_utc": "2025-09-16T00:00:00Z",
"reissued_invoice": "INV-2025-0812"
}
``` Documents: Sales footnotes 56% right
```json
{
"q3_total_usd": 15741000,
"q2_total_usd": 15464000,
"q2_central_originally_reported_usd": 3047000,
"q2_to_q3_change_pct": 1.8,
"top_region_q3": "East",
"fastest_growing_region_q1_to_q3": "International",
"regions_declining_q2_to_q3": ["Central"],
"international_q3_organic_usd": 1731000,
"west_excluding_mountain_q3_usd": 4201000
}
``` Size: 8.9B parameters. First tested OCT 10.
Comments
Sign in with GitHub to comment. Spam and abuse are hidden automatically.