- Llama 4 Scout is a free model from Meta that you can run on your own computer. In our tests it's not one we'd recommend right now: 34 out of 100, #49 of 63.
- It solved 9 of 30 coding jobs and scored 38 on reading documents. On our hardest tasks it scored 12.
- Runs on a Mac with 96 GB.
Coding
Our coding test is 30 programming jobs, from small ones like reading time durations or cleaning up messy data to harder ones like a config-file parser or a double-entry ledger. We run each answer against tests the model never sees, and a job only counts if everything passes. Llama 4 Scout got 9 of 30 right. The best local coders solved 29 of 30.
Reading documents
The second test hands the model things like an expense claim thread, a pay stub or an insurance statement, and asks for specific numbers and dates. Many questions need a bit of math, or noticing a correction further down the email. Llama 4 Scout scored 38; the best model scored 100.
| Test | Score | Public questions | Secret questions |
|---|---|---|---|
| Coding | 30 | 29 | 30 |
| Reading documents | 38 | 39 | 38 |
| Decisions | 81 | 83 | 80 |
On the 18 hardest tasks (included in the scores above) it scored 12. This number separates the top models.
We tested the full-size version online. The smaller versions you download usually score a little lower.
Can your computer run it?
You can download Llama 4 Scout in several sizes. Smaller versions fit on more computers and lose a little quality, so pick the biggest one that fits.
| Your computer | Runs it? | Version to download |
|---|---|---|
| Graphics card, 8 GB | No | – |
| Graphics card, 12 GB | No | – |
| Graphics card, 16 GB | No | – |
| Graphics card, 24 GB | No | – |
| Graphics card, 32 GB | No | – |
| Mac, 16 GB | No | – |
| Mac, 24 GB | No | – |
| Mac, 32 GB | No | – |
| Mac, 48 GB | No | – |
| Mac, 64 GB | No | – |
| Mac, 96 GB | Yes | Q4 |
| Mac, 128 GB | Yes | Q6 |
For normal-length chats. Long documents need more memory. Q4 or Q8? · How we worked this out
How to try it
Search for "Llama 4 Scout GGUF" in LM Studio or on Hugging Face and pick the version that fits your computer. New to this? Start with our Ollama guide.
Want the strongest all-rounder instead? Qwen3.6 27B scored 97.
Its actual answers
What Llama 4 Scout wrote for our public questions, word for word.
Coding: INI parser (hard) Didn't work
```python
import re
from typing import Dict
def parse_ini(text: str) -> Dict[str, Dict[str, str]]:
"""
Parse an INI file and return a dictionary of sections with their keys and values.
:param text: The content of the INI file.
:return: A dictionary where each key is a section name and each value is another dictionary.
The inner dictionary maps keys to values.
"""
config: Dict[str, Dict[str, str]] = {}
current_section: str | None = None
stack: list[tuple[str, str | None]] = []
for line in text.splitlines():
line = line.rstrip()
if not line or line.lstrip().startswith(('#', ';')):
continue
# Section header
if line.startswith('[') and line.endswith(']'):
name = line[1:-1].strip()
if name in config:
raise ValueError(f"Section {name!r} already exists")
config[name] = {}
current_section = name
continue
# Key line
match = re.match(r'\s*([^\s=:]*)(\s*[:=])\s*(.*)', line)
if match:
key, _, value = match.groups()
key = key.strip().lower()
if current_section is None:
raise ValueError("Key before section header")
# Continuation line
if key == '':
if current_section is None:
raise ValueError("Continuation line before section header")
if not config[current_section]:
raise ValueError("Continuation line without preceding key")
config[current_section][list(config[current_section].keys())[-1]] += '\n' + value.strip()
continue
# Inline comment
comment_match = re.search(r'\s*[;#](.*)', value)
if comment_match:
value = value[:comment_match.start()].strip()
# Quoted value
if value.startswith('"') and value.endswith('"'):
value = value[1:-1].replace('\\"', '"')
# Interpolation
value = interpolate(config, current_section, key, value)
config[current_section][key] = value
continue
raise ValueError(f"Malformed line: {line!r}")
return config
def interpolate(config: Dict[str, Dict[str, str]], current_section: str | None, key: str, value: str) -> str:
"""
Perform interpolation on a value.
:param config: The configuration dictionary.
:param current_section: The current section.
:param key: The key.
:param value: The value.
:return: The interpolated value.
"""
def replace(match: re.Match[str]) -> str:
full_match = match.group(0)
section, key = (match.group(1), match.group(2)) if match.group(1) else (current_section, match.group(2))
if section is None:
raise KeyError(f"Missing section in interpolation: {full_match!r}")
if key not in config[section]:
if section == 'DEFAULT':
raise KeyError(f"Missing key {key!r} in DEFAULT section")
if section not in config:
raise KeyError(f"Missing section {section!r}")
if key not in config[section]:
raise KeyError(f"Missing key {key!r} in section {section!r}")
return config[section][key]
# Detect reference cycle
stack.append((current_section, key))
try:
value = re.sub(r'\${(.*?)}', lambda match: replace(match), value)
finally:
stack.pop()
if len(stack) > 1 and (stack[-1][0], stack[-1][1]) == stack[-2]:
raise ValueError("Interpolation reference cycle detected")
return value
``` Coding: TTL LRU (hard) Didn't work
```python
from collections import OrderedDict
from typing import Callable, Dict, Optional
class TTLCache:
def __init__(self, capacity: int, ttl: float, clock: Callable[[], float]):
self.capacity = capacity
self.ttl = ttl
self.clock = clock
self.cache: OrderedDict[str, (float, object)] = OrderedDict()
self.stats = {"hits": 0, "misses": 0, "evictions": 0, "expirations": 0}
def _remove_expired(self) -> None:
now = self.clock()
while self.cache and self.cache[next(iter(self.cache))] [0] <= now:
key, _ = self.cache.popitem(last=False)
self.stats["expirations"] += 1
def _evict(self) -> None:
key, _ = self.cache.popitem(last=False)
self.stats["evictions"] += 1
def put(self, key: str, value: object) -> None:
if self.capacity == 0:
return
now = self.clock()
self._remove_expired()
if len(self.cache) >= self.capacity:
self._evict()
self.cache[key] = (now + self.ttl, value)
def get(self, key: str, default: Optional[object] = None) -> Optional[object]:
if self.capacity == 0:
self.stats["misses"] += 1
return default
now = self.clock()
self._remove_expired()
if key in self.cache:
expiry, value = self.cache.pop(key)
if expiry > now:
self.cache[key] = (expiry, value)
self.stats["hits"] += 1
return value
else:
self.stats["expirations"] += 1
self.stats["misses"] += 1
return default
def __len__(self) -> int:
now = self.clock()
return sum(1 for expiry, _ in self.cache.values() if expiry > now)
def stats(self) -> Dict[str, int]:
return self.stats.copy()
``` Coding: Parse duration Didn't work
```python
import re
from typing import Dict
def parse_duration(s: str) -> int:
"""
Converts a duration string into a total number of seconds.
Args:
s (str): A duration string.
Returns:
int: The total number of seconds.
Raises:
ValueError: If the input string is empty/blank, or if it contains invalid parts.
"""
# Remove leading and trailing whitespace
s = s.strip()
# Check for empty/blank strings
if not s:
raise ValueError("Input string is empty")
# Regular expression pattern to match a duration part
pattern = r"(\d+)([hms])"
parts = re.findall(pattern, s)
# Check for invalid parts
if len(parts) != len(set(part[1] for part in parts)):
raise ValueError("Repeated units are not allowed")
# Mapping of units to their conversion factors
units: Dict[str, int] = {"h": 3600, "m": 60, "s": 1}
# Check for unknown units and units out of order
previous_unit = None
total_seconds = 0
for num, unit in parts:
num = int(num)
if unit.lower() not in units:
raise ValueError(f"Unknown unit: {unit}")
if previous_unit and units[unit.lower()] <= units[previous_unit]:
raise ValueError("Units must appear in the order h, m, s")
total_seconds += num * units[unit.lower()]
previous_unit = unit.lower()
return total_seconds
``` Coding: Summarize ranges Worked
```python
def summarize_ranges(nums: list[int]) -> str:
"""
This function takes a list of integers, removes duplicates, sorts them,
and then collapses runs of 3 or more consecutive integers into "a..b".
Numbers in shorter runs (1 or 2 numbers) are listed individually.
Args:
nums (list[int]): A list of integers.
Returns:
str: A string representing the summarized ranges of integers.
"""
# Remove duplicates and sort the list
nums = sorted(set(nums))
# Initialize variables to store the result and the start of the current range
result = []
range_start = None
# Iterate over the sorted list of numbers
for i, num in enumerate(nums):
# If this is the first number or it's not consecutive to the previous number,
# it starts a new range
if i == 0 or num != nums[i - 1] + 1:
# If we've been building a range, add it to the result
if range_start is not None:
if nums[i - 1] == range_start:
result.append(str(range_start))
elif nums[i - 1] == range_start + 1:
result.extend([str(range_start), str(range_start + 1)])
else:
result.append(f"{range_start}..{nums[i - 1]}")
range_start = None
# Start a new range
range_start = num
# Add the last range to the result
if range_start is not None:
if nums[-1] == range_start:
result.append(str(range_start))
elif nums[-1] == range_start + 1:
result.extend([str(range_start), str(range_start + 1)])
else:
result.append(f"{range_start}..{nums[-1]}")
# Join the result with commas and return it
return ",".join(result)
``` Coding: Top customers Didn't work
```python
from typing import List, Dict, Tuple
from collections import defaultdict
import re
def top_customers(rows: List[Dict], n: int) -> List[Tuple[str, float]]:
"""
Returns the top n customers by total amount.
:param rows: A list of dictionaries, each containing a customer name and an amount.
:param n: The number of top customers to return.
:return: A list of tuples, each containing a customer name and their total amount.
"""
# Create a dictionary to store the total amount for each customer
customer_amounts = defaultdict(float)
# Regular expression to match and extract the amount from a string
amount_regex = re.compile(r'^-?\$?(\d{1,3}(?:,\d{3})*)(?:\.\d+)?$')
# Iterate over each row
for row in rows:
# Get the customer name and amount from the row
customer = row.get("customer", "").strip()
amount_str = row.get("amount", "")
# Skip rows with empty customer name or amount
if not customer or not amount_str:
continue
# Try to parse the amount
match = amount_regex.match(amount_str)
if match:
# Remove commas and convert to float
amount = float(match.group(1).replace(",", ""))
# Update the customer's total amount
customer_amounts[customer] += amount
# Round totals to 2 decimals and store in a list of tuples
customer_amounts = [(customer, round(amount, 2)) for customer, amount in customer_amounts.items()]
# Sort the customers by total amount descending, then name ascending
customer_amounts.sort(key=lambda x: (-x[1], x[0]))
# Return the top n customers
return customer_amounts[:n]
``` Coding: Slugify Didn't work
```python
import unicodedata
import re
def slugify(title: str, max_len: int = 50) -> str:
"""
Builds a URL slug.
1. Transliterate accented characters to ASCII (é -> e, ü -> u) and drop any other non-ASCII characters.
2. Lowercase.
3. Replace every run of characters other than a-z and 0-9 with a single "-", and strip leading/trailing "-".
4. If the slug is longer than `max_len`, shorten it without cutting a word: keep the longest prefix of whole words
(words are separated by "-") whose length is <= max_len. If even the first word is longer than max_len,
hard-cut it to max_len characters.
5. The result never ends with "-".
Args:
title (str): The title to slugify.
max_len (int, optional): The maximum length of the slug. Defaults to 50.
Returns:
str: The slugified title.
"""
# Transliterate accented characters to ASCII and drop any other non-ASCII characters
title = ''.join(c for c in unicodedata.normalize('NFD', title)
if unicodedata.category(c) != 'Mn' and c.isascii())
# Lowercase
title = title.lower()
# Replace every run of characters other than a-z and 0-9 with a single "-", and strip leading/trailing "-"
title = re.sub(r'[^\w]+', '-', title).strip('-')
# If the slug is longer than `max_len`, shorten it without cutting a word
if len(title) > max_len:
words = title.split('-')
prefix_len = 0
for word in words:
if prefix_len + len(word) + 1 > max_len:
break
prefix_len += len(word) + 1
title = '-'.join(words[:len(words) - 1] + [words[-1][:max_len - prefix_len]]) if prefix_len < max_len else title[:max_len]
return title
``` Coding: Token bucket Worked
```python
from typing import Callable
class TokenBucket:
def __init__(self, capacity: float, refill_per_sec: float, clock: Callable[[], float]):
self.capacity = capacity
self.refill_per_sec = refill_per_sec
self.clock = clock
self.tokens = capacity
self.last_refill = clock()
@property
def tokens(self) -> float:
self.refill()
return self._tokens
@tokens.setter
def tokens(self, value: float):
self._tokens = value
def refill(self) -> None:
current_time = self.clock()
elapsed_time = current_time - self.last_refill
self.last_refill = current_time
refill_amount = elapsed_time * self.refill_per_sec
self._tokens = min(self.capacity, self._tokens + refill_amount)
def allow(self, cost: float = 1) -> bool:
self.refill()
if self._tokens >= cost:
self._tokens -= cost
return True
return False
``` Decisions: Refund window (hard) 50% right
```json
{
"outcome": {
"full_refund": 0,
"partial_refund": 0.75,
"store_credit": 0.25,
"nothing": 0
},
"defective": {
"true": 0,
"false": 1
}
}
``` Decisions: Incident matrix (hard) 100% right
```
{
"severity": {
"0": 0,
"1": 0.7,
"2": 0.3,
"3": 0
},
"page": {
"true": 0.3,
"false": 0.7
}
}
``` Decisions: Tool followup (hard) 100% right
```
{
"tool": {
"web_search": 0,
"calculator": 0,
"calendar": 1,
"email": 0,
"none": 0
},
"confirm": {
"true": 0.5,
"false": 0.5
}
}
``` Decisions: Legit security alert (hard) 100% right
```
{
"phishing": {
"true": 0.05,
"false": 0.95
},
"action_needed": {
"true": 0.05,
"false": 0.95
}
}
``` Decisions: Meeting slot (hard) 0% right
```json
{
"slot": {
"A": 0.0,
"B": 0.25,
"C": 0.75,
"D": 0.0
},
"raj_last": {
"true": 0.0,
"false": 1.0
}
}
``` Decisions: Review mixed (hard) 100% right
```json
{
"hardware": {
"true": 0.8,
"false": 0.2
},
"support": {
"true": 0.9,
"false": 0.1
}
}
``` Decisions: Support checkout down 67% right
```
{
"department": {
"billing": 0.7,
"technical": 0.3,
"account": 0,
"shipping": 0,
"sales": 0
},
"urgency": {
"0": 0,
"1": 0,
"2": 0.2,
"3": 0.8
},
"outage": {
"true": 0.9,
"false": 0.1
}
}
``` Decisions: Refund wrong plan 100% right
```
{
"department": {
"billing": 0.9,
"technical": 0.05,
"account": 0.05,
"shipping": 0,
"sales": 0
},
"refund": {
"true": 0.8,
"false": 0.2
},
"tone": {
"frustrated": 0.1,
"calm": 0.9
}
}
``` Decisions: Moderation doxxing 100% right
```json
{
"policy": {
"none": 0,
"harassment": 0.9,
"hate": 0,
"spam": 0,
"self_harm": 0.1
},
"personal_info": {
"true": 0.9,
"false": 0.1
}
}
``` Decisions: Route calendar 100% right
```
{
"tool": {
"web_search": 0,
"calculator": 0,
"calendar": 0.9,
"email": 0,
"none": 0.1
},
"confirm": {
"true": 0.8,
"false": 0.2
}
}
``` Decisions: Doc invoice missing due 100% right
```json
{
"doc_type": {
"invoice": 0.99,
"resume": 0.005,
"contract": 0.0,
"bank_statement": 0.0,
"other": 0.005
},
"missing_due_date": {
"true": 0.8,
"false": 0.2
}
}
``` Decisions: Phishing paypal 100% right
```
{
"phishing": {
"true": 0.9,
"false": 0.1
},
"risk": {
"0": 0.05,
"1": 0.05,
"2": 0.3,
"3": 0.6
}
}
``` Decisions: Pii ssn email 50% right
```json
{
"data_kind": {
"none": 0,
"contact": 0.33,
"financial": 0.33,
"government_id": 0.33,
"health": 0
},
"sensitive": {
"true": 1,
"false": 0
}
}
``` Decisions: Review mixed 100% right
```json
{
"sentiment": {
"positive": 0.05,
"neutral": 0.05,
"negative": 0.9
},
"defect": {
"true": 0.9,
"false": 0.1
},
"recommend": {
"true": 0.05,
"false": 0.95
}
}
``` Documents: Saas escalator (hard) 20% right
```
{
"year2_price_per_seat_month": 47.7,
"year3_price_per_seat_month": 47.66,
"year1_invoice": 43200.00,
"year2_invoice": 45816.00,
"addon_months_billed": 6,
"addon_invoice": 28596.00,
"year3_invoice": 17153.92,
"year3_discount_percent": 15,
"total_contract_value": 134765.92,
"contract_end_date": "2027-02-29"
}
``` Documents: Expense thread 0% right
```
{
"employee_id": "EMP-20417",
"destination_city": "Lisbon",
"trip_start": "2025-02-24",
"trip_end": "2025-02-27",
"approved_items": [
{
"date": "2025-02-24",
"category": "airfare",
"amount_usd": 1184.60
},
{
"date": "2025-02-24",
"category": "ground_transport",
"amount_usd": 36.00 * 1.08
},
{
"date": "2025-02-25",
"category": "meals",
"amount_usd": 212.40 * 1.08
},
{
"date": "2025-02-26",
"category": "lodging",
"amount_usd": 432.00 * 1.08
},
{
"date": "2025-02-27",
"category": "ground_transport",
"amount_usd": 41.50 * 1.08
}
],
"rejected_item_count": 2,
"per_diem_days": 2,
"per_diem_usd": 65.00 * 2,
"total_reimbursable_usd": null,
"approver_email": "priya.raman@corvane.com"
}
``` Documents: Lease amendment 83% right
```json
{
"tenants": ["Marcus Lin", "Sofia Lin"],
"landlord": "Ridgeline Property Group LLC",
"zip": "97205",
"lease_end": "2025-11-30",
"original_monthly_rent": 2150.00,
"monthly_rent_from_2025_06_01": 2236.00,
"late_fee_from_2025_06_01": 111.80,
"security_deposit": 2150.00,
"total_pet_deposits": 800.00,
"total_monthly_payment_july_2025": 2287.00,
"move_in_payment": 4300.00
}
``` Documents: Ticket SLA 75% right
```
{
"ticket_id": "48213",
"account_id": "ACC-7731",
"open_issue": "inventory_sync",
"resolved_issues": ["billing_address", "invoice_pdf"],
"affected_orders": ["SO-99812", "SO-99820", "SO-99827"],
"priority": "P2",
"sla_due_local": "2025-09-12T17:00",
"sla_due_utc": "2025-09-13T00:00:00Z",
"reissued_invoice": "INV-2025-0812"
}
``` Documents: Sales footnotes 18% right
```
{
"q3_total_usd": 15458,
"q2_total_usd": 14264,
"q2_central_originally_reported_usd": 3047,
"q2_to_q3_change_pct": 8.4,
"top_region_q3": "East",
"fastest_growing_region_q1_to_q3": "International",
"regions_declining_q2_to_q3": ["Central", "East", "West"],
"international_q3_organic_usd": 1731,
"west_excluding_mountain_q3_usd": 4201
}
``` Size: 109B parameters. First tested OCT 11.
Comments
Sign in with GitHub to comment. Spam and abuse are hidden automatically.