- Nemotron 3.5 Lightning is a free model from NVIDIA that you can run on your own computer. In our tests it's not one we'd recommend right now: 57 out of 100, #26 of 56.
- It solved 13 of 30 coding jobs and scored 72 on reading documents. On our hardest tasks it scored 43.
- Runs on a 24 GB graphics card or a Mac with 32 GB.
Coding
Our coding test is 30 programming jobs, from small ones like reading time durations or cleaning up messy data to harder ones like a config-file parser or a double-entry ledger. We run each answer against tests the model never sees, and a job only counts if everything passes. Nemotron 3.5 Lightning got 13 of 30 right. The best local coders solved 29 of 30.
Reading documents
The second test hands the model things like an expense claim thread, a pay stub or an insurance statement, and asks for specific numbers and dates. Many questions need a bit of math, or noticing a correction further down the email. Nemotron 3.5 Lightning scored 72; the best model scored 100.
| Test | Score | Public questions | Secret questions |
|---|---|---|---|
| Coding | 43 | 43 | 44 |
| Reading documents | 72 | 76 | 70 |
| Decisions | 80 | 89 | 77 |
On the 18 hardest tasks (included in the scores above) it scored 43. This number separates the top models.
We tested the full-size version online. The smaller versions you download usually score a little lower.
Can your computer run it?
You can download Nemotron 3.5 Lightning in several sizes. Smaller versions fit on more computers and lose a little quality, so pick the biggest one that fits.
| Your computer | Runs it? | Version to download |
|---|---|---|
| Graphics card, 8 GB | No | – |
| Graphics card, 12 GB | No | – |
| Graphics card, 16 GB | No | – |
| Graphics card, 24 GB | Yes | Q5 |
| Graphics card, 32 GB | Yes | Q6 |
| Mac, 16 GB | No | – |
| Mac, 24 GB | No | – |
| Mac, 32 GB | Yes | Q4 |
| Mac, 48 GB | Yes | Q8 |
| Mac, 64 GB | Yes | Q8 |
| Mac, 96 GB | Yes | Q8 |
| Mac, 128 GB | Yes | Q8 |
For normal-length chats. Long documents need more memory. Q4 or Q8? · How we worked this out
How to try it
Search for "Nemotron 3.5 Lightning GGUF" in LM Studio or on Hugging Face and pick the version that fits your computer. New to this? Start with our Ollama guide.
Want the strongest all-rounder instead? Qwen3.6 27B scored 97.
Its actual answers
What Nemotron 3.5 Lightning wrote for our public questions, word for word.
Coding: INI parser (hard) Didn't work
Coding: TTL LRU (hard) Didn't work
```python
from __future__ import annotations
from typing import Any
class TTLCache:
"""LRU cache with per-entry TTL expiration.
Args:
capacity: Maximum number of entries. If 0 the cache never stores anything.
ttl: Time-to-live in seconds; an entry expires when ``now >= expiry``.
clock: Zero-argument callable returning the current time in seconds.
"""
def __init__(self, capacity: int, ttl: float, clock: Any) -> None:
if capacity < 0:
raise ValueError("capacity must be non-negative")
if ttl < 0:
raise ValueError("ttl must be non-negative")
self._capacity = capacity
self._ttl = ttl
self._clock = clock
# Ordered dict preserves insertion/access order for LRU.
# Each value is a tuple (stored_value, expiry_time).
self._data: dict[str, tuple[Any, float]] = {}
# Stats counters.
self._hits = 0
self._misses = 0
self._evictions = 0
self._expirations = 0
# ------------------------------------------------------------------
# Helper: purge all entries that have expired as of the current clock.
# ------------------------------------------------------------------
def _expire_stale(self) -> None:
now = self._clock()
# Remove every entry whose expiry time has been reached.
expired_keys = [k for k, (_, exp) in self._data.items() if exp <= now]
for k in expired_keys:
self._expirations += 1
del self._data[k]
# ------------------------------------------------------------------
# put
# ------------------------------------------------------------------
def put(self, key: str, value: Any) -> None:
# capacity 0: nothing is stored.
if self._capacity == 0:
return
now = self._clock()
# If the key already exists, refresh it and move it to the front
# (most-recently used). We treat it as an update rather than an
# insertion for capacity purposes.
if key in self._data:
# Bump the expiry to now + ttl.
self._data[key] = (value, now + self._ttl)
# Move the key to the end of the ordered dict (most recent).
# OrderedDict.move_to_end is O(1).
self._data.move_to_end(key)
return
# Key is new – first purge expired entries to free space.
self._expire_stale()
# If we are still at capacity, evict the least recently used entry.
if len(self._data) >= self._capacity:
# popitem(last=False) removes the LRU (first) item.
lru_key, _ = self._data.popitem(last=False)
self._evictions += 1
# Insert the new entry with expiry now + ttl.
self._data[key] = (value, now + self._ttl)
# It is already at the end (most recent) because we inserted after
# evicting, but we explicitly move it to be safe.
self._data.move_to_end(key)
# ------------------------------------------------------------------
# get
# ------------------------------------------------------------------
def get(self, key: str, default: Any = None) -> Any:
if self._capacity == 0:
self._misses += 1
return default
now = self._clock()
if key not in self._data:
self._misses += 1
return default
value, expiry = self._data[key]
# Entry expired?
if now >= expiry:
# Remove it, count as an expiration AND a miss.
self._expirations += 1
del self._data[key]
self._misses += 1
return default
# Entry is valid – promote to most-recently used.
self._data.move_to_end(key)
self._hits += 1
return value
# ------------------------------------------------------------------
# __len__
# ------------------------------------------------------------------
def __len__(self) -> int:
# Count only non‑expired entries.
now = self._clock()
count = 0
for _key, (_, exp) in self._data.items():
if exp > now:
count += 1
return count
# ------------------------------------------------------------------
# stats
# ------------------------------------------------------------------
def stats(self) -> dict[str, int]:
return {
"hits": self._hits,
"misses": self._misses,
"evictions": self._evictions,
"expirations": self._expirations,
}
``` Coding: Parse duration Didn't work
Coding: Summarize ranges Worked
```python
def summarize_ranges(nums: list[int]) -> str:
if not nums:
return ""
# Sort and remove duplicates
sorted_nums = sorted(set(nums))
# Group into runs of consecutive integers
groups: list[list[int]] = []
current: list[int] = [sorted_nums[0]]
for n in sorted_nums[1:]:
if n == current[-1] + 1:
current.append(n)
else:
groups.append(current)
current = [n]
groups.append(current)
# Build the result string
parts: list[str] = []
for g in groups:
if len(g) >= 3:
parts.append(f"{g[0]}..{g[-1]}")
else:
parts.append(",".join(str(x) for x in g))
return ",".join(parts)
``` Coding: Top customers Didn't work
```python
import re
from collections import OrderedDict
from typing import Any
def top_customers(rows: list[dict[str, Any]], n: int) -> list[tuple[str, float]]:
"""
Return top n customers as (name, total) tuples sorted by total descending,
then name ascending.
- Customer names are trimmed and matched case-insensitively; the trimmed
spelling of the first valid row for each customer is used.
- Rows with an empty customer name or an unparsable amount are skipped.
- Amounts are strings such as "$1,234.50", "1234.5", "-$5.00" (negative = refund).
- Totals are rounded to 2 decimal places.
"""
# Normalise: strip whitespace from name/amount and keep first occurrence per normalized name
customers: dict[str, tuple[str, float]] = {} # key -> (display_name, total)
for row in rows:
# Guard against missing keys
name_raw = row.get("customer", "")
amount_raw = row.get("amount", "")
# Trim the name
name = name_raw.strip()
if not name:
continue # skip empty customer name
# Normalise name for matching: lower + stripped
name_key = name.lower().strip()
# Parse amount
amount_str = amount_raw.strip()
if not amount_str:
continue # skip unparsable amount
# Remove surrounding whitespace, then strip leading $ if present
# The regex below expects optional leading minus, optional $, digits, optional comma, optional decimal
# We'll just attempt a conversion after cleaning.
# Remove commas for float parsing
cleaned = amount_str.replace(",", "")
# Attempt float conversion; if it fails, skip row
try:
value = float(cleaned)
except ValueError:
continue
# Ensure we actually have a numeric result (e.g., string was just "$")
# float() on "$12.3" would raise, but we guard above.
# Determine if this is the first time we see this customer
if name_key not in customers:
# Store the trimmed spelling of the first valid row
customers[name_key] = (name, value)
else:
# Add to existing total
existing_name, existing_total = customers[name_key]
new_total = round(existing_total + value, 2)
customers[name_key] = (existing_name, new_total)
# Build list of (name, total) tuples
result: list[tuple[str, float]] = [
(display_name, total) for display_name, total in customers.values()
]
# Sort by total descending, then name ascending (case‑insensitive but using stored name)
result.sort(key=lambda x: (-x[1], x[0].lower()))
# Return top n (or fewer if not enough customers)
return result[:n]
``` Coding: Slugify Worked
```python
import unicodedata
import re
def slugify(title: str, max_len: int = 50) -> str:
# Step 1: Transliterate accented characters to ASCII and drop other non-ASCII.
normalized = unicodedata.normalize("NFKD", title)
ascii_only = normalized.encode("ascii", "ignore").decode("ascii")
# Step 2: Lowercase.
lowered = ascii_only.lower()
# Step 3: Replace every run of non-alphanumeric characters with a single "-"
# and strip leading/trailing "-".
replaced = re.sub(r'[^a-z0-9]+', '-', lowered)
stripped = replaced.strip('-')
# If the result is empty, return an empty string.
if not stripped:
return ''
# Step 4: If longer than max_len, shorten without cutting a word.
if len(stripped) > max_len:
words = stripped.split('-')
# Collect whole words from the start whose total length (including separators)
# does not exceed max_len.
selected = []
current_len = 0
for word in words:
# Adding a word: if we already have words, we need a '-' separator.
added_len = len(word) + (1 if selected else 0)
if current_len + added_len <= max_len:
selected.append(word)
current_len += added_len
else:
break
if selected:
slug = '-'.join(selected)
else:
# Even the first word is too long; hard-cut it to max_len characters.
slug = stripped[:max_len]
else:
slug = stripped
# Step 5: The result never ends with "-".
slug = slug.rstrip('-')
return slug
``` Coding: Token bucket Worked
```python
class TokenBucket:
def __init__(self, capacity: float, refill_per_sec: float, clock) -> None:
self._capacity = capacity
self._refill_per_sec = refill_per_sec
self._clock = clock
self._tokens = capacity
self._last_refill = clock()
def _refill(self) -> None:
now = self._clock()
elapsed = now - self._last_refill
if elapsed > 0:
self._tokens = min(self._capacity, self._tokens + elapsed * self._refill_per_sec)
self._last_refill = now
def allow(self, cost: float = 1) -> bool:
self._refill()
if self._tokens >= cost:
self._tokens -= cost
return True
return False
@property
def tokens(self) -> float:
self._refill()
return self._tokens
``` Decisions: Refund window (hard) 100% right
{
"outcome": {
"full_refund": 0,
"partial_refund": 0,
"store_credit": 1,
"nothing": 0
},
"defective": {
"true": 0,
"false": 1
}
} Decisions: Incident matrix (hard) 50% right
{
"severity": {
"0": 0,
"1": 0,
"2": 0.1,
"3": 0.9
},
"page": {
"true": 0,
"false": 1
}
} Decisions: Tool followup (hard) 0% right
{
"tool": "calendar",
"confirm": false
} Decisions: Legit security alert (hard) 100% right
{
"phishing": {"true": 0.001, "false": 0.999},
"action_needed": {"true": 0.001, "false": 0.999}
} Decisions: Meeting slot (hard) 100% right
{
"slot": {"A": 0, "B": 1, "C": 0, "D": 0},
"raj_last": {"true": 1, "false": 0}
} Decisions: Review mixed (hard) 100% right
{
"hardware": {"true": 0.78, "false": 0.22},
"support": {"true": 0.92, "false": 0.08}
} Decisions: Support checkout down 100% right
{
"department": {
"technical": 0.8,
"billing": 0.15,
"account": 0.02,
"shipping": 0.02,
"sales": 0.01
},
"urgency": {
"3": 0.95,
"2": 0.05,
"1": 0,
"0": 0
},
"outage": {
"true": 1.0,
"false": 0.0
}
} Decisions: Refund wrong plan 100% right
{
"department": {"billing": 1, "technical": 0, "account": 0, "shipping": 0, "sales": 0},
"refund": {"true": 1, "false": 0},
"tone": {"frustrated": 0, "calm": 1}
} Decisions: Moderation doxxing 100% right
{"policy":{"none":0.0,"harassment":1.0,"hate":0.0,"spam":0.0,"self_harm":0.0},"personal_info":{"true":1.0,"false":0.0}} Decisions: Route calendar 100% right
{"tool":{"web_search":0,"calculator":0,"calendar":1,"email":0,"none":0},"confirm":{"true":1,"false":0}} Decisions: Doc invoice missing due 100% right
{
"doc_type": {
"invoice": 1.0,
"resume": 0.0,
"contract": 0.0,
"bank_statement": 0.0,
"other": 0.0
},
"missing_due_date": {
"true": 1.0,
"false": 0.0
}
} Decisions: Phishing paypal 100% right
{
"phishing": {
"false": 0.05,
"true": 0.95
},
"risk": {
"0": 0.02,
"1": 0.03,
"2": 0.15,
"3": 0.8
}
} Decisions: Pii ssn email 100% right
{
"data_kind": {
"government_id": 1.0
},
"sensitive": {
"true": 1.0
}
} Decisions: Review mixed 100% right
{
"sentiment": {
"positive": 0.1,
"neutral": 0.2,
"negative": 0.7
},
"defect": {
"true": 0.95,
"false": 0.05
},
"recommend": {
"true": 0.1,
"false": 0.9
}
} Documents: Saas escalator (hard) 18% right
{
"year1_price_per_seat_month": 45.00,
"year2_price_per_seat_month": 46.53,
"year3_price_per_seat_month": 47.41,
"year1_invoice": 5400.00,
"year2_invoice": 5425.48,
"addon_months_billed": 6,
"addon_invoice": 7308.84,
"year3_invoice": 5945.85,
"year3_discount_percent": 10.00,
"total_contract_value": 22515.14,
"contract_end_date": "2027-02-28"
} Documents: Expense thread 79% right
{
"employee_id": "EMP-20417",
"destination_city": "Lisbon",
"trip_start": "2025-02-24",
"trip_end": "2025-02-27",
"approved_items": [
{
"date": "2025-02-24",
"category": "airfare",
"amount_usd": 1184.6
},
{
"date": "2025-02-24",
"category": "ground_transport",
"amount_usd": 38.89
},
{
"date": "2025-02-25",
"category": "meals",
"amount_usd": 228.93
},
{
"date": "2025-02-26",
"category": "lodging",
"amount_usd": 461.76
},
{
"date": "2025-02-27",
"category": "ground_transport",
"amount_usd": 44.67
}
],
"rejected_item_count": 1,
"per_diem_days": 3,
"per_diem_usd": 195,
"total_reimbursable_usd": 2583.84,
"approver_email": "priya.raman@corvane.com"
} Documents: Lease amendment 83% right
{
"tenants": ["Marcus Lin", "Sofia Lin"],
"landlord": "Ridgeline Property Group LLC",
"zip": "97205",
"lease_end": "2025-11-30",
"original_monthly_rent": 2150.00,
"monthly_rent_from_2025_06_01": 2236.00,
"late_fee_from_2025_06_01": 111.80,
"security_deposit": 2150.00,
"total_pet_deposits": 800.00,
"total_monthly_payment_july_2025": 2607.00,
"move_in_payment": 3500.00
} Documents: Ticket SLA 100% right
{
"ticket_id": "48213",
"account_id": "ACC-7731",
"open_issue": "inventory_sync",
"resolved_issues": ["billing_address"],
"affected_orders": ["SO-99812", "SO-99820", "SO-99827"],
"priority": "P2",
"sla_due_local": "2025-09-15T15:30",
"sla_due_utc": "2025-09-15T20:30:00Z",
"reissued_invoice": "INV-2025-0812"
} Documents: Sales footnotes 100% right
{
"q3_total_usd": 15346000,
"q2_total_usd": 14464000,
"q2_central_originally_reported_usd": 3047000,
"q2_to_q3_change_pct": 6.1,
"top_region_q3": "East",
"fastest_growing_region_q1_to_q3": "International",
"regions_declining_q2_to_q3": ["East"],
"international_q3_organic_usd": 1731000,
"west_excluding_mountain_q3_usd": 4201000
} Size: 32B parameters. First tested OCT 10.
Comments
Sign in with GitHub to comment. Spam and abuse are hidden automatically.