The short version
- Solar Decide is a decision model from Upstage. You give it a situation and multiple-choice questions, and it returns a probability for every option.
- In our tests it's the most accurate decision model we've tested: right on 96% of calls, 95% of the hard cases, calibration 96.
- It's closed: you can only use it through an API.
- 16 regular models you can run at home did better, led by Qwen3.5-27B (100%).
How it did
Our decision test has 56 situations: routing support tickets, spotting phishing, picking the right tool, applying a refund or expense policy, triaging incidents. 19 of them are hard cases that need a little date, time-zone or money math.
| Measure | Solar Decide |
|---|---|
| Accuracy | 96 |
| Hard cases | 95 |
| Calibration | 96 |
| Public questions | 93 |
| Secret questions | 96 |
| API cost per 1,000 calls | – |
Calibration shows how honest its confidence is: 100 means always right and always sure. We called it through OpenRouter's Decisions API.
How to try it
- Through OpenRouter:
upstage/solar-decide(Decisions API).
Its actual answers
What Solar Decide picked on our public questions, next to the right answer.
Refund window (hard) 100% right
| Question | It picked | Right answer |
|---|---|---|
| What does the policy give this customer? | Store credit only | Store credit only |
| Does the item count as defective under the policy? | No | No |
Incident matrix (hard) 50% right
| Question | It picked | Right answer |
|---|---|---|
| Which severity applies under the matrix? | SEV3 | SEV3 |
| Should on-call be paged under the policy? | Yes | No |
Tool followup (hard) 100% right
| Question | It picked | Right answer |
|---|---|---|
| Which single tool should the assistant use for the user's actual remaining request? | Create, move or look up calendar events | Create, move or look up calendar events |
| Would handling the request change something on the user's behalf, so it should be confirmed first? | Yes | Yes |
Legit security alert (hard) 50% right
| Question | It picked | Right answer |
|---|---|---|
| Is this email a phishing attempt? | No | No |
| Given the context, does the recipient need to do anything about this email? | Yes | No |
Meeting slot (hard) 100% right
| Question | It picked | Right answer |
|---|---|---|
| Which candidate start time works for everyone? | 12:30 UTC | 12:30 UTC |
| Is Raj's working day the one that ends earliest (in UTC)? | Yes | Yes |
Review mixed (hard) 100% right
| Question | It picked | Right answer |
|---|---|---|
| Should this go to the hardware team? | Yes | Yes |
| Should this go to support leadership? | Yes | Yes |
Support checkout down 100% right
| Question | It picked | Right answer |
|---|---|---|
| Which team should handle this message? | Bugs, errors, outages, integrations | Bugs, errors, outages, integrations |
| How urgently does this need a response? | Immediately: business is blocked | Immediately: business is blocked |
| Is a production service down or unusable for this customer? | Yes | Yes |
Refund wrong plan 100% right
| Question | It picked | Right answer |
|---|---|---|
| Which team should handle this message? | Charges, invoices, refunds, plans and payments | Charges, invoices, refunds, plans and payments |
| Is the customer asking for money back? | Yes | Yes |
| What is the writer's tone? | Polite, no frustration | Polite, no frustration |
Moderation doxxing 100% right
| Question | It picked | Right answer |
|---|---|---|
| Which content policy, if any, does this message violate? | Targets a person with abuse, threats or exposing their private information | Targets a person with abuse, threats or exposing their private information |
| Does the message reveal someone's private personal information? | Yes | Yes |
Route calendar 100% right
| Question | It picked | Right answer |
|---|---|---|
| Which single tool should the assistant use to handle the user's latest request? | Create, move or look up calendar events | Create, move or look up calendar events |
| Would handling the request change something on the user's behalf (send, create, delete or book something), so it should be confirmed first? | Yes | Yes |
Doc invoice missing due 100% right
| Question | It picked | Right answer |
|---|---|---|
| What kind of document is this? | A bill asking for payment | A bill asking for payment |
| Is the payment due date missing from the document? | Yes | Yes |
Phishing paypal 100% right
| Question | It picked | Right answer |
|---|---|---|
| Is this email a phishing attempt? | Yes | Yes |
| How likely is this email to be phishing? | High: clearly phishing | High: clearly phishing |
Pii ssn email 100% right
| Question | It picked | Right answer |
|---|---|---|
| What is the most sensitive kind of personal data in this message? | SSN, passport or national ID numbers | SSN, passport or national ID numbers |
| Does the message contain government IDs, financial account numbers or health information? | Yes | Yes |
Review mixed 100% right
| Question | It picked | Right answer |
|---|---|---|
| What is the writer's overall sentiment about the product? | Mostly unhappy | Mostly unhappy |
| Does the review report a product defect or breakage? | Yes | Yes |
| Would the writer recommend the product? | No | No |
Comments
Sign in with GitHub to comment. Spam and abuse are hidden automatically.