Clef and the Decisions API: Choosing Thresholds for Auto-Routing

When classifying text such as customer inquiries, there is an alternative to extracting a classification result from generated text: receiving a probability for each option directly. The underlying design, returning a probability or score and comparing it with a cutoff to classify, has long been used in conventional machine learning.

On October 1, 2026, Cloudflare released Clef and Clef-flash, which it calls “decision models.” They take the input to be classified, such as a piece of text, along with questions that specify the answer format, and return a probability for each option. Perplexity also began offering its Decisions API, which likewise returns answers as probabilities, in October 2026.

When you use these probabilities to separate automated processing from human review, you set a cutoff value called a threshold. Using the routing of support tickets (records that track each inquiry individually) as an example, this article looks at how to choose that threshold based on business requirements and your own data.

The focus is on classification tasks with a small number of possible categories. Cloudflare’s fine-tuning service based on reinforcement learning (RL), performance on image inputs, and other uses such as tool selection are out of scope.

Features and deployment options

In common: send questions with a specified answer format, get probabilities back

Cloudflare describes decision models as models that use probability-based classification to support decisions about an agent’s next action or a handoff to a person. In the example in its blog post, a support inquiry is sent along with a yes/no question, “Is this request urgent?”, and a multiple-choice question, “Which team should handle it?” (billing, technical, or sales), among others. What comes back is not prose but an answer to each question with its probability.

Perplexity’s documentation describes nearly the same structure. Three types of questions (yes/no, multiple choice, and scoring against predefined criteria) can be bundled into a single request. For multiple-choice questions, the response includes a probability for each specified option as well as the model’s confidence.

Perplexity’s documentation states explicitly that the model does not write a reply and does not explain the reasoning behind its output. Cloudflare likewise describes a design in which the model outputs only probabilities rather than generating text.

Differences: how they are offered and pricing

Cloudflare says it has released the trained parameters (weights) of both models on Hugging Face under the Apache 2.0 license. They can be run in your own environment or used on Workers AI, which Cloudflare operates. Perplexity’s Decisions API is available through the company’s hosted service.

The official pricing page, checked on October 8, 2026, lists two models, pplx-decider-v1.1-27b and pplx-decider-v1-27b, both at $0.02 per million input tokens, with output free of charge. Cloudflare’s blog post does not mention pricing, but the official Workers AI model page lists Clef at $0.24 per million input tokens (as of October 8, 2026).

At Perplexity’s price listed on October 8, 2026, one million tickets with 1,000 input tokens each would cost about $20 in API input charges. This assumes the 1,000 tokens include the inquiry, questions, and options. To estimate the total operating cost, also account for retries, processing before and after the API call, and human review and correction.

Cloudflare’s published benchmarks and your own evaluation

Cloudflare says Clef leads the Jev Decision Index. It also reports that Clef outperformed Jev, the decision model used for comparison, in three of four business domains in an evaluation suite from Typesafe, Jev’s developer.

The figures below come from evaluations that Cloudflare conducted and published. MIF has not independently reproduced these results.

Even within the same table, Clef does not come out on top in every item. For example, on an evaluation of when a tool should be called (When2Call, accuracy), Jev scores 80.97, ahead of Clef’s 72.37.

The performance gap between Clef and Clef-flash varies by task. On intent classification that includes out-of-scope inquiries (CLINC150+OOS), Clef scores 97.43 and Clef-flash 66.77, a large gap. The metric is macro-F1, the unweighted average of the F1 scores across classes, so classes with few examples count just as much as the others. Because it is calculated differently from simple accuracy, it should not be compared directly with figures for other metrics.

According to Cloudflare, Clef is based on Qwen3.8-27B and Clef-flash on Qwen3.5-9B. Since this is not a comparison in which only model size was changed, the table alone does not let you attribute the differences to size.

Cloudflare also reports a median latency of 38.8 milliseconds for Clef-flash. That figure was measured under Cloudflare’s own evaluation conditions. It should be kept separate from the processing time of the business workflow as a whole, which includes retrieving the ticket, API communication, logging the result, and changing the assignee.

In our view, published evaluations like these are useful for narrowing down candidate models. Performance on your own inquiries and classification criteria, however, needs to be measured using your own data.

Separate prediction, handling decisions, and execution

A model returns a prediction such as “this inquiry is likely to be billing-related.” The next step is deciding how to handle the ticket: whether to assign a team provisionally, present the candidate teams to a reviewer, or send it to a special review queue. Changing the assignee field in the ticketing system according to that decision is execution. Permissions and approval requirements for that operation can be set separately from the decision about how to handle the ticket.

The official documentation for scikit-learn, a machine learning library, also explains classification by separating the problem of predicting probabilities from the problem of choosing an action based on those predictions.

MIF adds a third design consideration: executing the chosen action under the required permissions and approvals. We recommend treating prediction, decisions about how to handle the ticket, and execution as separate functions.

Refunds show where this distinction matters. A prediction that “this is a billing inquiry” does not lead to the conclusion that “a refund is allowed.” A refund depends on other inputs: the transaction in question, the contract terms, identity verification, the amount, and whether approval has been given. Separating the three means that refunds and contractual decisions are not authorized on the basis of this classification result alone.

Note that receiving probabilities does not mean you always need a threshold for automated processing. You can also have people review everything. What follows applies when you use probabilities to separate automated processing from human review.

Set business requirements first, then choose thresholds from measurements

A threshold is not a number to be set by staff intuition alone. First, decide the scope of the work to be automated, the errors you can accept, how many items people can review per day, and who approves the adoption of a threshold. Then select candidate thresholds that meet those requirements from validation data.

Machine learning libraries such as scikit-learn also provide tools that calculate and tune thresholds given a specified evaluation metric or error cost. Even when you use such tools, decide the conditions for adopting a threshold and who is responsible for changing it.

Not all errors have the same impact

With support tickets, the following two errors are not equally serious:

• Sending an ordinary inquiry to the wrong team.
• Treating an urgent outage report as an ordinary inquiry and putting it off.

“Is the top predicted probability high?” and “Are we missing urgent cases?” are separate questions. A high predicted probability for a particular team does not mean the ticket is non-urgent.

And while the routing itself can be reversed, the delay in handling while the ticket sat with the wrong team cannot be undone. You need to look not only at how easily a decision can be reversed, but also at how long it takes to notice and correct an error, and at the impact in the meantime.

How to check calibration depends on the question type

Returning probabilities does not guarantee that they are well calibrated. Calibration concerns whether predicted probabilities match how often the corresponding outcomes occur across many cases.

Cloudflare says it used a loss function intended to improve calibration (Brier loss) in training. The blog post alone, however, cannot tell you whether the probabilities are well calibrated for your own inquiries.

How you check depends on the question type.

• For yes/no questions, look at the share of actual “yes” outcomes in each predicted-probability bin (an interval such as 0.7 to 0.8). For example, for a group of tickets whose “probability of being urgent” is around 0.2, check whether about 20% of them were actually urgent.
• For multiple-choice questions, look at the top probability and check, for each bin, the share of cases in which that choice was correct.

In both cases, also look at the number of items and the average predicted probability in each bin. A high agreement rate on a small number of cases is not sufficient confirmation.

Moreover, good calibration is not the same as being fit to automate. Even if predictions near 0.9 are correct about 90% of the time, you still need to decide whether the roughly 10% error rate in that group is acceptable for the task.

Perplexity’s documentation also includes an operational caution. The second decimal place can occasionally change for the same request, so it advises leaving some margin in thresholds. It also explains that confidence for multiple-choice questions is not the top probability itself, but a value that drops when the gap to the second option is small.

Check human judgments before using them as reference answers

Before deployment, one approach is to keep existing operations running while recording the model’s predictions in parallel and comparing them with human judgments. This approach, in which the model’s predictions are not applied to actual processing, is called “shadow mode.”

If each staff member applies different classification criteria, however, what you are measuring is not correctness but agreement with that particular staff member. Check the reference answers against your classification criteria and resolve disagreements before using them for evaluation.

For inquiries involving both billing and technical issues, decide in advance whether one team or multiple teams can count as the correct assignment. You need that definition to interpret the evaluation results.

An example process

Based on the above, here is one example of the process we propose.

1. Define the target work and evaluation criteria. Organize the routing destinations, how urgent cases are handled, the criteria for the correct answer, and the scope within which automated processing is allowed.
2. Evaluate without affecting operations. Continue actual processing with existing methods and record the model’s predictions in parallel. In addition to typical cases, check cases that span multiple teams, cases with insufficient information, and out-of-scope inquiries.
3. Compare candidate thresholds. Compare the correctness of automated routing, missed urgent cases, and the number of items people must review. Evaluate the selected thresholds on a separate set of data, not only on the data used to tune them.
4. Start with a limited rollout and continue reviewing the results. Sample not only the cases sent to people but also the cases processed automatically. Re-evaluate when inquiry trends, classification criteria, the model, or the question wording change.

An example routing table: apply the first matching rule

The following is one example proposed by MIF, assuming the task of routing support tickets to the responsible team. Adjust it based on your own evaluation results.

In the table, p1 and p2 are the two highest predicted probabilities. T is the minimum required value of p1, and M is the minimum required gap between p1 and p2. Check the table from top to bottom, and apply the action in the first row that matches.

Check in order	Example conditions	Response	Mainly implemented in
1. Outside the scope of automated routing?	Cases designated in advance for human handling, such as refund, contract, or legal inquiries; cases requiring urgent attention	Route to the designated person or queue	Internal policies, permission settings, and ticketing system rules
2. Missing information or a result that requires review?	Missing information, inputs outside the defined categories, API errors, or uncertain predictions that meet your own “needs review” conditions (for example, a yes/no urgency probability near 0.5)	Mark as “needs review” for a staff member to check	Rules in your own systems, outside the model
3. Automated routing criteria met?	p1 ≥ T and p1 − p2 ≥ M, with all other business conditions also met	Provisionally assign the ticket to a team within the permitted scope	Ticketing system rules
4. Otherwise	None of rows 1–3 applies	Present the candidate teams for a staff member to choose from	Staff interface and internal procedures

Choose the values of T and M from your own evaluation results, following the steps in the previous section. Using both the probability and the gap is itself only one example; it does not mean this combination is always the best choice.

Treat uncertain predictions as requiring review, not as evidence that none of the categories applies. For example, if the probabilities for three options are 0.34, 0.33, and 0.33, that means the gap between the candidates is small. It does not mean the model judged that none of them applies. A yes/no probability near 0.5 likewise signals uncertainty about that question. It is not a third answer.

If you want to use “other” or “out of scope” as business categories, make them explicit as options or as separate review conditions, and include them in the evaluation. A high top probability alone will not necessarily reveal out-of-scope inputs. Your validation should also cover the possibility that an inquiry matching none of the options gets assigned with high probability to one of the existing candidates.

Likewise, how you will detect the “cases requiring urgent attention” in row 1 needs an evaluation of its own.

Logging and review

So that each routing decision can be traced later, record the inquiry identifier, the model used, the versions of the questions, classification criteria, and thresholds, the output probabilities and classification results, the action taken, the execution result, and any human corrections.

Inquiries may contain personal information, credentials, or contract details. When storing input text, define what to retain, how to mask sensitive information, who can access it, and how long to keep it. Manage data kept for re-evaluation under the same conditions. The OWASP Logging Cheat Sheet, a widely referenced security practice resource, also advises against logging sensitive information as-is and calls for managing access permissions and retention periods.

Use actual results to check whether your thresholds still meet business requirements. Review them regularly and whenever the model or question wording changes. Specify in advance who conducts the reviews and who is accountable for them.

Plan for changes to models and decision criteria

A threshold is a number chosen for a specific combination of model and questions at a given time. If you switch to a new model and only set the threshold back to its old value, you will not necessarily get the same results as before.

The predicted probabilities can also change when you change the question wording, the option definitions, or how inputs are constructed, not only when you change the model. These changes, too, call for re-evaluation. For that reason, we recommend tracking the model, questions and options, preprocessing, any adjustments to the probabilities, thresholds, and business decision rules together, and recording the exact combination that was evaluated and approved for use.

Each call to Perplexity’s Decisions API requires a model name. At the time of our check, the documentation listed two available models: pplx-decider-v1.1-27b and pplx-decider-v1-27b. We recommend recording which version you evaluated, and re-evaluating before switching versions.

How long older versions will remain available could not be determined from the materials we checked. A version number in a model name does not, on its own, tell you how long that version will be available or whether its behavior is fixed. Because the weights of Cloudflare’s models are public, another option is to pin a version and run it in your own environment.

In either case, plan how to return to a validated configuration where possible. If that is not feasible, be ready to stop automated processing and switch to manual handling or your existing procedures. Returning to shadow mode means the model’s predictions are no longer applied to operations, so you also need to decide who will handle tickets during that period.

Summary

At MIF, we recommend designing prediction, decisions about how to handle a ticket, and execution as separate functions. Before using probabilities for automated routing, define the target work and the errors you can accept, then use your own data to validate thresholds against those requirements.

Evaluate routing accuracy, missed urgent cases, the volume of human review, and how cases that do not meet the criteria for automated routing are handled.

Record the model, questions, classification criteria, and thresholds approved for use, and re-evaluate the setup when any of them changes. Plan how to return to a validated configuration, or stop automated processing and use existing procedures, when business requirements are no longer met.

Sources and Verification

This article is based on information confirmed as of October 8, 2026. Its primary sources are Cloudflare’s official blog post “Clef decision models” (published October 1, 2026), the Clef model card on Hugging Face, the Clef model page for Cloudflare Workers AI, and Perplexity’s Decisions API documentation, official changelog, and pricing page.

For the distinction between predicting probabilities and choosing actions, and for how to check calibration, we referred to scikit-learn’s official documentation. For the handling of logs, we referred to the OWASP Logging Cheat Sheet.

Descriptions of product features, pricing, and published evaluations are based on each company’s published information. MIF has not independently verified the performance of these APIs. The approach to choosing thresholds, the routing table, and the logging and switchover procedures are MIF’s design proposals.

Pricing, supported models, and how the products are offered may change. Before using either service, check the provider’s current official documentation.