A cheaper recipe meets every nutrient limit and clears the predicted palatability threshold. The optimizer ranks it first. Before making the prototype, the formulation team has another question: did the search identify a useful recipe, or one whose performance the model overestimates?
Our companion article on palatability prediction and validation defines the response, comparator, and evidence behind a prediction. Here, the question is how that prediction should influence formulation optimization. A model-supported candidate is a proposal to evaluate; its predicted feasibility is not an observed biological outcome.
Should palatability be a constraint or an objective?
Use a constraint for a required performance level and an objective when improvement is part of the brief. The same attribute can serve both roles.
For a cost-reduction project, the brief might minimize ingredient cost while requiring a predicted intake ratio of at least 0.55 against the current product. This illustrative threshold concerns the defined trial-level response; it is not a universal significance threshold.
That remains a single-objective problem if cost is the only objective. If the brief also seeks higher predicted preference, cost and palatability become competing objectives, with the required floor retained. Ranking feasible candidates afterward is a separate decision.
In FormuLogic, an integrated palatability prediction can be an explicit solver constraint. A cheaper candidate that misses a hard threshold remains infeasible under that brief. Any change to the requirement needs a deliberate review.
Why can optimization expose weaknesses in a validated model?
Optimization selects favorable predictions, so errors among selected candidates may differ from errors across the validation set.
Smith and Winkler describe the optimizer's curse: selecting alternatives using uncertain value estimates can produce optimistic estimates for the selected alternative, even when estimates were unbiased before selection. The selection process itself creates the effect. [1]
A related problem arises when a search reaches formulations poorly represented in the training data. Research on model-based optimization shows how overestimated predictions under distribution shift can direct a search toward poor designs. That research supports the methodological concern; it is not evidence of a measured failure rate in pet food formulation. [2]
Applied to our cost-reduction brief, the concern is not limited to maximizing palatability. A cheap recipe can appear to satisfy a palatability constraint because its prediction is optimistic. Overall validation accuracy alone does not establish reliable performance for the recipes the optimizer selects.
For example, separate inclusion limits may all be respected while their combined ingredient pattern is unfamiliar to the model. For this reason, review the candidate as a complete formulation, including the process assumptions used for prediction. An ingredient-by-ingredient range check is only part of that review.
What does a cost–palatability comparison reveal?
It shows what additional predicted performance costs, after mandatory requirements are applied. Keep the comparator and response definition identical across candidates.
Worked example: screening three reformulations. All values below are hypothetical, not customer results or FormuLogic outputs. Assume the candidates satisfy other modeled requirements and fall within the model's intended scope. The required predicted intake ratio against control B is 0.55.
A1 is excluded. A2 is the cheapest listed candidate meeting the modeled requirement. A3 costs another $0.03/lb for a predicted intake-ratio increase of 0.04. Across one million lb, that is $30,000 in additional ingredient cost, assuming unchanged unit costs and excluding implementation costs.
The table does not establish that A3 will outperform A2 in a trial. Nor does A2's margin of 0.02 above the threshold establish a probability of success. Those questions require evidence beyond the point estimates.
How should uncertainty affect candidate selection?
Match the treatment of uncertainty to the intended decision and the evidence available. A confidence label without a defined meaning cannot resolve a close comparison.
Distinguish uncertainty about an expected response from variation in a future trial result. Prediction intervals for future observations account for additional variation beyond uncertainty in the estimated mean; their interpretation depends on the model and assumptions. [3]
For this workflow, practical options include restricting the search to supported formulation regions, testing sensitivity to justified prediction margins, and requiring additional evidence for candidates near the threshold. Where appropriately validated uncertainty estimates exist, a conservative bound or probability requirement may be considered.
These are methodological options, not claims that FormuLogic automatically implements each one. A margin chosen without evidence does not become a statistical guarantee. Likewise, interval performance on ordinary validation examples should not be assumed to hold unchanged after optimization selects candidates.
For A2, ask whether the evidence supports treating 0.57 as sufficiently above 0.55. If plausible prediction error changes its screening status, retain that uncertainty in the decision rather than rounding it away. A3's larger predicted margin may justify investigation, but the larger number alone does not establish greater reliability.
Why compare multiple formulation solutions?
Multiple solutions let the team compare different ways of meeting the same brief. The useful question is what each candidate offers in cost, predicted performance, ingredient choices, and implementation requirements.
In the worked example, A2 is cheaper while A3 has a higher predicted intake ratio. Both clear the modeled palatability threshold. Choosing between them requires assessing whether the predicted difference merits the additional cost and which candidate has stronger supporting evidence.
FormuLogic's multi-objective, multi-solution optimization generates up to 50 ranked solutions and allows three to be selected for deeper comparison. Review meaningful ingredient and sourcing differences alongside predicted outcomes. Fifty nearly identical recipes may offer little practical choice.
Keep cost, predicted response, and the reasons for ranking visible. Review the priorities behind the ranking so the team can understand why a candidate is preferred and which changes to the brief would alter that choice.
Which candidates should become physical prototypes?
Select a shortlist that answers the commercial question and the most consequential remaining uncertainties. The highest predicted response need not be the most useful next experiment.
For the worked example, A2 addresses the lowest-cost feasible option; A3 tests whether the higher predicted response merits its premium. A different ingredient combination might be more informative than another small variation of either recipe.
Agree on the comparator, protocol, response calculation, and acceptance criteria before testing. Record the candidate and model versions and their predictions. Compare observed outcomes with those predictions, investigate discrepancies, and decide whether further testing or model revision is warranted.
The scientific question is whether the proposed formulations perform as expected. The manufacturing decision also includes ingredient qualification, process performance, and implementation economics.
Keep exploratory candidates distinguishable from candidates proposed for implementation. A formulation outside the model's supported scope can still be worth testing to learn about a new ingredient combination. Its experimental value does not make its predicted score dependable enough for a production decision.
What does this require from a FormuLogic engagement?
A prediction model suited to the customer's data and development scope, connected to a clearly defined formulation brief. FormuLogic engagements involve custom model development and validation using customer data, or tailored integration of an existing customer model.
Retain the control, endpoint, model version, validation limits, hard requirements, and ranking rationale with the decision. This makes the selected recipe's predicted advantage reviewable and gives subsequent experiments a precise claim to test.
If no candidate satisfies the modeled requirements, investigate the cause: ingredient restrictions, the required response, conflicting specifications, or limitations in the predictor. Record any revised brief before rerunning the search. Quietly relaxing the palatability threshold would change the question the team intended the solver to answer.
Explore FormuLogic and our AI-powered formulation overview for the broader workflow connecting proprietary predictions, formulation alternatives, and development decisions.
Sources
- Smith JE, Winkler RL. The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis. Management Science. 2006;52(3):311–322. DOI: 10.1287/mnsc.1050.0451.
- Trabucco B, Kumar A, Geng X, Levine S. Conservative Objective Models for Effective Offline Model-Based Optimization. Proceedings of the 38th International Conference on Machine Learning. 2021;139:10358–10368. Research on optimization using learned models; not a pet food study.
- NIST/SEMATECH. How can I predict the value and estimate the uncertainty of a single response? e-Handbook of Statistical Methods. Mean-response uncertainty versus future-observation uncertainty.
