Constrain the inventory policy before training the agent

A retailer has a forecast for next week’s demand and still has to choose an order. The choice depends on current stock, lead time, supplier limits, substitution, service targets and what happens to unsold units. A more accurate forecast can help, but it does not settle the policy that turns a forecast into a purchase order.

That distinction matters before an inventory agent is introduced. The buyer should specify the allowed actions, the costs of being wrong, the fallback and the operating boundaries. Otherwise the learning system will optimise an underspecified objective and the business will discover its policy only after stock has moved.

Prediction and action fail differently

The earlier article on demand-forecast overrides asks when a human should change a prediction. The question here is downstream: given a prediction, which replenishment action is permitted and how is its effect measured? A forecast error is a statement about information. An order error changes cash, stock availability and sometimes the next observation of demand. Stockouts hide sales the firm could have made, so future training data can also reflect yesterday’s policy rather than true demand.

The DeepStock paper reports reinforcement learning for inventory management on Alibaba’s Tmall platform. Its authors regularise learned policies using classical inventory ideas, including a base-stock policy, and report deployment on that platform. The useful design lesson is not that every retailer should reproduce their system. It is that a policy can learn within a structure informed by how inventory decisions work, rather than treating every possible order as equally sensible. The paper’s synthetic comparisons and one deployment do not establish a return for another assortment or supply chain.

Draw the policy boundary first

Define the action space in business terms. Which products may be reordered? How far ahead can the agent commit? What supplier minimums, shelf-life rules, storage limits or cash ceilings apply? Which items require approval because they are scarce, regulated or strategically important? These are not features to discover from sales history. They are choices the operator must make.

Give the system a stable incumbent policy to beat. A base-stock or reorder rule, even if imperfect, gives the buyer a comprehensible baseline. Compare the proposed policy on service level, stockouts, holding cost, waste, expedite cost and working capital, not a single reward that can conceal a harmful trade-off. Preserve the action recommended, action taken and reason for each override. That record is needed to evaluate the policy when staff intervene.

Separate simulation from release. A backtest using historic sales must account for missing demand when shelves were empty and for actions the old policy never tried. A simulation can expose a fragile policy but cannot prove that customers and suppliers will behave identically in production. Begin with a bounded assortment, approval thresholds and a manual route if data or supply conditions move outside the tested range.

Finally, name the pause conditions. An unexpected supplier delay, new product family, promotion or major cost change can make the old transition data a poor guide. The agent should not treat those events as a request to explore with unlimited stock. It should escalate, constrain orders or return to the incumbent rule until the operator reviews the boundary.

Consider an item with a strong demand forecast but a long, uncertain supplier lead time. A forecast alone suggests ordering enough to cover expected sales. The policy must also decide how much cash to commit, what stockout risk the firm accepts and whether an approved substitute changes the order. If the supplier misses one delivery, a learner trained on normal lead times may increase the next order precisely when the buyer wants a manual decision.

Write that response as a constraint before optimisation. The agent can order within a band under ordinary conditions, but a lead-time breach sends the recommendation to the inventory owner. Preserve the forecast, proposed order, permitted band, actual order and reason for override. Without those records, the team cannot tell whether later performance came from the policy or from staff correcting it. A human override is evidence about the decision, not just an inconvenience in the training data.

The release comparison should include the incumbent rule on the same items and period, accounting for promotions and supplier shocks. If a simulator assumes every missed sale was lost demand, document that assumption and test how the result changes when customers substitute or return later. A policy that wins only under one unobservable demand estimate has not yet earned authority to place live orders.

The limit of the claim

DeepStock is evidence that a structured learned policy can be deployed in one large ecommerce setting. It is not evidence that reinforcement learning is necessary for a smaller inventory problem. Many firms should first improve stock records, lead-time estimates and an ordinary reorder rule. The extra complexity is justified only when a measured decision can improve after those basics are working.

The operations director should be able to sign the permitted policy before a model is trained. Then the model has a bounded job: choose better orders within a service and risk envelope the business meant to authorise.