Monotonicity Constraints
Description
Monotonicity constraints require a model's prediction to move only one way as a chosen input rises, with the other inputs held fixed, so that, for example, predicted salary never falls as years of experience increase. The direction for each constrained feature is set from domain knowledge before training, and the learning algorithm or model structure then rules out any fit that breaks it, so the model cannot learn reversals that come from noise or sampling quirks in the data. Boosted-tree libraries apply the constraint when they choose splits and monotonic neural networks build it into their structure, while penalty-based methods only discourage violations. Because a penalty, a constraint set on the wrong column or a later processing step can still let a reversal through, testers then vary each constrained input across its range over many generated or real cases, with the other inputs fixed, and count any prediction that moves the wrong way. The evidence is that violation count, expected to be zero, together with a comparison of held-out accuracy with and without the constraint.
Example Use Cases
Transparency
A mortgage lender constrains its approval model so that higher income, longer employment and a higher credit score can only raise the approval score. Loan officers can then tell an applicant that improving any of these factors can only help the application, and the lender can state each factor's direction to its regulator.
A water utility's pipe-burst risk model is constrained so that risk rises with pipe age and operating pressure and falls with the quality of recent maintenance. Asset managers can then explain the ranked inspection list to the regulator through three directional rules, instead of an opaque score.
Reliability
A hospital analytics team builds a tree-based model that predicts the cost of care for each patient and constrains the prediction never to fall as age or number of chronic conditions rises. Before release they sweep each constrained feature across its range for thousands of generated patient records, with the other features held fixed, and confirm that no prediction moves the wrong way. They also compare held-out accuracy with and without the constraints.
A utility's day-ahead demand forecast for office buildings is constrained so that predicted demand never falls as occupancy or floor area rises. The validation team writes this as an automated test that draws random building profiles, raises one constrained input at a time and fails the build if any forecast drops, so a retrained model that breaks the rule is caught before deployment.
Limitations
- Forces a one-way relationship even where the real one reverses, such as an inverted U between experience and performance. The model then fits that feature worse and cannot show the turning point, so held-out accuracy should be compared with and without the constraint.
- Choosing which features to constrain, and in which direction, needs domain expertise. A wrong constraint builds a false belief into the model, and the check on the trained model will still pass, because it shows only that the constraint holds, not that it is right.
- The rule covers one feature with the others held fixed, and says nothing about the size of the effect. A model can meet it by ignoring the feature, and when correlated inputs move together a person's overall score can still fall as one favourable input rises, so the constraint does not make the whole model easy to explain.
- Support differs by model family. Tree-boosting libraries such as XGBoost accept constraints directly, while neural networks need a special architecture or a penalty term. With histogram-based tree methods, constraints can remove every candidate split and leave shallow trees that underfit.
- A sweep or property-based test samples the input space. It can find a violation but cannot prove there is none, and a pass depends on how well the generated inputs cover realistic and rare combinations of features.
Resources
Research Papers
Monotonic Networks
Monotonicity is a constraint which arises in many application do(cid:173) mains. We present a machine learning model, the monotonic net(cid:173) work, for which monotonicity can be enforced exactly, i.e., by virtue offunctional form . A straightforward method for implementing and training a monotonic network is described. Monotonic networks are proven to be universal approximators of continuous, differen(cid:173) tiable monotonic functions. We apply monotonic networks to a real-world task in corporate bond rating prediction and compare them to other approaches.
Application of the Back Propagation Neural Network Algorithm with Monotonicity Constraints for Two‐Group Classification Problems*
Neural network techniques are widely used in solving pattern recognition or classification problems. However, when statistical data are used in supervised training of a neural network employing the back‐propagation least mean square algorithm, the behavior of the classification boundary during training is often unpredictable. This research suggests the application of monotonicity constraints to the back propagation learning algorithm. When the training sample set is preprocessed by a linear classification function, neural network performance and efficiency can be improved in classification applications where the feature vector is related monotonically to the pattern vector. Since most classification problems in business possess monotonic properties, this technique is useful in those problems where any assumptions about the properties of the data are inappropriate.