Choosing an explainability technique for a black-box model comes down to three questions: do you need to explain one prediction or the whole model, what kind of data are you working with, and how much computation can you afford? For most projects, the answer lands on one of three model-agnostic methods. LIME delivers fast, local explanations of individual predictions. SHAP offers mathematically grounded feature attribution that works both locally and globally, at a higher computational price. Permutation Feature Importance gives a straightforward global view of which features matter, with no per-instance explanations at all.
These methods share one defining property: they treat the model as a function that maps inputs to outputs, without needing to inspect its internal architecture, as practitioner guides to explainable AI emphasize. That is what makes them portable across algorithms. This guide compares the three techniques across interpretability scope, core mechanism, typical application, and computational cost, then walks through the decision points that should drive your selection.
LIME: Local Explanations for Any Model
Local Interpretable Model-Agnostic Explanations rests on a simple insight: while a complex model's global behavior may be incomprehensible, its local behavior around any specific prediction can be approximated by a simple, interpretable model. LIME fits a local linear model around the prediction you want to explain, and the coefficients of that surrogate tell you which features pushed the output in which direction.
The scope is strictly local. A perspective piece comparing SHAP and LIME, published on arXiv, notes that LIME generates one plot per instance, while SHAP produces several plots reporting outcomes both locally and globally. That per-instance character is LIME's strength and its limitation: it is ideal when a stakeholder asks "why did the model make this prediction?" but poorly suited to summarizing overall model behavior.
Its speed is a genuine advantage. The same comparison finds LIME much faster than SHAP, especially with tree-based models, with computational complexity on the order of O(NLD), where N is the number of perturbations, L the features, and D the dimensions. The trade-off is fidelity: because LIME fits a local linear model, it can fail to capture non-linear associations, whereas SHAP may detect them depending on the underlying model.
In practice, LIME has been used to assess prediction reliability in high-stakes settings. A systematic review of model-agnostic XAI in finance, authored by Farhina Sardar Khan and colleagues and published in Artificial Intelligence Review, cites work by Çelik et al. (2023) that used LIME to assess prediction reliability and prevent erroneous decision-making in stock market forecasting on the KOSPI dataset. That pattern—spot-checking individual predictions before acting on them—is where LIME fits best.
SHAP: Game Theory for Global and Local Insights
SHapley Additive exPlanations takes a different foundation: cooperative game theory. It attributes each feature's contribution to a prediction using Shapley values, which consider all combinations, or coalitions, of features to cover every case where a feature acts alone or alongside subsets of the others. A biomarker-identification study published through PMC describes Shapley value-based methods like SHAP as providing localized feature characterization based on game theory.
The payoff is consistency. Because every coalition is accounted for, the attribution is fair by construction, and the same framework yields both local explanations for individual predictions and global summaries of model behavior. The arXiv perspective notes SHAP has been applied widely across domains to explain model outcomes either locally or globally, and that it may capture non-linear associations depending on the model used.
The cost is computational. Exact Shapley values grow in complexity as the number of features increases, which motivated the Kernel SHAP approximation. Even so, the PMC study characterizes Shapley value-based methods as computationally intensive, and the arXiv comparison gives Kernel SHAP complexity on the order of O(TLD), where T is the number of samples, L the features, and D the coalitions. For large datasets, this can become a bottleneck.
Two caveats deserve attention. The arXiv authors flag that SHAP is a model-dependent method in how its outputs should be interpreted, and that its behavior warrants care when features are collinear. The PMC study adds that Shapley-based methods do not guarantee a valid statistical test of importance. SHAP is nonetheless the method of record in domains where rigor matters: the finance review by Khan and colleagues cites Bussmann et al. (2020) using Shapley values to interpret AI predictions for peer-to-peer lending risk management.
Permutation Feature Importance: Simplicity and Robustness
Permutation Feature Importance answers a narrower question: how much does the model's performance depend on each feature? The mechanism is direct—shuffle one feature's values, breaking its relationship with the target, and measure the resulting increase in prediction error. A feature whose shuffling degrades performance substantially is important; one whose shuffling changes little is not.
Christoph Molnar's Interpretable Machine Learning book, the standard practitioner reference on the method, situates permutation importance among model-agnostic importance measures, distinguishing it from model-specific alternatives like Gini importance for random forests or standardized regression coefficients for regression models. It also notes a model-agnostic alternative in variance-based measures, and a loss-based alternative in LOFO, which omits a feature from training data and retrains.
The scope is global. PFI tells you which features matter to the model overall, but it produces no explanation for any individual prediction. That makes it a complement to, not a substitute for, LIME or SHAP when per-instance justification is required. Its strengths are transparency of mechanism and low conceptual overhead—there is no surrogate model to mis-specify and no game-theoretic machinery to explain to stakeholders.
The finance review by Khan and colleagues records permutation feature importance in active use, citing Kumar et al. (2024) applying it, alongside integrated gradients, to improve the interpretability of stock market prediction models. For an initial audit of which features drive a model, PFI is often the cheapest defensible first step.
XAI Technique Selection Matrix
The matrix below compares the three techniques across the criteria that drive selection: interpretability scope, core principle, typical application, and computational cost. Reading across a row reveals each method's profile; reading down a column reveals the trade-off you are making.
| Technique | Interpretability Scope | Core Principle | Typical Application | Computational Cost |
|---|---|---|---|---|
| LIME | Local only; one plot per instance | Approximates the black-box model locally with an interpretable linear model | Assessing reliability of individual predictions, e.g., stock market forecasting (KOSPI dataset) | Lower; roughly O(NLD), faster than SHAP especially on tree-based models |
| SHAP | Both local and global | Shapley values from game theory; considers all feature coalitions | High-stakes interpretation, e.g., peer-to-peer lending risk management | High; Kernel SHAP roughly O(TLD), a bottleneck on large datasets |
| Permutation Feature Importance | Global only; no per-instance explanations | Shuffles a feature's values and measures the increase in prediction error | Global feature relevance audits, e.g., financial forecasting models | Relatively low; repeated model evaluations per feature |
Three patterns emerge from the comparison. First, scope and cost move together: the methods that explain more (SHAP's dual local-global coverage) cost more. Second, LIME and PFI occupy complementary niches—fast local checks and cheap global audits—while SHAP spans both at a premium. Third, each method carries a documented weakness: LIME's linear surrogate may miss non-linear associations, SHAP is computationally intensive and does not guarantee a valid importance test, and PFI cannot explain any single prediction.
Making Your XAI Choice: A Decision Framework
Select the XAI technique that best aligns with the project's interpretability goals (local vs. global), data characteristics, and available computational resources. Work through the decision points in order.
- Local or global question? If you must justify individual predictions—to a regulator, a domain expert, or an affected party—LIME or SHAP is required. If you only need to know which features drive the model overall, permutation importance is sufficient and cheaper.
- Both scopes needed? If you need per-instance explanations and global summaries from one framework, SHAP is the only technique of the three that provides both, provided you can absorb its computational cost.
- Computational budget? With constrained resources, the arXiv comparison indicates LIME is much faster than SHAP, especially with tree-based models. Kernel SHAP's O(TLD) complexity can be a bottleneck on large datasets.
- Non-linear relationships suspected? LIME's local linear approximation may fail to capture non-linear associations; SHAP may detect them depending on the model. If feature importance is your goal and non-linearity is a concern, PFI sidesteps the surrogate-model question entirely.
- Audience and stakes? The finance review by Khan and colleagues links these choices to transparency and regulatory compliance in high-stakes domains, where SHAP's rigorous attribution has been the method of record in applications like peer-to-peer lending.
A common pattern is to combine methods: run permutation importance first for a global audit, then apply LIME or SHAP to the individual predictions that matter most. The measurable indicator of a sound choice is simple: the chosen XAI technique successfully generates explanations that are understandable by the target audience and meet the project's specific interpretability requirements. If your explanations answer the question your stakeholders actually ask—about one prediction or about the model as a whole—within your compute budget, the selection is right.
Sources
- Explainable AI Techniques for Black Box Models - ML Journey
- A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME — ARXIV
- Permutation-based identification of important biomarkers for complex diseases via machine learning models - PMC
- 23 Permutation Feature Importance – Interpretable Machine Learning — Christophm
