What are the most effective methods for selecting biomarker features?

1 Filter methods

Filter methods are based on statistical or information-theoretic criteria that measure the relevance or redundancy of each feature. They do not depend on the choice of a specific classifier or model, and they are usually fast and scalable. However, they may ignore the interactions among features or the effect of the model complexity. Some examples of filter methods are correlation analysis, mutual information, chi-square test, and variance threshold.

2 Wrapper methods

Wrapper methods are based on the performance of a specific classifier or model that uses a subset of features. They search for the optimal subset that maximizes the accuracy or minimizes the error of the model. They can capture the interactions among features and the effect of the model complexity, but they may also overfit the data or be computationally expensive. Some examples of wrapper methods are forward selection, backward elimination, recursive feature elimination, and genetic algorithms.

3 Embedded methods

Embedded methods are based on the incorporation of feature selection within the learning process of a specific classifier or model. They use regularization or penalty terms to shrink or eliminate irrelevant or redundant features. They can balance the trade-off between relevance and redundancy, and they are usually more efficient than wrapper methods. However, they may also depend on the choice of the regularization parameter or the penalty function. Some examples of embedded methods are lasso, ridge, elastic net, and decision trees.

4 Hybrid methods

Hybrid methods are based on the combination of filter and wrapper or embedded methods. They aim to exploit the advantages of both approaches and overcome their limitations. They can improve the robustness and stability of feature selection, and they can handle high-dimensional and heterogeneous data. However, they may also increase the complexity and diversity of feature selection, and they may require more parameters or assumptions. Some examples of hybrid methods are reliefF, mRMR, Boruta, and stability selection.

5 Evaluation methods

Evaluation methods are based on the assessment of the quality and validity of the selected features. They use internal or external criteria to measure the consistency, stability, reproducibility, or generalizability of the feature selection results. They can help to compare different methods or models, and they can provide feedback for improvement or refinement. However, they may also depend on the choice of the evaluation metric or the validation strategy. Some examples of evaluation methods are cross-validation, bootstrapping, permutation testing, and ROC analysis.

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?