## Overview Random Forest is an ensemble learning algorithm that builds multiple decision trees and combines their outputs to improve classification accuracy and reduce overfitting. It is widely used for both classification and regression tasks. ## Key Components 1. **Ensemble Learning** - Combines multiple weak learners (decision trees) into a strong learner 2. **Bootstrap Aggregation (Bagging)** - Each tree is trained on a random subset of data with replacement 3. **Feature Randomness** - Each split in a tree considers only a random subset of features 4. **Majority Voting** - Final prediction is based on majority voting from multiple trees ## How It Works 1. **Data Sampling** - Randomly selects samples from training data (bootstrap sampling) 2. **Tree Building** - Constructs multiple decision trees using different feature subsets 3. **Prediction Aggregation** - For classification, takes majority vote of all trees - For regression, takes average of predictions ## Implementation Example python ```python from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score #Initialize model rf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42) # Train model rf.fit(X_train, y_train) # Make predictions predictions = rf.predict(X_test) accuracy = accuracy_score(y_test, predictions) print(f'Accuracy: {accuracy:.2f}')` ``` ## Advantages - Handles high-dimensional data efficiently - Reduces overfitting compared to single decision trees - Works well with both numerical and categorical data - Robust to missing values and noise - Provides feature importance scores ## Disadvantages - Computationally expensive for large datasets - Less interpretable than a single decision tree - Can overfit if the number of trees is too large ## Hyperparameters 1. **Number of Trees (`n_estimators`)** - More trees improve accuracy but increase computation time 2. **Max Depth (`max_depth`)** - Limits tree depth to prevent overfitting 3. **Min Samples Split (`min_samples_split`)** - Minimum samples required to split a node 4. **Max Features (`max_features`)** - Number of features considered for each split - `sqrt` (default for classification) → Uses square root of features - `log2` → Uses log base 2 of features ## Best Practices 1. **Feature Selection** - Use feature importance to remove less relevant features 2. **Hyperparameter Tuning** - Use Grid Search or Randomized Search for optimal settings 3. **Handling Imbalanced Data** - Use `class_weight='balanced'` to adjust class distributions ## Common Applications - Fraud detection - Customer segmentation - Image classification - Medical diagnosis - Credit risk assessment ## Performance Optimization 1. **Reduce Overfitting** - Set `max_depth` and `min_samples_split` appropriately 2. **Increase Computational Efficiency** - Use `n_jobs=-1` to parallelize training 3. **Combine with Feature Selection** - Remove irrelevant features to improve speed and accuracy ## Evaluation Metrics - Accuracy - Precision - Recall - F1 Score - ROC-AUC curve