{"id":59,"date":"2023-08-10T07:01:00","date_gmt":"2023-08-10T07:01:00","guid":{"rendered":"https:\/\/amirhooshang.com\/blog\/?p=59"},"modified":"2025-05-14T08:53:09","modified_gmt":"2025-05-14T08:53:09","slug":"how-to-evaluate-machine-learning-models-from-prediction-to-clustering","status":"publish","type":"post","link":"https:\/\/amirhooshang.com\/blog\/2023\/08\/10\/how-to-evaluate-machine-learning-models-from-prediction-to-clustering\/","title":{"rendered":"How to Evaluate Machine Learning Models: From Prediction to Clustering"},"content":{"rendered":"\n

Machine Learning (ML) is everywhere these days, from predicting house prices to diagnosing diseases! But when we build a model, how do we know if it\u2019s actually performing well? This is where evaluation metrics<\/strong> come into play. In this article, we\u2019ll explore how to measure the performance of different types of ML models\u2014whether they\u2019re trained with labeled data, discover patterns on their own, or use a mix of both.<\/p>\n\n\n\n

\n\n\n\n

1. Types of Machine Learning Models Simplified<\/strong><\/h2>\n\n\n\n

ML models can be divided into three main categories:<\/p>\n\n\n\n

    \n
  • Supervised Models<\/strong>: Like a student who gets answers to all practice questions. These models learn from labeled data (e.g., images of cats and dogs with their names tagged).<\/li>\n\n\n\n
  • Unsupervised Models<\/strong>: Like someone trying to find patterns in a messy room. These models analyze unlabeled data (e.g., customer purchase histories) to discover groupings or patterns.<\/li>\n\n\n\n
  • Semi-Supervised Models<\/strong>: A hybrid approach. For example, using 20 labeled samples and 80 unlabeled ones to train a model.<\/li>\n<\/ul>\n\n\n\n

    Now, let\u2019s see how to evaluate each type!<\/p>\n\n\n\n

    \n\n\n\n

    2. Evaluating Supervised Models: From Predicting Numbers to Classifying Categories<\/strong><\/h2>\n\n\n\n

    A) Regression Models (Predicting Numbers)<\/strong><\/h3>\n\n\n\n

    Suppose you\u2019re predicting house prices. How do you measure your model\u2019s accuracy?<\/p>\n\n\n\n

      \n
    • Mean Squared Error (MSE)<\/strong>: Calculate the squared difference between predicted and actual values, then average them. Lower MSE = better model!<\/li>\n\n\n\n
    • Root Mean Squared Error (RMSE)<\/strong>: The square root of MSE. For example, an RMSE of $50k means your model is off by ~$50k on average.<\/li>\n\n\n\n
    • Mean Absolute Error (MAE)<\/strong>: Average of absolute differences (no squaring). Lower is better.<\/li>\n\n\n\n
    • R-squared (R\u00b2)<\/strong>: Explains how much variance your model captures (0 to 1). An R\u00b2 of 0.8 means 80% of the data\u2019s variation is explained by the model.<\/li>\n<\/ul>\n\n\n\n

      B) Classification Models (e.g., Spam Detection)<\/strong><\/h3>\n\n\n\n

      Here, the model categorizes data (e.g., spam vs. non-spam). Key metrics:<\/p>\n\n\n\n

        \n
      • Accuracy<\/strong>: Percentage of correct predictions. But beware! If 95% of data is non-spam, a model that always predicts “non-spam” will have 95% accuracy but is useless.<\/li>\n\n\n\n
      • Precision & Recall<\/strong>:<\/li>\n\n\n\n
      • Precision<\/strong>: Of all samples predicted as positive (e.g., spam), how many were truly positive?<\/li>\n\n\n\n
      • Recall<\/strong>: Of all actual positives, how many did the model correctly identify?<\/li>\n\n\n\n
      • F1-Score<\/strong>: Harmonic mean of Precision and Recall. Use this if both metrics matter.<\/li>\n\n\n\n
      • Confusion Matrix<\/strong>: A table showing True Positives, False Positives, True Negatives, and False Negatives.<\/li>\n<\/ul>\n\n\n\n
        \n\n\n\n

        3. Evaluating Unsupervised Models: From Clustering to Dimensionality Reduction<\/strong><\/h2>\n\n\n\n

        A) Clustering (e.g., Customer Segmentation)<\/strong><\/h3>\n\n\n\n

        How do we know if the model grouped data well?<\/p>\n\n\n\n

          \n
        • Silhouette Score<\/strong>: Assigns each data point a score between -1 and 1. Closer to 1 means the point fits well in its cluster and is far from others.<\/li>\n\n\n\n
        • Davies-Bouldin Index<\/strong>: Lower values mean better-separated clusters.<\/li>\n\n\n\n
        • Calinski-Harabasz Index<\/strong>: Measures the ratio of between-cluster to within-cluster variance. Higher is better.<\/li>\n<\/ul>\n\n\n\n

          B) Dimensionality Reduction (e.g., PCA)<\/strong><\/h3>\n\n\n\n

          When reducing 100D data to 2D, how do we ensure important information is retained?<\/p>\n\n\n\n

            \n
          • Explained Variance<\/strong>: For example, 80% means 80% of the original data\u2019s information is preserved in the reduced dimensions.<\/li>\n<\/ul>\n\n\n\n
            \n\n\n\n

            4. Semi-Supervised Models: Combining Both Worlds<\/strong><\/h2>\n\n\n\n

            These models use both labeled and unlabeled data. Metrics like Accuracy or MSE (from supervised learning) are used, but the goal is to leverage unlabeled data to improve performance.<\/p>\n\n\n\n

            \n\n\n\n

            5. Final Takeaway: Choose the Right Metric!<\/strong><\/h2>\n\n\n\n

            No single metric works for all models. For example:<\/p>\n\n\n\n

              \n
            • Use RMSE or MAE for regression (predicting numbers).<\/li>\n\n\n\n
            • Prioritize Precision and Recall over Accuracy for classification (e.g., spam detection).<\/li>\n\n\n\n
            • Use Silhouette Score to evaluate clustering quality.<\/li>\n<\/ul>\n\n\n\n

              Always ask: \u201cWhat\u2019s the goal of this model?\u201d<\/strong> Then pick the metric that aligns with it.<\/p>\n\n\n\n

              <\/p>\n","protected":false},"excerpt":{"rendered":"

              Machine Learning (ML) is everywhere these days, from predicting house prices to diagnosing diseases! But when we build a model, how do we know if it\u2019s actually performing well? This…<\/p>\n","protected":false},"author":1,"featured_media":72,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[61,62],"tags":[53,52,58,56,49,50,51,54,60,55,57,59],"class_list":["post-59","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","category-model-evaluation","tag-classification-metrics","tag-data-clustering","tag-dimensionality-reduction","tag-f1-score","tag-machine-learning-model-evaluation","tag-model-accuracy","tag-prediction-error","tag-r-squared","tag-semi-supervised-learning","tag-silhouette-score","tag-supervised-learning","tag-unsupervised-learning"],"_links":{"self":[{"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/posts\/59","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/comments?post=59"}],"version-history":[{"count":3,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/posts\/59\/revisions"}],"predecessor-version":[{"id":64,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/posts\/59\/revisions\/64"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/media\/72"}],"wp:attachment":[{"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/media?parent=59"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/categories?post=59"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/amirhooshang.com\/blog\/wp-json\/wp\/v2\/tags?post=59"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}