Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

03 — Step 2: independent random-forest replication over Colombia

LifeWatch ERIC

Our own implementation of the paper’s final spatial model (ten biophysical predictors), trained on Fagan et al. (2022) labels in Colombia, compared with the paper on accuracy and with the authors’ published map on area and spatial pattern.

Model. sklearn.ensemble.RandomForestClassifier set to R randomForest classification defaults: 500 trees, mtry = floor(sqrt(10)) = 3 (max_features=3), nodesize = 1 (min_samples_leaf=1), bootstrap samples of size n with replacement, Gini splits. Fully grown trees have pure leaves, so predict_proba equals R’s vote fraction. Categorical predictors (land cover, biome) are integer-coded (sklearn has no native categorical splits in random forests; R does).

Sample size. Paper: final model on 1,000,000 balanced records, validation on 4.87 M independent balanced points. Scaled by Colombia’s share of the Fagan regrowth patches (188,921 / 4.78 M = 3.95 %): 39,523 training records. Validation uses every point in our independent validation set.

Areas. Exact WGS84 area of each 30 m pixel (primary); Mollweide (sphere R = a) and nominal 0.09 ha as sensitivity columns. Areas are estimated from the 1-in-100 systematic grid of 02b (each grid pixel stands for 10 × 10 pixels).

Calibration (extension; the paper does not calibrate). The RF is trained on 50/50 samples, so its vote fractions are not probabilities at Colombia’s true regrowth prevalence π. We report expected area (i) uncalibrated, as in the paper, and (ii) after the prior-shift correction p′ = π·p / (π·p + (1 − π)(1 − p)), with π = exact Fagan-regrowth area / exact area of the sampling domain (class-0 ∪ class-1 masks, 30 m, from 02b). An isotonic calibration fitted on one half of the validation set, weighted to prevalence π, is a cross-check; the other half is used for evaluation (Brier score, reliability diagram).

set         label
train_pool  0        75990
            1        75990
validation  0        61678
            1        61678
dtype: int64 
 12161726 grid points; {'TREE_THRESHOLD': 30, 'DENSITY_RADIUS_M': 1000.0, 'DIST_CAP_M': 25000.0, 'GRID_STRIDE': 10, 'TARGET_PER_CLASS': 214809, 'RATE0': 0.0007049645526895149, 'RATE1': 0.09729694934810547, 'n_candidates': 455224, 'n_per_class': 137668, 'n_tiles': 121}
training records: 39522 {0: 19761, 1: 19761} | validation: 123356
OOB accuracy 0.8844 (paper 0.878); validation accuracy 0.8867 (paper 0.879)
       map 0  map 1
ref 0  56766   4912
ref 1   9066  52612
Gini importance:
 dist_forest_2000       0.2732
forest_density_2000    0.1927
phihox                 0.1142
pc2                    0.0853
pc1                    0.0780
pc3                    0.0749
pc4                    0.0743
ocdens                 0.0712
lc_2000                0.0287
biome                  0.0075

Prevalence and calibration

prevalence pi = 251,707 ha regrowth / 15.307 Mha sampling domain = 0.01644
Brier score on the prevalence-weighted evaluation half: {'uncalibrated': 0.0708721991620386, 'prior_shift': 0.008147165560779194, 'isotonic': 0.00806318948344988}
<Figure size 550x500 with 1 Axes>

Prediction on the systematic grid and areas

prediction-domain grid points 2,468,623; with complete predictors 2,456,636 (99.5144%)

Spatial agreement with the authors’ map (same 30 m pixels)

pixels with authors' continuous value: 2,405,222  Pearson 0.678  Spearman 0.754
pixels with authors' binary value: 2,456,636  agreement 0.489  kappa 0.112; kappa vs pct>50: 0.130
share of our prediction-domain grid points that are NoData in the authors' continuous map: 0.021; binary: 0.000

HEALPix depth-8 aggregation (WGS84) and cell-level agreement

HEALPix d8: 1451 cells (>50 % of a full cell in the domain); Pearson expected-area fraction 0.857; binary fraction 0.538
cross-check vs stage-A full-resolution authors' product (whole country, area sums): Pearson 0.865
                                            metric   value     se  mollweide_sphere  nominal_009ha  paper_value                                                               note
                                      oob_accuracy  0.8844    NaN               NaN            NaN       0.8780                                                                   
                               validation_accuracy  0.8867    NaN               NaN            NaN       0.8790                       n=123356 balanced; paper Neotropics OA 0.865
                validation_users_accuracy_regrowth  0.9146    NaN               NaN            NaN       0.9200                                               paper SI A1.2 global
            validation_producers_accuracy_regrowth  0.8530    NaN               NaN            NaN       0.8520                                 paper SI A1.2 global (count based)
                                     prevalence_pi  0.0164    NaN               NaN            NaN          NaN             Fagan regrowth area / sampling-domain area, exact 30 m
                                brier_uncalibrated  0.0709    NaN               NaN            NaN          NaN              evaluation half of validation, weighted to prevalence
                                 brier_prior_shift  0.0081    NaN               NaN            NaN          NaN              evaluation half of validation, weighted to prevalence
                                    brier_isotonic  0.0081    NaN               NaN            NaN          NaN              evaluation half of validation, weighted to prevalence
                    expected_area_uncalibrated_mha  3.9887 0.0025            4.0150         4.6953      11.1900                                      sum p x area, as in the paper
                     expected_area_prior_shift_mha  0.1883 0.0004            0.1895         0.2212          NaN                                           calibrated to prevalence
                        expected_area_isotonic_mha  0.2097 0.0005            0.2111         0.2464          NaN                                                        cross-check
                    area_p_gt_0.5_uncalibrated_mha  2.0259 0.0037            2.0392         2.3845      13.7000                                                                   
                     area_p_gt_0.5_prior_shift_mha  0.0286 0.0005            0.0288         0.0335          NaN                                                                   
                        area_p_gt_0.5_isotonic_mha  0.0697 0.0007            0.0702         0.0816          NaN                                                                   
                        prediction_domain_area_mha 18.8847    NaN           19.0088        22.2257      93.7800                                                   exact 30 m count
                             study_domain_area_mha 93.2409    NaN           93.8606       109.4531      93.7800                                              Colombia x biomes 1-3
              authors_expected_in_domain_exact_mha 10.5070    NaN           10.5761        12.3650      11.1900 authors' map, exact 30 m sums within our study / prediction domain
                authors_expected_in_pred_exact_mha 10.2809    NaN           10.3485        12.0991      11.1900 authors' map, exact 30 m sums within our study / prediction domain
                authors_binary_in_domain_exact_mha 11.0515    NaN           11.1242        13.0060      13.7000 authors' map, exact 30 m sums within our study / prediction domain
                  authors_binary_in_pred_exact_mha 11.0515    NaN           11.1242        13.0060      13.7000 authors' map, exact 30 m sums within our study / prediction domain
          authors_binary_valid_in_domain_exact_mha 93.2409    NaN           93.8606       109.4531      93.7800 authors' map, exact 30 m sums within our study / prediction domain
            authors_binary_valid_in_pred_exact_mha 18.8847    NaN           19.0088        22.2257      93.7800 authors' map, exact 30 m sums within our study / prediction domain
      authors_continuous_valid_in_domain_exact_mha 18.7811    NaN           18.9046        22.1037      93.7800 authors' map, exact 30 m sums within our study / prediction domain
        authors_continuous_valid_in_pred_exact_mha 18.4003    NaN           18.5212        21.6557      93.7800 authors' map, exact 30 m sums within our study / prediction domain
                    pixel_pearson_p_vs_authors_pct  0.6782    NaN               NaN            NaN          NaN                                                          n=2405222
                   pixel_spearman_p_vs_authors_pct  0.7541    NaN               NaN            NaN          NaN                                                                   
             pixel_binary_agreement_vs_authors_bin  0.4888    NaN               NaN            NaN          NaN                                                          n=2456636
                        pixel_kappa_vs_authors_bin  0.1119    NaN               NaN            NaN          NaN                                                                   
                   pixel_kappa_vs_authors_pct_gt50  0.1297    NaN               NaN            NaN          NaN                                                                   
              share_pred_domain_authors_pct_nodata  0.0209    NaN               NaN            NaN          NaN                                                                   
              healpix_d8_pearson_expected_fraction  0.8569    NaN               NaN            NaN          NaN                                                         1451 cells
                healpix_d8_pearson_binary_fraction  0.5378    NaN               NaN            NaN          NaN                                                                   
healpix_d8_pearson_expected_area_vs_stageA_product  0.8645    NaN               NaN            NaN          NaN                           stage A = all 30 m pixels, whole country
<Figure size 1600x600 with 4 Axes>