MAT-427: KNN + Preprocessing

Eric Friedlander

Computational Setup

library(tidyverse)
library(tidymodels)
library(knitr)
library(modeldata) # contains ames dataset

tidymodels_prefer()

Comparing Models: Data Splitting with tidymodels

set.seed(427) # Why?

ames_split <- initial_split(ames, prop = 0.70, strata = Sale_Price) # initialize 70/30
ames_split
<Training/Testing/Total>
<2049/881/2930>
ames_train <- training(ames_split) # get training data
ames_test <- testing(ames_split) # get test data

K-Nearest Neighbors Regression (multiple predictors)

ames |>
  select(Sale_Price, Gr_Liv_Area, Bedroom_AbvGr) |>
  head() |> 
  kable()
Sale_Price Gr_Liv_Area Bedroom_AbvGr
215000 1656 3
105000 896 2
172000 1329 3
244000 2110 3
189900 1629 3
195500 1604 3
  • Should 1 square foot count the same as 1 bedroom?
  • Need to center and scale (freq. just say scale)
    • subtract mean from each predictor
    • divide by standard deviation of each predictor
    • compares apples-to-apples

Why center and scale

New observation

Code 
new_house <- tibble(Gr_Liv_Area = 2000, Bedroom_AbvGr = 2)
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr, color = Sale_Price)) + 
  geom_point() +
  geom_point(data = new_house, color = "red")
  • Where do you think the 10 nearest neighbors should be?

Computing Distances

ames_dist <- ames_train |> 
  mutate(dist = sqrt((Gr_Liv_Area - 2000)^2 + (Bedroom_AbvGr-2)^2)) |> 
  arrange(dist)

10 “Nearest Neighbors”

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr, color = Sale_Price)) + 
  geom_point() +
  geom_point(data = new_house, color = "red") +
  geom_point(data = ames_dist |> slice(1:10), color = "green")
  • Are they where you thought they should be?

Looking at distances

ames_dist |>
  select(Gr_Liv_Area, Bedroom_AbvGr, dist) |> 
  kable()
Gr_Liv_Area Bedroom_AbvGr dist
2000 3 1.000000
2000 3 1.000000
2004 4 4.472136
1995 4 5.385165
2006 2 6.000000
2007 3 7.071068
2007 3 7.071068
1992 3 8.062258
2008 3 8.062258
1991 3 9.055385
1991 4 9.219544
2009 4 9.219544
2009 4 9.219544
1988 4 12.165525
1987 2 13.000000
2014 2 14.000000
1986 3 14.035669
2014 3 14.035669
1986 4 14.142136
2014 5 14.317821
2016 4 16.124516
1984 5 16.278821
1983 3 17.029386
2018 3 18.027756
1982 3 18.027756
2019 3 19.026298
1981 4 19.104973
1981 4 19.104973
2020 3 20.024984
2020 3 20.024984
2020 3 20.024984
1980 3 20.024984
2021 3 21.023796
1978 2 22.000000
1978 4 22.090722
2022 4 22.090722
1977 3 23.021729
1976 2 24.000000
1973 3 27.018512
2028 2 28.000000
1971 3 29.017236
1970 3 30.016662
2031 3 31.016125
2031 3 31.016125
1968 4 32.062439
1968 4 32.062439
2034 2 34.000000
2034 3 34.014703
1966 1 34.014703
2035 3 35.014283
2036 3 36.013886
1964 3 36.013886
1964 3 36.013886
1964 4 36.055513
1964 4 36.055513
2037 3 37.013511
1962 3 38.013156
1962 4 38.052595
1960 3 40.012498
1960 4 40.049969
1959 4 41.048752
1959 5 41.109610
2042 3 42.011903
1958 3 42.011903
2042 3 42.011903
1958 3 42.011903
1958 4 42.047592
1955 4 45.044423
2046 2 46.000000
2046 3 46.010868
2046 3 46.010868
1954 3 46.010868
1953 3 47.010637
2048 3 48.010416
1952 4 48.041649
2048 5 48.093659
2049 4 49.040799
2050 4 50.039984
2052 3 52.009614
1948 4 52.038447
1947 3 53.009433
2054 3 54.009259
1944 3 56.008928
1944 3 56.008928
1944 4 56.035703
2057 3 57.008771
2058 3 58.008620
1940 3 60.008333
2060 3 60.008333
2060 4 60.033324
2061 3 61.008196
1939 3 61.008196
1939 3 61.008196
2061 4 61.032778
1938 4 62.032250
2064 3 64.007812
1936 3 64.007812
2064 4 64.031242
1935 3 65.007692
1935 3 65.007692
1934 3 66.007575
2067 3 67.007462
1933 4 67.029844
1932 2 68.000000
2068 3 68.007352
1932 3 68.007352
2069 4 69.028979
1930 4 70.028566
1929 3 71.007042
2071 4 71.028163
1928 4 72.027772
1928 4 72.027772
2073 3 73.006849
1923 3 77.006493
1922 2 78.000000
2078 3 78.006410
1922 3 78.006410
1922 4 78.025637
1921 4 79.025312
1920 4 80.024996
1920 4 80.024996
1920 4 80.024996
1916 3 84.005952
1915 3 85.005882
1914 3 86.005814
1913 3 87.005747
1912 3 88.005682
2088 4 88.022724
1911 3 89.005618
2090 3 90.005555
2090 3 90.005555
1909 4 91.021975
1908 4 92.021737
1908 4 92.021737
2093 3 93.005376
1906 3 94.005319
1905 3 95.005263
1904 3 96.005208
1904 3 96.005208
2097 3 97.005155
2097 1 97.005155
2097 3 97.005155
2098 3 98.005102
1902 4 98.020406
1902 4 98.020406
2098 4 98.020406
2100 3 100.005000
1898 3 102.004902
2104 5 104.043260
1895 3 105.004762
1894 4 106.018866
2108 4 108.018517
1891 3 109.004587
2110 3 110.004545
1889 4 111.018017
1886 4 114.017543
1884 2 116.000000
1884 2 116.000000
1884 4 116.017240
1882 4 118.016948
1880 3 120.004167
1879 3 121.004132
2121 3 121.004132
2122 4 122.016392
1877 4 123.016259
1876 4 124.016128
2125 4 125.015999
2126 3 126.003968
1874 3 126.003968
2127 3 127.003937
1873 3 127.003937
1873 4 127.015747
1872 3 128.003906
2128 4 128.015624
1872 4 128.015624
1869 2 131.000000
1869 2 131.000000
1868 3 132.003788
2132 1 132.003788
1868 4 132.015151
1867 2 133.000000
2133 4 133.015037
1866 2 134.000000
1866 4 134.014925
2134 5 134.033578
1865 3 135.003704
2136 4 136.014705
1864 6 136.058811
1863 4 137.014598
1861 3 139.003597
2140 3 140.003571
2140 4 140.014285
2142 4 142.014084
1856 1 144.003472
2144 4 144.013888
1852 3 148.003378
2149 1 149.003356
1848 3 152.003289
2153 3 153.003268
1846 3 154.003247
1846 3 154.003247
1845 3 155.003226
1844 2 156.000000
1844 3 156.003205
1844 3 156.003205
2156 3 156.003205
1844 3 156.003205
2157 4 157.012738
1842 3 158.003164
2158 4 158.012658
1840 3 160.003125
2161 3 161.003106
1839 4 161.012422
1839 4 161.012422
1838 3 162.003086
1838 3 162.003086
1836 3 164.003049
1836 3 164.003049
1836 4 164.012195
2167 3 167.002994
1832 4 168.011904
2168 5 168.026784
2169 3 169.002959
1830 3 170.002941
1829 4 171.011696
1828 3 172.002907
1828 3 172.002907
1828 3 172.002907
1828 3 172.002907
2172 4 172.011628
1827 2 173.000000
1826 3 174.002873
1824 3 176.002841
1824 4 176.011363
1824 5 176.025566
1823 3 177.002825
2177 5 177.025422
1822 3 178.002809
1822 4 178.011236
1820 3 180.002778
1820 3 180.002778
2180 3 180.002778
1820 4 180.011111
1820 4 180.011111
1818 3 182.002747
1818 4 182.010989
2183 4 183.010929
2184 3 184.002717
1816 3 184.002717
2184 4 184.010869
1812 3 188.002660
1812 3 188.002660
1812 4 188.010638
1811 2 189.000000
1809 3 191.002618
2192 4 192.010416
1806 4 194.010309
1804 3 196.002551
2196 3 196.002551
1803 3 197.002538
1803 3 197.002538
2197 4 197.010152
1802 3 198.002525
1802 3 198.002525
1802 4 198.010101
2198 4 198.010101
1801 3 199.002512
1801 3 199.002512
1800 2 200.000000
2200 3 200.002500
1800 3 200.002500
1798 4 202.009901
2202 4 202.009901
1797 3 203.002463
1797 3 203.002463
1797 3 203.002463
1796 3 204.002451
1795 3 205.002439
1795 3 205.002439
1795 4 205.009756
2207 3 207.002415
1792 2 208.000000
1792 3 208.002404
1792 3 208.002404
1792 4 208.009615
1790 4 210.009524
1789 3 211.002370
1788 3 212.002359
2212 3 212.002359
1788 5 212.021225
1787 3 213.002347
1786 3 214.002336
1786 3 214.002336
1786 3 214.002336
1786 3 214.002336
2214 3 214.002336
1784 4 216.009259
2217 4 217.009216
1782 3 218.002294
1782 3 218.002294
1780 3 220.002273
1779 3 221.002262
1779 3 221.002262
1778 2 222.000000
2223 3 223.002242
1776 3 224.002232
1776 3 224.002232
1776 4 224.008928
2224 4 224.008928
1775 3 225.002222
1775 3 225.002222
2225 4 225.008889
1774 4 226.008849
1773 3 227.002203
1772 3 228.002193
2228 6 228.035085
1771 3 229.002183
1771 3 229.002183
2229 5 229.019650
2230 5 230.019564
1768 2 232.000000
1768 3 232.002155
1768 3 232.002155
1768 3 232.002155
1768 4 232.008621
1767 2 233.000000
2233 5 233.019313
1766 3 234.002137
2234 3 234.002137
1764 3 236.002119
1764 4 236.008474
2236 4 236.008474
1762 3 238.002101
1762 4 238.008403
1760 4 240.008333
2240 6 240.033331
2243 4 243.008230
1756 3 244.002049
1755 3 245.002041
1755 3 245.002041
1755 4 245.008163
1752 4 248.008064
1750 3 250.002000
2250 3 250.002000
1750 4 250.008000
1749 3 251.001992
2253 3 253.001976
1746 3 254.001969
1746 3 254.001969
1744 3 256.001953
2256 4 256.007812
1743 3 257.001945
1743 3 257.001945
1743 0 257.007782
1742 2 258.000000
1742 4 258.007752
2259 3 259.001931
1740 2 260.000000
1740 2 260.000000
2260 3 260.001923
1740 3 260.001923
1740 4 260.007692
1740 4 260.007692
1738 4 262.007633
2262 4 262.007633
1737 3 263.001901
2263 3 263.001901
1734 2 266.000000
1734 3 266.001880
1734 3 266.001880
1734 3 266.001880
1733 3 267.001873
2267 3 267.001873
1733 4 267.007491
2267 4 267.007491
2268 3 268.001866
1730 3 270.001852
1728 3 272.001838
1728 3 272.001838
1728 3 272.001838
1728 3 272.001838
1728 3 272.001838
1728 4 272.007353
1728 4 272.007353
1728 6 272.029410
1728 6 272.029410
1728 6 272.029410
1728 6 272.029410
1728 6 272.029410
1726 3 274.001825
2274 5 274.016423
1725 3 275.001818
1724 3 276.001812
1724 3 276.001812
1724 3 276.001812
2276 3 276.001812
1721 3 279.001792
1721 3 279.001792
1721 3 279.001792
1721 4 279.007168
2279 4 279.007168
1720 3 280.001786
1720 3 280.001786
1719 1 281.001779
1718 3 282.001773
1718 3 282.001773
1718 3 282.001773
1717 2 283.000000
1717 3 283.001767
1717 3 283.001767
2283 4 283.007067
1716 2 284.000000
1716 3 284.001761
1716 3 284.001761
1716 4 284.007042
2285 4 285.007018
1714 3 286.001748
1714 3 286.001748
1712 4 288.006944
2288 4 288.006944
2290 2 290.000000
1710 3 290.001724
2290 4 290.006896
2290 4 290.006896
1709 2 291.000000
1709 3 291.001718
1709 3 291.001718
1709 3 291.001718
1709 3 291.001718
2291 4 291.006873
2291 4 291.006873
1708 3 292.001712
1707 2 293.000000
2294 5 294.015306
2295 4 295.006780
1704 3 296.001689
2296 3 296.001689
1702 1 298.001678
1701 3 299.001672
1701 4 299.006689
1700 2 300.000000
1700 4 300.006667
1699 3 301.001661
1698 3 302.001656
1696 3 304.001645
1696 3 304.001645
1696 3 304.001645
1694 2 306.000000
1694 3 306.001634
1694 3 306.001634
1694 3 306.001634
1694 4 306.006536
1692 3 308.001623
1691 2 309.000000
1691 3 309.001618
1690 3 310.001613
1690 3 310.001613
1690 4 310.006452
1689 3 311.001608
1689 3 311.001608
1689 3 311.001608
1688 2 312.000000
1688 4 312.006410
1687 3 313.001597
1686 3 314.001592
1686 1 314.001592
2314 3 314.001592
1685 2 315.000000
2315 4 315.006349
1684 2 316.000000
1683 3 317.001577
1682 3 318.001572
1682 1 318.001572
2318 3 318.001572
1680 2 320.000000
2320 2 320.000000
1680 3 320.001562
1680 3 320.001562
2320 3 320.001562
1680 4 320.006250
1680 4 320.006250
1680 4 320.006250
2322 4 322.006211
1678 6 322.024844
1677 3 323.001548
2324 4 324.006173
1675 3 325.001538
1675 3 325.001538
1674 3 326.001534
1673 3 327.001529
2327 4 327.006116
1671 3 329.001520
1671 3 329.001520
2329 4 329.006079
1671 4 329.006079
1670 2 330.000000
1670 3 330.001515
1670 4 330.006061
2331 3 331.001511
1669 4 331.006042
1668 3 332.001506
1668 3 332.001506
1668 3 332.001506
2332 3 332.001506
1668 3 332.001506
2332 4 332.006024
1666 2 334.000000
1666 3 334.001497
1666 3 334.001497
2334 3 334.001497
1665 3 335.001492
1664 3 336.001488
1664 3 336.001488
1664 4 336.005952
1664 4 336.005952
1663 3 337.001484
1662 3 338.001479
2338 4 338.005917
1661 3 339.001475
1660 3 340.001471
1660 3 340.001471
2340 4 340.005882
1659 2 341.000000
1659 3 341.001466
1658 3 342.001462
1657 3 343.001458
1657 3 343.001458
1656 2 344.000000
1656 3 344.001454
1654 3 346.001445
1654 3 346.001445
1654 4 346.005780
1653 3 347.001441
1652 2 348.000000
1652 2 348.000000
1652 3 348.001437
1652 3 348.001437
1652 3 348.001437
1652 4 348.005747
1652 4 348.005747
1651 4 349.005731
1648 2 352.000000
1647 3 353.001416
1647 3 353.001416
2353 4 353.005666
1646 2 354.000000
1646 2 354.000000
1646 2 354.000000
1646 2 354.000000
1646 3 354.001412
1646 3 354.001412
1644 3 356.001404
1642 3 358.001397
2358 4 358.005586
1642 4 358.005586
2358 4 358.005586
1641 3 359.001393
1641 3 359.001393
1640 3 360.001389
1640 3 360.001389
1640 3 360.001389
1640 3 360.001389
1639 3 361.001385
1638 2 362.000000
1638 3 362.001381
2362 3 362.001381
2364 2 364.000000
1636 3 364.001374
1636 3 364.001374
1635 2 365.000000
1635 3 365.001370
2365 3 365.001370
1634 3 366.001366
1633 3 367.001362
1632 3 368.001359
1632 4 368.005435
1632 4 368.005435
1632 4 368.005435
1630 3 370.001351
1630 3 370.001351
1630 3 370.001351
1629 3 371.001348
1629 3 371.001348
1629 3 371.001348
1628 3 372.001344
2372 4 372.005376
1627 4 373.005362
1626 2 374.000000
1626 2 374.000000
1626 2 374.000000
1626 3 374.001337
1626 3 374.001337
2374 4 374.005348
1625 2 375.000000
1625 2 375.000000
1624 2 376.000000
1624 2 376.000000
2376 4 376.005319
1621 3 379.001319
1621 3 379.001319
1620 2 380.000000
2380 3 380.001316
1620 4 380.005263
1620 4 380.005263
1618 2 382.000000
1616 2 384.000000
1616 3 384.001302
1615 3 385.001299
2385 3 385.001299
1614 3 386.001295
1614 3 386.001295
1614 3 386.001295
1612 2 388.000000
1612 3 388.001289
1612 3 388.001289
1611 3 389.001285
1611 4 389.005141
1610 3 390.001282
1610 4 390.005128
1609 3 391.001279
1609 3 391.001279
1609 5 391.011509
1608 3 392.001276
1608 3 392.001276
2392 3 392.001276
2392 3 392.001276
1606 3 394.001269
1605 3 395.001266
1604 3 396.001263
1604 3 396.001263
1604 3 396.001263
1604 3 396.001263
1604 3 396.001263
1603 4 397.005038
1602 3 398.001256
2398 3 398.001256
1601 3 399.001253
1601 3 399.001253
1600 2 400.000000
1600 3 400.001250
1600 3 400.001250
2400 4 400.005000
1598 3 402.001244
2403 4 403.004963
1596 3 404.001238
1595 2 405.000000
1595 2 405.000000
1594 2 406.000000
1594 3 406.001232
1594 3 406.001232
1593 3 407.001229
1593 3 407.001229
1592 3 408.001225
1590 3 410.001219
1589 2 411.000000
2411 4 411.004866
1588 3 412.001214
1588 3 412.001214
1588 3 412.001214
1588 5 412.010922
1587 3 413.001211
2414 3 414.001208
1586 3 414.001208
1585 3 415.001205
1584 4 416.004808
2417 4 417.004796
1582 3 418.001196
1582 3 418.001196
1582 3 418.001196
2418 3 418.001196
1582 4 418.004785
1580 3 420.001191
1580 3 420.001191
1578 3 422.001185
1578 3 422.001185
1578 3 422.001185
2422 4 422.004739
1577 2 423.000000
1577 3 423.001182
1576 2 424.000000
1576 3 424.001179
1576 3 424.001179
1576 3 424.001179
1575 2 425.000000
1575 2 425.000000
1574 3 426.001174
1574 3 426.001174
1573 3 427.001171
1573 3 427.001171
1573 3 427.001171
1573 3 427.001171
1573 3 427.001171
1572 3 428.001168
1572 3 428.001168
1571 3 429.001166
1571 3 429.001166
1569 1 431.001160
1568 3 432.001157
1568 3 432.001157
2432 4 432.004630
1567 1 433.001155
1566 5 434.010369
1565 2 435.000000
1565 2 435.000000
1564 2 436.000000
1564 3 436.001147
1564 3 436.001147
1561 2 439.000000
1561 3 439.001139
1560 3 440.001136
1560 3 440.001136
1560 3 440.001136
1560 4 440.004545
1560 4 440.004545
1559 2 441.000000
1558 3 442.001131
1556 3 444.001126
1556 3 444.001126
1556 4 444.004504
1556 4 444.004504
1555 2 445.000000
1554 2 446.000000
1553 3 447.001119
2447 4 447.004474
1552 3 448.001116
1552 3 448.001116
2448 3 448.001116
2448 4 448.004464
1551 3 449.001114
1550 3 450.001111
1549 3 451.001109
1548 2 452.000000
1548 3 452.001106
1548 3 452.001106
2452 3 452.001106
2454 3 454.001101
1546 3 454.001101
1544 3 456.001097
1541 3 459.001089
1540 3 460.001087
1540 3 460.001087
1540 4 460.004348
1539 3 461.001085
1539 3 461.001085
1538 3 462.001082
2462 4 462.004329
1537 3 463.001080
1536 3 464.001078
1536 3 464.001078
1536 3 464.001078
1536 4 464.004310
2464 4 464.004310
1535 3 465.001075
1535 4 465.004301
1535 4 465.004301
1534 3 466.001073
1533 2 467.000000
2468 4 468.004274
1531 2 469.000000
1531 3 469.001066
1530 3 470.001064
2470 1 470.001064
1530 3 470.001064
1528 3 472.001059
2473 4 473.004228
2473 4 473.004228
1526 2 474.000000
1526 4 474.004219
1525 3 475.001053
1525 3 475.001053
1525 3 475.001053
2475 4 475.004210
1524 2 476.000000
1524 3 476.001050
1523 3 477.001048
1522 4 478.004184
1521 3 479.001044
1521 4 479.004175
1520 3 480.001042
1520 3 480.001042
2480 5 480.009375
2482 4 482.004149
1517 3 483.001035
1516 3 484.001033
1515 3 485.001031
2486 5 486.009259
2486 5 486.009259
1513 2 487.000000
1513 4 487.004107
1513 4 487.004107
1512 2 488.000000
1512 2 488.000000
1512 2 488.000000
1512 3 488.001025
1511 2 489.000000
1511 3 489.001022
2490 2 490.000000
1510 3 490.001020
1510 3 490.001020
1510 4 490.004082
1509 3 491.001018
1508 1 492.001016
1507 4 493.004057
1506 2 494.000000
1506 3 494.001012
2494 4 494.004049
1505 2 495.000000
1505 3 495.001010
2495 4 495.004040
2495 5 495.009091
1504 2 496.000000
1504 1 496.001008
1504 3 496.001008
1504 3 496.001008
2497 2 497.000000
1502 3 498.001004
1502 3 498.001004
1502 1 498.001004
1501 2 499.000000
1501 2 499.000000
1501 3 499.001002
1501 3 499.001002
1501 3 499.001002
2499 4 499.004008
1500 2 500.000000
1500 3 500.001000
1500 3 500.001000
2500 5 500.009000
1499 3 501.000998
1499 3 501.000998
2501 4 501.003992
1498 2 502.000000
1498 3 502.000996
1497 3 503.000994
2503 3 503.000994
1496 3 504.000992
1496 3 504.000992
1496 3 504.000992
1496 3 504.000992
1495 3 505.000990
1495 3 505.000990
1494 2 506.000000
1494 3 506.000988
1494 3 506.000988
1494 3 506.000988
1494 3 506.000988
1494 3 506.000988
1493 3 507.000986
1492 3 508.000984
1492 3 508.000984
1492 3 508.000984
1491 3 509.000982
1491 3 509.000982
1490 3 510.000980
1490 3 510.000980
1489 3 511.000978
1489 3 511.000978
1489 3 511.000978
1489 3 511.000978
1488 2 512.000000
1488 3 512.000977
1488 3 512.000977
1488 3 512.000977
1487 3 513.000975
1486 3 514.000973
1486 3 514.000973
2514 4 514.003891
2515 4 515.003884
1484 3 516.000969
1484 3 516.000969
1484 3 516.000969
1482 3 518.000965
1482 4 518.003861
1481 3 519.000963
1480 3 520.000962
1480 4 520.003846
2520 4 520.003846
2520 5 520.008654
1479 3 521.000960
1479 3 521.000960
1479 3 521.000960
1479 3 521.000960
1479 3 521.000960
1479 5 521.008637
1478 2 522.000000
1478 3 522.000958
1478 3 522.000958
1476 3 524.000954
1476 3 524.000954
2524 4 524.003817
1475 4 525.003809
1474 3 526.000951
2526 4 526.003802
1473 1 527.000949
1472 2 528.000000
1472 3 528.000947
1472 3 528.000947
1471 3 529.000945
1470 2 530.000000
1470 3 530.000943
1470 3 530.000943
1470 4 530.003774
1468 2 532.000000
1468 3 532.000940
1466 3 534.000936
1466 3 534.000936
1465 3 535.000935
1465 3 535.000935
1464 3 536.000933
1464 3 536.000933
1464 4 536.003731
1463 3 537.000931
1458 2 542.000000
1458 3 542.000923
1458 3 542.000923
1458 3 542.000923
1456 2 544.000000
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 3 544.000919
1456 4 544.003676
1456 4 544.003676
2544 6 544.014706
1455 2 545.000000
1453 2 547.000000
1453 3 547.000914
1452 2 548.000000
1452 2 548.000000
1450 2 550.000000
1450 3 550.000909
2550 4 550.003636
1446 3 554.000903
1445 3 555.000901
1445 3 555.000901
1444 2 556.000000
1444 3 556.000899
1444 3 556.000899
1442 2 558.000000
1442 3 558.000896
1442 3 558.000896
1442 4 558.003584
1441 2 559.000000
1441 3 559.000894
2559 4 559.003578
1440 2 560.000000
1440 2 560.000000
1440 3 560.000893
1440 3 560.000893
1440 3 560.000893
1440 4 560.003571
1440 4 560.003571
1437 3 563.000888
1436 3 564.000886
1436 3 564.000886
1436 3 564.000886
1436 3 564.000886
1436 3 564.000886
1435 3 565.000885
1434 3 566.000883
1434 4 566.003534
1432 2 568.000000
1432 3 568.000880
1432 3 568.000880
1432 3 568.000880
1432 3 568.000880
1431 3 569.000879
1431 3 569.000879
1430 3 570.000877
1430 3 570.000877
1430 3 570.000877
1430 3 570.000877
1429 3 571.000876
1428 3 572.000874
1428 4 572.003496
1427 4 573.003490
1426 3 574.000871
2574 3 574.000871
1425 3 575.000870
1425 3 575.000870
1424 2 576.000000
1424 3 576.000868
2576 4 576.003472
1422 3 578.000865
1422 3 578.000865
1422 3 578.000865
1419 2 581.000000
1419 2 581.000000
1419 2 581.000000
1419 3 581.000861
1416 3 584.000856
1416 3 584.000856
1414 2 586.000000
1414 3 586.000853
1414 3 586.000853
1414 3 586.000853
1414 3 586.000853
1414 3 586.000853
1412 3 588.000850
1412 4 588.003401
1411 3 589.000849
1409 3 591.000846
1409 1 591.000846
2592 6 592.013513
1406 3 594.000842
1405 2 595.000000
1404 3 596.000839
1404 3 596.000839
2599 4 599.003339
1400 3 600.000833
1400 3 600.000833
2601 4 601.003328
1398 2 602.000000
1396 3 604.000828
1396 4 604.003311
1395 3 605.000826
1394 3 606.000825
1393 3 607.000824
1392 2 608.000000
1392 2 608.000000
1392 2 608.000000
1392 3 608.000822
1392 3 608.000822
1392 3 608.000822
1392 3 608.000822
1391 2 609.000000
1390 1 610.000820
2610 4 610.003279
1389 2 611.000000
1389 3 611.000818
1387 3 613.000816
1386 3 614.000814
2614 4 614.003257
1383 2 617.000000
1383 3 617.000810
1383 3 617.000810
1382 1 618.000809
1382 3 618.000809
1382 3 618.000809
1382 3 618.000809
1381 3 619.000808
2620 4 620.003226
1378 3 622.000804
2622 3 622.000804
1377 3 623.000803
1376 2 624.000000
1376 3 624.000801
1376 3 624.000801
2624 4 624.003205
1375 3 625.000800
1374 2 626.000000
1374 3 626.000799
1374 3 626.000799
1374 3 626.000799
1374 3 626.000799
1374 3 626.000799
1373 3 627.000797
1372 3 628.000796
1372 3 628.000796
1370 2 630.000000
1370 2 630.000000
1370 3 630.000794
1370 3 630.000794
2630 4 630.003175
1369 3 631.000792
1368 2 632.000000
1368 3 632.000791
1367 2 633.000000
1367 3 633.000790
2634 6 634.012618
1365 2 635.000000
1365 3 635.000787
1365 3 635.000787
1364 2 636.000000
1364 2 636.000000
1363 2 637.000000
1363 2 637.000000
1363 3 637.000785
1362 2 638.000000
1362 2 638.000000
1362 3 638.000784
1362 3 638.000784
1362 4 638.003135
1361 2 639.000000
1361 3 639.000783
1360 2 640.000000
1360 2 640.000000
1360 3 640.000781
1360 1 640.000781
2640 3 640.000781
2640 4 640.003125
2640 5 640.007031
1358 2 642.000000
1358 2 642.000000
1358 2 642.000000
1358 2 642.000000
1358 3 642.000779
1358 1 642.000779
1357 2 643.000000
2643 3 643.000778
1356 3 644.000776
1356 3 644.000776
1355 3 645.000775
1355 4 645.003101
2646 3 646.000774
2646 4 646.003096
1353 2 647.000000
1352 2 648.000000
1352 2 648.000000
1352 3 648.000772
1352 4 648.003086
1352 4 648.003086
1351 3 649.000770
2649 4 649.003082
1350 2 650.000000
1350 3 650.000769
1350 3 650.000769
2650 6 650.012308
1348 3 652.000767
1347 3 653.000766
1346 2 654.000000
1346 3 654.000764
2654 4 654.003058
2654 4 654.003058
1344 2 656.000000
1344 2 656.000000
1344 2 656.000000
1344 3 656.000762
1344 3 656.000762
1344 3 656.000762
1344 4 656.003049
1343 3 657.000761
1342 2 658.000000
1342 4 658.003039
1341 2 659.000000
1341 1 659.000759
1340 3 660.000758
1340 3 660.000758
1338 2 662.000000
1338 3 662.000755
1338 3 662.000755
1337 2 663.000000
1337 2 663.000000
1337 2 663.000000
1337 2 663.000000
1337 2 663.000000
1337 2 663.000000
1337 2 663.000000
1337 3 663.000754
1337 3 663.000754
1337 3 663.000754
1336 2 664.000000
1336 3 664.000753
1336 3 664.000753
1334 2 666.000000
1334 3 666.000751
1332 2 668.000000
2668 3 668.000748
1330 3 670.000746
1330 3 670.000746
1329 3 671.000745
1329 3 671.000745
1328 2 672.000000
1328 3 672.000744
1328 3 672.000744
1327 3 673.000743
2674 2 674.000000
1326 3 674.000742
1324 2 676.000000
1324 2 676.000000
1324 3 676.000740
1324 3 676.000740
1324 3 676.000740
1324 3 676.000740
1323 3 677.000739
1322 4 678.002950
1321 1 679.000736
1320 3 680.000735
1320 3 680.000735
1320 3 680.000735
1320 3 680.000735
1320 3 680.000735
1317 3 683.000732
2683 4 683.002928
1316 2 684.000000
1316 3 684.000731
1316 3 684.000731
1316 3 684.000731
1316 4 684.002924
1315 3 685.000730
1314 2 686.000000
1314 3 686.000729
1314 3 686.000729
1314 3 686.000729
1313 3 687.000728
2687 4 687.002911
2687 4 687.002911
1312 3 688.000727
1312 3 688.000727
1310 2 690.000000
1310 3 690.000725
2690 3 690.000725
1310 1 690.000725
2690 4 690.002899
1309 3 691.000724
1308 2 692.000000
1308 2 692.000000
1308 3 692.000723
1308 3 692.000723
1306 2 694.000000
1306 3 694.000721
1306 3 694.000721
1306 1 694.000721
1306 1 694.000721
1304 3 696.000718
1304 3 696.000718
2696 3 696.000718
2696 4 696.002874
1302 2 698.000000
1302 3 698.000716
1302 3 698.000716
1302 3 698.000716
1302 3 698.000716
1302 3 698.000716
1302 1 698.000716
1302 3 698.000716
2698 4 698.002865
1301 2 699.000000
1299 2 701.000000
1299 3 701.000713
1298 2 702.000000
1298 3 702.000712
1298 3 702.000712
1298 3 702.000712
1297 3 703.000711
1296 2 704.000000
1296 2 704.000000
1296 3 704.000710
2704 4 704.002841
1295 2 705.000000
1295 3 705.000709
1295 1 705.000709
1294 2 706.000000
1293 2 707.000000
1292 3 708.000706
1290 2 710.000000
1290 3 710.000704
1288 3 712.000702
1288 4 712.002809
1287 2 713.000000
1287 3 713.000701
1287 3 713.000701
2713 3 713.000701
1285 2 715.000000
1285 3 715.000699
1285 3 715.000699
2715 4 715.002797
1283 3 717.000697
1282 2 718.000000
1281 1 719.000695
1280 2 720.000000
1280 2 720.000000
1278 2 722.000000
1277 2 723.000000
2726 2 726.000000
1273 2 727.000000
1273 3 727.000688
2727 3 727.000688
2728 4 728.002747
1271 4 729.002743
1270 2 730.000000
2730 4 730.002740
1269 3 731.000684
1269 3 731.000684
1268 2 732.000000
1268 3 732.000683
1268 3 732.000683
1268 3 732.000683
1266 2 734.000000
1266 2 734.000000
1265 3 735.000680
1264 2 736.000000
1264 3 736.000679
1262 2 738.000000
1261 3 739.000677
1260 3 740.000676
1258 2 742.000000
1258 3 742.000674
1258 3 742.000674
1258 0 742.002695
1256 3 744.000672
1256 1 744.000672
1256 3 744.000672
1254 3 746.000670
1253 2 747.000000
1252 2 748.000000
1252 3 748.000668
1252 3 748.000668
1252 3 748.000668
1252 1 748.000668
1251 3 749.000668
1250 2 750.000000
1250 2 750.000000
1248 2 752.000000
1248 2 752.000000
1248 2 752.000000
1248 3 752.000665
1247 1 753.000664
1246 2 754.000000
1246 3 754.000663
1246 3 754.000663
1245 3 755.000662
1245 1 755.000662
1244 3 756.000661
1242 2 758.000000
1242 3 758.000660
1242 3 758.000660
2758 4 758.002638
1241 1 759.000659
1240 2 760.000000
1240 3 760.000658
1239 1 761.000657
1236 2 764.000000
1236 2 764.000000
1236 2 764.000000
1236 3 764.000654
1235 2 765.000000
1235 1 765.000654
1232 3 768.000651
1232 3 768.000651
1232 3 768.000651
1232 3 768.000651
1230 3 770.000649
1229 2 771.000000
1229 2 771.000000
1229 2 771.000000
1229 3 771.000649
1228 2 772.000000
1228 3 772.000648
1228 3 772.000648
2772 4 772.002591
1226 2 774.000000
1226 3 774.000646
1226 1 774.000646
1226 1 774.000646
1225 3 775.000645
1224 2 776.000000
1224 2 776.000000
1224 2 776.000000
1224 3 776.000644
1224 3 776.000644
1224 3 776.000644
1224 4 776.002577
1223 2 777.000000
1222 2 778.000000
1221 2 779.000000
1221 4 779.002567
1220 2 780.000000
1220 2 780.000000
1220 2 780.000000
1220 2 780.000000
1218 2 782.000000
1218 3 782.000639
1218 3 782.000639
1218 3 782.000639
1218 3 782.000639
1218 3 782.000639
1218 4 782.002557
1217 2 783.000000
1217 3 783.000639
1217 3 783.000639
1216 2 784.000000
1216 2 784.000000
1216 3 784.000638
1216 3 784.000638
1216 3 784.000638
2784 5 784.005740
1215 3 785.000637
2786 4 786.002544
2787 6 787.010165
2787 6 787.010165
1212 3 788.000635
1212 3 788.000635
1212 3 788.000635
1211 2 789.000000
1210 3 790.000633
1210 3 790.000633
2790 4 790.002532
1209 3 791.000632
1208 2 792.000000
1208 2 792.000000
1208 2 792.000000
1208 2 792.000000
1208 3 792.000631
1207 3 793.000631
2795 4 795.002516
1204 2 796.000000
1204 2 796.000000
1204 3 796.000628
1203 3 797.000627
2798 3 798.000627
1200 2 800.000000
1200 2 800.000000
1200 2 800.000000
1200 2 800.000000
1200 2 800.000000
1200 3 800.000625
1200 3 800.000625
1200 1 800.000625
1200 3 800.000625
1200 3 800.000625
1200 4 800.002500
1200 4 800.002500
1196 2 804.000000
1196 3 804.000622
1195 4 805.002485
1194 3 806.000620
1194 3 806.000620
1193 2 807.000000
1193 3 807.000620
1192 3 808.000619
1192 4 808.002475
1191 2 809.000000
1190 3 810.000617
1188 1 812.000616
1187 2 813.000000
1187 3 813.000615
1187 3 813.000615
2814 4 814.002457
1183 2 817.000000
1183 3 817.000612
1182 3 818.000611
1181 3 819.000610
1180 2 820.000000
1180 2 820.000000
1178 2 822.000000
1178 3 822.000608
1178 3 822.000608
2822 4 822.002433
1176 2 824.000000
1175 3 825.000606
1175 3 825.000606
1174 2 826.000000
1174 3 826.000605
2826 3 826.000605
1173 3 827.000605
1173 3 827.000605
1172 3 828.000604
1167 3 833.000600
1167 3 833.000600
1164 3 836.000598
1163 3 837.000597
2840 4 840.002381
1159 3 841.000595
1159 3 841.000595
1158 3 842.000594
1158 3 842.000594
1158 3 842.000594
1155 3 845.000592
1154 3 846.000591
1154 3 846.000591
1154 3 846.000591
1152 2 848.000000
1152 2 848.000000
1152 2 848.000000
1152 2 848.000000
1152 3 848.000590
1152 3 848.000590
1152 3 848.000590
1151 2 849.000000
1150 3 850.000588
1150 3 850.000588
1148 3 852.000587
1146 3 854.000586
1146 3 854.000586
1145 2 855.000000
2855 4 855.002339
1144 3 856.000584
1144 3 856.000584
1144 3 856.000584
1144 3 856.000584
1143 3 857.000583
1143 1 857.000583
1142 2 858.000000
1142 3 858.000583
1142 3 858.000583
1141 3 859.000582
1141 3 859.000582
1140 3 860.000581
1138 3 862.000580
1137 4 863.002318
1136 2 864.000000
1134 2 866.000000
1134 3 866.000577
1132 2 868.000000
1132 2 868.000000
2868 4 868.002304
1131 3 869.000575
1131 3 869.000575
1130 2 870.000000
1128 2 872.000000
1128 2 872.000000
1128 3 872.000573
1128 3 872.000573
2872 4 872.002294
2872 4 872.002294
1127 3 873.000573
1126 3 874.000572
1126 3 874.000572
1125 3 875.000571
1124 3 876.000571
1121 3 879.000569
1120 3 880.000568
1120 3 880.000568
1117 3 883.000566
1117 3 883.000566
1116 3 884.000566
1116 3 884.000566
1114 3 886.000564
1114 3 886.000564
1114 3 886.000564
1114 3 886.000564
1114 3 886.000564
1113 3 887.000564
1112 2 888.000000
1112 2 888.000000
1112 3 888.000563
1112 4 888.002252
1111 2 889.000000
1111 3 889.000562
1110 1 890.000562
1109 3 891.000561
1108 3 892.000561
1107 3 893.000560
1103 3 897.000557
1102 2 898.000000
2898 2 898.000000
1100 3 900.000556
1100 3 900.000556
1100 3 900.000556
1099 3 901.000555
1098 2 902.000000
1098 3 902.000554
1097 3 903.000554
1096 2 904.000000
1096 3 904.000553
1096 3 904.000553
1095 2 905.000000
1094 3 906.000552
1094 3 906.000552
1093 2 907.000000
1093 3 907.000551
1092 2 908.000000
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1092 3 908.000551
1091 2 909.000000
1090 2 910.000000
1089 3 911.000549
1088 2 912.000000
1088 2 912.000000
1086 3 914.000547
1086 3 914.000547
1082 3 918.000545
1081 3 919.000544
1080 2 920.000000
1080 2 920.000000
1080 3 920.000544
1080 3 920.000544
1078 2 922.000000
1078 3 922.000542
1077 2 923.000000
1077 2 923.000000
1077 3 923.000542
1075 3 925.000541
1074 3 926.000540
1073 2 927.000000
1073 3 927.000539
1073 3 927.000539
1073 3 927.000539
1072 2 928.000000
1072 2 928.000000
1072 2 928.000000
1072 2 928.000000
1072 2 928.000000
1072 2 928.000000
1069 2 931.000000
1069 2 931.000000
1069 2 931.000000
1068 2 932.000000
1068 3 932.000536
1067 2 933.000000
1064 2 936.000000
1064 3 936.000534
1063 3 937.000534
1062 3 938.000533
1061 1 939.000532
1060 3 940.000532
1060 1 940.000532
1060 3 940.000532
1059 3 941.000531
1057 3 943.000530
1057 3 943.000530
1057 3 943.000530
1056 2 944.000000
1056 3 944.000530
2944 3 944.000530
1056 3 944.000530
1056 3 944.000530
1056 0 944.002119
1055 2 945.000000
1055 2 945.000000
1055 2 945.000000
1055 2 945.000000
1055 2 945.000000
2945 3 945.000529
1054 3 946.000528
1054 3 946.000528
1053 2 947.000000
1053 3 947.000528
1053 3 947.000528
1052 3 948.000527
1052 3 948.000527
1051 3 949.000527
1050 2 950.000000
1050 3 950.000526
1050 3 950.000526
1049 2 951.000000
1049 2 951.000000
1048 2 952.000000
1048 3 952.000525
1045 2 955.000000
1045 3 955.000524
1045 3 955.000524
1044 3 956.000523
1043 2 957.000000
1041 3 959.000521
1040 2 960.000000
1040 2 960.000000
1040 2 960.000000
1040 2 960.000000
1040 2 960.000000
1040 2 960.000000
1040 2 960.000000
1040 2 960.000000
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1040 3 960.000521
1039 3 961.000520
1038 2 962.000000
1036 3 964.000519
1036 1 964.000519
1034 2 966.000000
1034 3 966.000518
1034 3 966.000518
1034 1 966.000518
1033 3 967.000517
1032 2 968.000000
1032 2 968.000000
1032 2 968.000000
1032 3 968.000517
1031 2 969.000000
1030 3 970.000516
1029 3 971.000515
1027 2 973.000000
1027 3 973.000514
1024 2 976.000000
1024 3 976.000512
1022 2 978.000000
2978 5 978.004601
1020 3 980.000510
1020 1 980.000510
1015 3 985.000508
1013 3 987.000507
1012 4 988.002024
1009 3 991.000505
1008 2 992.000000
1008 3 992.000504
1008 3 992.000504
1008 1 992.000504
1006 3 994.000503
1005 2 995.000000
1005 2 995.000000
1005 3 995.000503
1004 2 996.000000
1003 3 997.000502
1002 3 998.000501
1001 2 999.000000
999 3 1001.000500
999 3 1001.000500
998 3 1002.000499
996 2 1004.000000
996 3 1004.000498
3005 3 1005.000498
992 2 1008.000000
990 3 1010.000495
990 3 1010.000495
990 3 1010.000495
990 3 1010.000495
990 3 1010.000495
990 3 1010.000495
988 2 1012.000000
988 2 1012.000000
988 3 1012.000494
988 3 1012.000494
988 3 1012.000494
988 1 1012.000494
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
987 2 1013.000000
985 2 1015.000000
985 3 1015.000493
984 3 1016.000492
984 3 1016.000492
984 3 1016.000492
981 3 1019.000491
980 3 1020.000490
980 3 1020.000490
980 3 1020.000490
980 4 1020.001961
976 2 1024.000000
976 2 1024.000000
974 3 1026.000487
972 2 1028.000000
971 3 1029.000486
970 3 1030.000485
968 2 1032.000000
968 2 1032.000000
968 2 1032.000000
968 4 1032.001938
965 2 1035.000000
964 2 1036.000000
964 3 1036.000483
960 2 1040.000000
960 2 1040.000000
960 2 1040.000000
960 3 1040.000481
960 3 1040.000481
960 3 1040.000481
960 3 1040.000481
960 3 1040.000481
960 0 1040.001923
958 2 1042.000000
958 2 1042.000000
958 2 1042.000000
954 3 1046.000478
952 2 1048.000000
952 2 1048.000000
952 3 1048.000477
951 2 1049.000000
950 2 1050.000000
950 3 1050.000476
949 2 1051.000000
948 2 1052.000000
948 3 1052.000475
948 3 1052.000475
948 3 1052.000475
945 2 1055.000000
943 2 1057.000000
943 2 1057.000000
943 3 1057.000473
941 3 1059.000472
936 2 1064.000000
936 2 1064.000000
936 2 1064.000000
936 2 1064.000000
936 2 1064.000000
936 2 1064.000000
936 2 1064.000000
936 3 1064.000470
936 3 1064.000470
935 2 1065.000000
935 3 1065.000470
934 2 1066.000000
932 2 1068.000000
928 2 1072.000000
928 3 1072.000466
925 2 1075.000000
925 2 1075.000000
925 3 1075.000465
925 3 1075.000465
925 3 1075.000465
924 2 1076.000000
924 3 1076.000465
924 3 1076.000465
923 2 1077.000000
923 3 1077.000464
922 2 1078.000000
919 2 1081.000000
919 3 1081.000462
918 2 1082.000000
918 3 1082.000462
914 2 1086.000000
914 2 1086.000000
914 2 1086.000000
914 3 1086.000460
3086 3 1086.000460
3086 4 1086.001842
913 3 1087.000460
912 2 1088.000000
912 2 1088.000000
912 2 1088.000000
912 2 1088.000000
912 2 1088.000000
912 3 1088.000460
912 3 1088.000460
912 3 1088.000460
910 3 1090.000459
909 3 1091.000458
908 2 1092.000000
907 3 1093.000458
907 3 1093.000458
907 3 1093.000458
906 2 1094.000000
904 2 1096.000000
904 1 1096.000456
904 3 1096.000456
902 2 1098.000000
901 2 1099.000000
900 3 1100.000454
900 3 1100.000454
899 3 1101.000454
898 3 1102.000454
897 3 1103.000453
896 2 1104.000000
896 2 1104.000000
894 2 1106.000000
894 2 1106.000000
894 2 1106.000000
894 3 1106.000452
894 3 1106.000452
894 3 1106.000452
894 3 1106.000452
894 3 1106.000452
894 3 1106.000452
894 3 1106.000452
893 2 1107.000000
892 3 1108.000451
892 3 1108.000451
892 3 1108.000451
890 3 1110.000450
889 3 1111.000450
888 3 1112.000450
887 3 1113.000449
884 2 1116.000000
884 2 1116.000000
884 2 1116.000000
882 2 1118.000000
882 2 1118.000000
882 3 1118.000447
882 1 1118.000447
882 3 1118.000447
879 2 1121.000000
879 3 1121.000446
876 2 1124.000000
876 3 1124.000445
874 3 1126.000444
874 3 1126.000444
874 3 1126.000444
872 2 1128.000000
872 3 1128.000443
869 2 1131.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 2 1136.000000
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
864 3 1136.000440
861 1 1139.000439
3140 4 1140.001754
858 2 1142.000000
858 3 1142.000438
856 2 1144.000000
854 2 1146.000000
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
848 1 1152.000434
846 2 1154.000000
845 2 1155.000000
845 3 1155.000433
845 1 1155.000433
845 3 1155.000433
844 2 1156.000000
841 2 1159.000000
840 2 1160.000000
840 3 1160.000431
836 2 1164.000000
835 2 1165.000000
833 3 1167.000428
832 2 1168.000000
832 2 1168.000000
827 2 1173.000000
825 2 1175.000000
825 2 1175.000000
819 2 1181.000000
816 2 1184.000000
816 2 1184.000000
816 2 1184.000000
816 2 1184.000000
816 2 1184.000000
816 3 1184.000422
816 3 1184.000422
813 2 1187.000000
810 2 1190.000000
808 2 1192.000000
808 1 1192.000419
804 2 1196.000000
803 2 1197.000000
800 1 1200.000417
797 2 1203.000000
796 2 1204.000000
796 2 1204.000000
796 2 1204.000000
796 2 1204.000000
793 2 1207.000000
792 2 1208.000000
792 2 1208.000000
792 2 1208.000000
790 2 1210.000000
789 2 1211.000000
789 2 1211.000000
784 2 1216.000000
780 2 1220.000000
773 2 1227.000000
3228 4 1228.001629
768 2 1232.000000
768 2 1232.000000
768 2 1232.000000
768 1 1232.000406
767 1 1233.000405
765 2 1235.000000
764 2 1236.000000
759 1 1241.000403
756 2 1244.000000
754 2 1246.000000
752 2 1248.000000
747 2 1253.000000
747 1 1253.000399
733 2 1267.000000
732 2 1268.000000
725 1 1275.000392
3279 4 1279.001564
720 2 1280.000000
720 1 1280.000391
715 2 1285.000000
704 2 1296.000000
698 2 1302.000000
694 2 1306.000000
693 2 1307.000000
691 2 1309.000000
672 2 1328.000000
670 2 1330.000000
641 2 1359.000000
630 1 1370.000365
630 1 1370.000365
630 1 1370.000365
630 1 1370.000365
630 1 1370.000365
630 1 1370.000365
3390 5 1390.003237
605 2 1395.000000
3395 8 1395.012903
572 2 1428.000000
3447 4 1447.001382
540 1 1460.000342
3493 3 1493.000335
3500 4 1500.001333
498 1 1502.000333
480 1 1520.000329
438 1 1562.000320
407 1 1593.000314
3608 4 1608.001244
3672 5 1672.002691
3820 5 1820.002473
4316 4 2316.000864
4676 3 2676.000187
5095 2 3095.000000

Visualizing Centering and Scaling in 1-D

Original data

ggplot(ames_train, aes(x = Gr_Liv_Area)) +
  geom_histogram() +
  theme(text = element_text(size = 20))

Centering

  • What’s different between these two plots

Original data

Code 
ames_train |> 
ggplot(aes(x = Gr_Liv_Area)) +
  geom_histogram() +
  theme(text = element_text(size = 20))

Centered data

Code 
ames_train |> mutate(Gr_Liv_Area_Centered = Gr_Liv_Area - mean(Gr_Liv_Area)) |> 
ggplot(aes(x = Gr_Liv_Area_Centered)) +
  geom_histogram() +
  theme(text = element_text(size = 20))

Scaling

  • What’s different between these two plots

Original data

Code 
ames_train |> 
ggplot(aes(x = Gr_Liv_Area)) +
  geom_histogram() +
  theme(text = element_text(size = 20))

Normalized (centered and scaled) data

Code 
ames_train |> mutate(Gr_Liv_Area_Centered_Scaled = (Gr_Liv_Area - mean(Gr_Liv_Area))/sd(Gr_Liv_Area)) |> 
ggplot(aes(x = Gr_Liv_Area_Centered_Scaled)) +
  geom_histogram() +
  theme(text = element_text(size = 20))

Visualizing Centering and Scaling: 2D

Original Data

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point()

Centering

  • What’s different between these two plots

Original Data

Code 
ames_train |> 
ggplot(aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point()+
  theme(text = element_text(size = 20))

Centered data

Code 
ames_train |> mutate(Gr_Liv_Area_Centered = Gr_Liv_Area - mean(Gr_Liv_Area),
                     Bedroom_AbvGr_Centered = Bedroom_AbvGr - mean(Bedroom_AbvGr)) |> 
ggplot(aes(x = Gr_Liv_Area_Centered, y = Bedroom_AbvGr_Centered)) +
  geom_point() +
  theme(text = element_text(size = 20))

Scaling

  • What’s different between these two plots

Original data

Code 
ames_train |> 
ggplot(aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point()+
  theme(text = element_text(size = 20))

Normalized data

Code 
ames_train |> mutate(Gr_Liv_Area_Scaled = scale(Gr_Liv_Area),
                     Bedroom_AbvGr_Scaled = scale(Bedroom_AbvGr)) |> 
ggplot(aes(x = Gr_Liv_Area_Scaled, y = Bedroom_AbvGr_Scaled)) +
  geom_point() +
  theme(text = element_text(size = 20))

Dealing with new data

New Observation

Code 
new_house <- tibble(Gr_Liv_Area = 2000, Bedroom_AbvGr = 2)
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr, color = Sale_Price)) + 
  geom_point() +
  geom_point(data = new_house, color = "red")
  • Where should we put this point on the normalized plot?

Plotting new observation witout normalizing

Code 
new_house <- tibble(Gr_Liv_Area = 2000, Bedroom_AbvGr = 2)
ames_train |> mutate(Gr_Liv_Area_Scaled = scale(Gr_Liv_Area), Bedroom_AbvGr_Scaled = scale(Bedroom_AbvGr)) |> 
  ggplot(aes(x = Gr_Liv_Area_Scaled, y = Bedroom_AbvGr_Scaled)) +
  geom_point() +
  geom_point(data = new_house, mapping = aes(Gr_Liv_Area, y = Bedroom_AbvGr), color = "red")
  • Does this look right?

Normalized using training data

Original Data

Code 
new_house <- tibble(Gr_Liv_Area = 2000, Bedroom_AbvGr = 2)
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point() +
  geom_point(data = new_house, color = "red") +
  theme(text = element_text(size = 20))

Scaled data

Code 
ames_train_scaled <- 
  ames_train |> mutate(Gr_Liv_Area_Scaled = scale(Gr_Liv_Area), Bedroom_AbvGr_Scaled = scale(Bedroom_AbvGr))

new_house <- tibble(Gr_Liv_Area = 2000, Bedroom_AbvGr = 2) |> 
  mutate(Gr_Liv_Area_Scaled = (Gr_Liv_Area - mean(ames_train$Gr_Liv_Area))/sd(ames_train$Gr_Liv_Area), 
         Bedroom_AbvGr_Scaled = (Bedroom_AbvGr - mean(ames_train$Bedroom_AbvGr))/ sd(ames_train$Bedroom_AbvGr))

ames_train_scaled |> 
  ggplot(aes(x = Gr_Liv_Area_Scaled, y = Bedroom_AbvGr_Scaled)) +
  geom_point() +
  geom_point(data = new_house, color = "red")+
  theme(text = element_text(size = 20))

Compute new distances

new_house |> kable()
Gr_Liv_Area Bedroom_AbvGr Gr_Liv_Area_Scaled Bedroom_AbvGr_Scaled
2000 2 0.9900212 -1.041808 
ames_dist <- ames_train_scaled |> 
  mutate(orig_dist = sqrt((Gr_Liv_Area - 2000)^2 + (Bedroom_AbvGr-2)^2),
         scaled_dist = sqrt((Gr_Liv_Area_Scaled - 0.99)^2 + (Bedroom_AbvGr_Scaled-(-1.04))^2))

orig_closest <- ames_dist |> 
  arrange(orig_dist) |> 
  head(10)

scaled_closest <- ames_dist |> 
  arrange(scaled_dist) |> 
  head(10)

Scaled distances look better

orig_closest |> select(Gr_Liv_Area, Bedroom_AbvGr, Gr_Liv_Area_Scaled, Bedroom_AbvGr_Scaled, orig_dist, scaled_dist) |> kable()
Gr_Liv_Area Bedroom_AbvGr Gr_Liv_Area_Scaled Bedroom_AbvGr_Scaled orig_dist scaled_dist
2000 3 0.9900212 0.1621766 1.000000 1.2021766
2000 3 0.9900212 0.1621766 1.000000 1.2021766
2004 4 0.9979593 1.3661617 4.472136 2.4061749
1995 4 0.9800985 1.3661617 5.385165 2.4061821
2006 2 1.0019284 -1.0418085 6.000000 0.0120647
2007 3 1.0039129 0.1621766 7.071068 1.2022571
2007 3 1.0039129 0.1621766 7.071068 1.2022571
1992 3 0.9741448 0.1621766 8.062258 1.2022812
2008 3 1.0058975 0.1621766 8.062258 1.2022817
1991 3 0.9721603 0.1621766 9.055385 1.2023090 
scaled_closest |>  select(Gr_Liv_Area, Bedroom_AbvGr, Gr_Liv_Area_Scaled, Bedroom_AbvGr_Scaled, orig_dist, scaled_dist) |>  kable()
Gr_Liv_Area Bedroom_AbvGr Gr_Liv_Area_Scaled Bedroom_AbvGr_Scaled orig_dist scaled_dist
2006 2 1.0019284 -1.041809 6 0.01206470
1987 2 0.9642222 -1.041809 13 0.02584121
2014 2 1.0178047 -1.041809 14 0.02786344
1978 2 0.9463613 -1.041809 22 0.04367615
1976 2 0.9423922 -1.041809 24 0.04764210
2028 2 1.0455882 -1.041809 28 0.05561764
2034 2 1.0574955 -1.041809 34 0.06751968
2046 2 1.0813099 -1.041809 46 0.09132782
1932 2 0.8550726 -1.041809 68 0.13493957
1922 2 0.8352272 -1.041809 78 0.15478340

Which plot looks better

Original distances

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr, color = Sale_Price)) + 
  geom_point() +
  geom_point(data = new_house, color = "red") +
  geom_point(data = orig_closest, color = "green") +
  theme(text = element_text(size = 20))

Scaled distances

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr, color = Sale_Price)) + 
  geom_point() +
  geom_point(data = new_house, color = "red") +
  geom_point(data = scaled_closest, color = "green") +
  theme(text = element_text(size = 20))

Dealing with the test set

Scaling Test Set

  • Subtract mean of corresponding variable in training set
  • Divide by sd of corresponding variable in training set

Visualizing Training and Test Sets: Untransformed

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point(color = "red", alpha = 0.2) +
  geom_point(data = ames_test, color = "blue", alpha = 0.2) +
  theme(text = element_text(size = 20))

Visualizing Training and Test Sets: Self Centered

Original Data

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point(color = "red", alpha = 0.2) +
  geom_point(data = ames_test, color = "blue", alpha = 0.2) +
  theme(text = element_text(size = 20))

Scaled Data

Code 
ames_train_centered <- ames_train |> 
  mutate(Gr_Liv_Area_Scaled = scale(Gr_Liv_Area),
         Bedroom_AbvGr_Scaled = scale(Bedroom_AbvGr))
ames_test_self_centered <- ames_test |> 
  mutate(Gr_Liv_Area_Scaled = scale(Gr_Liv_Area),
         Bedroom_AbvGr_Scaled = scale(Bedroom_AbvGr))
ggplot(ames_train_centered, aes(x = Gr_Liv_Area_Scaled, y = Bedroom_AbvGr_Scaled)) + 
  geom_point(color = "red", alpha = 0.2) +
  geom_point(data = ames_test_self_centered, color = "blue", alpha = 0.2) +
  theme(text = element_text(size = 20))

Visualizing Training and Test Sets: Training Centered

Original data

Code 
ggplot(ames_train, aes(x = Gr_Liv_Area, y = Bedroom_AbvGr)) + 
  geom_point(color = "red", alpha = 0.2) +
  geom_point(data = ames_test, color = "blue", alpha = 0.2) +
  theme(text = element_text(size = 20))

Scaled data

Code 
ames_train_centered <- ames_train |> 
  mutate(Gr_Liv_Area_Scaled = scale(Gr_Liv_Area),
         Bedroom_AbvGr_Scaled = scale(Bedroom_AbvGr))
ames_test_self_centered <- ames_test |> 
  mutate(Gr_Liv_Area_Scaled = (Gr_Liv_Area - mean(ames_train$Gr_Liv_Area))/sd(ames_train$Gr_Liv_Area), 
         Bedroom_AbvGr_Scaled = (Bedroom_AbvGr - mean(ames_train$Bedroom_AbvGr))/ sd(ames_train$Bedroom_AbvGr))
ggplot(ames_train_centered, aes(x = Gr_Liv_Area_Scaled, y = Bedroom_AbvGr_Scaled)) + 
  geom_point(color = "red", alpha = 0.2) +
  geom_point(data = ames_test_self_centered, color = "blue", alpha = 0.2) +
  theme(text = element_text(size = 20))

Recipes and Workflows

Preprocessing + Feature Engineering

  • Preprocessing: reformatting and transforming data so it can be used for modeling
    • Centering and scaling
    • Converting categorical data into dummy variables or the correct format
    • Dealing with missing values
  • Feature Engineering: reformatting and transforming data to improve the performance of your model
    • Combining variables and dimension reduction techniques
    • Transforming variables (e.g. squaring or logging)
  • For recommended preprocessing steps see Appendix A of TMWR
  • recipes: combine feature engineering and preprocessing steps into single object that can be applied to different data sets

Model workflow

  • Start with raw data
  • Preprocessing and feature engineering steps
  • Fit “model”
  • Make predictions using model

Where does my model start?

  • Start with raw data
  • Model starts here
    • Preprocessing and feature engineering steps
    • Fit “model”
  • Make predictions using model

Model Assessment

  • Start with raw data
  • Split Data
  • Using training data
    • Preprocessing and feature engineering steps
    • Fit “model”
  • Make predictions on test set
  • Compute error metrics

workflows and recipes in tidymodels

  • tidymodels provides workflows that combine model fitting with preprocessing steps and consist of:
    • A model object
    • A recipe which combines all of the preprocessing steps

Creating a KNN model

knn10_model <- nearest_neighbor(neighbors = 10) |>   # 10-nn regression
  set_engine("kknn") |>
  set_mode("regression")

recipe’s in tidymodels

  1. Call receipe with R formula and training set to assign features roles (predictor vs. outcome).
  2. Sequence of step_* specifying the list of transformations.
  3. Apply recipe to new data using predict.

Center and Scale Recipe

ames_preproc <- recipe(Sale_Price ~ Bedroom_AbvGr + Gr_Liv_Area, data = ames_train) |> 
  step_normalize(all_numeric_predictors())

Create workflow

knn10_workflow <- workflow() |> 
  add_model(knn10_model) |> 
  add_recipe(ames_preproc)

Fitting Model

knnfit10 <- knn10_workflow |> 
  fit(data = ames_train)

Making predictions: Single observation

Test Point: Gr_Liv_area = 2000 square feet, and Bedroom_AbvGr = 3, then

new_house |> kable()
Gr_Liv_Area Bedroom_AbvGr Gr_Liv_Area_Scaled Bedroom_AbvGr_Scaled
2000 2 0.9900212 -1.041808 
# obtain 10-nn prediction
predict(knnfit10, new_data = new_house) |> kable()
.pred
342204.5

Making Predictions: Test Set

# obtain 10-nn prediction
predict(knnfit10, new_data = ames_test) |> head() |> kable()
.pred
178219.0
167930.0
177574.5
238122.2
126330.0
80109.5

Linear Regression vs K-Nearest Neighbors

  • Linear regression is a parametric approach (with restrictive assumptions), KNN is non-parametric.
  • Linear regression works for regression problems (\(Y\) numerical), KNN can be used for both regression and classification - i.e. \(Y\) qualitative
  • Linear regression is interpretable, KNN is not.
  • Linear regression can accommodate qualitative predictors and can be extended to include interaction terms as well while KNN does not allow for qualitative predictors
  • Performance: KNN can be pretty good for small \(p\), that is, \(p \le 4\) and large \(n\). Performance of KNN deteriorates as \(p\) increases - curse of dimensionality

Choosing a model

Linear regression

# Best model from last time
fit3 <- linear_reg() |> 
  set_engine("lm") |> 
  fit(Sale_Price ~ Gr_Liv_Area + Bedroom_AbvGr, data = ames_train)

5-NN

knn5_model <- nearest_neighbor(neighbors = 5) |> 
  set_engine("kknn") |> 
  set_mode("regression")

knn5_fit <- workflow() |> 
  add_model(knn5_model) |> 
  add_recipe(ames_preproc) |> 
  fit(data = ames_train)

Making test predictions

ames_test_preds <- ames_test |> 
  mutate(lr_preds = predict(fit3, new_data = ames_test)$.pred,
         knn5_preds = predict(knn5_fit, new_data = ames_test)$.pred,
         knn10_preds = predict(knnfit10, new_data = ames_test)$.pred)

Computing Error Metrics: RMSE

rmse(ames_test_preds, estimate = lr_preds, truth = Sale_Price) |> kable()
.metric .estimator .estimate
rmse standard 52272.26 
rmse(ames_test_preds, estimate = knn5_preds, truth = Sale_Price) |> kable()
.metric .estimator .estimate
rmse standard 50954.43 
rmse(ames_test_preds, estimate = knn10_preds, truth = Sale_Price) |> kable()
.metric .estimator .estimate
rmse standard 48926.63

Computing Error Metrics: \(R^2\)

rsq(ames_test_preds, estimate = lr_preds, truth = Sale_Price) |> kable()
.metric .estimator .estimate
rsq standard 0.574702 
rsq(ames_test_preds, estimate = knn5_preds, truth = Sale_Price) |> kable()
.metric .estimator .estimate
rsq standard 0.605208 
rsq(ames_test_preds, estimate = knn10_preds, truth = Sale_Price) |> kable()
.metric .estimator .estimate
rsq standard 0.6296263

🔗 MAT 427 - Spring 2025 - Schedule