Introduction to Gaussian Processes
An introduction to GPs, ending with PPL implementations.
'%3e%3cg%20transform='matrix(0.12928,0,0,0.12928,4056.614702,2455.921073)'%3e%3cg%20transform='matrix(1,0,0,1,-1041.666667,-1041.666667)'%3e%3cg%20transform='matrix(4.166667,0,0,4.166667,0,0)'%3e%3cpath%20d='M249.98,46C137.31,46%2046,137.353%2046,250.02C46,303.229%2066.377,351.67%2099.74,387.98C124.466,380.486%20149.437,373.515%20169.773,368.205C214.439,356.54%20283.163,338.753%20320.063,316.547C307.902,308.752%20287.034,300.28%20237.522,284.615C187.969,268.937%20133.012,252.361%20136.672,235.244C136.672,235.244%20136.804,234.766%20137.222,233.967C137.112,234.144%20136.94,234.31%20136.836,234.488C145.016,213.582%20169.729,158.832%20169.729,158.832C187.454,129.745%20265.922,108.936%20320.659,93.809C333.426,90.281%20348.675,85.949%20364.721,81.301C332.029,59.022%20292.521,45.998%20249.971,45.998L249.98,46ZM431.66,157.15C394.847,163.09%20278.71,184.171%20248.95,190.886C236.841,193.618%20220.205,197.188%20203.38,201.726C227.174,213.428%20234.595,219.28%20267.48,231.427C300.423,243.596%20349.822,271.346%20344.32,293.048C344.32,293.048%20344.083,293.676%20343.687,294.558C339.467,306.15%20326.097,335.964%20326.097,335.964C307.463,354.143%20222.777,374.304%20167.017,385.036C149.23,388.459%20128.928,392.416%20108.343,396.778C145.031,432.186%20194.955,453.998%20249.973,453.998C362.643,453.998%20453.993,362.685%20453.993,250.018C453.993,216.571%20445.923,185.006%20431.655,157.15L431.66,157.15ZM170.2,212.377C169.655,212.591%20169.122,212.808%20168.585,213.025C169.107,212.81%20169.66,212.593%20170.2,212.377M164.45,214.785C163.027,215.405%20161.638,216.036%20160.284,216.678C161.574,216.058%20163.015,215.421%20164.45,214.785M159.352,217.137C154.824,219.33%20150.757,221.651%20147.26,224.107C150.329,221.975%20154.26,219.633%20159.352,217.137M145.424,225.434C144.848,225.874%20144.246,226.309%20143.71,226.758C144.227,226.333%20144.829,225.882%20145.424,225.434'%20style='fill:rgb(65,65,65);fill-rule:nonzero;'/%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e){kind=small}
Stan '%3e%3cg%20transform='matrix(0.350002,0,0,0.350002,5868.906928,2674.234751)'%3e%3cg%20transform='matrix(1,0,0,1,-1508.333333,-1168.75)'%3e%3cg%20transform='matrix(4.166667,0,0,4.166667,0,0)'%3e%3cpath%20d='M361.453,485.937C162.329,485.937%200.906,377.828%200.906,244.469C0.906,111.109%20162.329,3%20361.453,3C560.578,3%20722,111.109%20722,244.469C722,377.828%20560.578,485.937%20361.453,485.937ZM416.641,97.406C265.289,97.406%20142.594,171.314%20142.594,262.484C142.594,353.654%20265.289,427.562%20416.641,427.562C567.992,427.562%20679.687,377.033%20679.687,262.484C679.687,147.971%20567.992,97.406%20416.641,97.406Z'%20style='fill:rgb(140,140,140);'/%3e%3c/g%3e%3cg%20transform='matrix(4.166667,0,0,4.166667,0,0)'%3e%3cpath%20d='M550,377C550,377%20571.822,383.585%20584.5,390C588.899,392.226%20596.51,396.668%20602,402.5C607.378,408.212%20610,414%20610,414L696,559L557,559.062L492,437C492,437%20478.69,414.131%20470.5,407.5C463.668,401.969%20460.755,400%20454,400L420.974,400L421,558.974L298,559.026L298,152.938L545,152.938C545,152.938%20657.5,154.967%20657.5,262C657.5,369.033%20550,377%20550,377ZM496.5,241.024L422.037,240.976L422,310.026L496.5,310.002C496.5,310.002%20531,309.895%20531,274.877C531,239.155%20496.5,241.024%20496.5,241.024Z'%20style='fill:rgb(64,64,64);'/%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e){kind=small}
R '%3e%3cg%20transform='matrix(2.62,0,0,2.62,5476.682446,2950.142713)'%3e%3cg%20transform='matrix(1,0,0,1,-50,-50)'%3e%3cg%20transform='matrix(4.166667,0,0,4.166667,0,0)'%3e%3cpath%20d='M9.86,2C8.291,2%207,3.291%207,4.86L7,6.54L11.29,6.54C11.68,6.54%2012,7.11%2012,7.5L4.86,7.5C3.291,7.5%202,8.791%202,10.36L2,14.141C2,15.71%203.291,17.001%204.86,17.001L6.04,17.001L6.04,14.321C6.04,14.318%206.04,14.314%206.04,14.311C6.04,12.748%207.327,11.461%208.89,11.461C8.89,11.461%2014.14,11.461%2014.14,11.461C15.72,11.461%2017,10.19%2017,8.61L17,4.86C17,3.291%2015.709,2%2014.14,2L9.86,2ZM9.14,3.61C9.54,3.61%209.86,3.73%209.86,4.32C9.86,4.91%209.54,5.211%209.14,5.211C8.75,5.211%208.43,4.911%208.43,4.321C8.43,3.731%208.75,3.61%209.14,3.61'%20style='fill:rgb(106,106,106);fill-rule:nonzero;'/%3e%3c/g%3e%3cg%20transform='matrix(4.166667,0,0,4.166667,0,0)'%3e%3cpath%20d='M17.959,7L17.959,9.68C17.959,9.683%2017.959,9.686%2017.959,9.689C17.959,11.252%2016.672,12.539%2015.109,12.539C15.109,12.539%209.86,12.539%209.86,12.539C9.857,12.539%209.853,12.539%209.85,12.539C8.287,12.539%207,13.826%207,15.389C7,15.389%207,19.139%207,19.139C7,20.708%208.291,21.999%209.86,21.999L14.14,21.999C15.709,21.999%2016.999,20.709%2017,19.14L17,17.46L12.709,17.46C12.319,17.46%2012,16.89%2012,16.5L19.14,16.5C20.709,16.5%2022,15.209%2022,13.64L22,9.86C22,8.291%2020.709,7%2019.14,7L17.959,7ZM8.32,11.513L8.316,11.517L8.354,11.513L8.32,11.513ZM14.86,18.789C15.25,18.789%2015.57,19.089%2015.57,19.679C15.57,20.068%2015.249,20.389%2014.86,20.389C14.46,20.389%2014.14,20.269%2014.14,19.679C14.14,19.089%2014.46,18.789%2014.86,18.789'%20style='fill:rgb(153,153,153);fill-rule:nonzero;'/%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e){kind=small}
Python '%3e%3ccircle%20cx='13.497'%20cy='281.63'%20r='9.555'%20fill='%23c62828'/%3e%3ccircle%20cx='36.081'%20cy='281.63'%20r='9.555'%20fill='%237e57c2'/%3e%3ccircle%20cx='24.722'%20cy='262.39'%20r='9.555'%20fill='%23388e3c'/%3e%3c/g%3e%3c/svg%3e){kind=small}
Julia We’ll explores how data can be modelled non-parametrically by placing probability distributions over functions. Over – … functions?! That still sounds wild to me even after writing this notebook.
In this notebook, I will walk through the math behind Gaussian Processes (GPs) and how they enable us to tackle this seemingly impossibly complex problem.
While working on this, I have heavily relied on what I learned from Mattias Villani’s Advanced Bayesian Learning course and the book Gaussian Processes for Machine Learning.
The data analysed in the examples towards the end of this notebook is Lidar data.