Got this email from VD. I’ve edited to remove any personal information and to add blog-standard style and links. I answered, and I remind all readers of the on-going claassre, but I thought I’d let readers have a go at answering, too.
I greatly appreciate the wealth of material contained on your website, and I am an avid reader of both your articles and papers and a consumer of your videos/lectures/podcasts on YouTube. You bring a clarity to the oft misunderstood, and—to an uncultured pleb such as myself—seemingly esoteric field of magical, complex formulae known as statistics.
I have a twofold question: First, do you have any plans to produce a textbook for students utilizing the principles within Uncertainty: The Soul of Modeling, Probability and Statistics—something along the lines of an updated Breaking the Law of Averages? I confess I have not yet read Uncertainty but assure you that it is at the top of my books-to-purchase list (although I’m under the impression much of the content therein is elucidated on your blog). If Uncertainty is the book I’m looking for then please let me know. I am also working through Breaking the Law and find it extremely helpful, lacking only in solutions to check my work.
If I simply need to go through Breaking the Law a few more times, please let me know if that’s the best route. In any event, I would appreciate a sequel that is an even better synthesis of the ideas since-developed and distilled in Uncertainty while also functioning as introductory-to-intermediate text on logical probability/objective Bayesian statistics. I appreciate your approach utilizing logic, common sense, and observation, to quantify the uncertainty for a given set of premises rather than becoming so consumed with parametrical fiddling that I forgot the nature of the problem I was trying to solve.
Second, if no new book is in the works, do you know of any good textbooks or resources for undiscerning novices such as myself for learning logical probability/objective Bayesian statistics that aren’t inundated with the baggage of frequentist ideals or the worst parts of classical statistics, baggage still dragged around by many of the currently available textbooks and outlets for learning statistics? It seems every other book or resource I pick up has at least a subset of the many errors and problems you’ve exposed and/or alluded to in your articles. If no such “pure” text exists, can you recommend one with a list of caveats? I also have found a copy of Jaynes’ Probability Theory, so I’ve added that to the pile of tomes to peruse. Since reading your blog I now make a conscious effort to mentally translate all instances of “random”, “chance”, “stochastic”, etc. to “unknown,” as well as actively oppose statements that “x entity is y-distributed (usually normally, of course!)” and recognize the fruits of the Deadly Sin of Reification (models and formulae, however elegant, are not reality).
I currently work to some degree as an analyst in Business Intelligence/Operations for a [large] company—a field where uncertainty, risk, and accurate predictive modeling are of paramount importance—and confess my grasp of mathematics and statistics is often lacking (I am in the process of reviewing my high school pre-calculus algebra and trigonometry so I can finally have a good-spirited go at calculus and hopefully other higher math). I think my strongest grasp at this point is philosophy (which I studied in undergrad with theology and language), and then logic and Boolean algebra, having spent a bit of time in web development and now coding Business Intelligence solutions. It’s the math and stats part that’s weak. If only I could go back 10 years and give myself a good talking to; hindsight’s 20-20 I suppose.
While not aiming to be an actuary by any measure, I want to be able to understand statements chock full of Bayesian terminology like the following excerpt from an actuarial paper on estimating loss. I want to discern whether such methods and statistics are correct:
“We will also be assuming that the prior distribution (that is, the credibility complement, in Bayesian terms) is normal as well, which is the common assumption. This is a conjugate prior and the resulting posterior distribution (that is, the credibility weighted result) will also be normal. Only when we assume normality for both the observations and the prior, Bayesian credibility produces the same results as Bühlmann-Straub credibility. The mean of this posterior normal distribution is equal to the weighted average of the actual and prior means, with weights equal to the inverse of the variances of each. As for the variance, the inverse of the variance is equal to the sum of the inverses of the within and between variances (Bolstad 2007).” (Uri Korn, “Credibility for Pricing Loss Ratios and Loss Costs,” Casualty Actuarial Society E-Forum, Fall 2015).
I understand maybe 25% of the previous citation.
My end goal is to professionally utilize the epistemological framework given on your blog and in Uncertainty. I want to be able to do modeling and statistics the right way, based on reality and observables, without the nuisances of parameters and infinity if they are not needed. I deal with mostly discrete events and quantifications bounded by intervals far smaller than (-infinity, +infinity) or (0, infinity),
I appreciate any advice you could share. Thank you sir!