Question

exam srm sample question 12

Answer 1

It has less variance than a single tree, but is still quite a flexible approach

Answer 2

The answer is E, if ordering methods by flexibility we roughly get Lasso and Ridge, Simple Linear, MLR, GLM, single decision trees, random forests and boosting (using bagging). So Linear is relatively inflexible, Lasso is less flexible than Linear, Bagging is relatively flexible, and flexibility is overall inversely related. Thus all 4 statements are false and the answer is E.

Answer 3

I agree that this is a bit of an unfortunate contradiction to the bias-variance trade-off setup, but just know that bagging is considered more flexible than regular regression trees grown with recursive binary splitting. There’s a graph on page 25 (sec. 2.1) of ISLR that helped me keep track of that.

Answer 4

Lower variance doesn't necessarily mean less flexibility. Bagging involves the bootstrap, which is quite flexible because it allows us to create "new" samples from the training data

Answer 5

The problem with bagging and random forests is that they are very difficult to interpret.  You get the benefit of having flexibility without the heavy cost of variance due to averaging but you get a more severe cost of loss of interpretability which simpler models provide.  

In more cases than not, flexibility leads to more variance but it also leads to a loss of interpretability.