Consistency of internal risk measure estimates

This paper concerns the computation of risk measures for financial data and asks how, given a risk measurement procedure, we can tell whether the answers it produces are correct. We draw the distinction between `external' and `internal' risk measures and concentrate on the latter, where we observe data in real time, make predictions and observe outcomes. It is argued that evaluation of such procedures is best addressed from the point of view of probability forecasting or Dawid's theory of `prequential statistics' [Dawid, JRSS(A)1984]. We introduce a concept of `consistency' of a risk measure, which is close to Dawid's `strong prequential principle', and examine its application to quantile forecasting (VaR -- value at risk) and to mean estimation (applicable to CVaR -- expected shortfall). We show in particular that VaR has special properties not shared by any other risk measure. In a final section we show that a simple data-driven feedback algorithm can produce VaR estimates on financial data that easily pass both the consistency test and a further newly-introduced statistical test for independence of a binary sequence.