Semi-automatic ABC: ordinary meeting

December 19, 2011

Last week I had the pleasure of taking part in an RSS ordinary meeting on my paper with Paul Fearnhead and I thought I’d take the chance to write up my notes on it. They were a bit hurried so apologies if I misrepresent what anyone said!

The afternoon began with a Young Statistician’s Section pre-ordinary meeting. Michael Stumpf gave an introductory talk about ABC, starting from biological applications with intractable models to motivate the use of ABC. On our paper he commented that when using noisy ABC we “actively destroy” part of the data, and made a link between regression adjustment and Richard Wilkinson’s ideas that ABC is exact given measurement (or model) error. Then I gave a (hopefully) accessible introductory talk on the paper, the slides of which are here.

The main meeting began with Paul Fearnhead presenting the paper, including an overview of likelihood free methods in general and how ABC relates to them. Mark Beaumont proposed the vote of thanks. Amongst other comments he asked what information is lost between having sufficient summary statistics and our idea of calibration, and what other features of the posterior we could get by using other loss functions rather than quadratic loss. These comments were echoed by many of the other discussants, including the seconder, Christian Robert (whose slides are available on his blog). He also questioned whether our asymptotic result on the curse of dimensionality (Lemma 1) really deserves the interpretation we give it of motivating minimally low dimensional summary statisics and mentioned that in practice using more summary statistics than parameters seemed useful. Paul’s reply mentioned that we’ve had some success looking at regression methods, in particular sliced inverse regression, which can provide multiple summaries for each response (Some work on this is in my thesis).

The discussants began with Richard Wilkinson, who gave a very interesting contribution on the difference between our use of calibration and its standard use in the literature (going back to Dawid’s prequential statistics). The idea is that we are calibrated against our prior beliefs rather than reality. Julien Cornebise mentioned that ABC forms a non-parametric estimate of the likelihood, while Simon Wood’s synthetic likelihood approach makes a Gaussian estimate, and wondered whether this idea could be used within ABC. Mark Girolami aksed if it may sometimes be of more intrinsic interest to find the posterior conditional on certain summaries rather than the full data, an idea also proposed in Simon Wood’s Nature paper. Anthony Lee presented some potentially very interesting parallel MCMC algorithms for ABC, although the details were too much for me to take in! Simon White talked about potential methods of using noisy ABC for sequential inference, specialising in iid data, rather than state space models as we looked at in the paper. Xia Yingeun started by describing ABC-like methods going back to Student! He wondered how our approach would do on a continuous time version of the Ricker model for a blowfly dataset, as analysed in Simon Wood’s paper.

Christophe Andrieu noted that intractable models typically have the form Y=\phi(\theta,U) i.e. the data is a function of parameters \theta and random variables U for which the distribution is known. This opens up the possibility of latent variable approaches (One related published work, specialised to some particular models, is Peter Neal’s idea of coupled ABC). Simon Wood commented on difficulties stably reproducing our results for the Ricker model taken from his Nature paper, but also some success in using the general idea of a regression on the parameter values to produce summary statistics. I didn’t catch the name of the final contributor, but he suggested focussing on inference for marginal parameter distributions, and using the notion of marginal sufficiency (not something I’d seen before).

There wasn’t time to read the written contributions, only a tantalising list of contributors, including Andrew Gelman, Ajay Jasra, Michael Blum and Olivier Francois, Brad Efron and Kanti Mardia.

Overall it was a great experience as a young researcher to get this much feedback on research I’d contributed to, and I look forward to working on the response!


Semi-automatic ABC – list of typos

December 8, 2011

I’m very pleased that next week my paper with Paul Fearnhead is being discussed at the Royal Statistical Society. Unfortunately there are still several typos in the manuscript so I’ll keep an up to date list of them here. This is expanded from a comment I wrote on Xian Robert’s blog – thanks to him and others for spotting these. I’ll post up a link to my summary slides on the paper for the pre-ordinary meeting when I’ve finished writing them…

Typos:

  • We write K[\ldots]/h in several places where K[\ldots/h] would be correct.
  • The definition of Algorithm 3 on page 6 should define s_\text{obs} = S(y_\text{obs}) + hx, not h(x).
  • Theorem 1 should state “Algorithm 3 produces an ABC posterior that is calibrated”, not Algorithm 1.
  • In the last equation in the proof of Theorem 2, the final \theta should be \theta_0.
  • In the proof of Theorem 2, there is a 1/h factor before an integral in two places. This should be 1/h^d (where d is the dimension of the summary statistics).
  • On line 44 of page 4 we write h(\theta), which should be a(\theta).