Feature: model parameter reparameterizations

[ihincks]

Sometimes the natural parameterization of your model and the parameterization that is best for sampling do not agree. There are a number of reasons this might be the case, off the top of my head:

• distributions are more similar to gaussian (which Liu-West likes) in unnatural parameterization (a beta distribution does not look gaussian in many regimes, but reparametrized through a sigmoid logistic it does)
• distributions are less correlated in unnatural parameterization
• model parameters are easier to bound in unnatural parametrization
• canonicalize and are_models_valid can be made easier to implement
• timestep updates are simpler to implement

MCMC libraries such as stan have built in functionality to reparameterize bounded parameters and objects such as simplices.

I was originally thinking this would make sense as a DerivedModel. After some more careful consideration, I think it makes much more sense as an abstract class ModelReparameterization with methods to_natural and from_natural. Instances of such are given to an instance of SMCUpdaterReparameterized which is subclassed from SMCUpdater.

Let me know what you think.

[cgranade]

I've actually got some rather poorly-written experimental code for this, following the strategy of using DerivedModel (can post on a branch or as a separate gist), mainly for use with spherical ↔ Cartesian transformations, but I hadn't thought of associating reparameterization with the SMC updater itself. I suppose on option could be do it both ways, using a ModelReparameterization or ModelTransformation abstract class, instances of which could then be passed as input to a DerivedModel or SMCUpdater subclass. That would increase complexity, but would buy us a transition path that allows reusing perf_test and other functionality currently specialized to SMCUpdater itself. Thoughts, @csferrie?

[ihincks]

I have lost motivation for now since I realized there is something better to do in my use case. My partial work can be found in the branch ihincks/python-qinfer/feature-reparameterizations.