Tricking your optimiser

I have found that optimisers (R’s optim to be precise) are sometime unstable. Not in a numerical sense, but in a more practical sense, i.e. choking on certain inputs (I wasn’t able to figure out when exactly) and in the end crashing, without the possibility of recovery or error-processing. Which is quite bad, when you’re in the mid of some long calculations, e.g. a Monte-Carlo experiment. I found that optim’s standard method, Nelder-Mead, was unsusceptible to such crashes (Please, geeks, explain me why!). But with Nelder-Mead, you cannot use parameter constraints, so you have to formulate your optimisation problem in an unconstrained way. Here are some of my tricks.

Strictly positive values: I learned this trick from the documentation of the dlm package by Petris Feed your optimiser with logs, which can be both negative and positive. Then, after optimisation calculate the actual parameters by plugging the optimiser’s output into the exponential function, you are then guaranteed to have positive values. This works very well for variances, period lengths or frequencies, anything that has to be strictly positive.

Values in a certain range: This one is a bit tricky, and probably there are more elegant ways to do this, but I use the arcus tangens function and scale and shift it accordingly. The atan has the basic shape I want, coming asymptotically from -\frac{\pi}{2}, crossing at the origin and going asymptotically to +\frac{\pi}{2}.
An example for a function that converts any real number to something in the range ]0;1[ would be:

y = \frac{1}{\pi}\cdot \arctan{(\lambda x)}+\frac{1}{2}

The parameter 0<\lambda<1 controls the steepness of the function. The lower, the flatter the function will become. I find a value of \lambda=0.01 to work pretty well.
You will then let the optimiser run to find the optimal x which can be any real number. However, the optimisation problem converts this x into something between zero and one and uses this for the calculations. This works for things like dampening factors or anything that is supposed to be in a certain range, without the boundaries. This function can easily be adopted to cover any arbitrary range, by adjusting the scaling factor  \frac{1}{\pi} and the vertical shift  \frac{1}{2}.

(to be continued …)

Published by


I write about economics and politics. I take Ordnungspolitik seriously. While not blogging, I study monetary unions for my doctorate.

Leave a Reply