throws Exception

There was a big debate at work around Exception declaration in a Java API. I was quite surprised that such an apparently simple subject could end up being so controversial. The controversy was around the choice of declaring in the interfaces:

void myMethod() throws Exception

instead of

void myMethod() throws MyAPIException
void myMethod() throws MyAPIRuntimeException
void myMethod()

where MyAPI represents either a generic API related exception or a specific exception related to the method in question.

The choice of "throws Exception" did not even occur to me as a possibility, but after some digging, I found that some relatively famous libraries actually followed that principle at one point, for example Apache Struts 1.x or Spring MVC.

More modern libraries, like Google Guava, commons-math 3.x, Struts 2.x generally favor MyAPIRuntimeException where MyAPI is actually context-specific. Some old popular libraries declare a checked Exception, for example the HibernateException in Hibernate.

This seems to be a recurring subject on Stackoverflow:
Stackoverflow - Java interface throws Exception best practice
Stackoverflow - What to put in the throws clause of an interface method

But those are quite poor in terms of explanations. The best comments on this subjects are from:
Anders Hejlsberg (C#, Delphi, Turbo Pascal creator) - The Trouble with Checked Exceptions
James Gosling (Java creator) - Failure and Exceptions


This comment from Anders is particularly acute:
"To work around this requirement, people do ridiculous things. For example, they decorate every method with, "throws Exception." That just completely defeats the feature, and you just made the programmer write more gobbledy gunk. That doesn't help anybody. "



Today I believe the API in question declares "throws Exception"...

throws Exception

There was a big debate at work around Exception declaration in a Java API. I was quite surprised that such an apparently simple subject could end up being so controversial. The controversy was around the choice of declaring in the interfaces:

void myMethod() throws Exception

instead of

void myMethod() throws MyAPIException
void myMethod() throws MyAPIRuntimeException
void myMethod()

where MyAPI represents either a generic API related exception or a specific exception related to the method in question.

The choice of "throws Exception" did not even occur to me as a possibility, but after some digging, I found that some relatively famous libraries actually followed that principle at one point, for example Apache Struts 1.x or Spring MVC.

More modern libraries, like Google Guava, commons-math 3.x, Struts 2.x generally favor MyAPIRuntimeException where MyAPI is actually context-specific. Some old popular libraries declare a checked Exception, for example the HibernateException in Hibernate.

This seems to be a recurring subject on Stackoverflow:
Stackoverflow - Java interface throws Exception best practice
Stackoverflow - What to put in the throws clause of an interface method

But those are quite poor in terms of explanations. The best comments on this subjects are from:
Anders Hejlsberg (C#, Delphi, Turbo Pascal creator) - The Trouble with Checked Exceptions
James Gosling (Java creator) - Failure and Exceptions


This comment from Anders is particularly acute:
"To work around this requirement, people do ridiculous things. For example, they decorate every method with, "throws Exception." That just completely defeats the feature, and you just made the programmer write more gobbledy gunk. That doesn't help anybody. "



Today I believe the API in question declares "throws Exception"...

KDE, XFCE, Gnome-Shell in 2014

Many people (and notoriously, Linus Torvald) complained about Gnome-shell, especially the early iterations. Similarly KDE 4 was a nightmare of instability and inflexibility when it came out. And XFCE has always sounded a bit too basic. the moves of Gnome and KDE were particularly shocking as the earlier iteration: Gnome 2 and KDE 3 were well appreciated, productive environments.

Gnome Shell 3.10

It took me a bit of time to get used to it, and in the early stages I went to KDE 4 for a while, only to come back to it later.

• Positive aspects: lots of space on the desktop, things don't get in the way, looks good,very good desktop overview (fast and well presented), a dock by default, great external monitor support (plug and play, remembers settings automatically), best OSD (volume) of all.
• Negative aspects: the notifications bar looks awkward and badly integrated (better with an extension), still unstable and big memory leaks (on Fedora 20, where the integration should be the best, it regularly crashes, starts with 300Mb and goes up to 1Gb in a couple of days), fallback-session completely useless as one can not customize it at all. But the killer for my work was  inability to share the desktop with Webex, while XFCE could.

KDE

I gave it a long try especially in 2012, it has not changed much in 2014. My opinion of it fell when I tried it a very short time after months of Gnome Shell, and even more so after seeing the trouble my parents had with it, compared to Gnome 2.

• Positive aspects: desktop search (needs to be configured in order to scan only the relevant folders, used to be slow and resource intensive, not so much in 2014) 
• Negative aspects: resource hog, awful start menu, too many shiny effects by default that only distract the user from his task, silly concepts like activities, every aspect of the desktop seems to require tweaking in non obvious ways for it to be more usable, looks ok but not great.

XFCE

On Fedora, the default XFCE is very very basic, so much that I could hardly see a difference with one from 10 years ago. On Xubuntu, it's much much better. When I came to it from Gnome-Shell, I was surprised at how good was the "old" desktop paradigm for productivity. I also surprisingly found multiple desktops more natural to use than on Gnome Shell/KDE.
On Fedora the way to make it like Xubuntu is to install elementary icons, the whisker menu and choose the greybird/bluebird themes.

yum groups install "Xfce Desktop"
yum install xfce4-mixer.x86_64 xfce4-whiskermenu-plugin.x86_64 xfce4-cpugraph-plugin.x86_64 xfce4-mount-plugin.x86_64 xfce4-icon-theme.noarch google-droid* elementary-xfce-icon-theme.noarch xfce4-volumed.x86_64 pavucontrol.x86_64

• Positive aspects: fast and lean, great start menu.
• Negative aspects: external monitor support could be more automatic like Gnome-Shell, no nice overview of all windows, default installation can be a bit too bare, sometimes not sexy (volume applet is ugly, xubuntu provides the unity indicators in xfce as a remedy), primitive OSD.

Cinnamon, Unity, Conclusion

I gave a short try to cinnamon as well, in hopes that it was more stable than gnome shell. In short, it was not. It's certainly less of a memory hog, but I had some strange behavior with an additional phantom panel sometimes appearing at the bottom at the screen. And overall it looks a lot less polished.

Unity is more interesting, but it's too Ubuntu centric, I don't like the start button equivalent (slow, badly presented, don't care about HUD), the windows overview is not as useful as Gnome shell, the dock, something I usually like, is strangely annoying.

This is a very subjective review, my feeling is that in 2014, people should not waste their time with KDE or Cinnamon. Gnome shell could be worth a try if you don't care so much about memory leaks and slight instability but value a distraction free desktop. Otherwise go for XFCE or Unity on (X)ubuntu.

KDE, XFCE, Gnome-Shell in 2014

Many people (and notoriously, Linus Torvald) complained about Gnome-shell, especially the early iterations. Similarly KDE 4 was a nightmare of instability and inflexibility when it came out. And XFCE has always sounded a bit too basic. the moves of Gnome and KDE were particularly shocking as the earlier iteration: Gnome 2 and KDE 3 were well appreciated, productive environments.

Gnome Shell 3.10

It took me a bit of time to get used to it, and in the early stages I went to KDE 4 for a while, only to come back to it later.

• Positive aspects: lots of space on the desktop, things don't get in the way, looks good,very good desktop overview (fast and well presented), a dock by default, great external monitor support (plug and play, remembers settings automatically), best OSD (volume) of all.
• Negative aspects: the notifications bar looks awkward and badly integrated (better with an extension), still unstable and big memory leaks (on Fedora 20, where the integration should be the best, it regularly crashes, starts with 300Mb and goes up to 1Gb in a couple of days), fallback-session completely useless as one can not customize it at all. But the killer for my work was  inability to share the desktop with Webex, while XFCE could.

KDE

I gave it a long try especially in 2012, it has not changed much in 2014. My opinion of it fell when I tried it a very short time after months of Gnome Shell, and even more so after seeing the trouble my parents had with it, compared to Gnome 2.

• Positive aspects: desktop search (needs to be configured in order to scan only the relevant folders, used to be slow and resource intensive, not so much in 2014) 
• Negative aspects: resource hog, awful start menu, too many shiny effects by default that only distract the user from his task, silly concepts like activities, every aspect of the desktop seems to require tweaking in non obvious ways for it to be more usable, looks ok but not great.

XFCE

On Fedora, the default XFCE is very very basic, so much that I could hardly see a difference with one from 10 years ago. On Xubuntu, it's much much better. When I came to it from Gnome-Shell, I was surprised at how good was the "old" desktop paradigm for productivity. I also surprisingly found multiple desktops more natural to use than on Gnome Shell/KDE.
On Fedora the way to make it like Xubuntu is to install elementary icons, the whisker menu and choose the greybird/bluebird themes.

yum groups install "Xfce Desktop"
yum install xfce4-mixer.x86_64 xfce4-whiskermenu-plugin.x86_64 xfce4-cpugraph-plugin.x86_64 xfce4-mount-plugin.x86_64 xfce4-icon-theme.noarch google-droid* elementary-xfce-icon-theme.noarch xfce4-volumed.x86_64 pavucontrol.x86_64

• Positive aspects: fast and lean, great start menu.
• Negative aspects: external monitor support could be more automatic like Gnome-Shell, no nice overview of all windows, default installation can be a bit too bare, sometimes not sexy (volume applet is ugly, xubuntu provides the unity indicators in xfce as a remedy), primitive OSD.

Cinnamon, Unity, Conclusion

I gave a short try to cinnamon as well, in hopes that it was more stable than gnome shell. In short, it was not. It's certainly less of a memory hog, but I had some strange behavior with an additional phantom panel sometimes appearing at the bottom at the screen. And overall it looks a lot less polished.

Unity is more interesting, but it's too Ubuntu centric, I don't like the start button equivalent (slow, badly presented, don't care about HUD), the windows overview is not as useful as Gnome shell, the dock, something I usually like, is strangely annoying.

This is a very subjective review, my feeling is that in 2014, people should not waste their time with KDE or Cinnamon. Gnome shell could be worth a try if you don't care so much about memory leaks and slight instability but value a distraction free desktop. Otherwise go for XFCE or Unity on (X)ubuntu.

Two SABR for the same smile

While playing around with differential evolution & SABR calibration, I noticed that sometimes, several set of parameters can lead to a very similar smile, usually the good one is for relatively low vol of vol and the bad one is for relatively high vol of vol. I first looked for errors in my implementation, but it's a real phenomenon.

I used the normal implied volatility formula with beta=1, then converted it to lognormal (Black) volatility. While it might not be a great idea to rely on the normal formula with beta=1, I noticed the same phenomenon with the arbitrage free PDE density approach, especially for long maturities. Interestingly, I did not notice such behavior before with other stochastic volatility models like Heston or Schobel-Zhu: I suspect it has to do with the approximations rather than with the true behavior of SABR.

Differential evolution is surprisingly good at finding the global minimum without much initial knowledge, however when there are close fits like this it can be more problematic, usually this requires pushing the population size up. I find that differential evolution is a neat way to test the robustness (as well as performance) of different SABR algorithms as it will try many crazy sets.

In practice, for real world calibration, there is not much use of differential evolution to calibrate SABR as it is relatively simple to find a good initial guess.

Two SABR for the same smile

While playing around with differential evolution & SABR calibration, I noticed that sometimes, several set of parameters can lead to a very similar smile, usually the good one is for relatively low vol of vol and the bad one is for relatively high vol of vol. I first looked for errors in my implementation, but it's a real phenomenon.

I used the normal implied volatility formula with beta=1, then converted it to lognormal (Black) volatility. While it might not be a great idea to rely on the normal formula with beta=1, I noticed the same phenomenon with the arbitrage free PDE density approach, especially for long maturities. Interestingly, I did not notice such behavior before with other stochastic volatility models like Heston or Schobel-Zhu: I suspect it has to do with the approximations rather than with the true behavior of SABR.

Differential evolution is surprisingly good at finding the global minimum without much initial knowledge, however when there are close fits like this it can be more problematic, usually this requires pushing the population size up. I find that differential evolution is a neat way to test the robustness (as well as performance) of different SABR algorithms as it will try many crazy sets.

In practice, for real world calibration, there is not much use of differential evolution to calibrate SABR as it is relatively simple to find a good initial guess.

Heston vs SABR slice by slice fit

Some people use Heston to fit one slice of a volatility surface. In this case, some parameters are clearly redundant. Still, I was wondering how it fared against SABR, which is always used to fit a slice. And what about Schobel-Zhu?

Aggregated error in fit per slice on 10 surfaces

With Heston, the calibration is actually slightly better with kappa=0, that is, without mean reversion, because the global optimization is easier and the mean reversion is fully redundant. It's still quite remarkable that 3 parameters result in a fit as good as 5 parameters.
This is however not the case for Schobel-Zhu, where each "redundant parameter" seem to make a slight difference in the quality of calibration. kappa = 0 deteriorate a little bit the fit (the mean error is clearly higher), and theta near 0 (so calibrating 4 parameters) is also a little worse (although better than kappa = 0). Also interestingly, the five parameters Schobel-Zhu fit is slightly better than Heston, but not so when one reduce the number of free parameters.

So what about Heston vs SABR. It is interesting to consider the case of general Beta and Beta=1: it turns out that as confirmed for equities, beta=1 is actually a better choice.

Aggregated error in fit per slice on 10 surfaces

Overall on my 10 surfaces composed each of around 10 slices, an admittedly small sample, Heston (without mean-reversion) fit is a little bit better than SABR. Also the SVI-SABR idea from Gatheral is not great: the fit is clearly worse than SABR with Beta=1 and even worse than a simple quadratic.
Of course the best overall fit is achieved with the classic SVI, because it has 6 parameters while the others have only 3.

All the calibrations so far were done slice by slice independently, using levenberg marquardt on an initial guess found by differential evolution. Some people advocate for speed or stability of parameters reasons the idea of calibrating each slice using the previous slice as initial guess with a local optimizer like levenberg marquardt, in a bootstrapping fashion.

The results can be quite different, especially for SVI, which then becomes the worst, even worse than SVI-SABR, which is actually a subset of SVI with fewer parameters. How can this be?

This is because as the number of parameters increases, the first slices optimizations have a disproportionate influence, and finding the real minimum is much more difficult, even with differential evolution for the first slice. It's easy to picture that you'll have much more chances to get stuck in some local minimum.
It's interesting to note that the real stochastic volatility models are actually better behaved in this regard, but I am not so sure that this kind of calibration is such a great idea in general.

In practice, the SVI parameters fitted independently evolve in a given surface on each slice in a smooth manner, mostly monotonically. It's just that to go from one set on one slice to the other on the next slice, you might have to do something more than a local optimization.

Heston vs SABR slice by slice fit

Some people use Heston to fit one slice of a volatility surface. In this case, some parameters are clearly redundant. Still, I was wondering how it fared against SABR, which is always used to fit a slice. And what about Schobel-Zhu?

Aggregated error in fit per slice on 10 surfaces

With Heston, the calibration is actually slightly better with kappa=0, that is, without mean reversion, because the global optimization is easier and the mean reversion is fully redundant. It's still quite remarkable that 3 parameters result in a fit as good as 5 parameters.
This is however not the case for Schobel-Zhu, where each "redundant parameter" seem to make a slight difference in the quality of calibration. kappa = 0 deteriorate a little bit the fit (the mean error is clearly higher), and theta near 0 (so calibrating 4 parameters) is also a little worse (although better than kappa = 0). Also interestingly, the five parameters Schobel-Zhu fit is slightly better than Heston, but not so when one reduce the number of free parameters.

So what about Heston vs SABR. It is interesting to consider the case of general Beta and Beta=1: it turns out that as confirmed for equities, beta=1 is actually a better choice.

Aggregated error in fit per slice on 10 surfaces

Overall on my 10 surfaces composed each of around 10 slices, an admittedly small sample, Heston (without mean-reversion) fit is a little bit better than SABR. Also the SVI-SABR idea from Gatheral is not great: the fit is clearly worse than SABR with Beta=1 and even worse than a simple quadratic.
Of course the best overall fit is achieved with the classic SVI, because it has 6 parameters while the others have only 3.

All the calibrations so far were done slice by slice independently, using levenberg marquardt on an initial guess found by differential evolution. Some people advocate for speed or stability of parameters reasons the idea of calibrating each slice using the previous slice as initial guess with a local optimizer like levenberg marquardt, in a bootstrapping fashion.

The results can be quite different, especially for SVI, which then becomes the worst, even worse than SVI-SABR, which is actually a subset of SVI with fewer parameters. How can this be?

This is because as the number of parameters increases, the first slices optimizations have a disproportionate influence, and finding the real minimum is much more difficult, even with differential evolution for the first slice. It's easy to picture that you'll have much more chances to get stuck in some local minimum.
It's interesting to note that the real stochastic volatility models are actually better behaved in this regard, but I am not so sure that this kind of calibration is such a great idea in general.

In practice, the SVI parameters fitted independently evolve in a given surface on each slice in a smooth manner, mostly monotonically. It's just that to go from one set on one slice to the other on the next slice, you might have to do something more than a local optimization.

Quadratic Spline with Knots at Mid-Points

Two months ago, I looked at arbitrage free interpolation using piecewise-constant density. This is equivalent to a piecewise quadratic polynomial in call prices where each piece is centered around each call strike.

I wondered at the time what a quadratic spline would look like on this problem, as it should be very close in theory, except that we can ensure that it is C1, a condition for a good looking implied volatility.

For a while, I did not find any references around splines where knots are in between two interpolation points and derived my own formula. And then I lost the paper, but out of curiosity, I looked at the excellent De Boor book "A Practical Guide to Splines" and found that there was actually a chapter around this: quadratic splines with knots at mid-points. Interestingly, it turns out that a quadratic spline on standard knots is not always well defined, which is why, if one does quadratic splines, the knots need to be moved.

The papers from this era are quite rudimentary in their presentation (the book is much better). I found the paper from Demko 1977 "Interpolation by Quadratic Splines" quite usable for coding. I adjusted the boundaries to make the first and last quadratic fit the first two/last two strikes (adding a first strike at 0 and a large last strike if necessary) and spend countless time worrying about indices. The result on a simple classic example is interesting.

On the non monotonic discrete density data of my earlier blog entry, this gives:

QSpline is the quadratic spline

Unfortunately, interpolating small prices with such a spline results in a highly oscillating interpolation: this is the Gibbs phenomenon for splines. We need to loose strict C1 continuity for practical applications, and use a first derivative approximation instead, very much like the Harmonic cubic spline.

On Jaeckel data, the quadratic spline on prices is highly oscillating

Quadratic Spline with Knots at Mid-Points

Two months ago, I looked at arbitrage free interpolation using piecewise-constant density. This is equivalent to a piecewise quadratic polynomial in call prices where each piece is centered around each call strike.

I wondered at the time what a quadratic spline would look like on this problem, as it should be very close in theory, except that we can ensure that it is C1, a condition for a good looking implied volatility.

For a while, I did not find any references around splines where knots are in between two interpolation points and derived my own formula. And then I lost the paper, but out of curiosity, I looked at the excellent De Boor book "A Practical Guide to Splines" and found that there was actually a chapter around this: quadratic splines with knots at mid-points. Interestingly, it turns out that a quadratic spline on standard knots is not always well defined, which is why, if one does quadratic splines, the knots need to be moved.

The papers from this era are quite rudimentary in their presentation (the book is much better). I found the paper from Demko 1977 "Interpolation by Quadratic Splines" quite usable for coding. I adjusted the boundaries to make the first and last quadratic fit the first two/last two strikes (adding a first strike at 0 and a large last strike if necessary) and spend countless time worrying about indices. The result on a simple classic example is interesting.

On the non monotonic discrete density data of my earlier blog entry, this gives:

QSpline is the quadratic spline

Unfortunately, interpolating small prices with such a spline results in a highly oscillating interpolation: this is the Gibbs phenomenon for splines. We need to loose strict C1 continuity for practical applications, and use a first derivative approximation instead, very much like the Harmonic cubic spline.

On Jaeckel data, the quadratic spline on prices is highly oscillating