You can implement "finally" as a higher order function in Haskell:
finally :: Either err a -> IO () -> (a -> IO (Either err b)) -> IO (Either err b)
finally (Left err) cleanup _k = cleanup >> return (Left err)
finally (Right x) _cleanup k = k x
You avoid having to unwind the stack because the user of finally has to manually convert into CPS in order to supply the higher order argument of type "a -> IO (Either err b)"
It doesn't seem like the non-finally version will necessarily run in every case. In the instance of try/finally, a runtime exception might occur as well as a checked exception. finally will always run. But a.cleaup will not.