Improve frame (specialize none type) and/or defer and/or code sections

[dumblob]

It seems, the behavior of frame{} is not so consistent as I'd expect it to be :)
This is currently not possible:

routine r() => int {
  if (rand() > 0.5) return 5 else error('<=0.5')
  return 3  # just for compiler
}
routine wrap() {
  # the goal is to silently return if an exception occured in r()
  x = frame(none) { r() }
  if (x == none) return;
  # do some useful stuff with x
  io.writeln(x)
}

Also variants with defer{} inside of the frame{} are not currently feasible as the behavior of frame is sometimes kind of unexpected:

x = frame { defer { 5 } 3 }
io.writeln('first', x)
x = frame { defer { 5; 9 } 3 }
io.writeln('second', x)
x = frame { defer { 5 } return 3 }
io.writeln('third', x)
x = frame { defer { 5; 9 } return 3 }
io.writeln('fourth', x)
x = frame { defer { none } 3 }
io.writeln('fifth', x)
x = frame { defer { none } return 3 }
io.writeln('sixth', x)
x = frame { defer { return none } 3 }
io.writeln('seventh', x)
x = frame { defer { return none } return 3 }
io.writeln('eighth', x)

It should be the same for every code section block, not only frame{}.

[Night-walker]

The fist example doesn't work because it triggers frame(@T)[ => @T] => @T. Maybe frame(@T)[ => @V] => @T|@V can be added to address this case.

Regarding defer: some frame() calls return zero, which is nonsense indeed. Others result in the value obtained from deferred block, which probably reflects how the code is arranged on the low level. It may still be quite unintuitive.

As we touched defer, I begin to question its fitness for Dao. It obviously disrupts the normal control flow and hinders reasoning about the code. Its ability to do some post-stuff at scope exit and even modify the returned value does not actually add much, as proper code structuring (functions, frames) should mostly remove the need for such quirks. So defer is mainly about exception handling, where it still turns the logic inside out.

In Go where defer originates from, it is basically a mean to cope with the lack of destructors and things like Ruby blocks and Python with operator -- which is not an issue for Dao. The use of deferring for exception handling in Go is likely the result of ascetic language design. Overall, while the versatility of defer in Dao is certainly attractive, the shift in use cases comparing to Go may benefit a more specialized solution.

I'm not about to propose to roll back to boring try-catch. But if code deferring per se is not very important for Dao, I may have an idea or two about exception handling.

[dumblob]

Well, I really like defer as it's flexible, simple and logical way of ensuring execution of something. But I agree, that it`s usage for exceptions is kind of odd, although straightforward.

I'm also convinced that defer has it's place in Dao (many times one want be sure to call something disregarding returns, exceptions, etc.). Therefore I'm curious what do you @Night-walker have in your mind regarding exception handling?

[daokoder]

The fist example doesn't work because it triggers frame(@T)[ => @T] => @T.

Right.

Maybe frame(@T)[ => @V] => @T|@V can be added to address this case.

Sounds good. I will changed it to this.

Regarding defer: some frame() calls return zero, which is nonsense indeed. Others result in the value obtained from deferred block, which probably reflects how the code is arranged on the low level. It may still be quite unintuitive.

The root of the issue is that the defers are executed after the function return (in order to support return value modification), and the return values of the defers accidentally overwrite that of the routine.

As we touched defer, I begin to question its fitness for Dao. It obviously disrupts the normal control flow and hinders reasoning about the code.

I don't find it difficult to reason (not more than user-defined code section method, anyway), as long as it does not modify the returning value of the routine.

Its ability to do some post-stuff at scope exit and even modify the returned value does not actually add much, as proper code structuring (functions, frames) should mostly remove the need for such quirks. So defer is mainly about exception handling, where it still turns the logic inside out.

Well, I do often do "proper code structuring" in C in order to simplify cleanup in functions. But it is not exactly convenient after all. So I prefer to keep defers. Its use is more than exception handling. But I am considering to remove the support for return value modification, as it does complicate things with marginal benefits.

But if code deferring per se is not very important for Dao, I may have an idea or two about exception handling.

Let's hear you out first.

[Night-walker]

Maybe frame(@t)[ => @v] => @t|@v can be added to address this case.
Sounds good. I will changed it to this.

Only if the typing system can properly resolve @T|@T to @T.

I don't find it difficult to reason (not more than user-defined code section method, anyway), as long as it does not modify the returning value of the routine.

User-defined code sections have normal control flow, just as higher-order functions they resemble -- you just pass one function as an argument to another. But deferred code executes at some implicit point different to where you see it. This results in somewhat odd, backward-order logic, e.g. "error handling first, then the code which may raise errors". The most puzzling for me is the use of defer inside loops, which turns the code logic into some kind of circus show.

Well, I do often do "proper code structuring" in C in order to simplify cleanup in functions. But it is not exactly convenient after all.

But we don't need that much cleanups in Dao as in C and Go. As I already pointed out, Dao has destructors and code sections like mutex::protect(){} which mostly remove the need to do something at scope exit. And instead of using defer one can always define an ad-hoc code section:

# deferring
file = io.open(path)
defer { file.close() }
...

# section
routine with(path: string)[file: io::stream]{
    f = io.open(path)
    yield(f)
    f.close()
}

with(path){ [file]
    ...
}

Surely you can point out that in the above example deferring is simpler. Well, it certainly is for one-time use. But when you have at least several similar cases, deferring obviously becomes a clunky boilerplate code with no modularity and abstraction. And the code section retains simplicity (no repetition), readability (straightforward control flow) and controllability (you can change the routine at any time without affecting its use). That is what I call "proper structuring", and I believe it is generally how code should be written in a modern high-level language -- more abstraction, less monkey-patching.

Now about alternative exception handling. The most trivial idea I have in mind may be implemented using code sections only -- without even a dedicated language operator. try()[ => @T] => @T|Error or something logically similar to it is minimally sufficient for exception handling. The only issue is the lack of selectivity -- you can't list the errors you want to catch -- but there may be ways to resolve it preserving the general idiom.

It is just a "prototype" at the moment -- comparing it technically to defer-recover is premature. I just wanted to highlight the basic principle -- switching from imperative exception handling to functional one. That is, viewing error as a result of execution alongside with normal return.

If you are interested, we could make something out of this -- instead of quirky defer which looks a bit hackish among other Dao idioms.

[dumblob]

I like the functional idea of handling exceptions, but regarding the general defer, your example implementation of with doesn't have not much in common with defer as defer guarantees it`s execution no matter what has happened.

[dumblob]

Btw my subjective perception of Pythons/Javas with-like statements is that it inherently demands forced/prescribed interfaces (especially naming) which I personally don't like becase it encourages to create ecosystems like around Java (absolutely no simplicity, focus on naming/interfaces/language_itself instead of high-quality results, readability etc.).

[daokoder]

But deferred code executes at some implicit point different to where you see it. This results in somewhat odd, backward-order logic, e.g. "error handling first, then the code which may raise errors". The most puzzling for me is the use of defer inside loops, which turns the code logic into some kind of circus show.

Just consider a defer block as a closure that is executed right after the routine returns for whatever reason, and different defers are executed in the reverse order of their creation. I don't find this logic hard to follow.

You example is good one showing how to use code section to clean up resources, but as @dumblob has pointed out, defer has the guarantee not offered by your example.

[dumblob]

From what I know, language designers already tried to abstract exception handling as much as possible (C++, Java, Python, ...) in last decade(s) and they've failed (if not totally) - this is imho a sign, that abstraction in this area (exception handling) is contra productive.

[Night-walker]

I like the functional idea of handling exceptions, but regarding the general defer, your example implementation of with doesn't have not much in common with defer as defer guarantees it`s execution no matter what has happened.

It's just a matter of catching a possible exception at yield, albeit nothing bad happens if it stays the current way.

Btw my subjective perception of Pythons/Javas with-like statements is that it inherently demands forced/prescribed interfaces (especially naming) which I personally don't like becase it encourages to create ecosystems like around Java (absolutely no simplicity, focus on naming/interfaces/language_itself instead of high-quality results, readability etc.).

But I don't propose Python's with.

Just consider a defer block as a closure that is executed right after the routine returns for whatever reason, and different defers are executed in the reverse order of their creation. I don't find this logic hard to follow.

"Reverse order" is hard to follow, as one have to reverse the parts of code in mind in order to reason about them.

You example is good one showing how to use code section to clean up resources, but as @dumblob has pointed out, defer has the guarantee not offered by your example.

There is nothing defer can do which cannot be done with a code section (except for modifying returned value, of course):

routine with(path: string)[file: io::stream]{
    f = io.open(path)
    res = try { yield(f) } # assuming the 'try' I proposed
    f.close()
    if (res ?< Error) error(res)
}

Usually, however, you don't need a real cleanup in case of unhandled exception in Dao -- for instance, the file descriptor in the example would have been automatically closed by the garbage collector anyway.

From what I know, language designers already tried to abstract exception handling as much as possible (C++, Java, Python, ...) in last decade(s) and they've failed (if not totally) - this is imho a sign, that abstraction in this area (exception handling) is contra productive.

C++, Java and Python have completely identical exception handling in its essence -- and conceptually different comparing to defer, which by your logic implies that deferring is "contra productive" :)

[Night-walker]

By the way, regarding guaranteed execution of defer. While it may make sense for cleanup, which is rather unlikely to require to be carried out manually in Dao, it may be undesirable for certain other cases.

For instance, you may occasionally write an "optimistic" deferred code which is supposed to run if everything went OK. But if there was an uncaught error, bringing this code up may lead to undesirable consequences.

[dumblob]

There is nothing defer can do which cannot be done with a code section (except for modifying returned value, of course):

Really? If f = io.open(path) (or any exception-throwing call) throws an exception, no other code in the code section will be executed. Shouldn't the code be rather like this:

routine with(path: string)[file: io::stream]{
    (res, f) = try { io.open(path) } # assuming the 'try' I proposed
    if (! (res ?< Error)) (res2, _) = try { yield(f) }
    f.close()
    if (res ?< Error) error(res)
    if (res2 ?< Error) error(res2)
}

As you may have noticed - I'm actually mimicking the `defer``s backwards-execution, but manually :(

which by your logic implies that deferring is "contra productive" :)

Limbo/Go is the only exception, in the others exceptions are identical, but handling not (consider not only syntax, but also meta-programming e.g. in Python/C++, destructors in C++, their lack in Java, and some other constructs like class statements throws in Java etc.).

But if there was an uncaught error, bringing this code up may lead to undesirable consequences.

This is the same as handling more than one inner-section return codes and so it's nothing new nor complicated.

And regarding your try, it's pretty much the same as get_throwed_exceptions_as_list which could be useful imho, but it shouldn't imply removal of defer.

[Night-walker]

If f = io.open(path) (or any exception-throwing call) throws an exception, no other code in the code section will be executed.

If f = io.open(path) throws an exception, f.close() should not be executed (and it will not be executed with defer as well), so no extra manual handling is required here. As for executing the code after yield (or defer), it may not necessary be desired at all, and I already addressed such case in one of the previous comments.

But if there was an uncaught error, bringing this code up may lead to undesirable consequences.

This is the same as handling more than one inner-section return codes and so it's nothing new nor complicated.

Not at all. Consider the following case:

routine writeData(data: list<some>, cn: SQLConnection){
    cn.run('BEGIN TRAN') # transactional insert is meant
    defer { cn.run('COMMIT TRAN') } # an optimistic outcome

    for (row in data)
        cn.run('INSERT ...') # writing row into a table
}

If an unexpected exception occurs during the insertion, the transaction must not be committed -- and it will not with a section-based solution which simply omits error handling. But defer will run the inner code regardless of the outcome, which would then result in committing an incomplete transaction to the database.

What I am trying to say: in Dao, defer is much more likely to be utilized to carry out some main logic rather then a cleanup, and such use may be prone to unwanted, erroneous behavior.

[daokoder]

"Reverse order" is hard to follow, as one have to reverse the parts of code in mind in order to reason about them.

This shouldn't be a problem, if one places defers in the right places.

routine with(path: string)[file: io::stream]{
    f = io.open(path)
    res = try { yield(f) } # assuming the 'try' I proposed
    f.close()
    if (res ?< Error) error(res)
}

I have to say, this is a lot more convoluted than using defer.

routine writeData(data: list<some>, cn: SQLConnection){
    cn.run('BEGIN TRAN') # transactional insert is meant
    defer { cn.run('COMMIT TRAN') } # an optimistic outcome
    for (row in data)
        cn.run('INSERT ...') # writing row into a table
}

Of course, this is a wrong way to do transaction. It should have been something like this:

routine writeData(data: list<some>, cn: SQLConnection){
    cn.run('BEGIN TRAN') # transactional insert is meant
    defer {
       if( % recover() ){
          cn.run('ROLLBACK TRAN')
       }else{
          cn.run('COMMIT TRAN')
       }
    }
    for (row in data)
        cn.run('INSERT ...') # writing row into a table
}

[Night-walker]

routine with(path: string)[file: io::stream]{
f = io.open(path)
res = try { yield(f) } # assuming the 'try' I proposed
f.close()
if (res ?< Error) error(res)
}
I have to say, this is a lot more convoluted than using defer.

But one rarely need to resort to it -- just because cleanup in Dao is a rather uncommon need.

Of course, this is a wrong way to do transaction. It should have been something like this:

routine writeData(data: list, cn: SQLConnection){
cn.run('BEGIN TRAN') # transactional insert is meant
defer {
if( % recover() ){
cn.run('ROLLBACK TRAN')
}else{
cn.run('COMMIT TRAN')
}
}
for (row in data)
cn.run('INSERT ...') # writing row into a table
}

It should have, but it's so easy to make such a mistake presuming that nothing can go wrong. How often people check for exceptions? Quite rare, mostly when they want to handle them in some special way. In all other cases exceptions result in hard failure, where cleanup is doubtfully needed and everything else should simply not get executed.

If you use defer for the so-called business logic (which, again, may be the most frequent case in Dao), you then need to always check that you don't proceed when things run amok. I have to say, this is a lot more convoluted than using code sections :) Missing cleanup is much less dangerous, virtually harmless comparing to the possibility of body continuing to move when head is already dead. Headless zomby, ruthless and unpredictable -- I'm already scared to use defer :)

I already mentioned the saying: "I don't want a language which allows to write good code, I want one which disallows bad code". Not that I'm trying to prove by all means that defer is inherently wrong, but I didn't think about the possibility of "zomby code" before, and it surely worries me now :)

[dumblob]

If f = io.open(path) throws an exception, f.close() should not be executed (and it will not be executed with defer as well)

In this case yes, but think of some general case like chaining (which is actually almost the only way where exceptions make sense if one doesn't want to create empty objects). Also, did you really wanted to let the with statement throw IO exception? To catch it, one would need to use another try{} or frame{ defer{ if( %recover() ) ... } }, i.e. one more nesting => ugly unnecessary clutter.

and it surely worries me now :)

It's interesting as when I started to learn Go, defer was one of the scariest things, but after a short time, I've started to understand it as one of the most predictable and clear things in the whole language - just because it's always the same - it just always executes no matter what. This is the only thing one has to know. There is no conditional, no jumping, no exceptions, just always. Pretty simple and transparent - as I said before, lack of abstraction in this exception/closure cases is apparently much more useful.

As I said, we can add a method to catch all exceptions and return them as a list (now recover() sort of does it, but we could add a code-section variant), but this is definitely not meant as substitution for defer.