Figure out a way for exception handling with C wrappers

[abinashmeher999]

If the C++ function is wrapped with an extern "C" block, and is called as a C function, there doesn't seem to be a way to deal with the exceptions the underlying C++ function might throw. We should discuss what might be the proper approach for it? So that the ruby wrapper code throws the corresponding exceptions.

[certik]

What are the options?

Sent from my mobile phone.
On Mar 25, 2015 6:58 PM, "Abinash Meher" notifications@github.com wrote:

If the C++ function is wrapped with an extern "C" block, and is called as
a C function, there doesn't seem to be a way to deal with the exceptions
the underlying C++ function might throw. We should discuss what might be
the proper approach for it? So that the ruby wrapper code throws the
corresponding exceptions.

—
Reply to this email directly or view it on GitHub
#418.

[abinashmeher999]

We have two options as far I could find:

• Going with a handler that terminates the code as soon as it catches an exception. http://stackoverflow.com/questions/4448677/can-a-c-program-handle-c-exceptions (the one with 8 upvotes)
  but this doesn't clean up before termination. A really bad way. And I couldn't understand what was really happening in the example provided in the link.
• Or intermixing c wrapper code and Ruby wrapper code.
  The file where I define the underlying C++ functions for their C wrapper counterparts, I can include a try/catch block within the c code. And raise a ruby exception whenever I catch an exception. Here we won't be able to completely separate the ruby and C wrappers. Like the 7th point under the heading Execution

[isuruf]

I remember from the mailing thread that the Ruby wrapper code can be compiled using C++ and expose them with extern "C" blocks. If that is true, then the C++ exceptions thrown from the C wrapper code can be caught at the ruby wrapper code level right?

[abinashmeher999]

I remember from the mailing thread that the Ruby wrapper code can be compiled using C++

Yes it can be

and expose them with extern "C" blocks

This is not true for this case. The C++ functions whose ruby equivalent are needed is exposed with extern block

If that is true, then the C++ exceptions thrown from the C wrapper code can be caught at the ruby wrapper code level right?

I am not sure with this. We are exposing the C++ functions to C, which are compiled with a cpp compiler. And we are interfacing them with ruby functions that take C functions only and define a ruby equivalent for it, which again can be compiled by cpp compiler. The thing is there are no function definitions in the ruby wrapper code. We just mention what will be the name of the function in ruby for a particular one. All the error handling has to be done in implementation of the C wrappers I think, where there are functions being defined and within them the C++ functions being called.

I had a possible idea in mind. But that depends on if we need to have a separate C wrappers that are actually going to be used in C programs. What I was thinking earlier is, since I am going to write the C wrapper anyway for the ruby wrappers, and it's highly unlikely someone will be doing them again, I will make sure that the C wrappers can be used on their own. However, it seems I will have to mix some ruby wrapper code with it. It is better to have two implementations for the same functions. So is it possible that while compilation for C wrappers, the header files for c wrappers compile with the c files of non-ruby ones and while compilation for ruby wrappers, the header files compile with the c files that have ruby code?

[certik]

I think the best is to use C++ to compile the wrappers, not C. Then you can
catch the exception and reraise.

Sent from my mobile phone.

We have two options as far I could find:

• Going with a handler that terminates the code as soon as it catches
  an exception.
  http://stackoverflow.com/questions/4448677/can-a-c-program-handle-c-exceptions
  (the one with 8 upvotes) but this doesn't clean up before termination. A
  really bad way. And I couldn't understand what was really happening in the
  example provided in the link.
• Or intermixing c wrapper code and Ruby wrapper code.
  The file where I define the underlying C++ functions for their C wrapper
  counterparts, I can include a try/catch block within the c code. And raise
  a ruby exception whenever I catch an exception. Here we won't be able to
  completely separate the ruby and C wrappers.

—
Reply to this email directly or view it on GitHub
#418 (comment).

[certik]

@abinashmeher999 the way I see it, there are two ways to do this.

1. One is to always go through our C wrappers (cwrapper.h, cwrapper.cpp) and we would extend them, for example we can make all of the functions return 0 on success and non-zero if there was an exception thrown in C++, and then we can provide some functionality to query what exception was thrown. If we do not want it to be thread safe, then we can just set some global flag, similar to how Python works. We would need to think a bit harder to make it thread safe. This approach is completely language agnostic, and can be used from Ruby or other languages as well.
2. The other approach is what we currently use with Python --- the Python wrappers are specified in a Cython file csympy_wrapper.pyx, which gets compiled by Cython into a csympy_wrapper.cpp file. That file uses Python C/API (with extern "C") to expose all the functionality into Python, but the C functions use C++ inside them and you have to use a C++ compiler to compile it. Cython automatically catches any C++ exceptions and converts them to the appropriate Python exceptions. Cython understands some C++, but we still have to use some pretty ugly code, e.g. here.

Ruby doesn't have a tool like Cython, as far as I know, so one can emulate 2. by doing it by hand, I think that's what you were suggesting. But since you will be doing it by hand, it might make sense to do the approach 1., as it won't be that much harder, and more languages will benefit (we can simplify the Python wrappers to just use this common C interface as well). We can write some macros to catch and convert C++ exceptions (there could also be a global flag, that if enabled, it will simply abort the program if there is an exception -- this is useful if you don't want to bother about always checking that the function succeeded), etc.

In general, I don't think we use C++ exceptions for anything important --- we use it for "Not implemented yet.", and I think that's it in Release mode (the default), though we might use them in the future to check that parameters to some function are correct (or in the given range), so in Ruby and Python it's a good habit to return an exception rather than abort everything (though a lot of these "input" errors can be checked in the wrapper itself, e.g. if some function requires an integer > 1, you can check in the wrappers that the user provided an integer > 1 --- then if it aborts, it's a sign that there is a bug in the wrappers that should be fixed). As far as the "not implemented yet", those are generally just temporary, but they are used for example in eval_double for types that we didn't anticipate --- though I think ultimately it should be implemented for all types, so it should never raise.
In Debug mode, we use them in the assert statements, but then you want to abort the program anyway.

So it seems to me that if an exception is raised, it typically means that there is some kind of a bug, either in C++ or in the wrappers, that should be fixed, and once it is fixed, it won't raise again. But I think even in such cases, it's a good habit to reraise the exception to Ruby or Python, so that the user gets a nicer stacktrace, instead of an abort.

[abinashmeher999]

@certik Thanks for such a detailed explanation.
Yeah, that's what I meant. But what I was thinking is since I am going to do it by hand, there won't be much changes to accommodate 2 from 1. Also, returning an error code is not always fool proof. We might need to set different unattainable values as error codes for different functions that return values like integers. However, we can still keep it while we use the same model in wrappers to catch and raise exceptions from C++ code.

so in Ruby and Python it's a good habit to return an exception rather than abort everything (though a lot of these "input" errors can be checked in the wrapper itself, e.g. if some function requires an integer > 1, you can check in the wrappers that the user provided an integer > 1 --- then if it aborts, it's a sign that there is a bug in the wrappers that should be fixed)

Are you sure? because that way it will work but we need to do it everytime we write a wrapper. Also shouldn't it be a part of the same function?

[certik]

What I had in mind is that each function in C will return values through arguments. The actual return value of the function will only be used as an error indicator. We can either to just zero, non-zero and use some global state to query the exception, or we can return zero for success and a particular non-zero code for a particular C++ exception. Or perhaps we can return a pointer -- NULL means success, and non-null will point to an allocated object with the exception info (the caller is then responsible for freeing it).

Are you sure? because that way it will work but we need to do it everytime we write a wrapper. Also shouldn't it be a part of the same function?

I thought about it some more in the meantime and I think I agree with you: I think we want to wrap all exceptions from C++. The C++ code should raise if the input is incorrect, in the user level functions: the low level functions might not always be checked in Release mode for efficiency reasons, only in Debug mode using asserts, but the higher level functions should always check, even in Release mode.

[abinashmeher999]

@certik Now I understand. This seems the way to go. The actual return value can return as it should and two parameters, one that contains the error code that tells the kind of exception and the other points to the allocated object that contains the exception info. or maybe a custom struct that contains both and is passed as a single parameter. The second way would be more comprehensible to new developers.

[certik]

So you propose to return the error code through arguments:

int error_code, i;
i = gcd(8, 4, &error_code)

as opposed to through the return value:

int error_code, i;
error_code = gcd(8, 4, &i)

Both should work. I think the standard in C is to use return values for this. But for example Lapack uses an info argument (the last argument) for error codes.

[abinashmeher999]

We should go with the standard way of returning the error_code. This on stack overflow provides enough reasons to go with the style where we return the error_code.

[certik]

Here is an interesting blogpost about error codes: http://blog.sduto.it/2014/05/a-c-error-handling-style-that-plays.html

[rajithv]

@isuruf @abinashmeher999 @certik I'm about to get into the exception handling part.

From this thread, and the links given, I see that it is necessary to change the method signature of almost all the functions, how will this affect the other wrappers? (I'll be of course be modifying the ruby wrapper as needed).

I was also thinking of having a common function in the Ruby API, which takes a function pointer, and relevant arguments, and handles the raising of exceptions. So the other parts of the Ruby Wrapper will stay more or less unchanged.

What do you think?

[abinashmeher999]

@rajithv Have you handled validation of user input for all the functions
exposed through Ruby API? If not, that should be done first.

After that if we are going to change the function signature for all the
functions according to the convention of returning the error code, it might
require refactoring in all the wrappers. So a good idea would be to do it
in chunks. Change the prototype of related functions, see where all the
builds break and repair them.

For your last approach, I have my own apprehensions that the readability
will pay the price. A new developer would expect the C exception handling
in the Ruby wrappers at a place where the C API function is called.
Debugging might also get difficult. IMHO it should go with the function
itself. @isuruf @certik What do you think?

On Mon, Jul 18, 2016, 4:38 PM Rajith Vidanaarachchi <
notifications@github.com> wrote:

@isuruf https://github.com/isuruf @abinashmeher999
https://github.com/abinashmeher999 @certik https://github.com/certik
I'm about to get into the exception handling part.

From this thread, and the links given, I see that it is necessary to
change the method signature of almost all the functions, how will this
affect the other wrappers? (I'll be of course be modifying the ruby wrapper
as needed).

I was also thinking of having a common function in the Ruby API, which
takes a function pointer, and relevant arguments, and handles the raising
of exceptions. So the other parts of the Ruby Wrapper will stay more or
less unchanged.

What do you think?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#418 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGmQ68J9kztXhjkNOju1u2DdhpDw2DKGks5qW16lgaJpZM4D061Z
.