Python – Is it a good practice to use try-except-else in Python

exceptionexception handlingpythontry-catch

From time to time in Python, I see the block:

try:
   try_this(whatever)
except SomeException as exception:
   #Handle exception
else:
   return something

What is the reason for the try-except-else to exist?

I do not like that kind of programming, as it is using exceptions to perform flow control. However, if it is included in the language, there must be a good reason for it, isn't it?

It is my understanding that exceptions are not errors, and that they should only be used for exceptional conditions (e.g. I try to write a file into disk and there is no more space, or maybe I do not have permission), and not for flow control.

Normally I handle exceptions as:

something = some_default_value
try:
    something = try_this(whatever)
except SomeException as exception:
    #Handle exception
finally:
    return something

Or if I really do not want to return anything if an exception happens, then:

try:
    something = try_this(whatever)
    return something
except SomeException as exception:
    #Handle exception

Best Answer

"I do not know if it is out of ignorance, but I do not like that kind of programming, as it is using exceptions to perform flow control."

In the Python world, using exceptions for flow control is common and normal.

Even the Python core developers use exceptions for flow-control and that style is heavily baked into the language (i.e. the iterator protocol uses StopIteration to signal loop termination).

In addition, the try-except-style is used to prevent the race-conditions inherent in some of the "look-before-you-leap" constructs. For example, testing os.path.exists results in information that may be out-of-date by the time you use it. Likewise, Queue.full returns information that may be stale. The try-except-else style will produce more reliable code in these cases.

"It my understanding that exceptions are not errors, they should only be used for exceptional conditions"

In some other languages, that rule reflects their cultural norms as reflected in their libraries. The "rule" is also based in-part on performance considerations for those languages.

The Python cultural norm is somewhat different. In many cases, you must use exceptions for control-flow. Also, the use of exceptions in Python does not slow the surrounding code and calling code as it does in some compiled languages (i.e. CPython already implements code for exception checking at every step, regardless of whether you actually use exceptions or not).

In other words, your understanding that "exceptions are for the exceptional" is a rule that makes sense in some other languages, but not for Python.

"However, if it is included in the language itself, there must be a good reason for it, isn't it?"

Besides helping to avoid race-conditions, exceptions are also very useful for pulling error-handling outside loops. This is a necessary optimization in interpreted languages which do not tend to have automatic loop invariant code motion.

Also, exceptions can simplify code quite a bit in common situations where the ability to handle an issue is far removed from where the issue arose. For example, it is common to have top level user-interface code calling code for business logic which in turn calls low-level routines. Situations arising in the low-level routines (such as duplicate records for unique keys in database accesses) can only be handled in top-level code (such as asking the user for a new key that doesn't conflict with existing keys). The use of exceptions for this kind of control-flow allows the mid-level routines to completely ignore the issue and be nicely decoupled from that aspect of flow-control.

There is a nice blog post on the indispensibility of exceptions here.

Also, see this Stack Overflow answer: Are exceptions really for exceptional errors?

"What is the reason for the try-except-else to exist?"

The else-clause itself is interesting. It runs when there is no exception but before the finally-clause. That is its primary purpose.

Without the else-clause, the only option to run additional code before finalization would be the clumsy practice of adding the code to the try-clause. That is clumsy because it risks raising exceptions in code that wasn't intended to be protected by the try-block.

The use-case of running additional unprotected code prior to finalization doesn't arise very often. So, don't expect to see many examples in published code. It is somewhat rare.

Another use-case for the else-clause is to perform actions that must occur when no exception occurs and that do not occur when exceptions are handled. For example:

recip = float('Inf')
try:
    recip = 1 / f(x)
except ZeroDivisionError:
    logging.info('Infinite result')
else:
    logging.info('Finite result')

Another example occurs in unittest runners:

try:
    tests_run += 1
    run_testcase(case)
except Exception:
    tests_failed += 1
    logging.exception('Failing test case: %r', case)
    print('F', end='')
else:
    logging.info('Successful test case: %r', case)
    print('.', end='')

Lastly, the most common use of an else-clause in a try-block is for a bit of beautification (aligning the exceptional outcomes and non-exceptional outcomes at the same level of indentation). This use is always optional and isn't strictly necessary.

Related Solutions

Java – What are the effects of exceptions on performance in Java

It depends how exceptions are implemented. The simplest way is using setjmp and longjmp. That means all registers of the CPU are written to the stack (which already takes some time) and possibly some other data needs to be created... all this already happens in the try statement. The throw statement needs to unwind the stack and restore the values of all registers (and possible other values in the VM). So try and throw are equally slow, and that is pretty slow, however if no exception is thrown, exiting the try block takes no time whatsoever in most cases (as everything is put on the stack which cleans up automatically if the method exists).

Sun and others recognized, that this is possibly suboptimal and of course VMs get faster and faster over the time. There is another way to implement exceptions, which makes try itself lightning fast (actually nothing happens for try at all in general - everything that needs to happen is already done when the class is loaded by the VM) and it makes throw not quite as slow. I don't know which JVM uses this new, better technique...

...but are you writing in Java so your code later on only runs on one JVM on one specific system? Since if it may ever run on any other platform or any other JVM version (possibly of any other vendor), who says they also use the fast implementation? The fast one is more complicated than the slow one and not easily possible on all systems. You want to stay portable? Then don't rely on exceptions being fast.

It also makes a big difference what you do within a try block. If you open a try block and never call any method from within this try block, the try block will be ultra fast, as the JIT can then actually treat a throw like a simple goto. It neither needs to save stack-state nor does it need to unwind the stack if an exception is thrown (it only needs to jump to the catch handlers). However, this is not what you usually do. Usually you open a try block and then call a method that might throw an exception, right? And even if you just use the try block within your method, what kind of method will this be, that does not call any other method? Will it just calculate a number? Then what for do you need exceptions? There are much more elegant ways to regulate program flow. For pretty much anything else but simple math, you will have to call an external method and this already destroys the advantage of a local try block.

See the following test code:

public class Test {
    int value;


    public int getValue() {
        return value;
    }

    public void reset() {
        value = 0;
    }

    // Calculates without exception
    public void method1(int i) {
        value = ((value + i) / i) << 1;
        // Will never be true
        if ((i & 0xFFFFFFF) == 1000000000) {
            System.out.println("You'll never see this!");
        }
    }

    // Could in theory throw one, but never will
    public void method2(int i) throws Exception {
        value = ((value + i) / i) << 1;
        // Will never be true
        if ((i & 0xFFFFFFF) == 1000000000) {
            throw new Exception();
        }
    }

    // This one will regularly throw one
    public void method3(int i) throws Exception {
        value = ((value + i) / i) << 1;
        // i & 1 is equally fast to calculate as i & 0xFFFFFFF; it is both
        // an AND operation between two integers. The size of the number plays
        // no role. AND on 32 BIT always ANDs all 32 bits
        if ((i & 0x1) == 1) {
            throw new Exception();
        }
    }

    public static void main(String[] args) {
        int i;
        long l;
        Test t = new Test();

        l = System.currentTimeMillis();
        t.reset();
        for (i = 1; i < 100000000; i++) {
            t.method1(i);
        }
        l = System.currentTimeMillis() - l;
        System.out.println(
            "method1 took " + l + " ms, result was " + t.getValue()
        );

        l = System.currentTimeMillis();
        t.reset();
        for (i = 1; i < 100000000; i++) {
            try {
                t.method2(i);
            } catch (Exception e) {
                System.out.println("You'll never see this!");
            }
        }
        l = System.currentTimeMillis() - l;
        System.out.println(
            "method2 took " + l + " ms, result was " + t.getValue()
        );

        l = System.currentTimeMillis();
        t.reset();
        for (i = 1; i < 100000000; i++) {
            try {
                t.method3(i);
            } catch (Exception e) {
                // Do nothing here, as we will get here
            }
        }
        l = System.currentTimeMillis() - l;
        System.out.println(
            "method3 took " + l + " ms, result was " + t.getValue()
        );
    }
}

Result:

method1 took 972 ms, result was 2
method2 took 1003 ms, result was 2
method3 took 66716 ms, result was 2

The slowdown from the try block is too small to rule out confounding factors such as background processes. But the catch block killed everything and made it 66 times slower!

As I said, the result will not be that bad if you put try/catch and throw all within the same method (method3), but this is a special JIT optimization I would not rely upon. And even when using this optimization, the throw is still pretty slow. So I don't know what you are trying to do here, but there is definitely a better way of doing it than using try/catch/throw.

Java – The case against checked exceptions

I think I read the same Bruce Eckel interview that you did - and it's always bugged me. In fact, the argument was made by the interviewee (if this is indeed the post you're talking about) Anders Hejlsberg, the MS genius behind .NET and C#.

http://www.artima.com/intv/handcuffs.html

Fan though I am of Hejlsberg and his work, this argument has always struck me as bogus. It basically boils down to:

"Checked exceptions are bad because programmers just abuse them by always catching them and dismissing them which leads to problems being hidden and ignored that would otherwise be presented to the user".

By "otherwise presented to the user" I mean if you use a runtime exception the lazy programmer will just ignore it (versus catching it with an empty catch block) and the user will see it.

The summary of the argument is that "Programmers won't use them properly and not using them properly is worse than not having them".

There is some truth to this argument and in fact, I suspect Goslings motivation for not putting operator overrides in Java comes from a similar argument - they confuse the programmer because they are often abused.

But in the end, I find it a bogus argument of Hejlsberg's and possibly a post-hoc one created to explain the lack rather than a well thought out decision.

I would argue that while the over-use of checked exceptions is a bad thing and tends to lead to sloppy handling by users, but the proper use of them allows the API programmer to give great benefit to the API client programmer.

Now the API programmer has to be careful not to throw checked exceptions all over the place, or they will simply annoy the client programmer. The very lazy client programmer will resort to catch (Exception) {} as Hejlsberg warns and all benefit will be lost and hell will ensue. But in some circumstances, there's just no substitute for a good checked exception.

For me, the classic example is the file-open API. Every programming language in the history of languages (on file systems at least) has an API somewhere that lets you open a file. And every client programmer using this API knows that they have to deal with the case that the file they are trying to open doesn't exist. Let me rephrase that: Every client programmer using this API should know that they have to deal with this case. And there's the rub: can the API programmer help them know they should deal with it through commenting alone or can they indeed insist the client deal with it.

In C the idiom goes something like

  if (f = fopen("goodluckfindingthisfile")) { ... } 
  else { // file not found ...

where fopen indicates failure by returning 0 and C (foolishly) lets you treat 0 as a boolean and... Basically, you learn this idiom and you're okay. But what if you're a noob and you didn't learn the idiom. Then, of course, you start out with

   f = fopen("goodluckfindingthisfile");
   f.read(); // BANG!

and learn the hard way.

Note that we're only talking about strongly typed languages here: There's a clear idea of what an API is in a strongly typed language: It's a smorgasbord of functionality (methods) for you to use with a clearly defined protocol for each one.

That clearly defined protocol is typically defined by a method signature. Here fopen requires that you pass it a string (or a char* in the case of C). If you give it something else you get a compile-time error. You didn't follow the protocol - you're not using the API properly.

In some (obscure) languages the return type is part of the protocol too. If you try to call the equivalent of fopen() in some languages without assigning it to a variable you'll also get a compile-time error (you can only do that with void functions).

The point I'm trying to make is that: In a statically typed language the API programmer encourages the client to use the API properly by preventing their client code from compiling if it makes any obvious mistakes.

(In a dynamically typed language, like Ruby, you can pass anything, say a float, as the file name - and it will compile. Why hassle the user with checked exceptions if you're not even going to control the method arguments. The arguments made here apply to statically-typed languages only.)

So, what about checked exceptions?

Well here's one of the Java APIs you can use for opening a file.

try {
  f = new FileInputStream("goodluckfindingthisfile");
}
catch (FileNotFoundException e) {
  // deal with it. No really, deal with it!
  ... // this is me dealing with it
}

See that catch? Here's the signature for that API method:

public FileInputStream(String name)
                throws FileNotFoundException

Note that FileNotFoundException is a checked exception.

The API programmer is saying this to you: "You may use this constructor to create a new FileInputStream but you

a) must pass in the file name as a String
b) must accept the possibility that the file might not be found at runtime"

And that's the whole point as far as I'm concerned.

The key is basically what the question states as "Things that are out of the programmer's control". My first thought was that he/she means things that are out of the API programmers control. But in fact, checked exceptions when used properly should really be for things that are out of both the client programmer's and the API programmer's control. I think this is the key to not abusing checked exceptions.

I think the file-open illustrates the point nicely. The API programmer knows you might give them a file name that turns out to be nonexistent at the time the API is called, and that they won't be able to return you what you wanted, but will have to throw an exception. They also know that this will happen pretty regularly and that the client programmer might expect the file name to be correct at the time they wrote the call, but it might be wrong at runtime for reasons beyond their control too.

So the API makes it explicit: There will be cases where this file doesn't exist at the time you call me and you had damn well better deal with it.

This would be clearer with a counter-case. Imagine I'm writing a table API. I have the table model somewhere with an API including this method:

public RowData getRowData(int row)

Now as an API programmer I know there will be cases where some client passes in a negative value for the row or a row value outside of the table. So I might be tempted to throw a checked exception and force the client to deal with it:

public RowData getRowData(int row) throws CheckedInvalidRowNumberException

(I wouldn't really call it "Checked" of course.)

This is bad use of checked exceptions. The client code is going to be full of calls to fetch row data, every one of which is going to have to use a try/catch, and for what? Are they going to report to the user that the wrong row was sought? Probably not - because whatever the UI surrounding my table view is, it shouldn't let the user get into a state where an illegal row is being requested. So it's a bug on the part of the client programmer.

The API programmer can still predict that the client will code such bugs and should handle it with a runtime exception like an IllegalArgumentException.

With a checked exception in getRowData, this is clearly a case that's going to lead to Hejlsberg's lazy programmer simply adding empty catches. When that happens, the illegal row values will not be obvious even to the tester or the client developer debugging, rather they'll lead to knock-on errors that are hard to pinpoint the source of. Arianne rockets will blow up after launch.

Okay, so here's the problem: I'm saying that the checked exception FileNotFoundException is not just a good thing but an essential tool in the API programmers toolbox for defining the API in the most useful way for the client programmer. But the CheckedInvalidRowNumberException is a big inconvenience, leading to bad programming and should be avoided. But how to tell the difference.

I guess it's not an exact science and I guess that underlies and perhaps justifies to a certain extent Hejlsberg's argument. But I'm not happy throwing the baby out with the bathwater here, so allow me to extract some rules here to distinguish good checked exceptions from bad:

Out of client's control or Closed vs Open:

Checked exceptions should only be used where the error case is out of control of both the API and the client programmer. This has to do with how open or closed the system is. In a constrained UI where the client programmer has control, say, over all of the buttons, keyboard commands etc that add and delete rows from the table view (a closed system), it is a client programming bug if it attempts to fetch data from a nonexistent row. In a file-based operating system where any number of users/applications can add and delete files (an open system), it is conceivable that the file the client is requesting has been deleted without their knowledge so they should be expected to deal with it.
Ubiquity:

Checked exceptions should not be used on an API call that is made frequently by the client. By frequently I mean from a lot of places in the client code - not frequently in time. So a client code doesn't tend to try to open the same file a lot, but my table view gets RowData all over the place from different methods. In particular, I'm going to be writing a lot of code like
```
if (model.getRowData().getCell(0).isEmpty())
```

and it will be painful to have to wrap in try/catch every time.

Informing the User:

Checked exceptions should be used in cases where you can imagine a useful error message being presented to the end user. This is the "and what will you do when it happens?" question I raised above. It also relates to item 1. Since you can predict that something outside of your client-API system might cause the file to not be there, you can reasonably tell the user about it:
```
"Error: could not find the file 'goodluckfindingthisfile'"
```
Since your illegal row number was caused by an internal bug and through no fault of the user, there's really no useful information you can give them. If your app doesn't let runtime exceptions fall through to the console it will probably end up giving them some ugly message like:
```
"Internal error occured: IllegalArgumentException in ...."
```
In short, if you don't think your client programmer can explain your exception in a way that helps the user, then you should probably not be using a checked exception.

So those are my rules. Somewhat contrived, and there will doubtless be exceptions (please help me refine them if you will). But my main argument is that there are cases like FileNotFoundException where the checked exception is as important and useful a part of the API contract as the parameter types. So we should not dispense with it just because it is misused.

Sorry, didn't mean to make this so long and waffly. Let me finish with two suggestions:

A: API programmers: use checked exceptions sparingly to preserve their usefulness. When in doubt use an unchecked exception.

B: Client programmers: get in the habit of creating a wrapped exception (google it) early on in your development. JDK 1.4 and later provide a constructor in RuntimeException for this, but you can easily create your own too. Here's the constructor:

public RuntimeException(Throwable cause)

Then get in the habit of whenever you have to handle a checked exception and you're feeling lazy (or you think the API programmer was overzealous in using the checked exception in the first place), don't just swallow the exception, wrap it and rethrow it.

try {
  overzealousAPI(thisArgumentWontWork);
}
catch (OverzealousCheckedException exception) {
  throw new RuntimeException(exception);  
}

Put this in one of your IDE's little code templates and use it when you're feeling lazy. This way if you really need to handle the checked exception you'll be forced to come back and deal with it after seeing the problem at runtime. Because, believe me (and Anders Hejlsberg), you're never going to come back to that TODO in your

catch (Exception e) { /* TODO deal with this at some point (yeah right) */}

Best Answer

Related Solutions

Java – What are the effects of exceptions on performance in Java

Java – The case against checked exceptions

Related Topic