Categories Performance Monitoring

Being Nothingness: Comparing to None in Python

AppNeta no longer blogs on DevOps topics like this one.

Feel free to enjoy it, and check out what we can do for monitoring end user experience of the apps you use to drive your business at www.appneta.com.

The “is None” Idiom

Consider these two snippets from PEP-0008:

“Comparisons to singletons like None should always be done with is or is not, never the equality operators.

Also, beware of writing if x when you really mean if x is not None — e.g. when testing whether a variable or argument that defaults to None was set to some other value. The other value might have a type (such as a container) that could be false in a boolean context!”

Later in that document, we see:

“Don’t compare boolean values to True or False using ==.

Yes:   if greeting:
No:    if greeting == True:
Worse: if greeting is True:

So, singletons like None are supposed to be compared using “is” and “is not”. But the language’s other two singletons, True and False, are supposed to be used differently. Are there other singletons? Or more importantly, are there other singletons like None, which we should compare with “is” and “is not”.

The suggestion to write “x is None” instead of “x == None” is quite old now. The very first draft of PEP-0290 recommends converting x == None and x != None into the is and is not alternatives. At about the same time, PEP-0008 changed to include the selections above. These changes were in 2002, when Python 2.2 was current and 2.3 is under development. So for over a decade, the Python community has preferred all comparisons to None, to be done with the is operator. The formulation “x is None” is now considered idiomatic Python.

That That “is” Costs

But this idiom has a cost. Very rarely does a programmer actually want to use the “is” operator. Identical objects are not guaranteed to compare True under “is” — not even identical strings. These operators have a very narrow utility. When beginners try to employ it, the results are not intuitive. Even to veteran programmers, the results of “is” comparisons can be surprising.

>>> 999 is 999
>>> x = 999 ; y = 999
>>> x is y
>>> z = 999
>>> x is z

Confusing, right? This behavior is also implementation-specific: in general, integers are not singletons like None, and two integers with the same value will be represented by different objects in memory.

The “is not” operator introduces another semantic confusion. It’s the only Python operator that is spelled with two words, and what’s more, both those words are also operators when encountered on their own. This leads to a separate confusion, documented around the web:

>>> False is not None
>>> False is (not None)

Though this snippet may appear to describe something about the associativity of the “is” or “not” operator, it’s actually something different. In the first case, the expression uses the “is not” operator; in the second case, the expression uses the “is” and “not” operators. It’s unlikely that too many programmers will encounter bugs based on this distinction, but it’s a distinction that may not be obvious.

But Why?

One of the arguments in favor of using “is None” has been that an object might override “__eq__” such that it compares as equal to None. I find it strange that this would argue in favor of using the “is” idiom. Truthiness and duck typing are well-established concepts in idiomatic Python. Truthiness is documented in the  “Truth Value Testing” section of the library reference, and duck typing is touted as a language feature. Testing with an identity operator goes against both of these trends, even if it is just for the case of None.

In PEP-0290, the logic offered is that an “is” test is slightly faster in the case of built-in objects, and much faster in the event of objects that have a custom “__eq__” method. In other words, this idiom has entered the language largely as an optimization pattern.

>>> min(timeit.Timer('x is None', setup="x = 1").repeat(10, 1000000))
>>> min(timeit.Timer('x == None', setup="x = 1").repeat(10, 1000000))

Okay, I guess that will help.

Geoff Gerrietts: Geoff is a Python enthusiast with almost twenty years of industry experience in a career that has included running network cables, documenting X.25 drivers, and performance-tuning data pipelines. His affection for significant whitespace has drawn him from one coast to the other and back again; today he applies his talents to AppNeta's TraceView.

View Comments

  • There is a very straightforward reason why True and False are treated differently from other singletons in if statements: if statements' conditional clauses are boolean expressions, and True and False, unlike other singletons, are boolean expressions. If 'if x is True :' seems OK, then what about 'if ( x is True ) is True:' and so on, ad infinitum?

    The 'is' operator has a straightforward, unambiguous meaning to any programmer who understands that Python variables are references (a person who does not is not yet a Python programmer, and will find all sorts of things confusing.) The distinction between identity and equality is a consequence of this fact, and if you confuse the test for one for the test for the other, you are going to create problems. I think the point about __eq__ overloading is that if the situation calls for an identity test, then that is what you should use, so you don't even have to consider the possibility that you will get something different. It is reasonable that a class' author can modify the equality test, which is an aspect of the semantics of the class, but not the identity test, which is in the language domain, not that of the users.

    I agree that the 'is not' operator is at best redundant and probably harmful.

  • Your performance metrics don't seem to pan out. I think a more fair comparison would be 'if x: pass' vs 'if x is not None: pass', in which case 'if x' comes out on top, slightly.