Stop checking for NULL pointers!

One day you are tasked with writing a function called dup_to_upper(). The function takes a NUL-terminated C string, and returns a newly-allocated string with a copy of the original (similar to strdup()), but in which all lower-case letters have been converted to their upper-case counterparts (the function is essential for implementing a shouting mode plugin for some social media website).

The task is pretty straight-forward, and in little time you come up with a first version:

char *
dup_to_upper(char const * const src)
{
    size_t len = strlen(src) + 1; 
    char * const dst = malloc(len); 
    if (dst == NULL) {     
        return NULL; 
    } 

    for (size_t i = 0; i < len; i++) {
        dst[i] = toupper(src[i]); 
    } 

    return dst;
}

You take pride in your work, and congratulate yourself on checking malloc() for an out-of-memory error. You’ve done well.

But then it strikes you that there is another error condition you hadn’t covered: what if the string passed to the function is a NULL pointer? Sooner or later the function is going to dereference that pointer and most likely crash the program that called the function. You quickly add some code to address this concern:

char *
dup_to_upper(char const * const src)
{
    if (src == NULL) {
        errno = EINVAL;
        return NULL;
    }

    size_t len = strlen(src) + 1; 
    char * const dst = malloc(len); 
    if (dst == NULL) {     
        return NULL; 
    } 

    for (size_t i = 0; i < len; i++) {
        dst[i] = toupper(src[i]); 
    } 

    return dst;
}

Surely, you have done very well now, and overall increased the average quality of C code in the world. But did you?

Let’s take a closer look at the newly added check. Why is the function testing the pointer against NULL? Some functions check for NULL because the interface specifies it explicitly as a possible pointer value, often as a way to tell the implementation not to look at the value. For example, the sigaction() function can have either its act argument as NULL, in which case a new action should not be installed, or its oact argument as NULL, to indicate that the caller is not interested in knowing what the old action was.

This is not the case here, though: NULL has no special semantic meaning for dup_to_upper(). Our coder added the check not because it was specified by the interface, but because it is considered by some to be good defensive programming. The check for NULL was added in order to catch an invalid pointer. I contend that such a check is wrong for two reasons:

  1. NULL is by far not the only invalid pointer.
  2. NULL is not always an invalid pointer.

Let’s start by examining what it means for a pointer to be invalid. The question can be answered according to multiple criteria:

  1. Non-canonical addresses: Many architectures, especially in the 64-bit world, divide the range of addresses to “canonical” and “non-canonical”. A non-canonical address is one that the architecture cannot handle. For example, in the x86-64 architecture, only 48 bits out of the possible 64 address bits can be used to generate a canonical address. The range is split in two, which means that canonical addresses are those in the ranges 0x0000000000000000-0x00007fffffffffff and 0xffff800000000000-0xffffffffffffffff.
  2. Unmapped addresses: not all canonical addresses have a mapping installed in the process’ address space, and those that do can be subjected to access restrictions. When a process starts, mappings are created for each code and data segment required by the binary and any linked libraries. Further calls to map memory (such as using the POSIX mmap() function) map more ranges into the address space. It is rare, however, for a process to use all of the available addresses in its address space. Any attempt to read from, write to or execute an unmapped address results in a translation failure, which typically ends up aborting the process.
  3. Inaccessible addresses: these are addresses for which there is a mapping in the address space, but the page tables enforce some form of restriction on access. Restrictions can prevent an address from being accessible at all (neither read, nor write), allow it to be read but not written, or prevent it from being executed. Additionally, most architectures provide a way to designate addresses as only accessible from higher privilege levels.
  4. Semantically-wrong addresses: an address is used to identify an object in memory by its location. When a function is called with a pointer as an argument the caller intends for the function to work on that object. But the same coder that made the mistake of calling the function with a NULL pointer can make the mistake of calling the function with a pointer to a different object in memory.

The last category is the most interesting one. When I call dup_to_upper() I am expected to provide it with a pointer to a C string for which I want an upper-case copy. For example, I can ask the user for a string and then convert it:

char *str = NULL;
size_t len = 0;
getline(&str, &len, stdin);
char * const upper = dup_to_upper(str);

But what if I write instead

char * const upper = dup_to_upper(&str);

The pointer I gave no longer identifies the string I intended to pass to the function, but an object in memory corresponding to the address of that pointer. In this particular case the compiler will probably complain, but since C is weakly-typed it is easy to make such mistakes, for example with functions that take void * arguments.
Here is another example:

char *str1 = NULL; 
char *str2; 
size_t len = 0; 
getline(&str1, &len, stdin); 
char * const upper = dup_to_upper(str2);

Since str2 was not initialized, it may point anywhere. If I’m lucky it holds an address that is not canonical, not mapped or not accessible. If I’m unlucky it points at some arbitrary mapped address (at least dup_to_upper() doesn’t change the contents of the memory at which str2 points!). Again, the compiler may complain here, if it can detect that str2 was not initialized, but how about:

char *str = NULL; 
char *password = "Password1!"; 
size_t len = 0; 
getline(&str, &len, stdin); 
char * const upper = dup_to_upper(password);

There is no reason for the compiler to complain now – the argument passed to the function is syntactically correct. It’s just not what I wanted. You may claim that I have a bug in my code, but that claim applies equally well to passing a NULL pointer as the argument.

So what makes NULL special in this case? Some people who argue for NULL checks contend that it is a common-enough mistake to deserve special handling. It is true that static variables in C are initialized to 0 if no other value is given, and that malloc() returns NULL if it fails (and most programmers do not check for malloc() failures). On the other hand, automatic variables in C hold arbitrary values if not initialized, and the mmap() call returns MAP_FAILED on failure. This value, as will be explained below, is not a NULL pointer. Even when using static variables, or memory allocated with malloc(), invalid pointers can often have non-NULL values, e.g.:

struct foo_s {
    int a;
    int b;
};

struct foo_s * const foo = malloc(sizeof(*foo));
get_number(&foo->b);

If malloc() fails, then &foo->b is the address 0x4.This may seem like a contrived example, but this pattern happens quite often in the real world.

But surely, you now say, there is nothing wrong with the added check for NULL? Granted, it doesn’t catch all cases of invalid pointers, but it at least catches some.

The problem with the extra check is that it creates a contract in the API that the function cannot abide by. When it comes to documenting the new function the documentation will likely have a section about error codes, in which the author will say that the function returns NULL and sets errno to EINVAL if the argument is invalid. But, as we have just seen, that is only true for a very small subset of invalid arguments. The function will not return NULL and set errno to EINVAL if the given argument is 0x4, MAP_FAILED, an address obtained from mmap() with PROT_NONE or an address in the privileged range of the address space (many operating systems keep a range of the address space for use by the kernel). In all of these cases the function will likely result in an access violation and terminate the process. Not only does the function not fulfill its contract with the caller, it behaves differently for different values of invalid pointers: a NULL pointer causes it to return with an error, a non-canonical/unmapped/inaccessible pointer causes it to abort the process, and a semantically-invalid address may cause it to return a string with unexpected content.

The discussion so far assumed that NULL is always an invalid pointer. But is it? The C standard defines a NULL pointer as a pointer with the value 0. Earlier I mentioned that the mmap() function returns the constant MAP_FAILED in case of an error, and that this constant is not 0. There is a very good reason for that: mmap() should be able to return the address 0, as this address may, in fact, be valid. It is not common for address 0 to be valid, but it can happen. For example:

  1. Some boards may have RAM starting at physical address 0. Before the MMU is turned on, or if the board is running with the MMU disabled (or is a micro-controller without an MMU) then software needs to be able to access that address. For example, it should be possible to use memset() to initialize the memory at that address.
  2. I once implemented a system in which a second copy of the operating system was started on one core after the first system booted. This system ran on an ARMv7 board without a hypervisor. In order to initialize the second instance it was necessary to place an exception vector at a location that did not conflict with the exception vector of the first instance. In the ARMv7 architecture the exception vector can be placed at one of two addresses, one of which is 0. Consequently, the process that started the second instance had to map virtual address 0 in order to place the exception vector there, prior to handing over control to the second instance’s kernel.

In conclusion, when passing a pointer to a C function, there is only one semantically-correct value for that pointer, as opposed to 264-1 invalid values. NULL may or may not be one of these invalid values. Let the MMU and operating system handle it.

3 thoughts on “Stop checking for NULL pointers!

  1. Pingback: Stop checking for NULL pointers! | ProgClub

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s