The Unreasonable sem_open() Function

I have recently re-implemented named semaphores for the QNX OS. For those unfamiliar with the mechanism, a named semaphore is a semaphore that is associated with a unique name that can be accessed by multiple processes. This makes named semaphores handy for synchronization among threads in different processes.

The POSIX standard defines a few functions for dealing with named semaphores, which are modelled after the basic functions for dealing with files:

  • sem_open() – either creates a new named semaphore, or opens an existing one. If successful, the function returns a pointer to a sem_t object, which can then be used with the standard semaphore functions sem_post() and sem_wait(). The sem_t object itself is an opaque handle, defined differently by each operating system.
  • sem_close() – invalidates the handle returned by sem_open() and deallocates all system resources associated with it (the importance of this part of the specification will become clear shortly).
  • sem_unlink() – deletes the name, which prevents future calls to sem_open() from succeeding for that name, but does not affect any existing handles.

Various functional and performance tests showed that my implementation was good – multiple processes can open a semaphore and use it to synchronize the activity of various threads. But then, disaster struck – one of the POSIX conformance tests was failing. As it turns out, POSIX has this requirement as part of the specification for sem_open():

If a process makes multiple successful calls to sem_open() with the same value for name, the same semaphore address shall be returned for each such successful call, provided that there have been no calls to sem_unlink() for this semaphore, and at least one previous successful sem_open() call for this semaphore has not been matched with a sem_close() call.

In other words, POSIX mandates that in the following code, sem1 and sem2 should not only represent the same semaphore, but actually have the same virtual address:

sem_t * const sem1 = sem_open("foo", 0);
sem_t * const sem2 = sem_open("foo", 0);

What could possibly be the reason for such a requirement? There is nothing in the operation of a semaphore, especially one intended for sharing across processes, that would necessitate or even logically entail such a restriction. Even worse, it can easily lead to bugs. Remember the highlighted part of the description of sem_close()? According to POSIX

sem_close(sem1);

should release all resources for both sem1 and sem2. The same does not happen with file descriptors, for example:

int const fd1 = open("foo", O_RDONLY);
int const fd2 = open("foo", O_RDONLY);
close(fd1);

At this point fd2 is still fully functional.

In February I logged the following objection to sem_open() with the Austin Group, which is the body responsible for the standard, though I haven’t heard anything yet:

The current specification mandates that two calls to sem_open() with the same name made by the same process return the same virtual address, so long as no process called sem_unlink() in between the two calls.
I believe that this is an unreasonable requirement, for the following reasons:

  1. There is no dependency by any other sem_() function on this requirement. So long as the two sem_t pointers returned by the calls refer to the same underlying semaphore all sem_() functions will behave correctly when passed these pointers.
  2. It puts an unnecessary burden on the system to track virtual address usage by the calling process. The system should only need to track the association of any given sem_t pointer to the underlying object. If, for example, the sem_t pointer holds a file descriptor to an open semaphore, then the system only needs to track the file descriptor.
  3. Since sem_close() is documented as releasing all resources for the semaphore and making the pointer invalid for future use, the requirement promotes an unsafe “open twice, close once” paradigm.
  4. The requirement deviates from the standard approach to resource allocation, where multiple calls provide different handles, even if those handles refer to the same object (e.g., open(), shm_open(), mmap() with the same file descriptor and offset)
  5. The requirement may conflict with the following future direction: “A future version might require the sem_open() and sem_unlink() functions to have semantics similar to normal file system operations.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s