How should I write ISO C++ Standard conformant custom new and delete operators?

Part I

This C++ FAQ entry explained why one might want to overload new and delete operators for one's own class. This present FAQ tries to explain how one does so in a standard-conforming way.

Implementing a custom new operator

The C++ standard (§18.4.1.1) defines operator new as:

void* operator new (std::size_t size) throw (std::bad_alloc);

The C++ standard specifies the semantics that custom versions of these operators have to obey in §3.7.3 and §18.4.1

Let us summarize the requirements.

Requirement #1: It should dynamically allocate at least size bytes of memory and return a pointer to the allocated memory. Quote from the C++ standard, section 3.7.4.1.3:

The allocation function attempts to allocate the requested amount of storage. If it is successful, it shall return the address of the start of a block of storage whose length in bytes shall be at least as large as the requested size...

The standard further imposes:

...The pointer returned shall be suitably aligned so that it can be converted to a pointer of any complete object type and then used to access the object or array in the storage allocated (until the storage is explicitly deallocated by a call to a corresponding deallocation function). Even if the size of the space requested is zero, the request can fail. If the request succeeds, the value returned shall be a non-null pointer value (4.10) p0 different from any previously returned value p1, unless that value p1 was sub-sequently passed to an operator delete.

This gives us further important requirements:

Requirement #2: The memory allocation function we use (usually malloc() or some other custom allocator) should return a suitably aligned pointer to the allocated memory, which can be converted to a pointer of an complete object type and used to access the object.

Requirement #3: Our custom operator new must return a legitimate pointer even when zero bytes are requested.

One of the evident requirements that can even be inferred from new prototype is:

Requirement #4: If new cannot allocate dynamic memory of the requested size, then it should throw an exception of type std::bad_alloc.

But! There is more to that than what meets the eye: If you take a closer look at the new operator documentation (citation from standard follows further down), it states:

If set_new_handler has been used to define a new_handler function, this new_handler function is called by the standard default definition of operator new if it cannot allocate the requested storage by its own.

To understand how our custom new needs to support this requirement, we should understand:

What is the new_handler and set_new_handler?

new_handler is a typedef for a pointer to a function that takes and returns nothing, and set_new_handler is a function that takes and returns a new_handler.

set_new_handler's parameter is a pointer to the function operator new should call if it can't allocate the requested memory. Its return value is a pointer to the previously registered handler function, or null if there was no previous handler.

An opportune moment for an code sample to make things clear:

#include <iostream>
#include <cstdlib>

// function to call if operator new can't allocate enough memory or error arises
void outOfMemHandler()
{
    std::cerr << "Unable to satisfy request for memory\n";

    std::abort();
}

int main()
{
    //set the new_handler
    std::set_new_handler(outOfMemHandler);

    //Request huge memory size, that will cause ::operator new to fail
    int *pBigDataArray = new int[100000000L];

    return 0;
}

In the above example, operator new (most likely) will be unable to allocate space for 100,000,000 integers, and the function outOfMemHandler() will be called, and the program will abort after issuing an error message.

It is important to note here that when operator new is unable to fulfill a memory request, it calls the new-handler function repeatedly until it can find enough memory or there is no more new handlers. In the above example, unless we call std::abort(), outOfMemHandler() would be called repeatedly. Therefore, the handler should either ensure that the next allocation succeeds, or register another handler, or register no handler, or not return (i.e. terminate the program). If there is no new handler and the allocation fails, the operator will throw an exception.

Continuation 1

---

Part II

... continued

Given the behavior of operator new from the example, a well designed new_handler must do one of the following:

Make more memory available: This may allows the next memory allocation attempt inside operator new's loop to succeed. One way to implement this is to allocate a large block of memory at program start-up, then release it for use in the program the first time the new-handler is invoked.

Install a different new-handler: If the current new-handler can't make any more memory available, and of there is another new-handler that can, then the current new-handler can install the other new-handler in its place (by calling set_new_handler). The next time operator new calls the new-handler function, it will get the one most recently installed.

(A variation on this theme is for a new-handler to modify its own behavior, so the next time it's invoked, it does something different. One way to achieve this is to have the new-handler modify static, namespace-specific, or global data that affects the new-handler's behavior.)

Uninstall the new-handler: This is done by passing a null pointer to set_new_handler. With no new-handler installed, operator new will throw an exception ((convertible to) std::bad_alloc) when memory allocation is unsuccessful.

Throw an exception convertible to std::bad_alloc. Such exceptions are not be caught by operator new, but will propagate to the site originating the request for memory.

Not return: By calling abort or exit.

To implement an class-specific new_handler we have to provide a class with its own versions of set_new_handler and operator new. The class's set_new_handler allows clients to specify the new-handler for the class (exactly like the standard set_new_handlerallows clients to specify the global new-handler). The class's operator new ensures that the class-specific new-handler is used in place of the global new-handler when memory for class objects is allocated.

Now that we understand new_handler & set_new_handler better we are able to modify the Requirement #4 suitably as:

Requirement #4 (Enhanced):
Our operator new should try to allocate memory more than once, calling the new-handling function after each failure. The assumption here is that the new-handling function might be able to do something to free up some memory. Only when the pointer to the new-handling function is null does operator new throw an exception.

As promised, the citation from the Standard: Section 3.7.4.1.3:

An allocation function that fails to allocate storage can invoke the currently installed new_handler(18.4.2.2), if any. [Note: A program-supplied allocation function can obtain the address of the currently installed new_handler using the set_new_handler function (18.4.2.3).] If an allocation function declared with an empty exception-specification (15.4), throw(), fails to allocate storage, it shall return a null pointer. Any other allocation function that fails to allocate storage shall only indicate failure by throw-ing an exception of class std::bad_alloc (18.4.2.1) or a class derived from std::bad_alloc.

Armed with the #4 requirements, let us attempt the pseudo code for our new operator:

void * operator new(std::size_t size) throw(std::bad_alloc)
{  
   // custom operator new might take additional params(3.7.3.1.1)

    using namespace std;                 
    if (size == 0)                     // handle 0-byte requests
    {                     
        size = 1;                      // by treating them as
    }                                  // 1-byte requests

    while (true) 
    {
        //attempt to allocate size bytes;

        //if (the allocation was successful)

        //return (a pointer to the memory);

        //allocation was unsuccessful; find out what the current new-handling function is (see below)
        new_handler globalHandler = set_new_handler(0);

        set_new_handler(globalHandler);


        if (globalHandler)             //If new_hander is registered call it
             (*globalHandler)();
        else 
             throw std::bad_alloc();   //No handler is registered throw an exception

    }

}

Continuation 2

---

Part III

... continued

Note that we cannot get the new handler function pointer directly, we have to call set_new_handler to find out what it is. This is crude but effective, at least for single-threaded code. In a multithreaded environment, probably some kind of lock to safely manipulate the (global) data structures behind the new-handling function will be needed. (More citation/details are welcome on this.)

Also, we have an infinite loop and the only way out of the loop is for memory to be successfully allocated, or for the new-handling function to do one of the things we inferred before. Unless the new_handler does one of those things, this loop inside new operator will never terminate.

A caveat: Note that the standard (§3.7.4.1.3, quoted above) does not explicitly say that the overloaded new operator must implement an infinite loop, but it merely says that such is the default behaviour. So this detail is open to interpretation, but most of the compilers (GCC and Microsoft Visual C++) do implement this loop functionality (you can compile the code samples provided earlier). Also, since an C++ authory such as Scott Meyers suggests this approach, it is reasonable enough.

Special scenarios

Let us consider the following scenario.

class Base
{
    public:
        static void * operator new(std::size_t size) throw(std::bad_alloc);
};

class Derived: public Base
{
   //Derived doesn't declare operator new
};

int main()
{
    // This calls Base::operator new!
    Derived *p = new Derived;

    return 0;
}

As this FAQ, explains, a common reason for writing a custom memory manager is to optimize allocation for objects of a specific class, not for a class or any of its derived classes, which basically means that our operator new for the Base class is typically tuned for objects of size sizeof(Base) -nothing larger and nothing smaller.

In the above sample, because of inheritance the derived class Derived inherits the new operator of the Base class. This makes calling operator new in a base class to allocate memory for an object of a derived class possible. The best way for our operator new to handle this situation is to divert such calls requesting the "wrong" amount of memory to the standard operator new, like this:

void * Base::operator new(std::size_t size) throw(std::bad_alloc)
{
    if (size != sizeof(Base))          // If size is "wrong,", that is, != sizeof Base class
    {
         return ::operator new(size);  // Let std::new handle this request
    }
    else
    {
         //Our implementation
    }
}

Note that, the check for size also incoprporates our requirement #3. This is because all freestanding objects have a non-zero size in C++, so sizeof(Base) can never be zero, so if size is zero, the request will be forwarded to ::operator new, and it is gauranteed that it will handle it in standard compliant way.

Citation: From the creator of C++ himself, Dr Bjarne Stroustrup.

---

Implementing a custom delete operator

The C++ Standard(§18.4.1.1) library defines operator delete as:

void operator delete(void*) throw();

Let us repeat the exercise of gathering the requirements for writing our custom operator delete:

Requirement #1: It shall return void and its first parameter shall be void*. A custom delete operator can have more than one parameter as well but well we just need one parameter to pass the pointer pointing to the allocated memory.

Citation from the C++ Standard:

Section §3.7.3.2.2:

"Each deallocation function shall return void and its first parameter shall be void*. A deallocation function can have more than one parameter....."

Requirement #2: It should guarantee that it is safe to delete a null pointer passed as an argument.

Citation from C++ Standard: Section §3.7.3.2.3:

The value of the first argument supplied to one of the deallocation functions provided in the standard library may be a null pointer value; if so, the call to the deallocation function has no effect. Otherwise, the value supplied to operator delete(void*) in the standard library shall be one of the values returned by a previous invocation of either operator new(size_t) or operator new(size_t, const std::nothrow_t&) in the standard library, and the value supplied to operator delete[](void*) in the standard library shall be one of the values returned by a previous invocation of either operator new[](size_t) or operator new[](size_t, const std::nothrow_t&) in the standard library.

Requirement #3: If the pointer being passed is not null, then the delete operator should deallocate the dynamic memory allocated and assigned to the pointer.

Citation from C++ Standard: Section §3.7.3.2.4:

If the argument given to a deallocation function in the standard library is a pointer that is not the null pointer value (4.10), the deallocation function shall deallocate the storage referenced by the pointer, render-ing invalid all pointers referring to any part of the deallocated storage.

Requirement #4: Also, since our class-specific operator new forwards requests of the "wrong" size to ::operator new, We MUST forward "wrongly sized" deletion requests to ::operator delete.

So based on the requirements we summarized above here is an standard conformant pseudo code for a custom delete operator:

class Base
{
    public:
        //Same as before
        static void * operator new(std::size_t size) throw(std::bad_alloc);
        //delete declaration
        static void operator delete(void *rawMemory, std::size_t size) throw();

        void Base::operator delete(void *rawMemory, std::size_t size) throw()
        {
            if (rawMemory == 0)
            {
                return;                            // No-Op is null pointer
            }

            if (size != sizeof(Base))
            {
                // if size is "wrong,"
                ::operator delete(rawMemory);      //Delegate to std::delete
                return;
            }
            //If we reach here means we have correct sized pointer for deallocation
            //deallocate the memory pointed to by rawMemory;

            return;
        }
};

---