Report a bug If you spot a problem with this page, click here to create a Bugzilla issue. Improve this page Quickly fork, edit online, and submit a pull request for this page. Requires a signed-in GitHub account. This works well for small changes. If you'd like to make larger changes you may want to consider using local clone.

std.experimental.allocator

High-level interface for allocators. Implements bundled allocation/creation and destruction/deallocation of data including structs and classes, and also array primitives related to allocation. This module is the entry point for both making use of allocators and for their documentation.

Synopsis: 

// Allocate an int, initialize it with 42
int* p = theAllocator.make!int(42);
assert(*p == 42);
// Destroy and deallocate it
theAllocator.dispose(p);

// Allocate using the global process allocator
p = processAllocator.make!int(100);
assert(*p == 100);
// Destroy and deallocate
processAllocator.dispose(p);

// Create an array of 50 doubles initialized to -1.0
double[] arr = theAllocator.makeArray!double(50, -1.0);
// Append two zeros to it
theAllocator.expandArray(arr, 2, 0.0);
// On second thought, take that back
theAllocator.shrinkArray(arr, 2);
// Destroy and deallocate
theAllocator.dispose(arr);

Layered Structure

D's allocators have a layered structure in both implementation and documentation:

  1. A high-level, dynamically-typed layer (described further down in this module). It consists of an interface called IAllocator, which concrete allocators need to implement. The interface primitives themselves are oblivious to the type of the objects being allocated; they only deal in void[], by necessity of the interface being dynamic (as opposed to type-parameterized). Each thread has a current allocator it uses by default, which is a thread-local variable theAllocator of type IAllocator. The process has a global allocator called processAllocator, also of type IAllocator. When a new thread is created, processAllocator is copied into theAllocator. An application can change the objects to which these references point. By default, at application startup, processAllocator refers to an object that uses D's garbage collected heap. This layer also include high-level functions such as make and dispose that comfortably allocate/create and respectively destroy/deallocate objects. This layer is all needed for most casual uses of allocation primitives.

  2. A mid-level, statically-typed layer for assembling several allocators into one. It uses properties of the type of the objects being created to route allocation requests to possibly specialized allocators. This layer is relatively thin and implemented and documented in the std.experimental.allocator.typed module. It allows an interested user to e.g. use different allocators for arrays versus fixed-sized objects, to the end of better overall performance.

  3. A low-level collection of highly generic heap building blocks —  Lego-like pieces that can be used to assemble application-specific allocators. The real allocation smarts are occurring at this level. This layer is of interest to advanced applications that want to configure their own allocators. A good illustration of typical uses of these building blocks is module std.experimental.allocator.showcase which defines a collection of frequently- used preassembled allocator objects. The implementation and documentation entry point is std.experimental.allocator.building_blocks. By design, the primitives of the static interface have the same signatures as the IAllocator primitives but are for the most part optional and driven by static introspection. The parameterized class CAllocatorImpl offers an immediate and useful means to package a static low-level allocator into an implementation of IAllocator.

  4. Core allocator objects that interface with D's garbage collected heap (std.experimental.allocator.gc_allocator), the C malloc family (std.experimental.allocator.mallocator), and the OS (std.experimental.allocator.mmap_allocator). Most custom allocators would ultimately obtain memory from one of these core allocators.

Idiomatic Use of std.experimental.allocator

As of this time, std.experimental.allocator is not integrated with D's built-in operators that allocate memory, such as new, array literals, or array concatenation operators. That means std.experimental.allocator is opt-in — applications need to make explicit use of it.

For casual creation and disposal of dynamically-allocated objects, use make, dispose, and the array-specific functions makeArray, expandArray, and shrinkArray. These use by default D's garbage collected heap, but open the application to better configuration options. These primitives work either with theAllocator but also with any allocator obtained by combining heap building blocks. For example: 

void fun(size_t n)
{
    // Use the current allocator
    int[] a1 = theAllocator.makeArray!int(n);
    scope(exit) theAllocator.dispose(a1);
    ...
}

To experiment with alternative allocators, set theAllocator for the current thread. For example, consider an application that allocates many 8-byte objects. These are not well supported by the default allocator, so a free list allocator would be recommended. To install one in main, the application would use: 

void main()
{
    import std.experimental.allocator.building_blocks.free_list
        : FreeList;
    theAllocator = allocatorObject(FreeList!8());
    ...
}

Saving the IAllocator Reference For Later Use

As with any global resource, setting theAllocator and processAllocator should not be done often and casually. In particular, allocating memory with one allocator and deallocating with another causes undefined behavior. Typically, these variables are set during application initialization phase and last through the application.

To avoid this, long-lived objects that need to perform allocations, reallocations, and deallocations relatively often may want to store a reference to the allocator object they use throughout their lifetime. Then, instead of using theAllocator for internal allocation-related tasks, they'd use the internally held reference. For example, consider a user-defined hash table: 

struct HashTable
{
    private IAllocator _allocator;
    this(size_t buckets, IAllocator allocator = theAllocator) {
        this._allocator = allocator;
        ...
    }
    // Getter and setter
    IAllocator allocator() { return _allocator; }
    void allocator(IAllocator a) { assert(empty); _allocator = a; }
}

Following initialization, the HashTable object would consistently use its allocator object for acquiring memory. Furthermore, setting HashTable.allocator to point to a different allocator should be legal but only if the object is empty; otherwise, the object wouldn't be able to deallocate its existing state.

Using Allocators without IAllocator

Allocators assembled from the heap building blocks don't need to go through IAllocator to be usable. They have the same primitives as IAllocator and they work with make, makeArray, dispose etc. So it suffice to create allocator objects wherever fit and use them appropriately: 

void fun(size_t n)
{
    // Use a stack-installed allocator for up to 64KB
    StackFront!65536 myAllocator;
    int[] a2 = myAllocator.makeArray!int(n);
    scope(exit) theAllocator.dispose(a2);
    ...
}

In this case, myAllocator does not obey the IAllocator interface, but implements its primitives so it can work with makeArray by means of duck typing.

One important thing to note about this setup is that statically-typed assembled allocators are almost always faster than allocators that go through IAllocator. An important rule of thumb is: "assemble allocator first, adapt to IAllocator after". A good allocator implements intricate logic by means of template assembly, and gets wrapped with IAllocator (usually by means of allocatorObject) only once, at client level.
License:
Boost License 1.0.
Authors:
Andrei Alexandrescu

Source: std/experimental/allocator

interface IAllocator;
Dynamic allocator interface. Code that defines allocators ultimately implements this interface. This should be used wherever a uniform type is required for encapsulating various allocator implementations.
Composition of allocators is not recommended at this level due to inflexibility of dynamic interfaces and inefficiencies caused by cascaded multiple calls. Instead, compose allocators using the static interface defined in std.experimental.allocator.building_blocks, then adapt the composed allocator to IAllocator (possibly by using CAllocatorImpl below).

Methods returning Ternary return Ternary.yes upon success, Ternary.no upon failure, and Ternary.unknown if the primitive is not implemented by the allocator instance.
abstract @property uint alignment();
Returns the alignment offered.
abstract size_t goodAllocSize(size_t s);
Returns the good allocation size that guarantees zero internal fragmentation.
abstract void[] allocate(size_t, TypeInfo ti = null);
Allocates n bytes of memory.
abstract void[] alignedAllocate(size_t n, uint a);
Allocates n bytes of memory with specified alignment a. Implementations that do not support this primitive should always return null.
abstract void[] allocateAll();
Allocates and returns all memory available to this allocator. Implementations that do not support this primitive should always return null.
abstract bool expand(ref void[], size_t);
Expands a memory block in place and returns true if successful. Implementations that don't support this primitive should always return false.
abstract bool reallocate(ref void[], size_t);
Reallocates a memory block.
abstract bool alignedReallocate(ref void[] b, size_t size, uint alignment);
Reallocates a memory block with specified alignment.
abstract Ternary owns(void[] b);
Returns Ternary.yes if the allocator owns b, Ternary.no if the allocator doesn't own b, and Ternary.unknown if ownership cannot be determined. Implementations that don't support this primitive should always return Ternary.unknown.
abstract Ternary resolveInternalPointer(void* p, ref void[] result);
Resolves an internal pointer to the full block allocated. Implementations that don't support this primitive should always return Ternary.unknown.
abstract bool deallocate(void[] b);
Deallocates a memory block. Implementations that don't support this primitive should always return false. A simple way to check that an allocator supports deallocation is to call deallocate(null).
abstract bool deallocateAll();
Deallocates all memory. Implementations that don't support this primitive should always return false.
abstract Ternary empty();
Returns Ternary.yes if no memory is currently allocated from this allocator, Ternary.no if some allocations are currently active, or Ternary.unknown if not supported.
@property IAllocator theAllocator();
@property void theAllocator(IAllocator a);
Gets/sets the allocator for the current thread. This is the default allocator that should be used for allocating thread-local memory. For allocating memory to be shared across threads, use processAllocator (below). By default, theAllocator ultimately fetches memory from processAllocator, which in turn uses the garbage collected heap.
Examples: 
// Install a new allocator that is faster for 128-byte allocations.
import std.experimental.allocator.building_blocks.free_list : FreeList;
import std.experimental.allocator.gc_allocator : GCAllocator;
auto oldAllocator = theAllocator;
scope(exit) theAllocator = oldAllocator;
theAllocator = allocatorObject(FreeList!(GCAllocator, 128)());
// Use the now changed allocator to allocate an array
const ubyte[] arr = theAllocator.makeArray!ubyte(128);
assert(arr.ptr);
//...
@property IAllocator processAllocator();
@property void processAllocator(IAllocator a);
Gets/sets the allocator for the current process. This allocator must be used for allocating memory shared across threads. Objects created using this allocator can be cast to shared.
auto make(T, Allocator, A...)(auto ref Allocator alloc, auto ref A args);
Dynamically allocates (using alloc) and then creates in the memory allocated an object of type T, using args (if any) for its initialization. Initialization occurs in the memory allocated and is otherwise semantically the same as T(args). (Note that using alloc.make!(T[]) creates a pointer to an (empty) array of Ts, not an array. To use an allocator to allocate and initialize an array, use alloc.makeArray!T described below.)
Parameters:
T Type of the object being created.
Allocator alloc The allocator used for getting the needed memory. It may be an object implementing the static interface for allocators, or an IAllocator reference.
A args Optional arguments used for initializing the created object. If not present, the object is default constructed.
Returns:
If T is a class type, returns a reference to the created T object. Otherwise, returns a T* pointing to the created object. In all cases, returns null if allocation failed.
Throws:
If T's constructor throws, deallocates the allocated memory and propagates the exception.
Examples: 
// Dynamically allocate one integer
const int* p1 = theAllocator.make!int;
// It's implicitly initialized with its .init value
assert(*p1 == 0);
// Dynamically allocate one double, initialize to 42.5
const double* p2 = theAllocator.make!double(42.5);
assert(*p2 == 42.5);

// Dynamically allocate a struct
static struct Point
{
    int x, y, z;
}
// Use the generated constructor taking field values in order
const Point* p = theAllocator.make!Point(1, 2);
assert(p.x == 1 && p.y == 2 && p.z == 0);

// Dynamically allocate a class object
static class Customer
{
    uint id = uint.max;
    this() {}
    this(uint id) { this.id = id; }
    // ...
}
Customer cust = theAllocator.make!Customer;
assert(cust.id == uint.max); // default initialized
cust = theAllocator.make!Customer(42);
assert(cust.id == 42);
T[] makeArray(T, Allocator)(auto ref Allocator alloc, size_t length);
T[] makeArray(T, Allocator)(auto ref Allocator alloc, size_t length, auto ref T init);
T[] makeArray(T, Allocator, R)(auto ref Allocator alloc, R range) if (isInputRange!R);
Create an array of T with length elements using alloc. The array is either default-initialized, filled with copies of init, or initialized with values fetched from range.
Parameters:
T element type of the array being created
Allocator alloc the allocator used for getting memory
size_t length length of the newly created array
T init element used for filling the array
R range range used for initializing the array elements
Returns:
The newly-created array, or null if either length was 0 or allocation failed.
Throws:
The first two overloads throw only if alloc's primitives do. The overloads that involve copy initialization deallocate memory and propagate the exception if the copy operation throws.
Examples: 
int[] a = theAllocator.makeArray!int(2);
assert(a == [0, 0]);
a = theAllocator.makeArray!int(3, 42);
assert(a == [42, 42, 42]);
import std.range : only;
a = theAllocator.makeArray!int(only(42, 43, 44));
assert(a == [42, 43, 44]);
bool expandArray(T, Allocator)(auto ref Allocator alloc, ref T[] array, size_t delta);
bool expandArray(T, Allocator)(auto ref Allocator alloc, T[] array, size_t delta, auto ref T init);
bool expandArray(T, Allocator, R)(auto ref Allocator alloc, ref T[] array, R range) if (isInputRange!R);
Grows array by appending delta more elements. The needed memory is allocated using alloc. The extra elements added are either default- initialized, filled with copies of init, or initialized with values fetched from range.
Parameters:
T element type of the array being created
Allocator alloc the allocator used for getting memory
T[] array a reference to the array being grown
size_t delta number of elements to add (upon success the new length of array is array.length + delta)
T init element used for filling the array
R range range used for initializing the array elements
Returns:
true upon success, false if memory could not be allocated. In the latter case array is left unaffected.
Throws:
The first two overloads throw only if alloc's primitives do. The overloads that involve copy initialization deallocate memory and propagate the exception if the copy operation throws.
Examples: 
auto arr = theAllocator.makeArray!int([1, 2, 3]);
assert(theAllocator.expandArray(arr, 2));
assert(arr == [1, 2, 3, 0, 0]);
import std.range : only;
assert(theAllocator.expandArray(arr, only(4, 5)));
assert(arr == [1, 2, 3, 0, 0, 4, 5]);

ForcedInputRange r;
int[] b = [ 1, 2, 3, 4 ];
auto temp = b;
r.array = &temp;
assert(theAllocator.expandArray(arr, r));
assert(arr == [1, 2, 3, 0, 0, 4, 5, 1, 2, 3, 4]);
bool shrinkArray(T, Allocator)(auto ref Allocator alloc, ref T[] array, size_t delta);
Shrinks an array by delta elements.
If array.length < delta, does nothing and returns false. Otherwise, destroys the last array.length - delta elements in the array and then reallocates the array's buffer. If reallocation fails, fills the array with default-initialized data.
Parameters:
T element type of the array being created
Allocator alloc the allocator used for getting memory
T[] array a reference to the array being shrunk
size_t delta number of elements to remove (upon success the new length of array is array.length - delta)
Returns:
true upon success, false if memory could not be reallocated. In the latter case, the slice array[$ - delta .. $] is left with default-initialized elements.
Throws:
The first two overloads throw only if alloc's primitives do. The overloads that involve copy initialization deallocate memory and propagate the exception if the copy operation throws.
Examples: 
int[] a = theAllocator.makeArray!int(100, 42);
assert(a.length == 100);
assert(theAllocator.shrinkArray(a, 98));
assert(a.length == 2);
assert(a == [42, 42]);
void dispose(A, T)(auto ref A alloc, T* p);
void dispose(A, T)(auto ref A alloc, T p) if (is(T == class) || is(T == interface));
void dispose(A, T)(auto ref A alloc, T[] array);
Destroys and then deallocates (using alloc) the object pointed to by a pointer, the class object referred to by a class or interface reference, or an entire array. It is assumed the respective entities had been allocated with the same allocator.
CAllocatorImpl!A allocatorObject(A)(auto ref A a) if (!isPointer!A);
CAllocatorImpl!(A, Yes.indirect) allocatorObject(A)(A* pa);
Returns a dynamically-typed CAllocator built around a given statically- typed allocator a of type A. Passing a pointer to the allocator creates a dynamic allocator around the allocator pointed to by the pointer, without attempting to copy or move it. Passing the allocator by value or reference behaves as follows.
  • If A has no state, the resulting object is allocated in static shared storage.
  • If A has state and is copyable, the result will store a copy of it within. The result itself is allocated in its own statically-typed allocator.
  • If A has state and is not copyable, the result will move the passed-in argument into the result. The result itself is allocated in its own statically-typed allocator.
Examples: 
import std.experimental.allocator.mallocator : Mallocator;
IAllocator a = allocatorObject(Mallocator.instance);
auto b = a.allocate(100);
assert(b.length == 100);
assert(a.deallocate(b));

// The in-situ region must be used by pointer
import std.experimental.allocator.building_blocks.region : InSituRegion;
auto r = InSituRegion!1024();
a = allocatorObject(&r);
b = a.allocate(200);
assert(b.length == 200);
// In-situ regions can deallocate the last allocation
assert(a.deallocate(b));
class CAllocatorImpl(Allocator, Flag!"indirect" indirect = No.indirect): IAllocator;
Implementation of IAllocator using Allocator. This adapts a statically-built allocator type to IAllocator that is directly usable by non-templated code.
Usually CAllocatorImpl is used indirectly by calling theAllocator.
ref Allocator impl();
The implementation is available as a public member.
this(Allocator* pa);
The implementation is available as a public member.
@property uint alignment();
Returns impl.alignment.
size_t goodAllocSize(size_t s);
Returns impl.goodAllocSize(s).
void[] allocate(size_t s, TypeInfo ti = null);
Returns impl.allocate(s).
void[] alignedAllocate(size_t s, uint a);
If impl.alignedAllocate exists, calls it and returns the result. Otherwise, always returns null.
Ternary owns(void[] b);
If Allocator implements owns, forwards to it. Otherwise, returns Ternary.unknown.
bool expand(ref void[] b, size_t s);
Returns impl.expand(b, s) if defined, false otherwise.
bool reallocate(ref void[] b, size_t s);
Returns impl.reallocate(b, s).
bool alignedReallocate(ref void[] b, size_t s, uint a);
Forwards to impl.alignedReallocate.
bool deallocate(void[] b);
If impl.deallocate is not defined, returns Ternary.unknown. If impl.deallocate returns void (the common case), calls it and returns Ternary.yes. If impl.deallocate returns bool, calls it and returns Ternary.yes for true, Ternary.no for false.
bool deallocateAll();
Calls impl.deallocateAll() and returns Ternary.yes if defined, otherwise returns Ternary.unknown.
Ternary empty();
Forwards to impl.empty() if defined, otherwise returns Ternary.unknown.
void[] allocateAll();
Returns impl.allocateAll() if present, null otherwise.