std.experimental.allocator.building_blocks.stats_collector

import std.experimental.allocator.gc_allocator : GCAllocator;
import std.experimental.allocator.building_blocks.free_list : FreeList;
alias Allocator = StatsCollector!(GCAllocator, Options.bytesUsed);

enum Options: ulong;

Options for StatsCollector defined below. Each enables during compilation one specific counter, statistic, or other piece of information.

numOwns

Counts the number of calls to owns.

numAllocate

Counts the number of calls to allocate. All calls are counted, including requests for zero bytes or failed requests.

numAllocateOK

Counts the number of calls to allocate that succeeded, i.e. they returned a block as large as requested. (N.B. requests for zero bytes count as successful.)

numExpand

Counts the number of calls to expand, regardless of arguments or result.

numExpandOK

Counts the number of calls to expand that resulted in a successful expansion.

numReallocate

Counts the number of calls to reallocate, regardless of arguments or result.

numReallocateOK

Counts the number of calls to reallocate that succeeded. (Reallocations to zero bytes count as successful.)

numReallocateInPlace

Counts the number of calls to reallocate that resulted in an in-place reallocation (no memory moved). If this number is close to the total number of reallocations, that indicates the allocator finds room at the current block's end in a large fraction of the cases, but also that internal fragmentation may be high (the size of the unit of allocation is large compared to the typical allocation size of the application).

numDeallocate

Counts the number of calls to deallocate.

numDeallocateAll

Counts the number of calls to deallocateAll.

numAll

Chooses all numXxx flags.

bytesUsed

Tracks bytes currently allocated by this allocator. This number goes up and down as memory is allocated and deallocated, and is zero if the allocator currently has no active allocation.

bytesAllocated

Tracks total cumulative bytes allocated by means of allocate, expand, and reallocate (when resulting in an expansion). This number always grows and indicates allocation traffic. To compute bytes deallocated cumulatively, subtract bytesUsed from bytesAllocated.

bytesExpanded

Tracks the sum of all delta values in calls of the form expand(b, delta) that succeed (return true).

bytesContracted

Tracks the sum of all b.length - s with b.length > s in calls of the form realloc(b, s) that succeed (return true). In per-call statistics, also unambiguously counts the bytes deallocated with deallocate.

bytesMoved

Tracks the sum of all bytes moved as a result of calls to realloc that were unable to reallocate in place. A large number (relative to bytesAllocated) indicates that the application should use larger preallocations.

bytesNotMoved

Tracks the sum of all bytes NOT moved as result of calls to realloc that managed to reallocate in place. A large number (relative to bytesAllocated) indicates that the application is expansion-intensive and is saving a good amount of moves. However, if this number is relatively small and bytesSlack is high, it means the application is overallocating for little benefit.

bytesSlack

Measures the sum of extra bytes allocated beyond the bytes requested, i.e. the internal fragmentation. This is the current effective number of slack bytes, and it goes up and down with time.

bytesHighTide

Measures the maximum bytes allocated over the time. This is useful for dimensioning allocators.

bytesAll

Chooses all byteXxx flags.

all

Combines all flags above.

struct StatsCollector(Allocator, ulong flags = Options.all, ulong perCallFlags = 0);

Allocator that collects extra data about allocations. Since each piece of information adds size and time overhead, statistics can be individually enabled or disabled through compile-time flags.

All stats of the form numXxx record counts of events occurring, such as calls to functions and specific results. The stats of the form bytesXxx collect cumulative sizes.

In addition, the data callerSize, callerModule, callerFile, callerLine, and callerTime is associated with each specific allocation. This data prefixes each allocation.

Examples:

import std.experimental.allocator.gc_allocator : GCAllocator;
import std.experimental.allocator.building_blocks.free_list : FreeList;
alias Allocator = StatsCollector!(GCAllocator, Options.all, Options.all);

Allocator alloc;
auto b = alloc.allocate(10);
alloc.reallocate(b, 20);
alloc.deallocate(b);

import std.file : deleteme, remove;
import std.stdio : File;
import std.range : walkLength;

auto f = deleteme ~ "-dlang.std.experimental.allocator.stats_collector.txt";
scope(exit) remove(f);
Allocator.reportPerCallStatistics(File(f, "w"));
alloc.reportStatistics(File(f, "a"));
writeln(File(f).byLine.walkLength); // 22

Allocator parent;

The parent allocator is publicly accessible either as a direct member if it holds state, or as an alias to Allocator.instance otherwise. One may use it for making calls that won't count toward statistics collection.

alias alignment = Allocator.alignment;

Alignment offered is equal to Allocator.alignment.

Ternary owns(void[] b);

Increments numOwns (per instance and and per call) and forwards to parent.owns(b).

void[] allocate(size_t n);

Forwards to parent.allocate. Affects per instance: numAllocate, bytesUsed, bytesAllocated, bytesSlack, numAllocateOK, and bytesHighTide. Affects per call: numAllocate, numAllocateOK, and bytesAllocated.

bool expand(ref void[] b, size_t delta);

Defined whether or not Allocator.expand is defined. Affects per instance: numExpand, numExpandOK, bytesExpanded, bytesSlack, bytesAllocated, and bytesUsed. Affects per call: numExpand, numExpandOK, bytesExpanded, and bytesAllocated.

bool reallocate(ref void[] b, size_t s);

Defined whether or not Allocator.reallocate is defined. Affects per instance: numReallocate, numReallocateOK, numReallocateInPlace, bytesNotMoved, bytesAllocated, bytesSlack, bytesExpanded, and bytesContracted. Affects per call: numReallocate, numReallocateOK, numReallocateInPlace, bytesNotMoved, bytesExpanded, bytesContracted, and bytesMoved.

bool deallocate(void[] b);

Defined whether or not Allocator.deallocate is defined. Affects per instance: numDeallocate, bytesUsed, and bytesSlack. Affects per call: numDeallocate and bytesContracted.

bool deallocateAll();

Defined only if Allocator.deallocateAll is defined. Affects per instance and per call numDeallocateAll.

Ternary empty();

Defined only if Options.bytesUsed is defined. Returns bytesUsed == 0.

void reportStatistics(R)(auto ref R output);

Reports per instance statistics to output (e.g. stdout). The format is simple: one kind and value per line, separated by a colon, e.g. bytesAllocated:7395404

struct PerCallStatistics;

Defined if perCallFlags is nonzero.

string file;

uint line;

The file and line of the call.

Options[] opts;

The options corresponding to the statistics collected.

ulong[] values;

The values of the statistics. Has the same length as opts.

const string toString();

Format to a string such as: mymodule.d(655): [numAllocate:21, numAllocateOK:21, bytesAllocated:324202].

static auto byFileLine();

Defined if perCallFlags is nonzero. Iterates all monitored file/line instances. The order of iteration is not meaningful (items are inserted at the front of a list upon the first call), so preprocessing the statistics after collection might be appropriate.

void reportPerCallStatistics(R)(auto ref R output);

Defined if perCallFlags is nonzero. Outputs (e.g. to a File) a simple report of the collected per-call statistics.