Change Log: 2.099.0

This adds the ability for an alias declaration inside a template to be assigned a new value. For example, the recursive template:

template staticMap(alias F, T...)
{
    static if (T.length == 0)
        alias staticMap = AliasSym!();
    else
        alias staticMap = AliasSym!(F!(T[0]), staticMap!(T[0 .. T.length]));
}

can now be reworked into an iterative template:

template staticMap(alias F, T...)
{
    alias A = AliasSeq!();
    static foreach (t; T)
        A = AliasSeq!(A, F!t); // alias assignment here
    alias staticMap = A;
}

Using the iterative approach will eliminate the combinatorial explosion of recursive template instantiations, eliminating the associated high memory and runtime costs, as well as eliminating the issues with limits on the nesting depth of templates. It will eliminate the obtuse error messages generated when deep in recursion.

The grammar:

AliasAssign:
    Identifier = Type;

is added to the expansion of DeclDef. The Identifier must resolve to a lexically preceding AliasDeclaration:

alias Identifier = Type;

where the Identifier's match, and both are members of the same TemplateDeclaration. Upon semantic processing, when the AliasAssign is encountered the Type in the AliasAssign replaces the Type from the corresponding AliasDeclaration or any previous matching AliasAssign.

The AliasAssign grammar was previously rejected by the parser, so adding it should not break existing code.

One of D's best features is easy integration with C code. There's almost a one-to-one mapping between C and a subset of D code (known as DasBetterC). D and C code can call each other directly.

But D cannot read C code directly. In particular, the interface to most C code comes in the form of a .h (or "header") file. To access the declarations in the .h file and make them available to D code, the C declarations in the .h file must somehow be translated into D. Although hand translating the .h files to D is not difficult, it is tedious, annoying, and definitely a barrier to using D with existing C code.

Why can't the D compiler simply read the .h file and extract its declarations? Why doesn't it "just work"? D has had great success with integrating documentation generation into the language, as well as unit testing. Despite the existence of many documentation generators and testing frameworks, the simplicity of it being built in and "just working" is transformative.

The C Preprocessor

Is its own language that is completely distinct from the C language itself. It has its own grammar, its own tokens, and its own rules. The C compiler is not even aware of the existence of the preprocessor. (The two can be integrated, but that doesn't change the fact that the two semantically know nothing about each other.) The preprocessor is different enough from D that there's no hope of translating preprocessor directives to D in anything but the most superficial manner, any more than the preprocessor directives can be replaced with C.

Previous Solutions

htod by Walter Bright

htod converts a C .h file to a D source file, suitable for importing into D code. htod is built from the front end of the Digital Mars C and C++ compiler. It works just like a C or C++ compiler except that its output is source code for a D module rather than object code.

DStep by Jacob Carlborg

DStep code

DStep Article

From the Article: "DStep is a tool for automatically generating D bindings for C and Objective-C libraries. This is implemented by processing C or Objective-C header files and outputting D modules. DStep uses the Clang compiler as a library (libclang) to process the header files."

dpp by Átila Neves

dpp code

dpp Article

From the Article: "dpp is a compiler wrapper that will parse a D source file with the .dpp extension and expand in place any #include directives it encounters, translating all of the C or C++ symbols to D, and then pass the result to a D compiler (DMD by default)."

Like DStep, dpp relies on libclang.

Introducing ImportC, an ISO C11 Compiler

Here is the next step:

• Forget about the C preprocessor.
• Overlook C++.
• Put a real ISO C11 compiler in the D front end.
• Call it ImportC to distinguish this unique capability.

In detail:

1. Compile C code directly, but only C code that has already been run through

gcc -E -P stdio.h >stdio.c

and in the D source file:

import stdio; // reads stdio.c and compiles it

With gcc doing all the preprocessor work, it becomes 100% behavior compatible.

2. A C compiler front end, stripped of its integrated preprocessor, is a simple

3. The D part of dmd will have no idea it originated as C code. It would be

Using ImportC As A C Compiler

Instead of importing C code, dmd can be used to compile C code like a standalone C compiler.

Create the file hello.c:

int printf(const char*, ...);

int main()
{
    printf("hello world\n");
}

Compile and run it:

dmd hello.c
./hello
hello world!

For C code using C includes you can preprocess the C source with a C preprocessor. Create the file testcode.c:

#include <stdint.h>

uint32_t someCodeInC(uint32_t a, uint32_t b)
{
    return a + b;
}

Preprocess the C code with the C preprocessor:

gcc -E -P testcode.c >testcode.i

Create D file d_main.d:

import std.stdio;
import testcode;

void main()
{
    writeln("Result of someCodeInC(3,4) = ", someCodeInC(3, 4) );
}

Compile and run it:

dmd d_main.d testcode.i
./d_main
Result of someCodeInC(3,4) = 7

The '.i' file extension can be used instead of '.c'. This makes it clear, that preprocessed imtermediate C code is in use. The '.i' files can be created by some generator script or a Makefile.

Implementation Details

User Experience

• Error messages are spare and utilitarian.

• Recovering after encountering an error is not well developed.

• Pretty-printing code is done in D syntax, not C. Also,

• No warnings are emitted. ImportC doesn't care about coding style,

• If the ImportC code corrupts memory, overflows buffers, etc.,

• Symbolic debugging support is there.

Variance From ISO C11

• Doesn't have a C Preprocessor.

• Tag symbols are part of the global symbol table, not the special tag

• Multiple chars in char literal are not supported.

• _Atomic as type qualifier is ignored.

• _Atomic as type specifier is ignored.

• _Alignof is ignored.

• _Generic is not implemented.

• const is transitive, as in D. A pointer to a const pointer to a mutable

• { initializer-list } is not implemented.

• ( type-name ) { initializer-list } is not implemented.

• Complex numbers are not implemented.

• Forward referencing works in ImportC. All global symbols are visible

• Semantics applied after C parsing are D semantics, not C semantics,

Implementation Defined Behavior

The C11 Standard allows for many instances of implementation defined behavior.

• volatile, restrict, register, _Noreturn are accepted and ignored

• char is unsigned

• inline is ignored, as the D compiler inlines what it feels like inlining.

• long double matches what the host C compiler does, which is not necessarily

Extensions

Using a D compiler for semantic processing offers many temptations to add in better D semantics. Inventing yet another new dialect of C is not the point of ImportC. However, some things are useful:

• Compile Time Function Execution

This works. It comes in handy for writing test cases for ImportC using _Static_assert. It also means that the D compiler can execute the ImportC functions in imports just like it can for D.

Unimplemented Extensions

• Much of C code makes use of extensions provided by the host C compiler.

• Alternative keywords are not implemented. You can define the alternate keywords as macros to remove or replace them with standard keywords.

#define __attribute __attribute__
#define __asm       asm
#define __asm__     asm
#define __const     const
#define __const__   const
#define __inline    inline
#define __inline__  inline
#define __extension__

#include <stdlib.h>

Future Directions

C Preprocessor

Some means of incorporating this may be practical. For now, use cpp, warp or spp.

Translation of C macros to D needs to be done, as htod, DStep and dpp all do this.

ImportC++ ?

No. Use dpp.

ImportObjective-C ?

No. Use DStep.

Documentation

More info on ImportC will be written on its page in the specification.

Newcomers from languages with built-in delegates (such as JavaScript and C#) would often use (args) => { /* body */ } for delegate/function literals.

However, in D, this syntax results in a delegate that returns a delegate, without any other side effects. This may trigger hard-to-debug bugs, therefore it is now deprecated.

If a delegate returning a delegate is indeed the intended usage, use either (args) { return () => /* body */; } or (args) => () { /* body */ }.

The following features/bugfixes/improvements were implemented for the experimental C++ header generator:

• Declarations from template mixins are emitted into the instantation context
• (Superclass) methods aliases are emitted as using ... instead of typedef ...
• extern(D) symbols are no longer emitted by accident in certain situations
• extern(D) structs and classes are emitted if referenced by an exported symbol
• Symbols referenced from template declarations are emitted before their first use
• Forward declarations consistently include template<...>
• extern(C++, class), extern(C++, struct) affects forward declarations
• Types used in reference parameters are forward declared if possible
• Renamed or local imports are emitted as using ... when possible
• Complex types are emitted as _Complex.
• Declarations are emitted before the first member access
• Global variables with invalid names in C++ are omitted
• Initializers of union variables/parameters omit non-active members
• Typecasts are emitted as C style, not D's cast(...) ...
• Structs are always tagged as final because D disallows struct inheritance
• No longer asserts for declarations using noreturn
• Symbol names always include template parameters and enclosing declarations when required
• Properly emits (static / enum) members of templated aggregates
• Properly emits static arrays in template declarations
• No longer omits the parent aggregate when referencing __gshared variables
• No longer uses D syntax for expressions nested in unary / binary expressions
• Does not emit public:, ... inside of anonymous structs/unions
• Does not emit typedef for aliases to declaration symbols

Note: The header generator is still considerer experimental, so please submit any bugs encountered to the bug tracker.

This preview ensures that partially constructed objects are properly destroyed when an exception is thrown inside of an constructor. It was introduced in 2.075 and is now enabled by default.

Note that code may fail to compile if a constructor with strict attributes may call a less qualified destructor:

struct Array
{
    int[] _payload;
    ~this()
    {
        import core.stdc.stdlib : free;
        free(_payload.ptr);
    }
}

class Scanner
{
    Array arr;
    this() @safe {} // Might call arr.~this()
}

Such code should either make the constructor nothrow s.t. the destructor will never be called, or adjust the field destructors.

The compiler will only issue a deprectation for attribute violations caused by the inserted destructor call unless the -preview=dtorfields flag is explicitly specified.

For integral vector types, add the .min and .max properties.

For floating point vector types, add the .min, .max, .min_normal, .nan, .infinity, and .epsilon properties.

The value of those properties is the value corresponsing to the vector element type broadcast to the vector type. I.e.:

import core.simd;

static assert(float4.max == cast(float4)float.max);

Variables that are const or immutable can be used as switch cases even if they are initialized at run-time, but using variables that are mutable has been deprecated since dmd 2.073.2. This deprecation has now been turned into an error:

void foo(int s, int x, const int y, immutable int z) {
    switch (s) {
        case x: // error
        case y: // allowed
        case z: // allowed
        default:
    }
}

It is advised to only use compile-time known values in case statements, because the presence of any run-time variable (even a const or immutable one) results in the entire switch being lowered to a series of if-else statements before code generation, instead of an efficient jump table. Prefer regular if-statements for comparing pairs of run-time known variables.

Errors resulting from bad indexing will now contain the length of the array, as well as the offending index for arr[bad], or offending indices arr[bad1 .. bad2] for bad slices.

For example:

void main()
{
    int[] a = [1, 2, 3];
    int b = a[7];
}

Previously this would yield the following error when compiled and run:

dmd -run main.d
[email protected](4): Range violation
––––––––––
??:? _d_arrayboundsp [0x555765c167f9]
??:? _Dmain [0x555765c16752]

It now yields:

dmd -run main.d
[email protected](4): index [7] exceeds array of length 3
––––––––––
??:? _d_arraybounds_indexp [0x5647250b980d]
??:? _Dmain [0x5647250b9751]

Similarly, in case of out of bounds slice operations:

void main()
{
    int[] a = [1, 2, 3];
    int[] b = a[2 .. 4];
}

dmd -run main.d
[email protected](4): slice [2 .. 4] extends past array of length 3
––––––––––
??:? _d_arraybounds_slicep [0x5647250b980d]
??:? _Dmain [0x5647250b9751]

The error messages for an out of bounds slice copy (arr[a..b] = arr[c..d]) or indexing a non-existent key in an Associative Array (table["bad"]) have not been updated.

Class allocators have been deprecated since v2.080.0.

class C
{
    new(size_t size)
    {
        return malloc(size);
    }
}

Starting with this release all class allocators not annotated with @disable will result in a compilation error. As the grammar will also be changing, there are new deprecation messages for the old-style allocator syntax, which accepted a non-empty parameter list and function body.

class C
{
    @disable new(size_t size)   // Deprecation: non-empty parameter list
    {
        return malloc(size);    // Deprecation: function definition
    }
}

Example of corrective action:

class C
{
    @disable new();
}

See the deprecated features page for more information.

The following code has been deprecated since 2.087.0.

module foo;
immutable int bar;
static this()
{
    bar = 42;
}

This is problematic because module constructors (static this) run each time a thread is spawned, and immutable data is implicitly shared, which led to immutable value being overriden every time a new thread was spawned.

The corrective action for any code that still does this is to use `shared static this over static this`, as the former is only run once per process.

The command line switch -target=<triple> can be used to specify the target the compiler should produce code for. The format for <triple> is <arch>-[<vendor>-]<os>[-<cenv>[-<cppenv]]. Specific values of the specifiers are shown in the tables below.

<arch>

The architecture and subarch features are concatenated, so x86_64+avx2 is 64bit with avx2 instructions.

<vendor>

<os>

<os> is the operating system specifier. It may be optionally followed by a version number as in darwin20.3.0 or freebsd12. For Freebsd, this sets the corresponding predefined version identifier, e.g. freebsd12 sets version(FreeBSD12). For other operating systems this is for easier interoperability with other compilers.

<cenv>

<cppenv>

<cppenv> is the C++ runtime environment. This specifier is optional. If this specifier is present the <cenv> specifier must also be present.

The following code has been deprecated since 2.088.0

union U
{
    int a;
    long b = 4;
}

This is problematic because unions are default initialized to whatever the initializer for the first field is, any other initializers present are ignored.

The corrective action is to declare the union field with the default initialization as the first field.

Previously, template arguments weren't fully qualified; they now are, implying longer names in that case.

TypeInfo_Struct instances now store the (potentially significantly shorter) mangled name only and demangle it lazily on the first name or toString() call (with a per-thread cache). So if you only need a unique string per struct TypeInfo, prefer mangledName over computed name (non-@nogc and non-pure).

Related breaking change: TypeInfo.toString() isn't pure anymore to account for the TypeInfo_Struct demangled name cache. TypeInfo_Class.toString() and others are still pure.

For Posix systems that support the fork() function (or the clone() on linux systems), the conservative/precise GC can be made concurrent by enabling the 'fork' GC options in the usual ways, e.g. by adding --DRT-gcopt=fork:1 to the command line or by embedding

extern(C) __gshared string[] rt_options = [ "gcopt=fork:1" ];

into your linked binary (see ../spec/garbage.html#gc_config).

The application continues execution and new memory is allocated from the system while the forked process is marking heap objects. Parallel marking is disabled for the forked process so it only uses a single thread for minimal impact on the concurrent execution of the application.

This reduces "stop the world" time at the cost of needing more memory and page-protection overhead when writing to memory currently being scanned.

The lwpid_t symbol was added to core/sys/linux/sys/procfs.d. The O_CLOEXEC symbol was added to core/sys/posix/fcntl.d.

A new function isValidCharacter has been added to std.utf. It can be used to check if a single character forms a valid code point. For example the char 0x80 is not a valid code point, because it can only be used in trailing characters of UTF8 sequences, whereas the wchar ä is a valid character:

assert(!isValidCharacter(cast(char) 0x80));
assert(isValidCharacter('ä'));

Dub will no longer build dynamicLibrary targetType's as staticLibrary.

Except for x86_omf. This has been disabled due to numerous issues that will lead this to not doing what is expected of it.

No compiler or linker flags have been added at this time, you will need to specify the relevant flag to get the compiler to link dynamically against Phobos.

The $DUB_BUILD_PATH variable is now defined inside the postBuildCommands section. It contains the absolute path in which the package was built, and can be used to copy by-products of the build process to their intended locations.

For example, if an executable exports symbols, you will want to make the resulting import library and symbols export file available somewhere. That can be done with a dub.json section like this:

    "postBuildCommands-windows": [
        "copy /y $DUB_BUILD_PATH\\$DUB_TARGET_NAME.lib $PACKAGE_DIR\\lib"
        "copy /y $DUB_BUILD_PATH\\$DUB_TARGET_NAME.exp $PACKAGE_DIR\\lib"
    ],

Now users can use the documented predefined variables inside custom command directives without the need for a wrapper shell script.

Before this would have failed:

"preBuildCommands": ["$DC -run foo.d"]

unless DC was defined as environment variable outside DUB.

It was before possible to run a script that used the $DC environment variable or on POSIX escape the $ with $$DC to make the shell substitute the variable. These workarounds are no longer needed now.

API change: none of the different command directives are no longer substituted with the process environment variables. You now access the raw commands as provided by the user in the recipe. dub describe has been adjusted and now also processes the predefined environment variables as well as the process environment variables.