Eigen Compiler Suite

Thanks for your ongoing interest in this project.

I found the following declarations in your code and am not sure what the intention is:

These are called context classes and are described in section A.2.4 in the User Manual. Usually they are defined in the source file of a single component, see tools/asmparser.cpp for example. The using Context = class ... construct defines a protected/private class as well as a public shorthand for member definitions of that class outside its body. You have identified instances where the class definition is needed in several source files which is why the definition is hosted by a separate header file. Yes, this trick depends on the order of includes but works as long as the source file includes the same headers as the separate header file.

Thanks for your ongoing interest in this project.

Well, I rather thank you again for this tremendous work; I am still stunned that I have not come across it long ago.

These are called context classes and are described in section A.2.4

I got some understanding of the context classes, including the special way of using "using" declarations for extended modularization; however, i find the combination with the preprocessor quite surprising.

Btw. meanwhile while migrating linkbin I came to a situation when even GCC 4.8 gives up with a message "internal compiler error: in gen_type_die_with_usage, at dwarf2out.c:19486", even if the code seems to be legal C++11.

Actuall the same effect (protected/private class) as you achieve with the "using Context = class ..." concept can also be realized in plain C++98 (see attached asmgenerator.cpp as an example). We have just to prefix the member names with Generator::Context instead of Context:: only, but which personally I rather see as an advantage than an annoyance, and also the intellisense of my LeanCreator (which only understands the easier parts of C++11) is happier with it.

In A.2.1 you write "the source code of the Eigen Compiler Suite is completely portable"; shouldn't that rather state "the source code of the Eigen Compiler Suite is completely ISO C++17 conformant", because apparently it's portability is restricted to a subset of existing compilers. And the more new standard versions the C++ committee throws at us (one each three years which is much longer than the usual adoption cycle), the worse the situation will become. I don't think there is a compiler around which implements to complete C++20 standard; even C++17 is obviously not yet fully supported everywhere.

PS: is there actually a way to edit my posts after I submitted them with this system? I should have written "which is much shorter" than "which is much longer", and there are usually a lot of typos which I only see much later.

We have just to prefix the member names with Generator::Context instead of Context::

That was the whole point of the type alias.

because apparently it's portability is restricted to a subset of existing compilers

Portability here rather means that it is a cross compiler by design. You will always find compilers that cannot compile conformant code.

rather means that it is a cross compiler by design

That it is both (in principle) plattform independent and also a cross-compiler for different platforms, is a great strenght of ECS.

Which parts of the source code do you consider the most and the least stable when looking e.g. two years into the future?

Can I assume that the files I use in my subset (https://github.com/rochus-keller/EiGen/blob/master/BUSY) are stable concerning architecture and features (i.e. the only requred changes are bug fixes if any)?

Which parts of the source code do you consider the most and the least stable when looking e.g. two years into the future?

I would consider everything that is listed under status as done to be stable and I don't expect much changes unless some significant overhaul is required.

Can I assume that the files I use in my subset are stable concerning architecture and features?

Yes, with the exception of the ARM A32 back-end which is still based on ARMv7 rather than ARMv8 like ARM A64 and T32.

Thanks, good to know. I'm mostly concerned about my subset, since the migration of linkbin showed that quite many changes were required to avoid the GCC 4.8 issues, and I expect that the migration to MSVC requires even more changes. Backporting single isolated fixes seems affordable, but in case of a significan overhaul I would likely have to redo the migration. Even this is affordable if not required each other month ;-)

May I ask, whether you consider to also support dynamic shared libraries in future? As far as I understood everything for it seems to be in place (e.g. to generate relocation tables an such).

to also support dynamic shared libraries

To be more precise: the generation of shared libraries using a tool like linkbin.

May I ask, whether you consider to also support dynamic shared libraries in future?

This is not planned. From the linkers view however, dynamic libraries are just binary files like executable ones. You may already know that linkers like linkbin do not know or assume anything about the target file format and just generate plain binary files. All format-dependent information like headers and segments are encoded using usual binary sections provided by files like runtime/amd32linux.asm, so I think it is definitely possible and probably not that difficult to provide shared libraries using the same technique, provided the binary code itself must not be relocatable once a shared dynamic library is loaded.

are encoded using usual binary sections provided by files like runtime/amd32linux.asm

I still have a long way to go before I have a complete overview and discover new features every day while studying the documentation and code that fascinate me.

It's not that shared libraries are an important topic for me at the moment, but in the longer term their generation will be unavoidable, as will, for example, the disassembly of object files generated with other compilers (e.g. to assemble and link them with ECS generated code).

so I think it is definitely possible and probably not that difficult to provide shared libraries using the same technique

Not sure yet; maybe with something like the #repeat directive; but I rather think that it would be easier to generate asm files with corresponding relocation and the other required tables for dynamic binding.

Update:

The migration appears to have been successfully completed. I finally managed to compile and run the tools correctly on GCC 4.8 and MSVC 2015 (compiler version 14.0).

A lot of effort went into debugging, because even when the code compiled correctly, the generated code crashed.

One fix which required several hours of debugging with MSVC is here: https://github.com/rochus-keller/EiGen/blob/e66115badc38c511fdd7065156babda569dcaee3/amd64.cpp#L711. Note the LL suffix; if I assign -2147483648 to an int64_t variable under MSVC, the actual value of the variable is 2147483648. Even a cast of int64_t(-2147483648) didn't help. The LL suffix seems to work with all tested compilers. Maybe you can add it also in your version of the code.

Congratulations to your success and thanks for the update and the bug fix, I just added it.

Even a cast of int64_t(-2147483648) didn't help.

-int64_t(2147483648) should do the trick.

Thanks. Is there a public repository somewhere with a commit log, where I could track your changes?

Is there a public repository somewhere with a commit log

No, sorry.

No, sorry.

Do you mind if I make an "inofficial copy" Github repository of your source trees published on https://software.openbrace.org/projects/ecs/files?

-int64_t(2147483648) should do the trick.

std::numeric_limits<int32_t>::min() seems also to work.

The compiler would actually issue a warning for -2147483648, but only if /Wall is enabled, and burried under tons of clutter.

Do you mind if I make an "inofficial copy" Github repository of your source trees?

I personally don't like seeing my code being published on some internet site, but since it is free software I can't prevent people from doing so.

std::numeric_limits<int32_t>::min() seems also to work.

Yes, but it obfuscates the intent. The number is not really bound to the int32_t type. It just happens not to be representable using the standard integer type of your implementation, but it is representable using its standard unsigned type for which the unary - operator still yields an unsigned value. Thanks nevertheless for your debugging efforts.

my code being published on some internet site

Well, it's Github, not just "some internet site". I could e.g. publish a "source" repository under this group: https://github.com/EigenCompilerSuite, and even add your email as the contact address, and enough information to make everyone understand that it is your work. Of course it would be even better if you had your own Github account and would publish your code yourself.

I can't prevent people from doing so.

Well, I will not do this against your will. As I said you should consider to do it yourself, but I can do it if you have not time. Anyway, as I already said I was very surprised I never heard of your excellent work before, even if I regularly scan "the market" for compiler backends and follow all the relevant sites about programming languages and compilers. At the moment, your ECS seems to be one of the best compiler kits available, but hardly anyone seems to know about it (correct me, if I'm wrong). People should be made aware that they should not waste any more time with QBE and co. when ECS is available. Have you thought about posting on Reddit or Hackernews? I can support you here too if you want.

Well, it's Github, not just "some internet site".

That wasn't meant to sound derogatory. I deliberately set up this site so that I don't have to depend on third parties.

and even add your email as the contact address

Please don't. I don't mind who gets the credit but I do care about privacy. But a link to the official website would be appreciated.

Well, I will not do this against your will.

If you think its helpful, please go ahead. As I said, it is free software after all, and thus literally your own now.

Have you thought about posting on Reddit or Hackernews?

Maybe when the compiler is self-hosting. At this point I think only the Oberon front-end is worth being published which is why I only posted on the Oberon mailing list to get some early feedback. Thank you for the kind words by the way.

Maybe when the compiler is self-hosting.

As already mentioned, in my humble optinion this is well a lifes work; if you set this goal as a benchmark for publication worthiness, the public will probably have to wait many more years to learn about your great work.

I think only the Oberon front-end is worth being published

In my view, the greatest benefit of your work is a cross-platform, multi-target compiler backend. If you also provide compilers, this is primarily helpful as a proof-of-concept and to explain how to use the backend. However, the community definitely has a great need for backends, as projects like LLVM, QBE, MIR, ACK, Firm, etc. demonstrate. However, with the exception of LLVM, which in my view is completely derailed and suffers from various disadvantages, none of the projects mentioned have the maturity and functional scope of the ECS backend.

If you think its helpful, please go ahead.

Thanks. I will set up something for you to check.

is literally your own

Well, you're the original author and thus the sole owner of the copyright of your code. You allow people to use your work under GPL, which doesn't make them the owner or the intellectual property of your code. We are co-owners for the parts of your code which I modified.

greatest benefit of your work is a cross-platform, multi-target compiler backend

Of course combined with your high development and test standard and the excellent documentation, which are important key performance indicators where your work also excels.

Thanks again. I am well aware of the legal meaning of the license. I just wanted to point out that since you now own a copy of free software, you also have the freedom to publish it, even without the explicit permission of the original author.

I do agree that the compilers are probably not that useful for power users, but they complement a toolchain which is hopefully more comprehensible and approachable than others. This is also why I always considered the back-ends a necessity rather than a deliverable. But your take on them is nevertheless interesting and appreciated.

Here is my proposal: https://github.com/EigenCompilerSuite

The source code is here: https://github.com/EigenCompilerSuite/sources

Looks great, thank you.

The links to the files will become invalid once a new version is published. Maybe I suggest linking to the files and news sections instead?