Forums » Customizing the ECS »
Standard OS interface required to implement the C standard library
Added by Rochus Keller about 1 year ago
The ECS runtime already implements essential services which can be used to construct a C runtime library.
I'm currently migrating the minimal features of the Newlib C runtime library (https://sourceware.org/newlib/) required to build and run the Lua 5.1 VM. The Lua VM is well suited as a test project of considerable size and popularity, and it also supports in-depth performance measurements.
Newlib has an interesting design that is worth considering in relation to the features of the ECR runtime. Essentially all functions of Newlib directly or indirectly depend on a minimal OS interface which consists of only 19 functions. On platforms where not all of these functions implementations are available, they can simply be stubbed.
Here is the specification of this minimal OS interface: https://sourceware.org/newlib/libc.html#Stubs
I think these functions are good candidates for integration in the ECS runtime. If so, migration of Newlib to ECS using ecc is straigt-forward, and with that a whole lot of C code (up tho whole operating systems) would be directly accessible (by compiling it directly to ECS IR).
Replies (16)
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
Thanks for the interesting specification. The long-term goal of the ECS is to provide its own C and C++ library but its implementation is still in a early stage. However, providing an implementation for the minimal OS interface yourself should already feasible now.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
Ok, I see. I had a look at the cpp library and the runtime assembler files. For some functions in the cstdlib file I found the implementation or reference to the shared libc, but not for all. Where is e.g. the implementation of strtof or lldiv?
Meanwhile I also found out that quite a lot of files in Newlib still use the old K&R syntax, which chibicc doesn't understand. So I switched to uClib and currently migrate functions from there.
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
The functions malloc
and free
are only referenced from libc in order to postpone the implementation of the memory manager. You can easily add references to missing functions using the C++ compiler:
#include "runtime/linuxlib.hpp" LIBRARY (libc, "libc.so.6") FUNCTION (libc, strtof, 2) FUNCTION (libc, ldiv, 2)
Please note that only integer and pointer types are supported as parameters and results for now.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
Ok, I see; am I right to assume that therefore not all function declarations in cstdlib currently have implementations, but that I can provide it via the LIBRARY and FUNCTION macros you mentioned? That's ok for me and I'm thinking of adding inline assembler support for my chibicc version to also support these macros (instead of adding a libc implementation in source form). Concerning malloc & co: do I have to expect that you add more libc symbols to the runtime library, i.e. I have to expect name collisions in future?
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
am I right to assume that therefore not all function declarations in cstdlib currently have implementations, but that I can provide it via the LIBRARY and FUNCTION macros you mentioned?
Yes, exactly. That should work for all functions that do not accept or return floating-point values, or call back some user-defined function due to the different calling convention.
Concerning malloc & co: do I have to expect that you add more libc symbols to the runtime library, i.e. I have to expect name collisions in future?
Yes, over time for sure. The goal is to have a completely free-standing and independent implementation of the library someday.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
Meanwhile I implemented inline assembler statements and sections, and I could successfully adopt your implementation of setjmp/longjmp.
provide it via the LIBRARY and FUNCTION macros
Would it be possible to implement the LIBRARY and FUNCTION macros in ECS IR only (i.e. not in the architecture specific assembler)?
Yes, over time for sure.
The problem is that this way the runtime library becomes a "moving target" with symbols possibly colliding with my C runtime library; i.e. all builds would have to know and check the ECS runtime defined symbols and version and variably build different parts otherwise.
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
Would it be possible to implement the LIBRARY and FUNCTION macros in ECS IR only (i.e. not in the architecture specific assembler)?
A function definition provides a trampoline that jumps to a function wrapper which takes care of the calling convention of the external library. The rest of its definitions is not specific to the architecture but descibes a part of the binary ELF file format, namely a relocation entry in the dynamic section. Both things are hard to achieve using intermediate code alone.
The problem is that this way the runtime library becomes a "moving target" with symbols possibly colliding with my C runtime library; i.e. all builds would have to know and check the ECS runtime defined symbols and version and variably build different parts otherwise.
In case there are collisions you can always mark the function as replaceable either in your library or in the ECS. Two symbols with the same name do not cause errors during linking, if one of them is marked as replaceable. The idea is to provide a default implementation for functions like the stubs described in the OS interface of newlib, which gets autmatically replaced if linked against a platform with a better suited implementation.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
Ok, thanks.
I've now seen in your C++ compiler that you either mark asm sections as ecs::code or not, and that the latter are apparently passed to the machine code generator 1:1.
I now provided a "target" option to the asm() declaration on segment level (i.e. outside of functions), by which the asm string is passed throught to the machine code generator. I assume it makes no sense to also provide this on statement level, isn't it?
can always mark the function as replaceable either in your library or in the ECS
Ok. I actually already use this attribute for tentative C delcarations (i.e. where it is not known yet whether it is extern or not); but as far as I understand, the attribute makes the symbol weaker, i.e. the one marked as "replacable" is not considered if a non-replacable one is found. Therefore I think it would be more prudent to mark the symbols in the runtime library as "replaceable". Anyway, if I implement a standard library by re-using existing C code, it would be a lot of effort to mark each declaration, and I neither think there is already a way to do this in C.
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
I assume it makes no sense to also provide this on statement level, isn't it?
Why not? What difference does it make?
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
Why not? What difference does it make?
It's already pretty challenging to fit explicit IR in a function; one has to know well the IR and the calling convention, and also the constraints imposed by pointer, alignment and other architecture specific features; looking at assembler, probably not that many people might have an idea how the IR is translated to specific machine code and how to write machine code statements that fit the given function context.
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
If you just compile intermediate code then most of the runtime library is not actually linked unless explicitly requested. The rest is a handful of functions defined in the runtime support for the environment which interfaces with the operating system and can easily be marked as replaceable such that your user-defined library takes preference.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
most of the runtime library is not actually linked unless explicitly requested.
But as far as I understand I don't get around linking with e.g. amd32linuxrun.asm or win32run.asm, because otherwise the ELF or PE headers (and other thing required that the result is recognized as an executable) would be missing, and the colliding symbols are exactly in these files.
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
There are only a handful of symbols in those files which can easily be marked as replaceable.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
Do you suggest that I implement my own version of amd32linuxrun.asm & co? I think this is pretty unpractical.
Why not separate the concerns up-front as you did e.g. with amd32libpthread or amd32libsdl?
Now there are arbitrary C standard library implementations in amd32linuxrun.obf and amd32run.obf, in the former e.g. many file handling routined, and in the latter surprisingly floor, sqrt, strcmp, tolower and toupper.
Wouldn't it be much better to have something like amd32libc.obf which would be linked if need be?
Can you also please give me a hint from which source files amd32run.obf is generated?
RE: Standard OS interface required to implement the C standard library - Added by Florian Negele about 1 year ago
Do you suggest that I implement my own version of amd32linuxrun.asm & co? I think this is pretty unpractical.
If name clashes are your primary concern, why not simply use a different naming convention? The linker just expects a code section called main
, all other sections can have arbitrary names.
Why not separate the concerns up-front as you did e.g. with amd32libpthread or amd32libsdl?
These are completely separate libraries. The purpose of amd32linuxrun.asm is to provide all platform-specific runtime support in one place. The upper part generates the ELF data structures as expected by the loader and provides interfaces for system calls. All standard functions in the lower part use these system calls which is why they are in the same file.
Can you also please give me a hint from which source files amd32run.obf is generated?
This file is generated from runtime.cpp and provides platform-independent but hardware-specific runtime support as well as some common standard functions that are used by multiple languages. The function strcmp
is implemented there for example to support string comparisons in Oberon without having to link against the complete C++ library.
RE: Standard OS interface required to implement the C standard library - Added by Rochus Keller about 1 year ago
In case you're interested: the C library (libm and libc) is now complete, and ecc/chibicc can successfully build both the Lua VM and a test app together with the C library and runtime. To avoid linker errors, I created my own version of amd32linuxrun and runtime.c, which no longer include C library symbols. Unfortunately though I get segfaults; at least the test app runs to the first "Hello" and then crashes. As it seems I not should implement debugging support to see what's happening.