Eigen Compiler Suite

Great work!

probably a new release with the patches included are not for from being released?

Yes, a new development release should ship in the next few months.

The library was now update with the following notable changes:

• Complete support for Linux with the additional sys-calls added to the Linux runtime
• StringPattern module added which is a port of the Pattern style string matching from Lua.
  This is much simplified from traditional regex implementation and is without any memory allocation.
  About 100 tests from the Lua test suite was added. The old Regex implementation was removed.
  The new code is much more suitable to run on smaller platforms with limited memory.
• LZ4 compression and decompression module added.
  This is a very small implementation which should be a good fit to run on smaller platforms.
  Used in a library with monochrome images a compression ration of 10x is observed.
• ADTTree module replaced with a cleaner Red/Black tree implementation.

I notice the compilation is much faster on the Linux platform.
It is really noticeable. Not sure why that could be. It could
maybe be the defaults of the Clang compiler used.
Otherwise a joy to work with the ECS tools.

Impressing work!

I notice the compilation is much faster on the Linux platform.

As compared to Windows? I also noticed that applications start much slower on Windows than on Linux even using the same compiler. When compiling small modules, the startup time is thus a considerable part of the overall run time.

Yes, Windows.

I suspect the high startup time is due to the unsigned .exe files
being slowed down by virus/malware detection.

On my computer I exempt the working folder where I use the
compiler. It could be some real-time check still running or use
of temporary files etc. Will investigate a bit this, because it is
very noticeable.

Best regards

The library was now updated with some smaller issues and I made
a small example application which implements a simple top 10 word counter utility.

Not very useful, but showcases many library features.
I would say the the ergometric is not bad at all:

MODULE test;

IN Std IMPORT ArrayOfChar, String, StringPattern, OS, OSStream, ADTStream;
IN Std IMPORT PairStrInt := ADTPair(String.STRING, LENGTH);
IN Std IMPORT TreePair := ADTTree(PairStrInt.Pair, PairStrInt.Compare);
IN Std IMPORT DictStrInt := ADTDictionary(String.STRING, LENGTH, String.Hash, String.Equal);

PROCEDURE PairCompare(fl-, fr- : String.STRING; sl-, sr- : LENGTH): INTEGER;
BEGIN
    IF (sl = sr) OR (sl < sr) THEN RETURN -1 END; (* If we return 0, pairs would be overwritten *)
    RETURN 1;
END PairCompare;

PROCEDURE ProcessFile(VAR fh : ADTStream.Stream);
VAR
    dict : DictStrInt.Dictionary;
    dit : DictStrInt.Iterator;
    tree : TreePair.Tree;
    tit : TreePair.Iterator;
    pair : PairStrInt.Pair;
    pat : StringPattern.Pattern;
    s, line, word: String.STRING;
    idx, pos, len, cnt : LENGTH;
BEGIN
    PairStrInt.compare := PairCompare;
    dict.Init(256);
    dict.duplicateKey := String.Duplicate; (* Dictionary insert copies of strings *)
    dict.disposeKey := String.Dispose; (* Dictionary disposes of strings *)
    WHILE fh.ReadLine(line) DO
        idx := 0;
        WHILE pat.Find("%w+", line^, idx) # -1 DO
            IGNORE(pat.Capture(0, pos, len));
            String.Extract(word, line^, pos, len);
            ArrayOfChar.LowerCase(word^);
            IF ~dict.Get(word, cnt) THEN cnt := 0 END;
            dict.Set(word, cnt + 1);
            idx := pos + len + 1;
        END;
    END;
    tree.Init();
    dict.First(dit);
    WHILE dit.NextItem(s, cnt) DO
        pair.first := s;
        pair.second := cnt;
        tree.Insert(pair);
    END;
    tree.Last(tit);
    cnt := 0;
    WHILE tit.Next(pair) & (cnt < 10) DO
        OSStream.StdOut.FormatInteger(pair.second, 3, {});
        OSStream.StdOut.WriteString(" : ");
        OSStream.StdOut.WriteString(pair.first^);
        OSStream.StdOut.WriteNL;
        INC(cnt);
    END;
    String.Dispose(line);
    String.Dispose(word);
    dict.Dispose();
    tree.Dispose();
END ProcessFile;

PROCEDURE Run;
VAR
    fh : OSStream.File;
    s : String.STRING;
BEGIN
    IF OS.Args() = 1 THEN
        ProcessFile(OSStream.StdIn);
    ELSIF OS.Args() = 2 THEN
        OS.Arg(s, 1);
        IF ~fh.Open(s^, OSStream.AccessRead) THEN
            OSStream.StdErr.WriteString("Failed to open file"); OSStream.StdErr.WriteNL;
            String.Dispose(s);
            OS.Exit(1);
        END;
        ProcessFile(fh);
        String.Dispose(s);
        fh.Close();
    ELSE
        OSStream.StdErr.WriteString("Usage : test [filename]"); OSStream.StdErr.WriteNL;
        OS.Exit(1);
    END;
END Run;

BEGIN
    Run;
END test.

It is very easy to make a mistake here if the VAR parameter is forgotten in the
ProcessFile procedure.

The code will then silently fail. I have done this mistake several time myself.

Perhaps the compiler should give a warning in these cases, as very seldom a RECORD
is intended to be passed by value?

Cheers.

Thanks for the update. Great work!

I am hesitant to add such a warning since it might be as helpful as it is annoying when a pass by value is actually intended. What do you think?

I agree it could be false positives here.

The mention XDS compiler had this warning
(also covering arrays), but this was dependent
on the size of object passed before triggering a
"possible performance degradation" warning.

It could be a possibility, but this is more one
the nice to have side and probably not straight
forward to implement.

Best regards,

It is quite straight forward to implement, see the attached patch file in case you want to apply it locally.

Old Oberon code typically uses pass by value for strings because they could otherwise not be passed to procedures. The resulting programs therefore copied a lot of strings in memory. The ECS introduced read-only parameters for that purpose and elides all copies of read-only value parameters. Maybe that helps in your case as well.

That is excellent.

I was aware of the read-only parameter (XDS compiler had the same extension).
With this patch I found three locations where it was missing in the library.

It will give some false positives, like for the Complex values in the library
where the RECORD is so small it probably does not matter.