vishapoberon/compiler
1
0
Fork 0
mirror of https://github.com/vishapoberon/compiler.git synced 2026-10-09 23:17:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

starting work on dynamically loaded modules and shell

Browse source
This commit is contained in:
Norayr Chilingarian 2026-10-09 03:13:59 +04:00
parent 531bfe168b
commit c18945bc70
15 changed files with 1390 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
ReadMe.md
View file

@ -153,6 +153,9 @@ Execute as usual on Linux (`./hello`) or Windows (`hello`).
For more details on compilation, see [**Compiling**](/doc/Compiling.md).
An optional Unix prototype, [**vloksh and shared modules**](/doc/SharedModules.md),
loads VOC modules on demand and provides a command shell with `.sym`-based completion.
### Viewing the interfaces of included modules.
In order to see the definition of a module's interface, use the "showdef" program.

158
doc/SharedModules.md Normal file
View file

@ -0,0 +1,158 @@
# Shared modules and vloksh (Unix prototype)
`vloksh` is a minimal VOC counterpart of polpo's loksh. It executes Oberon
`Module.Command` procedures in its own address space, loading native ELF `.so`
libraries on demand. The shell and loader are optional: the normal compiler
build and the installed runtime are unchanged.
## Try it
Requires an installed VOC with its shared runtime and development headers,
a C compiler, GNU Readline development files, Python 3 and binutils.
The current shell uses the default Oberon-2 type model (`-O2`).
```sh
make vloksh
make vloksh-test
make vloksh-demo
build/vloksh/vloksh -Pbuild/vloksh/strutils-demo
```
The demo reads sources from `../strutils` without modifying that repository.
Override its location with `STRUTILS=/path/to/strutils`.
Inside the shell:
```text
> StrDemo
StrDemo.Echo
StrDemo.Run
> StrDemo.Run
shared
modules
really
work
calls: 1
> StrDemo.Run
shared
modules
really
work
calls: 2
> StrDemo.Echo hello world
hello world
> quit
```
Tab completes module names, then their commands. Readline supplies editing,
session history and filename completion for arguments. Module and command
discovery does **not** load a library or run a module initializer. It reads the
existing `.sym` files, selecting exported, parameterless proper procedures,
as showdef's `OPT.Import` reader would. Already initialized modules are also
discoverable through VOC's command registry.
The shell also supports single-command and batch operation:
```sh
build/vloksh/vloksh -Pbuild/vloksh/strutils-demo StrDemo.Run
printf 'StrDemo.Run\nStrDemo.Run\n' | build/vloksh/vloksh -Pbuild/vloksh/strutils-demo
```
`-Ppath` or `-P path` overrides `VOC_MODULE_PATH`. Paths are colon-separated;
by default the loader searches the current directory and the directory of the
actually loaded `libvoc-O2.so`. Symbols are searched in those directories,
then `VOC_SYM_PATH`, or the installation's `2/sym` directory when it is unset.
`VOCROOT` can override the installation root for symbol discovery.
Build overrides for another installation:
```sh
make vloksh VOC=/opt/voc/bin/voc VOCROOT=/opt/voc VOCLIBDIR=/opt/voc/lib
```
Build products stay in `build/vloksh`. Nothing is installed by these targets.
## Building and packaging modules
The experimental helper builds **one module per shared object**, in import
order, in the current build directory:
```sh
python3 /path/to/compiler/src/tools/vloksh/voc-shared.py /path/to/strTypes.Mod
python3 /path/to/compiler/src/tools/vloksh/voc-shared.py /path/to/strUtils.Mod
python3 /path/to/compiler/src/tools/vloksh/voc-shared.py /path/to/StrDemo.Mod
```
For module `M` it produces `libvoc-M-O2.so`, plus the usual `M.sym`, `M.h` and
generated C. It translates with VOC and compiles explicitly with `-fPIC`.
Every external import becomes an ELF `DT_NEEDED` dependency; modules already
provided by `libvoc-O2.so` are not duplicated. `$ORIGIN` run paths let libraries
in the same directory find each other. Imports in another directory need a
system linker search path or `LD_LIBRARY_PATH` at runtime; `-P` locates the
requested module but cannot override the ELF linker's dependency search.
The helper supports `--root`, `--libdir`, `--voc`, `--cc`, `--model C`, `-I`
(additional C includes) and `-L` (shared dependencies). `OBERON`/`MODULES`
remain VOC's symbol search configuration. It honors `CC`, `CFLAGS`,
`LDFLAGS`, `LDLIBS`, `VOCROOT` and `VOCLIBDIR`.
A Gentoo package can install:
| File | Location / purpose |
| --- | --- |
| `libvoc-strTypes-O2.so`, `libvoc-strUtils-O2.so` | `/usr/$(get_libdir)`; runtime libraries |
| `strTypes.sym`, `strUtils.sym` | `/usr/share/voc/2/sym`; compilation and shell completion |
| `strTypes.h`, `strUtils.h` | `/usr/share/voc/2/include`; C backend compilation |
Static archives can coexist for existing consumers; `.sym` and `.h` are still
needed when compiling against a packaged module. Runtime execution itself
does not need symbols, but completion of an unloaded module does. No command
index or other completion metadata is required. The overlay ebuilds have not
been changed by this experiment.
## Loader contract and limitations
- `SharedModules.ThisMod` opens `libvoc-M-O2.so` with
`dlopen(RTLD_NOW | RTLD_GLOBAL)`, checks a small ABI descriptor and calls
`M__init`. That initializer registers the module, its commands and GC roots
in the existing shared VOC runtime. `ThisCommand` uses the existing registry.
Existing `Modules.ThisMod` is unchanged; clients needing on-demand lookup
use `SharedModules.ThisMod` in this prototype.
- Handles and pointer-sized values use `SYSTEM.ADDRESS`, not `LONGINT`.
- The helper embeds `M__voc_abi`, checking pointer/basic-type sizes and runtime
descriptor sizes. This is a **shape check, not a complete ABI/versioning or
imported-interface fingerprint check**. Use matching compiler/runtime,
architecture, type model and dependency interfaces. Unrelated Ofront `.so`
files are not compatible. Dependencies must be built with the same contract.
- The shell and all loaded modules must use **one shared** `libvoc-O2.so`.
Statically embedding separate runtime/heap copies in plugins is unsafe.
- Libraries remain loaded until process exit. The loader pins a directly loaded
module's registry entry so `Modules.Free` cannot remove its GC roots. Imported
modules remain referenced by their clients. There is no unload/reload yet:
VOC's static type descriptors, pointer enumerators and finalizers can outlive
a command, making a casual `dlclose` unsafe.
- Commands take no formal parameters or result; arguments are available as
`Oberon.Par.text` starting at `Oberon.Par.pos`. General exported functions
remain callable by importing modules, but are not shell commands.
- HALT, assertions and faults terminate the prototype; there is no loksh-style
trap recovery, process isolation, Unix pipeline syntax or job control yet.
Load only trusted native modules: they have the shell's full permissions.
- The existing registry limits module names to 19 and command names to 23
characters. The helper rejects names that would be truncated.
## Layout and later V4 work
`src/library/v4/SharedModules.Mod` is the reusable, UI-independent loader;
`CommandSymbols.Mod` is a read-only `.sym` command reader following
`src/compiler/OPM.Mod` and `OPT.Mod` (format `F7 84`). Unsupported or malformed
symbols return no partial completions. The small C support layer handles
POSIX library/path operations. The shell and Readline bridge live separately
in `src/tools/vloksh`.
A later graphical port fits under `src/library/v4/ui`, rather than mixing
UI dependencies into the existing headless library. Its entry command could
be launched by the same loader. However, VOC already supplies headless
`Texts` and `Oberon` modules: a loaded UI cannot simply replace those modules
with incompatible records/interfaces of the same names. The port must first
unify the common interfaces or use distinct CLI/UI modules and import aliases,
as polpo does. X11 bindings, fonts/text resources, event loop and trap handling
also need adaptation. This prototype establishes loading, not that UI port.

11
makefile
View file

@ -113,6 +113,17 @@ usage:
tags:
ctags -R --options=oberon.ctags --extras=+q
# Optional Unix command shell; does not rebuild or install the compiler.
.PHONY: vloksh vloksh-test vloksh-demo
vloksh:
$(MAKE) -f src/tools/vloksh/Makefile all
vloksh-test:
$(MAKE) -f src/tools/vloksh/Makefile test
vloksh-demo:
$(MAKE) -f src/tools/vloksh/Makefile demo
# Generate config files Configuration.Make and Configuration.Mod
FORCE:

175
src/library/v4/CommandSymbols.Mod Normal file
View file

@ -0,0 +1,175 @@
MODULE CommandSymbols;
(* Read-only command discovery from VOC .sym files (OPM/OPT format F7 84).
Like showdef, select exported XProc objects with no result/parameters.
Unlike OPT.Import, this reader builds no compiler symbol table and never
rewrites imported symbols. Keep the tags in sync with compiler/OPT.Mod. *)
IMPORT SYSTEM, Files;
TYPE
Visitor* = PROCEDURE(name: ARRAY OF CHAR);
Reader = RECORD
rider: Files.Rider;
length: LONGINT;
valid: BOOLEAN;
depth, refs: INTEGER
END;
Name = ARRAY 256 OF CHAR;
Entry = POINTER TO EntryDesc;
EntryDesc = RECORD next: Entry; name: Name END;
PROCEDURE Byte(VAR r: Reader): CHAR;
VAR ch: CHAR;
BEGIN
IF ~r.valid OR (Files.Pos(r.rider) >= r.length) THEN
r.valid := FALSE; RETURN 0X
END;
Files.Read(r.rider, ch);
IF r.rider.eof THEN r.valid := FALSE END;
RETURN ch
END Byte;
PROCEDURE Number(VAR r: Reader): SYSTEM.INT64;
VAR n, wide: SYSTEM.INT64; shift, b: INTEGER;
BEGIN
n := 0; shift := 0;
LOOP
b := ORD(Byte(r));
IF ~r.valid THEN RETURN 0 END;
IF b < 128 THEN
IF (shift = 63) & (b # 0) & (b # 127) THEN r.valid := FALSE; RETURN 0 END;
wide := b MOD 64 - b DIV 64 * 64;
RETURN n + ASH(wide, shift)
END;
IF shift >= 63 THEN r.valid := FALSE; RETURN 0 END;
wide := b - 128; n := n + ASH(wide, shift);
INC(shift, 7)
END
END Number;
PROCEDURE ReadName(VAR r: Reader; VAR name: Name);
VAR i: INTEGER; ch: CHAR;
BEGIN
i := 0;
REPEAT
ch := Byte(r);
IF i >= LEN(name) - 1 THEN r.valid := FALSE; name[0] := 0X; RETURN END;
name[i] := ch; INC(i)
UNTIL ~r.valid OR (ch = 0X)
END ReadName;
PROCEDURE Skip(VAR r: Reader; count: SYSTEM.INT64);
BEGIN
IF (count < 0) OR (count > r.length - Files.Pos(r.rider)) THEN r.valid := FALSE
ELSE Files.Set(r.rider, Files.Base(r.rider), Files.Pos(r.rider) + SHORT(count))
END
END Skip;
PROCEDURE ^Structure(VAR r: Reader): BOOLEAN;
PROCEDURE Signature(VAR r: Reader): BOOLEAN;
VAR noResult, noParams, ignored: BOOLEAN; tag, n: SYSTEM.INT64; name: Name;
BEGIN
noResult := Structure(r); tag := Number(r); noParams := tag = 18;
WHILE r.valid & (tag # 18) DO
IF (tag # 23) & (tag # 24) & (tag # 42) THEN r.valid := FALSE; RETURN FALSE END;
ignored := Structure(r); n := Number(r); ReadName(r, name); tag := Number(r)
END;
RETURN r.valid & noResult & noParams
END Signature;
PROCEDURE Structure(VAR r: Reader): BOOLEAN;
VAR tag, n: SYSTEM.INT64; ignored: BOOLEAN; name: Name;
BEGIN
tag := Number(r);
IF tag # 34 THEN
IF (tag >= 0) OR (-tag >= r.refs) THEN r.valid := FALSE END;
IF (tag = -4) OR (tag = -7) THEN n := Number(r) END;
RETURN r.valid & (tag = -10) (* OPT.NoTyp *)
END;
INC(r.depth); INC(r.refs);
IF (r.depth > 128) OR (r.refs > 16384) THEN r.valid := FALSE; RETURN FALSE END;
tag := Number(r); (* module index or Smname *)
IF tag = 16 THEN ReadName(r, name)
ELSIF tag > 0 THEN r.valid := FALSE
END;
ReadName(r, name); tag := Number(r);
IF tag = 35 THEN n := Number(r); tag := Number(r) END; (* Ssys *)
CASE SHORT(tag) OF
| 36, 38: ignored := Structure(r) (* pointer, dynamic array *)
| 37: ignored := Structure(r); n := Number(r) (* fixed array *)
| 40: ignored := Signature(r) (* procedure type *)
| 39: (* record: base, size, alignment, method count, fields/methods *)
ignored := Structure(r);
n := Number(r); n := Number(r); n := Number(r); tag := Number(r);
WHILE r.valid & (tag # 18) DO
CASE SHORT(tag) OF
| 25, 26: ignored := Structure(r); ReadName(r, name); n := Number(r)
| 27, 28, 30: n := Number(r)
| 29: ignored := Signature(r); ReadName(r, name); n := Number(r)
ELSE r.valid := FALSE
END;
tag := Number(r)
END
ELSE r.valid := FALSE
END;
DEC(r.depth);
RETURN FALSE
END Structure;
PROCEDURE Enumerate*(filename, module: ARRAY OF CHAR; visit: Visitor): BOOLEAN;
VAR f: Files.File; r: Reader; name: Name; tag, n: SYSTEM.INT64;
command, ignored: BOOLEAN; first, last, entry: Entry; ch: CHAR;
BEGIN
f := Files.Old(filename);
IF f = NIL THEN RETURN FALSE END;
Files.Set(r.rider, f, 0); r.length := Files.Length(f);
r.valid := TRUE; r.depth := 0; r.refs := 14; first := NIL; last := NIL;
ch := Byte(r); IF ch # 0F7X THEN r.valid := FALSE END;
ch := Byte(r); IF ch # 084X THEN r.valid := FALSE END;
tag := Number(r); IF tag # 16 THEN r.valid := FALSE END;
ReadName(r, name); IF name # module THEN r.valid := FALSE END;
(* Link names precede the exported objects. *)
REPEAT ReadName(r, name) UNTIL ~r.valid OR (name = "");
WHILE r.valid & (Files.Pos(r.rider) < r.length) DO
tag := Number(r); command := FALSE;
WHILE r.valid & (tag = 41) DO (* exported documentation *)
n := Number(r); Skip(r, n); tag := Number(r)
END;
IF (tag >= 1) & (tag <= 9) THEN
CASE SHORT(tag) OF
| 1, 2, 3: ch := Byte(r)
| 4, 7: n := Number(r); n := Number(r)
| 5: Skip(r, 4)
| 6: Skip(r, 8)
| 8: ReadName(r, name)
| 9: (* NIL *)
ELSE r.valid := FALSE
END;
ReadName(r, name)
ELSIF tag = 19 THEN ignored := Structure(r)
ELSIF (tag = 20) OR (tag = 21) OR (tag = 22) THEN
ignored := Structure(r); ReadName(r, name)
ELSIF (tag >= 31) & (tag <= 33) THEN
command := Signature(r) & (tag = 31);
IF tag = 33 THEN n := Number(r); Skip(r, n) END;
ReadName(r, name);
IF command & r.valid THEN
NEW(entry); entry.name := name;
IF last = NIL THEN first := entry ELSE last.next := entry END;
last := entry
END
ELSE r.valid := FALSE
END
END;
Files.Close(f);
(* No partial results from an unsupported or corrupt symbol file. *)
IF r.valid THEN
entry := first;
WHILE entry # NIL DO visit(entry.name); entry := entry.next END
END;
RETURN r.valid
END Enumerate;
END CommandSymbols.

107
src/library/v4/SharedModules.Mod Normal file
View file

@ -0,0 +1,107 @@
MODULE SharedModules;
(* Optional Unix shared-object loader. Uses VOC's existing module registry.
Libraries remain mapped for the lifetime of the process: the heap can
retain their enumPtrs procedures, type descriptors and finalizers. *)
IMPORT SYSTEM, Heap, Modules, CommandSymbols;
TYPE
Module* = Modules.Module;
Command* = Modules.Command;
Visitor* = CommandSymbols.Visitor;
ModuleBody = PROCEDURE(): SYSTEM.PTR;
VAR
path*: ARRAY 4096 OF CHAR;
res*: INTEGER;
resMsg*: ARRAY 1024 OF CHAR;
model: ARRAY 3 OF CHAR;
PROCEDURE -AInclude '#include "SharedModulesSupport.h"';
PROCEDURE -DefaultPath(VAR search: ARRAY OF CHAR)
'VOCShared_default_path((char*)search, search__len)';
PROCEDURE -Open(name, search, model: ARRAY OF CHAR;
shortSize, intSize, longSize, setSize: SYSTEM.INT32;
VAR error: ARRAY OF CHAR): SYSTEM.ADDRESS
'VOCShared_open((char*)name, (char*)search, (char*)model, shortSize, intSize, longSize, setSize, (char*)error, error__len)';
PROCEDURE -Body(handle: SYSTEM.ADDRESS; name: ARRAY OF CHAR): ModuleBody
'(SharedModules_ModuleBody)VOCShared_body(handle, (char*)name)';
PROCEDURE -DiskModules(search, model: ARRAY OF CHAR; visit: Visitor)
'VOCShared_modules((char*)search, (char*)model, visit)';
PROCEDURE -SymbolFile(name, search, model: ARRAY OF CHAR;
VAR filename: ARRAY OF CHAR): BOOLEAN
'VOCShared_symbol((char*)name, (char*)search, (char*)model, (char*)filename, filename__len)';
PROCEDURE First*(): Module;
BEGIN RETURN SYSTEM.VAL(Module, Heap.modules)
END First;
PROCEDURE ThisMod*(name: ARRAY OF CHAR): Module;
VAR m: Module; handle: SYSTEM.ADDRESS; body: ModuleBody;
BEGIN
m := Modules.ThisMod(name);
IF m = NIL THEN
handle := Open(name, path, model, SIZE(SHORTINT), SIZE(INTEGER),
SIZE(LONGINT), SIZE(SET), resMsg);
IF handle = 0 THEN res := 1; RETURN NIL END;
body := Body(handle, name);
m := SYSTEM.VAL(Module, body());
IF (m = NIL) OR (m.name # name) THEN
(* Do not unmap code after calling an initializer, even on failure. *)
res := 1; resMsg := "initializer did not register the requested module";
RETURN NIL
END;
Heap.INCREF(m) (* Pin its registry/GC roots; Modules.Free must not remove them. *)
END;
res := 0; resMsg := "";
RETURN m
END ThisMod;
PROCEDURE ThisCommand*(m: Module; name: ARRAY OF CHAR): Command;
VAR cmd: Command;
BEGIN
IF m = NIL THEN res := 1; resMsg := "module not loaded"; RETURN NIL END;
cmd := Modules.ThisCommand(m, name);
res := Modules.res; COPY(Modules.resMsg, resMsg);
RETURN cmd
END ThisCommand;
PROCEDURE EnumerateModules*(visit: Visitor);
VAR m: Module;
BEGIN
m := First();
WHILE m # NIL DO visit(m.name); m := m.next END;
DiskModules(path, model, visit)
END EnumerateModules;
PROCEDURE EnumerateCommands*(name: ARRAY OF CHAR; visit: Visitor): BOOLEAN;
VAR m: Module; c: Modules.Cmd; filename: ARRAY 8192 OF CHAR; valid: BOOLEAN;
BEGIN
m := Modules.ThisMod(name);
IF m # NIL THEN
c := m.cmds;
WHILE c # NIL DO visit(c.name); c := c.next END;
res := 0; resMsg := ""; RETURN TRUE
ELSE
(* Read symbols, without dlopen or module initialization. *)
IF SymbolFile(name, path, model, filename) THEN
valid := CommandSymbols.Enumerate(filename, name, visit);
IF valid THEN res := 0; resMsg := ""; RETURN TRUE END;
resMsg := "invalid or unsupported symbol file"
ELSE
resMsg := "symbol file not found (set VOC_SYM_PATH if symbols are elsewhere)"
END
END;
res := 1; RETURN FALSE
END EnumerateCommands;
BEGIN
IF SIZE(INTEGER) = 2 THEN model := "O2" ELSE model := "OC" END;
DefaultPath(path)
END SharedModules.

211
src/library/v4/SharedModulesSupport.c Normal file
View file

@ -0,0 +1,211 @@
#define _GNU_SOURCE
#include <dlfcn.h>
#include <dirent.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "SharedModulesSupport.h"
/* The handles returned by open are deliberately never closed after __init.
In particular, dlclose is NOT an implementation of Heap.FreeModule. */
static int identifier(const char *s, size_t maximum)
{
size_t n = 0;
if (!((*s >= 'A' && *s <= 'Z') || (*s >= 'a' && *s <= 'z'))) return 0;
for (; *s; s++, n++) {
if (!((*s >= 'A' && *s <= 'Z') || (*s >= 'a' && *s <= 'z') ||
(*s >= '0' && *s <= '9') || *s == '_')) return 0;
}
return n <= maximum;
}
/* Empty path components mean the current directory, as in loksh. */
static int directory(const char **path, char *dir, size_t capacity)
{
const char *end;
size_t n;
if (!*path) return 0;
end = strchr(*path, ':');
n = end ? (size_t)(end - *path) : strlen(*path);
if (n >= capacity) { *path = end ? end + 1 : NULL; return -1; }
memcpy(dir, *path, n);
dir[n] = 0;
if (!n) strcpy(dir, ".");
*path = end ? end + 1 : NULL;
return 1;
}
static int filename(char *out, size_t capacity, const char *dir,
const char *name, const char *model, const char *extension)
{
int n = snprintf(out, capacity, "%s/libvoc-%s-%s.%s", dir, name, model, extension);
return n >= 0 && (size_t)n < capacity;
}
void VOCShared_default_path(char *path, ADDRESS capacity)
{
Dl_info info;
char dir[4096], *slash;
const char *env = getenv("VOC_MODULE_PATH");
if (env) { snprintf(path, (size_t)capacity, "%s", env); return; }
snprintf(path, (size_t)capacity, ".");
/* Locate the actual shared runtime, not the executable or a compiled-in
/usr/lib versus /usr/lib64 guess. */
if (dladdr((void *)Heap_REGMOD, &info) && info.dli_fname) {
snprintf(dir, sizeof dir, "%s", info.dli_fname);
slash = strrchr(dir, '/');
if (slash) {
*slash = 0;
snprintf(path, (size_t)capacity, ".:%s", dir);
}
}
}
ADDRESS VOCShared_open(const char *name, const char *path, const char *model,
INT32 short_size, INT32 int_size, INT32 long_size, INT32 set_size,
char *error, ADDRESS capacity)
{
char dir[4096], file[8192], symbol[128];
const char *cursor = path;
void *handle = NULL;
const INT32 *abi;
const INT32 expected[] = {1, sizeof(ADDRESS), short_size, int_size,
long_size, set_size, sizeof(Heap_ModuleDesc), sizeof(Heap_CmdDesc)};
size_t i;
int status, found = 0;
if (!identifier(name, sizeof(Heap_ModuleName) - 1)) {
snprintf(error, (size_t)capacity, "invalid module name (maximum 19 characters)");
return 0;
}
while ((status = directory(&cursor, dir, sizeof dir))) {
if (status < 0 || !filename(file, sizeof file, dir, name, model, "so")) continue;
if (access(file, F_OK) == 0) {
found = 1;
handle = dlopen(file, RTLD_NOW | RTLD_GLOBAL);
break;
}
}
if (!found) {
snprintf(file, sizeof file, "libvoc-%s-%s.so", name, model);
handle = dlopen(file, RTLD_NOW | RTLD_GLOBAL);
}
if (!handle) {
const char *message = dlerror();
snprintf(error, (size_t)capacity, "%s", message ? message : "library not found");
return 0;
}
snprintf(symbol, sizeof symbol, "%s__voc_abi", name);
abi = dlsym(handle, symbol);
if (!abi) {
snprintf(error, (size_t)capacity, "%s: missing VOC shared-module ABI descriptor", file);
dlclose(handle);
return 0;
}
for (i = 0; i < sizeof expected / sizeof expected[0]; i++) {
if (abi[i] != expected[i]) {
snprintf(error, (size_t)capacity, "%s: incompatible VOC shared-module ABI", file);
dlclose(handle);
return 0;
}
}
snprintf(symbol, sizeof symbol, "%s__init", name);
if (!dlsym(handle, symbol)) {
snprintf(error, (size_t)capacity, "%s: missing %s", file, symbol);
dlclose(handle);
return 0;
}
error[0] = 0;
return (ADDRESS)handle;
}
void *VOCShared_body(ADDRESS handle, const char *name)
{
char symbol[128];
snprintf(symbol, sizeof symbol, "%s__init", name);
return dlsym((void *)handle, symbol);
}
static void symbol_path(char *path, size_t capacity, const char *model)
{
Dl_info info;
char prefix[4096], *slash;
const char *env = getenv("VOC_SYM_PATH");
const char *root = getenv("VOCROOT");
const char *subdir = !strcmp(model, "O2") ? "2" : "C";
if (env) { snprintf(path, capacity, "%s", env); return; }
if (root) { snprintf(path, capacity, "%s/%s/sym", root, subdir); return; }
path[0] = 0;
if (dladdr((void *)Heap_REGMOD, &info) && info.dli_fname) {
snprintf(prefix, sizeof prefix, "%s", info.dli_fname);
slash = strrchr(prefix, '/');
if (!slash) return;
*slash = 0;
slash = strrchr(prefix, '/');
if (!slash) return;
*slash = 0;
snprintf(path, capacity, "%s/%s/sym:%s/share/voc/%s/sym", prefix, subdir, prefix, subdir);
}
}
static void enumerate_directory(const char *dir, const char *model,
VOCShared_Visitor visit, int symbols)
{
char suffix[32], name[sizeof(Heap_ModuleName)];
DIR *stream;
struct dirent *entry;
size_t n, prefix_size = symbols ? 0 : 7, suffix_size;
if (!(stream = opendir(dir))) return;
if (symbols) snprintf(suffix, sizeof suffix, ".sym");
else snprintf(suffix, sizeof suffix, "-%s.so", model);
suffix_size = strlen(suffix);
while ((entry = readdir(stream))) {
n = strlen(entry->d_name);
if (n <= prefix_size + suffix_size ||
(!symbols && strncmp(entry->d_name, "libvoc-", prefix_size)) ||
strcmp(entry->d_name + n - suffix_size, suffix)) continue;
n -= prefix_size + suffix_size;
if (n >= sizeof name) continue;
memcpy(name, entry->d_name + prefix_size, n);
name[n] = 0;
if (identifier(name, sizeof name - 1)) visit((CHAR *)name, sizeof name);
}
closedir(stream);
}
void VOCShared_modules(const char *path, const char *model, VOCShared_Visitor visit)
{
char dir[4096], symbols[8192];
const char *cursor = path;
int status;
while ((status = directory(&cursor, dir, sizeof dir))) {
if (status < 0) continue;
enumerate_directory(dir, model, visit, 0);
enumerate_directory(dir, model, visit, 1);
}
symbol_path(symbols, sizeof symbols, model);
cursor = symbols;
while ((status = directory(&cursor, dir, sizeof dir)))
if (status > 0) enumerate_directory(dir, model, visit, 1);
}
int VOCShared_symbol(const char *name, const char *path, const char *model,
char *file, ADDRESS capacity)
{
char dir[4096], symbols[8192];
const char *cursor = path;
int pass, status, n;
if (!identifier(name, sizeof(Heap_ModuleName) - 1)) return 0;
symbol_path(symbols, sizeof symbols, model);
for (pass = 0; pass < 2; pass++, cursor = symbols) {
while ((status = directory(&cursor, dir, sizeof dir))) {
if (status < 0) continue;
n = snprintf(file, (size_t)capacity, "%s/%s.sym", dir, name);
if (n >= 0 && n < capacity && !access(file, R_OK)) return 1;
}
}
return 0;
}

17
src/library/v4/SharedModulesSupport.h Normal file
View file

@ -0,0 +1,17 @@
#ifndef VOC_SHARED_MODULES_SUPPORT_H
#define VOC_SHARED_MODULES_SUPPORT_H
#include "Heap.h"
typedef void (*VOCShared_Visitor)(CHAR *name, ADDRESS length);
void VOCShared_default_path(char *path, ADDRESS capacity);
ADDRESS VOCShared_open(const char *name, const char *path, const char *model,
INT32 short_size, INT32 int_size, INT32 long_size, INT32 set_size,
char *error, ADDRESS capacity);
void *VOCShared_body(ADDRESS handle, const char *name);
void VOCShared_modules(const char *path, const char *model, VOCShared_Visitor visit);
int VOCShared_symbol(const char *name, const char *path, const char *model,
char *filename, ADDRESS capacity);
#endif

34
src/tools/vloksh/Makefile Normal file
View file

@ -0,0 +1,34 @@
# Optional Unix prototype, built against an existing VOC installation.
VOC ?= voc
CC ?= cc
VOCROOT ?= /usr/share/voc
VOCLIBDIR ?= $(dir $(shell $(CC) -print-file-name=libvoc-O2.so))
ROOT := $(abspath $(dir $(lastword $(MAKEFILE_LIST)))/../../..)
BUILD ?= $(ROOT)/build/vloksh
SUPPORT := $(ROOT)/src/library/v4
TOOLS := $(ROOT)/src/tools/vloksh
INCLUDES := -I$(VOCROOT)/2/include -I$(SUPPORT) -I$(TOOLS) -I$(BUILD)
.PHONY: all test demo
all: $(BUILD)/vloksh
$(BUILD):
mkdir -p "$@"
$(BUILD)/CommandSymbols.c: $(SUPPORT)/CommandSymbols.Mod | $(BUILD)
cd "$(BUILD)" && VOCROOT="$(VOCROOT)" VOCLIBDIR="$(VOCLIBDIR)" $(VOC) -sFS "$(SUPPORT)/CommandSymbols.Mod"
$(BUILD)/SharedModules.c: $(SUPPORT)/SharedModules.Mod $(BUILD)/CommandSymbols.c
cd "$(BUILD)" && VOCROOT="$(VOCROOT)" VOCLIBDIR="$(VOCLIBDIR)" $(VOC) -sFS "$(SUPPORT)/SharedModules.Mod"
$(BUILD)/vloksh.c: $(TOOLS)/vloksh.Mod $(BUILD)/SharedModules.c
cd "$(BUILD)" && VOCROOT="$(VOCROOT)" VOCLIBDIR="$(VOCLIBDIR)" $(VOC) -sSm "$(TOOLS)/vloksh.Mod"
$(BUILD)/vloksh: $(BUILD)/vloksh.c $(BUILD)/SharedModules.c $(BUILD)/CommandSymbols.c $(SUPPORT)/SharedModulesSupport.c $(SUPPORT)/SharedModulesSupport.h $(TOOLS)/Terminal.c $(TOOLS)/Terminal.h
$(CC) $(CFLAGS) -fPIC -g -Wno-stringop-overflow $(INCLUDES) $(BUILD)/vloksh.c $(BUILD)/SharedModules.c $(BUILD)/CommandSymbols.c $(SUPPORT)/SharedModulesSupport.c $(TOOLS)/Terminal.c $(LDFLAGS) -L$(VOCLIBDIR) -Wl,-rpath,$(abspath $(VOCLIBDIR)) -o "$@" -lvoc-O2 -lreadline -ldl
test: all
VOC="$(VOC)" VOCROOT="$(VOCROOT)" VOCLIBDIR="$(VOCLIBDIR)" CC="$(CC)" python3 "$(TOOLS)/test.py" "$(BUILD)/vloksh"
demo: all
VOC="$(VOC)" VOCROOT="$(VOCROOT)" VOCLIBDIR="$(VOCLIBDIR)" CC="$(CC)" python3 "$(TOOLS)/demo.py" "$(BUILD)"

86
src/tools/vloksh/Terminal.c Normal file
View file

@ -0,0 +1,86 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <readline/readline.h>
#include <readline/history.h>
#include "Terminal.h"
static VlokshComplete complete;
static char **candidates;
static size_t count, next;
void Vloksh_add_completion(const char *name)
{
char **grown;
char *copy;
size_t i;
for (i = 0; i < count; i++) if (!strcmp(candidates[i], name)) return;
copy = strdup(name);
if (!copy) return;
grown = realloc(candidates, (count + 1) * sizeof *candidates);
if (!grown) { free(copy); return; }
candidates = grown;
candidates[count++] = copy;
}
static char *candidate(const char *text, int state)
{
(void)text;
if (!state) next = 0;
return next < count ? strdup(candidates[next++]) : NULL;
}
static char **matches(const char *text, int start, int end)
{
char **result;
size_t i;
(void)end;
/* First word: Oberon commands. Arguments: Readline's file completion. */
for (i = 0; i < (size_t)start; i++)
if (rl_line_buffer[i] != ' ' && rl_line_buffer[i] != '\t') return NULL;
rl_attempted_completion_over = 1;
/* A completed module ends in '.', not a space: the command follows. */
rl_completion_suppress_append = strchr(text, '.') == NULL;
count = 0;
complete((CHAR *)text, (ADDRESS)strlen(text) + 1);
result = rl_completion_matches(text, candidate);
for (i = 0; i < count; i++) free(candidates[i]);
free(candidates);
candidates = NULL;
count = 0;
return result;
}
void Vloksh_terminal_init(VlokshComplete callback)
{
complete = callback;
rl_readline_name = "vloksh";
rl_completer_word_break_characters = " \t\n";
rl_attempted_completion_function = matches;
using_history();
stifle_history(64);
}
int Vloksh_read_line(char *line, ADDRESS capacity)
{
char *input = NULL;
size_t allocated = 0, length;
ssize_t n;
if (isatty(STDIN_FILENO)) {
input = readline("> ");
if (!input) return 0;
if (*input) add_history(input);
length = strlen(input);
} else {
n = getline(&input, &allocated, stdin);
if (n < 0) { free(input); return 0; }
length = (size_t)n;
while (length && (input[length - 1] == '\n' || input[length - 1] == '\r')) --length;
input[length] = 0;
}
if (length >= (size_t)capacity) { free(input); return -1; }
memcpy(line, input, length + 1);
free(input);
return 1;
}

11
src/tools/vloksh/Terminal.h Normal file
View file

@ -0,0 +1,11 @@
#ifndef VLOKSH_TERMINAL_H
#define VLOKSH_TERMINAL_H
#include "SYSTEM.h"
typedef void (*VlokshComplete)(CHAR *prefix, ADDRESS length);
void Vloksh_terminal_init(VlokshComplete complete);
int Vloksh_read_line(char *line, ADDRESS capacity);
void Vloksh_add_completion(const char *name);
#endif

19
src/tools/vloksh/demo.py Normal file
View file

@ -0,0 +1,19 @@
#!/usr/bin/env python3
"""Build the optional strutils experiment without modifying that repository."""
import os
from pathlib import Path
import subprocess
import sys
tools = Path(__file__).resolve().parent
compiler = tools.parents[2]
strutils = Path(os.environ.get("STRUTILS", str(compiler.parent / "strutils")))
output = Path(sys.argv[1]).resolve() / "strutils-demo"
output.mkdir(parents=True, exist_ok=True)
for source in [strutils / "src/strTypes.Mod", strutils / "src/strUtils.Mod",
tools / "examples/StrDemo.Mod"]:
subprocess.run([sys.executable, str(tools / "voc-shared.py"), str(source)],
cwd=output, check=True)
print(f"\nRun: {output.parent / 'vloksh'} -P{output} StrDemo.Run")
print(f"Or: {output.parent / 'vloksh'} -P{output} (then StrDemo.<Tab>)")

28
src/tools/vloksh/examples/StrDemo.Mod Normal file
View file

@ -0,0 +1,28 @@
MODULE StrDemo;
(* A command module; neither it nor strUtils is linked into vloksh. *)
IMPORT strUtils, Out, Oberon, Texts;
VAR count: LONGINT;
PROCEDURE Run*;
VAR words: strUtils.pstrings; i: LONGINT;
BEGIN
words := strUtils.tokenize("shared modules really work", " ");
i := 0;
WHILE i < LEN(words^) DO
Out.String(words^[i]^); Out.Ln; INC(i)
END;
INC(count);
Out.String("calls: "); Out.Int(count, 0); Out.Ln
END Run;
PROCEDURE Echo*;
VAR r: Texts.Reader; ch: CHAR;
BEGIN
Texts.OpenReader(r, Oberon.Par.text, Oberon.Par.pos);
WHILE ~r.eot DO Texts.Read(r, ch); IF ~r.eot THEN Out.Char(ch) END END;
Out.Ln
END Echo;
END StrDemo.

206
src/tools/vloksh/test.py Normal file
View file

@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Integration tests using the installed VOC, ELF loader, and a real pty."""
import os
from pathlib import Path
import pty
import re
import select
import shlex
import subprocess
import sys
import tempfile
import time
tools = Path(__file__).resolve().parent
shell = Path(sys.argv[1]).resolve()
helper = tools / "voc-shared.py"
PROVIDER = '''MODULE Provider;
IMPORT Out;
VAR count: LONGINT;
PROCEDURE Next*(): LONGINT;
BEGIN INC(count); RETURN count END Next;
BEGIN Out.String("provider initialized"); Out.Ln
END Provider.
'''
CONSUMER = '''MODULE Consumer;
IMPORT Provider, Out, Oberon, Texts, Heap;
CONST A* = 1234567890; B* = {1, 3}; C* = "string"; D* = 1.5; E* = NIL;
TYPE Node* = POINTER TO NodeDesc;
NodeDesc* = RECORD next*: Node; n*: LONGINT END;
Handler* = PROCEDURE (VAR node: Node): Node;
Grid* = ARRAY 3, 4 OF Node;
VAR kept*: Node; handler*: Handler;
PROCEDURE (node: Node) Touch*;
BEGIN INC(node.n) END Touch;
PROCEDURE Function*(): LONGINT;
BEGIN RETURN 17 END Function;
PROCEDURE WithArg*(n: LONGINT);
BEGIN Out.Int(n, 0) END WithArg;
(** A command documented in the symbol file. *)
PROCEDURE Run*;
BEGIN
IF kept = NIL THEN NEW(kept); kept.n := 40 END;
kept.Touch; Heap.GC(TRUE);
Out.String("run "); Out.Int(Provider.Next(), 0);
Out.String(" kept "); Out.Int(kept.n, 0); Out.Ln
END Run;
PROCEDURE Echo*;
VAR r: Texts.Reader; ch: CHAR;
BEGIN
Texts.OpenReader(r, Oberon.Par.text, Oberon.Par.pos);
WHILE ~r.eot DO Texts.Read(r, ch); IF ~r.eot THEN Out.Char(ch) END END;
Out.Ln
END Echo;
BEGIN Out.String("consumer initialized"); Out.Ln
END Consumer.
'''
def run(*args, input=None, expected=0, env=None):
result = subprocess.run([str(shell), *map(str, args)], input=input, text=True,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
env=env, timeout=10)
assert result.returncode == expected, (args, result.returncode, result.stdout)
return result.stdout
def build(directory, name, source):
file = directory / f"{name}.Mod"
file.write_text(source)
subprocess.run([sys.executable, str(helper), str(file)], cwd=directory, check=True)
def interactive(directory):
pid, fd = pty.fork()
if pid == 0:
os.execv(str(shell), [str(shell), f"-P{directory}"])
collected = bytearray()
def until(needle):
data = bytearray()
deadline = time.monotonic() + 10
while needle not in data:
remaining = deadline - time.monotonic()
assert remaining > 0, (needle, bytes(data))
ready, _, _ = select.select([fd], [], [], remaining)
assert ready, (needle, bytes(data))
chunk = os.read(fd, 65536)
assert chunk, bytes(data)
data.extend(chunk)
collected.extend(data)
return bytes(data)
try:
until(b"> ")
# Completion reads symbols even when ELF dependencies cannot be loaded.
dependency = directory / "libvoc-Provider-O2.so"
hidden = dependency.with_suffix(".hidden")
dependency.rename(hidden)
try:
os.write(fd, b"Consu\t")
result = until(b"Consumer.")
assert b"initialized" not in result
os.write(fd, b"R\t")
result = until(b"Run")
assert b"initialized" not in result
finally:
hidden.rename(dependency)
os.write(fd, b"\n")
result = until(b"> ")
assert b"run 1 kept 41" in result, result
# History and in-process state survive the first call.
os.write(fd, b"\x1b[A\n")
result = until(b"> ")
assert b"run 2 kept 42" in result, result
os.write(fd, b"\x04")
_, status = os.waitpid(pid, 0)
assert os.waitstatus_to_exitcode(status) == 0
pid = None
finally:
os.close(fd)
if pid:
os.kill(pid, 9)
os.waitpid(pid, 0)
def main(directory):
build(directory, "Provider", PROVIDER)
build(directory, "Consumer", CONSUMER)
path = f"-P{directory}"
listed = run(path, "Consumer")
assert set(listed.splitlines()) == {" Consumer.Echo", " Consumer.Run"}, listed
assert "initialized" not in listed
assert "Function" not in listed and "WithArg" not in listed and "Touch" not in listed
# Discovery does not require any shared object at all.
library = directory / "libvoc-Consumer-O2.so"
library.rename(library.with_suffix(".hidden"))
assert run(path, "Consumer") == listed
library.with_suffix(".hidden").rename(library)
output = run(path, "Consumer.Run")
assert "provider initialized\nconsumer initialized\nrun 1 kept 41\n" == output, output
output = run(path, input="Consumer.Run\nConsumer.Run\nConsumer.Echo batch parameters\nquit\n")
assert output.count("consumer initialized") == 1, output
assert "run 1 kept 41\nrun 2 kept 42\nbatch parameters\n" in output, output
assert run(path, "Consumer.Echo", "hello", "world").endswith("hello world\n")
env = dict(os.environ, VOC_MODULE_PATH=str(directory))
assert "run 1 kept 41" in run("Consumer.Run", env=env)
assert "libvoc-Absent-O2.so" in run(path, "Absent.Run", expected=1)
assert "not found" in run(path, "Consumer.Absent", expected=1)
assert "symbol file not found" in run(path, "Absent", expected=1)
assert "too long" in run(path, input="x" * 4096 + "\n", expected=1)
assert "needs a module path" in run("-P", expected=1)
# Symbol reader handles the same rich interfaces as showdef.
root = Path(os.environ.get("VOCROOT", "/usr/share/voc"))
for name in ("Texts", "Files", "Heap", "Math", "oocStrings", "ulmObjects"):
interface = subprocess.check_output(["showdef", name], text=True)
expected = set(re.findall(r"^ PROCEDURE (\w+)\s*;", interface, re.MULTILINE))
# Use a fresh name so even linked-in core modules exercise the .sym
# parser rather than falling back to their in-memory command registry.
symbols = (root / "2/sym" / f"{name}.sym").read_bytes()
(directory / "SymbolProbe.sym").write_bytes(
symbols.replace(name.encode() + b"\0", b"SymbolProbe\0", 1))
actual = {line.strip().split(".", 1)[1] for line in run(path, "SymbolProbe").splitlines()}
assert actual == expected, (name, actual, expected)
# Scan the entire installed symbol corpus, not only the example's types.
for symbols in (root / "2/sym").glob("*.sym"):
(directory / "SymbolProbe.sym").write_bytes(symbols.read_bytes().replace(
symbols.stem.encode() + b"\0", b"SymbolProbe\0", 1))
run(path, "SymbolProbe")
# Fail closed on unsupported versions, truncated streams and malformed tags.
good = (directory / "Consumer.sym").read_bytes().replace(b"Consumer\0", b"Broken\0", 1)
(directory / "Broken.sym").write_bytes(good)
assert "Broken.Run" in run(path, "Broken")
for content in (good[:1], good[:20], good[:-1], good[:1] + b"\x00" + good[2:],
good[:2] + b"\x80" * 40):
(directory / "Broken.sym").write_bytes(content)
assert "invalid or unsupported" in run(path, "Broken", expected=1)
for name, descriptor in (("BadABI", "export const INT32 BadABI__voc_abi[8] = {1, 0};"),
("NoABI", "")):
source = directory / f"{name}.c"
source.write_text('#include "SYSTEM.h"\n' + descriptor +
f'\nexport void *{name}__init(void) {{ return 0; }}\n')
subprocess.run([*shlex.split(os.environ.get("CC", "cc")), "-fPIC", "-shared",
f"-I{root / '2/include'}", str(source), "-o",
str(directory / f"libvoc-{name}-O2.so")], check=True)
assert "ABI" in run(path, f"{name}.Run", expected=1)
dynamic = subprocess.check_output(["readelf", "-d", str(library)], text=True)
assert "libvoc-Provider-O2.so" in dynamic and "libvoc-O2.so" in dynamic
assert "$ORIGIN" in dynamic
interactive(directory)
print("vloksh tests passed: symbols, no-init completion, pty/history, shared dependencies, GC/state, errors, ABI")
if __name__ == "__main__":
temp_root = Path("/tmp/opencode")
temp_root.mkdir(exist_ok=True)
with tempfile.TemporaryDirectory(prefix="vloksh-test-", dir=temp_root) as directory:
main(Path(directory))

201
src/tools/vloksh/vloksh.Mod Normal file
View file

@ -0,0 +1,201 @@
MODULE vloksh;
(* Vishap Linux Oberon Kommand Shell. Commands execute in this process,
not in child processes, just as in loksh and the Oberon UI. *)
IMPORT SYSTEM, Modules, SharedModules, Oberon, Texts, Out, Heap, Platform;
VAR
prefix, modulePrefix: ARRAY 128 OF CHAR;
failed, quit: BOOLEAN;
PROCEDURE -AInclude '#include "Terminal.h"';
PROCEDURE -InitTerminal(complete: SharedModules.Visitor)
'Vloksh_terminal_init(complete)';
PROCEDURE -ReadLine(VAR line: ARRAY OF CHAR): SYSTEM.INT32
'Vloksh_read_line((char*)line, line__len)';
PROCEDURE -AddCompletion(name: ARRAY OF CHAR)
'Vloksh_add_completion((char*)name)';
PROCEDURE StartsWith(name, prefix: ARRAY OF CHAR): BOOLEAN;
VAR i: INTEGER;
BEGIN
i := 0;
WHILE (i < LEN(prefix)) & (prefix[i] # 0X) DO
IF (i >= LEN(name)) OR (name[i] # prefix[i]) THEN RETURN FALSE END;
INC(i)
END;
RETURN TRUE
END StartsWith;
PROCEDURE Append(source: ARRAY OF CHAR; VAR target: ARRAY OF CHAR): BOOLEAN;
VAR i, j: INTEGER;
BEGIN
i := 0; WHILE (i < LEN(target)) & (target[i] # 0X) DO INC(i) END;
j := 0;
WHILE (j < LEN(source)) & (source[j] # 0X) DO
IF i >= LEN(target) - 1 THEN RETURN FALSE END;
target[i] := source[j]; INC(i); INC(j)
END;
target[i] := 0X;
RETURN TRUE
END Append;
PROCEDURE CompleteModule(name: ARRAY OF CHAR);
VAR candidate: ARRAY 128 OF CHAR; ok: BOOLEAN;
BEGIN
IF StartsWith(name, prefix) THEN
COPY(name, candidate); ok := Append(".", candidate);
IF ok THEN AddCompletion(candidate) END
END
END CompleteModule;
PROCEDURE CompleteCommand(name: ARRAY OF CHAR);
VAR candidate: ARRAY 128 OF CHAR; ok: BOOLEAN;
BEGIN
COPY(modulePrefix, candidate);
ok := Append(".", candidate) & Append(name, candidate);
IF ok & StartsWith(candidate, prefix) THEN AddCompletion(candidate) END
END CompleteCommand;
PROCEDURE Complete(text: ARRAY OF CHAR);
VAR i, j: INTEGER; valid: BOOLEAN;
BEGIN
IF LEN(text) > LEN(prefix) THEN RETURN END;
COPY(text, prefix);
i := 0; WHILE (prefix[i] # 0X) & (prefix[i] # ".") DO INC(i) END;
IF prefix[i] = "." THEN
j := 0; WHILE j < i DO modulePrefix[j] := prefix[j]; INC(j) END;
modulePrefix[j] := 0X;
valid := SharedModules.EnumerateCommands(modulePrefix, CompleteCommand)
ELSE
IF StartsWith("help", prefix) THEN AddCompletion("help") END;
IF StartsWith("quit", prefix) THEN AddCompletion("quit") END;
IF StartsWith("exit", prefix) THEN AddCompletion("exit") END;
SharedModules.EnumerateModules(CompleteModule)
END
END Complete;
PROCEDURE Help;
BEGIN
Out.String("vloksh - Vishap Linux Oberon Kommand Shell"); Out.Ln;
Out.String(" vloksh [-Ppath] [Module.Command [arguments]]"); Out.Ln;
Out.String(" vloksh < script run a batch of commands"); Out.Ln;
Out.String(" Module list its commands without initializing it"); Out.Ln;
Out.String(" Tab: completion; arrows: history; quit/exit/Ctrl-D: leave"); Out.Ln;
Out.String(" VOC_MODULE_PATH: colon-separated module directories"); Out.Ln;
Out.String(" Modules stay loaded; HALT or a fault terminates this prototype."); Out.Ln
END Help;
PROCEDURE ShowCommand(name: ARRAY OF CHAR);
BEGIN
Out.String(" "); Out.String(modulePrefix); Out.Char("."); Out.String(name); Out.Ln
END ShowCommand;
PROCEDURE Error;
BEGIN
Out.String("vloksh: "); Out.String(SharedModules.resMsg); Out.Ln;
failed := TRUE
END Error;
PROCEDURE Execute(line: ARRAY OF CHAR);
VAR i, j, dot, start: INTEGER; name: ARRAY 128 OF CHAR;
command: ARRAY 64 OF CHAR; m: SharedModules.Module;
proc: SharedModules.Command; par, previous: Oberon.ParList;
W: Texts.Writer;
BEGIN
i := 0; WHILE (line[i] = " ") OR (line[i] = 09X) DO INC(i) END;
IF (line[i] = 0X) OR (line[i] = "#") THEN RETURN END;
j := 0; dot := -1;
WHILE (line[i] # 0X) & (line[i] > " ") DO
IF j >= LEN(name) - 1 THEN
SharedModules.resMsg := "command name too long"; Error; RETURN
END;
IF line[i] = "." THEN dot := j END;
name[j] := line[i]; INC(i); INC(j)
END;
name[j] := 0X;
IF (name = "quit") OR (name = "exit") THEN quit := TRUE; RETURN END;
IF (name = "help") OR (name = "--help") OR (name = "-h") THEN Help; RETURN END;
IF dot < 0 THEN
COPY(name, modulePrefix);
IF ~SharedModules.EnumerateCommands(name, ShowCommand) THEN Error END;
Out.Flush; RETURN
END;
name[dot] := 0X;
j := 0; start := dot + 1;
WHILE name[start] # 0X DO
IF j >= LEN(command) - 1 THEN
SharedModules.resMsg := "command name too long"; Error; RETURN
END;
command[j] := name[start]; INC(j); INC(start)
END;
command[j] := 0X;
WHILE (line[i] = " ") OR (line[i] = 09X) DO INC(i) END;
NEW(par); NEW(par.text); Texts.Open(par.text, ""); par.pos := 0;
Texts.OpenWriter(W);
WHILE line[i] # 0X DO Texts.Write(W, line[i]); INC(i) END;
Texts.Append(par.text, W.buf);
(* Set parameters before __init too, for modules that inspect them there. *)
previous := Oberon.Par; Oberon.Par := par;
m := SharedModules.ThisMod(name);
IF m # NIL THEN
proc := SharedModules.ThisCommand(m, command);
IF proc # NIL THEN proc ELSE Error END
ELSE Error
END;
Oberon.Par := previous;
Out.Flush;
Heap.GC(TRUE)
END Execute;
PROCEDURE Run;
VAR i, status: INTEGER; line: ARRAY 4096 OF CHAR;
arg: ARRAY 4096 OF CHAR; ok: BOOLEAN;
BEGIN
i := 1;
IF i < Modules.ArgCount THEN
Modules.GetArg(i, arg);
IF (arg[0] = "-") & (arg[1] = "P") THEN
IF arg[2] = 0X THEN
INC(i);
IF i >= Modules.ArgCount THEN
Out.String("vloksh: -P needs a module path"); Out.Ln; Platform.Exit(1)
END;
Modules.GetArg(i, SharedModules.path)
ELSE
status := 0;
WHILE arg[status + 2] # 0X DO
SharedModules.path[status] := arg[status + 2]; INC(status)
END;
SharedModules.path[status] := 0X
END;
INC(i)
END
END;
IF i < Modules.ArgCount THEN
line := "";
WHILE i < Modules.ArgCount DO
Modules.GetArg(i, arg);
ok := Append(arg, line);
IF i < Modules.ArgCount - 1 THEN ok := ok & Append(" ", line) END;
IF ~ok THEN Out.String("vloksh: command line too long"); Out.Ln; Platform.Exit(1) END;
INC(i)
END;
Execute(line)
ELSE
InitTerminal(Complete);
WHILE ~quit DO
status := SHORT(ReadLine(line));
IF status = 0 THEN quit := TRUE
ELSIF status < 0 THEN
Out.String("vloksh: input line too long"); Out.Ln; failed := TRUE
ELSE Execute(line)
END
END
END;
IF failed THEN Heap.FINALL; Platform.Exit(1) END
END Run;
BEGIN Run
END vloksh.

123
src/tools/vloksh/voc-shared.py Normal file
View file

@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""Build one VOC module per ELF shared object, with a single shared runtime."""
import argparse
import os
from pathlib import Path
import re
import shlex
import shutil
import subprocess
import sys
def installation(voc, model):
compiler = Path(shutil.which(voc) or voc).resolve()
prefix = compiler.parent.parent
roots = [prefix / "share/voc", prefix]
root = Path(os.environ["VOCROOT"]) if "VOCROOT" in os.environ else next(
(p for p in roots if (p / model / "include/SYSTEM.h").is_file()), roots[0]
)
library = f"libvoc-O{model}.so"
directories = [root / "lib", prefix / "lib64", prefix / "lib"]
libdir = Path(os.environ["VOCLIBDIR"]) if "VOCLIBDIR" in os.environ else next(
(p for p in directories if (p / library).is_file()), directories[0]
)
return root, libdir
def run(command, **kwargs):
print("+ " + shlex.join(str(part) for part in command), flush=True)
result = subprocess.run([str(part) for part in command], **kwargs)
if isinstance(result.stdout, str):
print(result.stdout, end="", flush=True)
result.check_returncode()
return result
def build(args):
root, libdir = installation(args.voc, args.model)
root = Path(args.root or root).resolve()
libdir = Path(args.libdir or libdir).resolve()
include = root / args.model / "include"
runtime = libdir / f"libvoc-O{args.model}.so"
if not (include / "SYSTEM.h").is_file() or not runtime.is_file():
raise ValueError("cannot find VOC installation; use --root and --libdir")
source = Path(args.source).resolve()
env = dict(os.environ, VOCROOT=str(root), VOCLIBDIR=str(libdir))
directories = [Path.cwd(), *map(Path, args.library_path), libdir]
directories = list(dict.fromkeys(p.resolve() for p in directories))
# -S translates only. Compile the generated C explicitly with PIC even if
# the installed compiler's default C command does not enable it.
result = run([args.voc, f"-O{args.model}", "-sFS", source], env=env,
stdout=subprocess.PIPE, text=True)
# Let VOC read ASCII or binary V4 source and report the MODULE name; the
# generated __REGMOD must agree. Never assume it matches the file name.
declared = re.search(r"\bCompiling ([A-Za-z][A-Za-z0-9_]*)\.", result.stdout)
if not declared:
raise ValueError("cannot determine VOC's generated module name")
name = declared[1]
text = Path(f"{name}.c").read_text()
registered = re.search(r'__REGMOD\("([A-Za-z][A-Za-z0-9_]*)",', text)
if not registered or registered[1] != name:
raise ValueError(f"{name} is not a non-main module")
commands = re.findall(r'__REGCMD\("([A-Za-z][A-Za-z0-9_]*)",', text)
if len(name) > 19 or any(len(command) > 23 for command in commands):
raise ValueError("VOC's runtime registry limits module names to 19 and commands to 23 characters")
symbols = subprocess.check_output(["nm", "-D", "--defined-only", str(runtime)], text=True)
runtime_symbols = {line.split()[-1] for line in symbols.splitlines() if line.split()}
imports = list(dict.fromkeys(re.findall(r"__MODULE_IMPORT\((\w+)\)", text)))
dependencies = []
for imported in imports:
library = f"libvoc-{imported}-O{args.model}.so"
if f"{imported}__init" in runtime_symbols:
# Never duplicate modules already provided by the shared runtime.
continue
if not any((directory / library).is_file() for directory in directories):
raise ValueError(f"missing shared dependency {library}; build {imported} first or use -L")
dependencies.append(f"-l:{library}")
shared = Path(f"{name}.shared.c")
shared.write_text(text + f'''\n/* Shared-module ABI shape, checked before calling __init. */
#include "Heap.h"
export const INT32 {name}__voc_abi[] = {{
1, sizeof(ADDRESS), sizeof(SHORTINT), sizeof(INTEGER), sizeof(LONGINT),
sizeof(SET), sizeof(Heap_ModuleDesc), sizeof(Heap_CmdDesc)
}};
''')
library = f"libvoc-{name}-O{args.model}.so"
cc = shlex.split(args.cc)
includes = [Path.cwd(), include, *map(Path, args.include)]
flags = shlex.split(os.environ.get("CFLAGS", ""))
ldflags = shlex.split(os.environ.get("LDFLAGS", ""))
extra_libs = shlex.split(os.environ.get("LDLIBS", ""))
run([*cc, *flags, "-fPIC", "-Wno-stringop-overflow", "-shared", shared,
*[f"-I{directory}" for directory in includes], *ldflags,
*[f"-L{directory}" for directory in directories],
"-Wl,-z,defs", f"-Wl,-soname,{library}", "-Wl,-rpath,$ORIGIN",
"-Wl,--no-as-needed", *dependencies, f"-lvoc-O{args.model}",
*extra_libs, "-o", library])
print(f"Built {library}; keep {name}.sym and {name}.h for development/completion.")
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("source", help="one non-main Oberon module")
parser.add_argument("--model", choices=("2", "C"), default="2")
parser.add_argument("--voc", default=os.environ.get("VOC", "voc"))
parser.add_argument("--cc", default=os.environ.get("CC", "cc"))
parser.add_argument("--root", help="VOC resource directory, e.g. /usr/share/voc")
parser.add_argument("--libdir", help="directory containing libvoc-O2.so")
parser.add_argument("-I", "--include", action="append", default=[])
parser.add_argument("-L", "--library-path", action="append", default=[])
try:
build(parser.parse_args())
except (ValueError, OSError, subprocess.CalledProcessError) as error:
parser.exit(1, f"voc-shared: {error}\n")
if __name__ == "__main__":
main()