On Jul 3, 3:22Â am, DavidM
nowhere.com> wrote:> Hi all,
>
> I've been working on an 'extension library' loader for FICL, which I plan
> to port to other forths.
>
> This loader is a bit like gForth's 'fflib' - it allows one to load shared
> C libraries (.so files on nix, .dll files on windows), extract C
> functions, and invoke them with correct arguments - all from within Forth
> code.
>
> But on considering the task of porting this loader to other forths, an
> issue came to mind...
>
> Many or most popular forths are 'image-based'. In other words, the forth
> words are compiled into an image file which is loaded on system startup,
> and can be saved and restarted as more words are defined.
>
> Here's the problem - if any dynamic library loading is performed outside
> of word definitions - in other words, if a forth source file performs
> library loading in interpret mode (as opposed to runtime, when words are
> invoked), then there will be forth words in the system containing live
> pointers to C entities - library handles, C function pointers, pointers
> to objects returned by C functions etc.
>
> Such words will work perfectly in the current session, but I can easily
> see that if one saves the 'image', then reloads it into a new session,
> then these words can easily end up with dangling pointers - pointing off
> into the boondocks - and the words will throw one into a sea of segfaults.
>
> Has anyone put any thought into this problem?
>
> So far, the only easy solution I can see is to abstain strictly from
> creating an image containing any references to dynamic C libraries or
> their contents or products. Then at run-time, one needs to load in afresh
> any required sources which interact with dynamic libraries.
>
> All this said, one thing I really like about FICL is that it avoids
> images completely. Instead it has a system of compiling a bunch of Forth
> sources into a single compact compressed string, embedded into a C source
> file which gets compiled into the system. At startup time, FICL can then
> do a high-speed decompression of this source string, so that all
> instructions contained in the source forth files get executed. While this
> sounds expensive, FICL starts up in 12ms, which compares favourably with
> other forths, especially gForth which can take upwards of 25ms.
>
> Any thoughts on this?
>
> Cheers
> Dave
Yes, Win32Forth has "fixed" this problem (although the image is an
executable, and no extra loader outside of the OS is required).
External routines in DLLs appear in a special dictionary as normal
Forth words. They look like DEFERed words, although they have extra
information associated with them, such as the name of the external
entry point. The XT for this word is set to a "resolving" word
initially. The first invocation of the word executes the resolving
word, which (1) finds the external address from the name (2) fixes up
the original word's XT to point to the external address (3) tail
executes it. On subsequent invocations, the now modified word just
calls straight into the external routine.
Step 1 in the "resolving" word is quite complex, as we need to find
both the library and the entry point. This requires a pair of calls
into the OS (LoadLibrary and GetProcAddress). The resolving word
requires to know these in advance, so a feature of the Windows loader
is used to provide them at load time, just for these two external
addresses.
Because all resolution apart from the two mentioned above is done at
run time, it's possible to refer to external libraries and routines
that don't exist on a given system. But the advantages of this dynamic
technique far outweigh the disadvantages.
When saving an image (a full executable in the case of Win32Forth),
the XTs are reset back to the original resolving word. Because the
external words are in a special dictionary and easily findable, it's
trivial to do so.
If you need a more detailed description, please ask.
--
Regards
Alex McDonald
