added some "immediate-noop" words, removed some "$if"

[urforth.git] / URFORTH.me

this is direct-threaded self-hosting x86 32-bit GNU/Linux Forth system.

UrForth level 1 doesn't need anything except UrForth itself to build the
system from the sources. there is a prebuilt binary in this repository
which can be used to bootstrap the system (with 'c.sh' shell script).

some notes about standards-compliance: i don't fuckin' care. pre-ANS
standards are too restrictive, and ANS standard is fubared with idiotic
"portability", and inability to get rid of obsolete concepts. this goes
up to 2012 (and then); just look at `FIND`! and fuck it, i don't expect
anybody (except me, of course) to use UrForth anyway.

also, about ANS "portability" crap: i strongly believe that default
architecture should be 32-bit 2-complement one, without any forced data
align rules. for non-conformant systems, an implementation should emulate
the abovementioned arch. this represents most architectures out there,
and if your arch is different, your Forth system should take care of that,
running standards-compliant code without any changes. and library authors
can avoid error-prone jumping through the hoops.

if you think that speed is more important than compatibility, there is
always an option to ignore any standards.

also, i'm not planning to create 64-bit versions of UrForth. "64-bitness"
is another thing i consider useless. and PIC code will not be supported.


more ANS idiocity: 2! and 2@ are storing 64-bit numbers as
big-dword-endian. i introduced "2!LE" and "2@LE", and left "standard"
words intact.

also, "cell" was another idiotic word choice. does "c@" operate on cells,
or on chars? i wonder if anybody there had even one working *brain* *cell*.
(got it? got a joke? it's a great joke!)


i added benchmark from BigForth. so far, UrForth is ~2.2 (2.1 with
debugger) times slower than BigForth on it. of course, benchmarks only
shows how good the system can pass a benchmark, but stil... it's not that
bad, considering that BigForth is subroutine-threaded (i.e. generates
native code), and somewhat optimised. also, it is only ~1.12 times slower
that unoptimised SPF4.


some advanced UrForth features, in no particular order:

* vocabularies has hastables for word names (256 bytes per vocabulary).
  this makes searches ~50-100 times faster:
    FORTH -- 799 words, 64 of 64 buckets used, 4 min items, 21 max items, average: 12 words per bucket
  considering that each word first checked for valid hash and length,
  the searcher usually does only one full string comparison.
  in other words: vocabulary searches are lightning fast.
  (ok, its not that advanced, because most Forth systems out there does
  that, but hey, it's my book, written by my rules!)

* number prefixes:
    $,#,0x,&h -- hex number (note that 2012 standard wants "#" for decimal)
    %,0b,&b -- binary number
    0o,&o -- octal number
    0d,&d -- decimal number

* number postfixes:
    nnnH -- hex
    nnnO -- octal
    nnnB -- binary (only for BASE<12)

* underscores in numbers are ignored:
    0x8000_00_00 is a valid number

* extended word search:
    you can use "a:b" to find word "b" in vocabulary "a".
    of course, "a:b:c" and such are allowed too.
    note that "colon access" will ignore "hidden" attribute. this is not a bug.

* UrForth has fully working BREAK and CONTINUE, they can be used in BEGIN and DO...LOOP.
    also, they know about CASE, so you can use BREAK/CONTINUE in OF/OTHERWISE clauses.

* BEGIN loops can contain arbitrary number of WHILE/NOT-WHILE parts,
  and they can be terminated with UNTIL/NOT-UNTIL even if WHILE is present.
  AGAIN is allowed too, for any kind of BEGIN loop.

* there is IFNOT in addition to IF. also, there are +IF and -IF, to check for
  positive and negative numbers respectively. zero is neither negative, nor
  positive. to include zero, use "+0IF" and "-0IF".

* the above positive/negative checks can be used with loops too ("+WHILE", and such).

* segfault handler will show you stack dump and backtrace.

* vocabularies supports public and hidden words. it is possible to create
  "nested" vocabulary, which will see all parent's words (including hidden ones).

* hidden words in "current" vocabulary are always visible. to access hidden words in
  other vocabularies, use "vocname:wordname" syntax.

* multiline comments: normal (* ... *), and nested (( ... ))
  nested multiline comment allows other nested comments

* x86 assembler with defered plug-in interfaces for memory r/w and label manager
  (can be used to create metacompilers, or even standalone assemblers).
  it is using normal intel syntax, not yoda-style. also, you don't need to
  separate operands with spaces, because assembler does its own input stream
  parsing.

* there are `[:` and `;]` to create cblocks. this feature can be used like this:
    : foreach-do ( cfa -- )  10 0 do i over execute loop drop ;
    : a  [: . cr ;] foreach-do ;
  internally, it compiles header-less word, and leaves its CFA on the stack.

* cblocks can be used in interpreter too, i.e. you can type this at REPL:
    [: 10 0 do i . cr loop ;] execute
  and it will work. and even this will work:
    [: ." hey!" cr [: 65 emit cr ;] execute ;] execute
  note that you cannot assign such cblocks to DEFERed words, because they
  will not live long enough.
  also, this feature is not really planned, it is just a side-effect of cblocks
  implementation. do not rely on it.

* internally, there is DP-TEMP variable. when it is 0, the normal DP is used
  for HERE. but when it is non-zero, all HERE-based words will use it instead.
  this is used to make cblocks working in interpreter, for example. no words
  except "HERE" and "N-ALLOT" are accessing "DP" directly.

* "OVERRIDE" word can be used to override Forth words with other Forth words.
  it works like this:
    : newdot  ( n old-xtoken )
      check-some-condition if
        OVERRIDE-EXECUTE
      else
        2drop
      endif
    ;
    OVERRIDE . newdot
   or
    ' . ' newdot (OVERRIDE)

  note that overriden word can be called as usual, and override forces even
  previously compiled words to call `newdot`. also note that `newdot` cannot
  be called as normal forth word anymore (no checks are made, it will just
  make your system unusable due to UB).

  currently, there is no way to "unoverride" the word. it may be added later.

  also, remember that "old-xtoken" cannot be called with "EXECUTE". but you
  can freely pass "old-xtoken" around and call it with "OVERRIDE-EXECUTE" at
  any moment, and at any place.

  this can be used to create things like metacompilers, for example, without
  duplicating all compiler word definitions again.

  if you will override already overriden word, old override will be replaced
  with the new one (i.e. the overrides are not chained).

  you can chain overrides like this, if you want to:
    0 value (prev-ovr)  (hidden)
    : ovr-new  ( ... xtoken -- )
      (prev-ovr) ?drop override-execute
    ;
    get-override . to (prev-ovr)
    override . ovr-new

  note that this feature is highly experimental, and may change/disappear.

* the kernel includes "scattered colon" feature (invented by M. Gassanenko).
  most of initialisation and other chained things are implemented with it.
  also, there are interpreter and "to" scattered colon hooks.
  see "samples/scolon.f" for example code.
  scattered colons are great, you *WILL* miss them in other Forth systems! ;-)

* "REPLACE" word can be used to perform system-wide word replacement:
    replace oldword newword
  WARNING! you can replace any word with any other word (including constants,
  code words, and so on, on both sides; the desired effect is up to you). i.e.
  this will work:
    69 constant fuck
    : hell 666 ;
    replace fuck hell
    fuck 666 = .  ( true )
  note that aliases may be optimised to the original words, so replacing an alias
  may not work for compiled code.

* you can set dstack/rstack value with POKE/RPOKE. this is complementary to
  PICK/RPICK (and i aliased those to PEEK/RPEEK).
  usage is: `value index POKE`, where `index` is the same as in PICK.

* there is FOR ... ENDFOR that accepts limit, and iterates [0..limit). it is
  safe to use with negative or zero limit -- the loop will be skipped in this case.
  i.e. `3 FOR I . ENDFOR` will print "0 1 2", and `-1 FOR I . ENDFOR` will print
  nothing.

* interpreter extension mechanics:
  there are two scattered colon words:
    (INTERPRET-WFIND)      ( addr count -- cfa 1 // cfa -1 // addr count false )
    (INTERPRET-NOT-FOUND)  ( addr count -- true // addr count false )

  "(interpret-wfind)" will be called to find a word.
  return `1` if cfa is immediate, `-1` if it is a normal word.
  if your chain handler succeeded, you MUST EXIT from it!
  you can leave various data on the stack if your handler returned immediate word.

  "(interpret-not-found)" will be called when a word cannot be found, and cannot
  be converted to a number.
  if your chain handler succeeded, you MUST EXIT from it!



some notable differences from ANS:

* number parser does not understand decimal dot, nor floating exponent; i.e. you cannot
  directly input double numbers or floats. this is because i don't see much use in
  that, and you can write your own prefix words if you need to (as fp did with "f#").
* TRUE is 1, not -1; there are LOGAND and LOGOR; NOT is logical, bitwise not is BITNOT.
* TIB is still there, no ANS SOURCE and such.
* TIB size is in variable #TIB; there's no need to finish TIB with any special byte. yet
  the parser will stop if it will see zero byte in TIB.
* current tib line is in variable TIB-LINE# (if it is 0, no line counting and debug info).
* do not read TIB manually, use "tib-peekch" and "tib-getch".
* DOES> words use extra bytes after PFA.
* there is [COMPILE], and no CHAR or ASCII (use [CHAR] instead).
* [CHAR] will error on any string that is not one-char.
* use WFIND ( addr count -- cfa -1 // cfa 1 // false ) instead of idiotic FIND.
* COUNT expects cell-counted string; for byte-counted, use BCOUNT.
* WORD returns cell-counted string (at HERE) (but don't use it, use PARSE and PARSE-NAME).
* there is N-ALLOT ( size -- start-addr ) low-level word, which is used by ALLOT and
  others. it returns starting address of the allocated dictionary memory (which can
  be in transient DP-TEMP).
* i don't fuckin' know what "address unit" is, and why it is necessary. so MOVE always
  works with bytes.
* FIG-style "CFA->NFA" and such are still there.
* S" always unescapes string, because i see no reason to not do it. also, terminating
  zero byte is always there (but it is not included in count).
  S" will use byte-counted string if it can (with a different LIT word).
* ." cannot print strings longer than 255 chars ('cmon!).
* there is FIG-style VAR and DEFERED is FIG-style (VARIABLE and DEFER are ANS)
* PAD is not relative to HERE, it resides in separate memory area.
* floating point words are supported only partially, via FPU+FPU stack; printing and
  parsing of floating point numbers are very inexact, and cannot be used for round-trips.
  this is something i'm planning to fix eventually (prolly by porting Ryu and plan9 parser).
  note: 32-bit floats seems to round-trip under the default FPU mode, but it's still better
  to not rely on that.
* do not even try that ANS-allowed idiocity like "BEGIN ... WHILE ... UNTIL ... THEN".
  this is hard to read, and plainly stupid. in UrForth, you can have as much "WHILE"
  (or "NOT-WHILE") secions as you want to, and you can finish "BEGIN" with any of
  "REPEAT", "AGAIN", "UNTIL". there is no need for extra "THEN" (and the compiler will
  complain if you'll try to do such thing). also, there are "BREAK" and "CONTINUE" words.
* `ABORT"` is unconditional abort. use `?ABORT"` instead -- this is more logical, and is
  in line with `?ERROR`. "use-lib: ans" will change that.
* ALIAS will copy source word attributes ("hidden" and "immediate"). not copying attributes
  is another standard idiocity, and a source of bugs. i've never ever encountered a case
  where i don't want an alias to not be immediate when a source word is immediate, for example.
  "use-lib: ans" will change that.



FAQ ;-)

Q: i am calling functions from .so, and they are segfaulting at random!
it doesn't happen with C code, UrForth has a bug there!

A: nope. what is happening is that your system is broken, and doesn't
follow ABI. 32-bit ABI doesn't require the stack to be aligned in any
particluar way (except being dword-aligned), but modern GCC not only
aligns the stack at 16 bytes, but generates code that expects the stack
to be always aligned like that. it is a violation of ABI, and a bug in
GCC. rebuild your system with "-mstackrealign" GCC flag to fix it.

Q: but everybody else is happy to do what GCC du..evelopers command them
to do! why can't you simply add stack aligning code to UrForth?!

A: because i see no reason to workaround GCC bugs in my code. the longer
we will tolerate GCC ABI breakage, the longer it will last.

Q: what the fuck with your assembler code? what is "ld", "jr", "jp",
"cp", and other shit?

A: i am sorry. i used to write for Z80, and i am too lazy to switch
mnemonics. conventional x86 mnemonics are supported too, so you can use
whichever you like. consult Z80 manual to understand the shit i wrote, if
you really need to.


have fun, and happy hacking
Ketmar Dark // Invisible Vector