Indentations break the feed.

[SquirrelJME.git] / assets / developer-notes / stephanie-gawroriski / 2016 / 07 / 06.mkd

# 2016/07/06

## 07:39

Today is my birthday. Yay.

## 07:46

The immutable configuration should match getters with the mutable one for
simplicity.

## 07:53

The simulations can act as a group. Instead of having individual simulations
there would just be a system simulation. If a system instance does not exist
then it will be created. When a program is requested to run on a given system
then it will be passed to it. So this way, multiple programs can run on any
given system but they would essentially be very standalone in their own
execution.

## 07:58

One thing to consider however is that the simulation could launch programs
that exist in the simulated filesystem or from the real filesystem. I should
likely just support it only from the simulated filesystem. With my current plan
there would be a system filesystem and a home one. However, some operating
systems may require a combined user based filesystem on top of a system based
one. Also, some operating systems do not have a filesystem at all. For example
Palm OS has a database filesystem while expansion cards act as traditional
filesystems. The Nintendo 64 has only ROM and block based storage (although if
a 64drive is used, the main cart can contain a filesystem). So having an
external filesystem support could be slightly odd. However, for a Linux
based system I should be able to run `fossil`. This way I can have a bootstrap
build environment (assuming I can also get the Java compiler and interpreter
also simulated). I will bump into a chicken and the egg problem however. Right
now I have no class library and no graphical interfaces. For the simulator to
work better, I need a graphical interface since for some systems such as
Palm OS, everything uses graphics. However for testing, I can have a virtual
serial port that the user can use which the operating system can output to
perhaps for a given process. So on Linux, the test program would pipe its
output to a virtual serial device, on Windows it would output via a COM1 or
such. On the Nintendo 64's 64drive, it would output to its USB connection.
This way I can have the test system output while being simulated be parseable
for errors and such. If a test fails on a real system then that is incorrect,
while if a test fails in just the simulator the simulator is incorrect.

## 11:10

Well, Google never wished me a Happy Birthday, but Microsoft did.

## 11:28

When it comes to the new `JITLogicAcceptor`, I should only create it when it
actually is needed when a method needs to get its logic handled. Before I was
definitely going to clutter `JITOutput` with logical operations, but that
would be nasty. However with the `JITLogicAcceptor` I can essentially have a
multiplexer which can output to multiple logical acceptors with the same
information. Then this way, I can handle caching and runtime JIT at the same
time. Thinking about other things, I wonder if in the game of life it would
be possible to create standalone simulations (not using the super cells that
run the same simulation) but a way where I can create a glider and a few other
objects at specific positions within the world. If that could be done then a
JIT would be possible in the game of life.

## 11:44

Actually what I need is some kind of cache creator callback that could be
specified in the output. So basically, there would be a (`Auto`)`Closeable`
class writer of sorts. `JITOutput` would return this. It would essentially be
a `beginClass` with the namespace and the name of the class. If the output is
to be written to a cache then that can be handled by `JITOutput`. Otherwise
if it is a binary that could also be handled too. So then `JITOutput` gets a
producing class. Finishing the class can be handled by whatever implements the
interface accordingly.

## 12:05

I am going to have to have a creator interface for cached forms. The config
will be given an interface which would create `OutputStream`s for writing to
the disk. One thing that I could do on Linux, is create actual object files
and then create a linker similar to `gcc` which can combine all of the object
files together with a set entry point which performs the work. If I match the
native format I could manually link all the classes together myself and set
an entry point so to speak. Although this is not required at all. It would
be interesting for hybrid programs however.

## 12:10

However for `JITCacheCreator` that can sort of be hidden in a way by the
`JITOutput`. However, `JITCacheCreator` would be sent to the configuration. If
it is sent then that means `JITOutput` should output to a cache. I just need
the creator because the `JITOutput` has no means of determining exactly where
to place the `OutputStream` and if it even wants to be stored on the disk.

## 13:21

So I need to copy over the class flag code and use that in the decoder since
that will be very important.

## 13:28

What I need though is a name. The CI code would basically be imported as-is
for the most part, except with some slight changes. Well, perhaps not changed
at all. The CI family will basically be going away with the new JIT since there
is no need to really keep it around. A future Java compiler could just write
the class data directly anyway.

## 13:40

JIT would not really work out well. `Class` would not work.

## 19:55

So basically what I need to determine now is how `beginClass` is to work. I
have thought of it before. Currently I just have caching used. However with a
multiplexing output for classes I would not need to worry about duplicating or
having lots of branches. However what would essentially happen is that when
class code is generated, the work would essentially be performed twice. One
for being cached and the other if directly executed. Since that would be a bit
of a waste I would suppose that there would be branched handlers for output.
Generally when directly executed just some basic class details are needed.
However the cached form can also be directly executed and placed in memory and
initialized. So what I really could just do is have just a cached form writer
for now. Also, with the generic operating system handling and such, it is
very possible to compile for multiple systems at the same time although
linking may be complicated by that fact. One consideration with the simulator
is that I could directly feed it blobs. However it should be able to run actual
OS code because that would need to be tested more than just the native machine
code that is being executed. So one thing to consider at this point, for
systems such as Linux, is if as noted before blobs should match the standard
compilable object format. Then the blobs could be linked into a static binary
and initialized with basic C code for now. However in my case, it would
essentially just be a blob wrapped in an ELF file.

## 20:05

One thing to determine is if the ELF symbol table is unique to a given OS or
if it is standard for the format. I believe symbol tables are non-standard.
Appears that the symbol table is standard. One thing though is, if I can get
away without needing a symbol table at all. However, going without one would
be a bit ugly. However, at least with ELFs and symbol tables though I can kind
of easily generate a binary even without linking. When it comes to system
specific virtual machine calls (`unsafe`) then those calls can just be bound
to special symbol names rather than their normally intended method. So
instead of calling a mangled name for the method, it is instead just a call
to say `__squirreljme_foo`. However likely it would be best to have an
alternative static OS implementation somewhere where methods are bound to
instead. So basically the JIT would turn `unsafe` calls into
`foo.bar.ImplementingClass` with compatible signatures. Then this implementing
class would have the magical code such as direct assembly access and such.
This way assembly can stay out of the main libraries and exist only in a small
portion of the code. One thing I need though is a kind of ELF output libary
and a common name mangling scheme which would work in C for the most part.
Technically I could just store everything in an ELF and then link the end
result anyway. Alternatively, I can always have a blob to ELF converter for
that. If all blobs have their size information, I could essentially just
concatenate every single blob, setup a base linker script, and create a small
wrapper program which initializes data from the blob.

## 20:14

So I would say then that blobs would be best which include their detailed
information. With a backwards linking chain of sizes, pointers to classes
could be created. Finding a class definition would be a bit slow, but there
could be a generic large table at the end which points to every blob that
exists similar to what I have thought up of before.

## 20:32

So I will need a good way to have an output class format. However, since
testing things will be a bit unknown while I have code generation, I should
switch to working on an interpretive handling of classes. I would like the
runtime classes to be purely interpreted (the JVM classes) in a kind of
bootstrap state, however that would be very complex. However, an alternative
would be to kind of target an abstract machine of sorts that can easily handle
the logic. I could for example output to plain C code. This would in a way
test how standalone the code is. While an intrepreter in Java will essentially
be a `JVM` instance. However since `JVM` is essentially the kernel, it would
need a way to setup sub-objects and processes. However the major issue is that
the interpreter would be completely unable to call into the normal code and
access kernel space objects as if they were user-space since the objects
would be in completely different domains. However it could still work if the
interpreter based machine is simple enough. The only initial problem is that
the run-time kind of will expect all of the classes to exist and be built
into the executable. Any dependencies would also be included too. There would
just be enough classes to have a JIT and to run the launcher. I just hope that
including all of them does not complicate and bloat it for some targets that
are extremely limited in size. However, all my JARs are essentially just
476KiB right now. For a wide range of systems that is enough. The actual size
is 386KiB however (due to sector round up). Since the blobs would essentially
have no debugging information, that would be a bit lighter. When running,
each class in its compilation state will need to access a lookup table which
refers to other classes and methods. One thing that would be very efficient
would there to be a completely merged table. Otherwise there are going to
be many instances to `java.lang.Object` for example. So I will say that when
it comes to `JITOutput` that it always merge all possible classes and place
it into a single index. Since `JITOutput` is initialized by something, the act
of calling `close` on the output could finish off anything it needs to perform.
Then this way to the JIT, all classes in a single namespace can be compiled at
once and essentially a JITted JAR would be loaded 100% at-run time or at least
memory mapped. Entire programs would be precompiled and resources included
also. This would also probably be the best result also. Since this is the
more efficient route and has much constant deduplication I must take it.
Wasting lots of space on the same copies of a string would be a waste.

## 20:47

So considering this, I should probably split the class and global tables to
namespaces. This way, single blobs get their own namespace. Doing it this way
would mean that `JITOutput` would not need to be closed. Any and all classes
in the same namespace would be compiled together and use the same constant
data. At least with it this way, when programs are running as user-space they
do not need to have all the namespaces the kernel uses visible to the process.
Only the `JVM` needs access to all of the default namespace areas. So
basically then a `JITNamespaceWriter` is created which then has a `close` and
a `beginClass` without a namespace. Since everything within a single JAR is
visible to each other this namespace split makes sense. It would also make it
easier for the launcher showing default namespaces JARs.

## 20:55

Then this also means that `JIT` gets `JITNamespaceWriter` to target namespaces.

## 21:02

Then for the builder, since the `JVM` and the builder will generally take an
input JAR and recompile all of it, I can instead have a generic class in `JIT`
which has a basic common interface to all of the JAR handling so that
namespace recompilation is handled using the same code. Because as it stands
right now, I would need to duplicate that work in the `JVM` and any changes to
it may change and potentially break apart.