begin to write developer's guide

3 files changed, 449 insertions, 0 deletions

diff --git a/doc/Makefile b/doc/Makefile
new file mode 100644
index 00000000..14202ea4
--- /dev/null
+++ b/doc/Makefile
@@ -0,0 +1,4 @@
+
+guide.html: guide.pandoc
+        pandoc --toc --css common.css $< -o $@
+
diff --git a/doc/common.css b/doc/common.css
new file mode 100644
index 00000000..ce60b17f
--- /dev/null
+++ b/doc/common.css
@@ -0,0 +1,207 @@
+
+/* slides */
+
+body.single_slide > * { display: none; visibility: hidden }
+body.single_slide .current { display: block; visibility: visible }
+body.single_slide div#TOC { display: none; visibility: hidden }
+body.single_slide div.handout { display: none; visibility: hidden }
+body.single_slide div#lang { display: block; visibility: visible }
+
+
+/* padding */
+
+body {
+  margin: auto;
+  padding: 1em 1em 1em 1em;
+  max-width: 50em; 
+  line-height: 130%;
+  background-color: white;
+  color: black;
+}
+
+h1 {
+  padding-top: 1.5em;
+}
+
+h2 {
+  padding-top: 1em;
+}
+
+h3 {
+  padding-top: 0.5em;
+}
+
+h1.title {
+  padding-top: 3em;
+  padding-bottom: 2em;
+  //text-align: center;
+  line-height: normal;
+  border-bottom: 0;
+  font-size: 200%;
+}
+
+hr {
+  height: 10px;
+}
+
+
+/* margin */
+
+div.indent, pre, table {
+  margin-left: 5%;
+  width: 95%;
+}
+
+div.indent, pre {
+  margin-top: 0.5em;
+  margin-bottom: 0.5em;
+}
+
+pre.normal {
+  margin: 0;
+  width: auto;
+}
+
+
+div.indent input {
+  margin-bottom: 0.5em;
+}
+
+input, textarea {
+  margin: 0;
+}
+
+
+/* border */
+
+h1, h2, h3 { 
+  border-bottom: 1px dotted black;
+}
+
+h1.cover, input, textarea, hr {
+  border-style: none;
+}
+
+
+/* font sizes */
+
+h1              { font-size: 130%; }
+h1.cover        { font-size: 200%; }
+h2              { font-size: 110%; }
+input, textarea { font-size: 100%; } /* monospace correction */
+
+
+/* font family, style, weight, text decoration */
+
+h1, h2, h3 {
+  font-weight: bold;
+}
+
+pre code, textarea, div.string {
+  font-family: monospace;
+  font-size: 130%;
+}
+
+pre.normal, pre.normal code, code, .error, input {
+  font-family: sans-serif;
+  font-size: 100%;
+}
+
+
+.wait {
+  font-style: italic;
+}
+
+a {
+  text-decoration: none;
+}
+
+
+
+div.serious {font-weight: bold;}
+div.info {color: gray;}
+
+
+/* colors other than highlighting */
+
+input           { background-color: #eeeeee; }
+textarea        { background-color: #ffffba; }
+hr              { background-color: #f5f0ee; }
+div.string span { background-color: yellow; }
+
+a               { color: #2a207a; }
+h1 a, h2 a, h3 a { color: black; }
+code            { color: red; }
+.wait           { color: gray; }
+.result         { color: green; }
+.type           { color: #0080ff; }
+.error          { color: #a030a0; }
+.comment        { color: gray; float: right; }
+
+
+/* misc */
+
+.result {
+  word-wrap: break-word;
+}
+
+p.caption {
+  display: none;
+}
+
+div#lang {
+  position:fixed;
+  z-index:1000;
+  right: 1%;
+  top: 1%;
+}
+
+div#info {
+  display: none;
+  position:fixed;
+  z-index:1000;
+  left: 40%;
+  width: 20%;
+  top: 40%;
+  height: 20%; 
+}
+
+.wait {
+  padding: 10px;
+  width: 8em;
+  border: 1px solid grey;
+}
+
+/*  search result colors  */
+
+code.search { color: black; }
+code.search b { color: gray; }
+
+.c0{background-color: #fcc;}
+.c1{background-color: #cfc;}
+.c2{background-color: #ccf;}
+.c3{background-color: #ffc;}
+.c4{background-color: #fcf;}
+.c5{background-color: #cff;}
+
+
+/*  highlighting css  */
+
+table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode, table.sourceCode 
+   { margin: 0; padding: 0; border: 0; vertical-align: baseline; border: none; }
+td.lineNumbers { border-right: 1px solid #AAAAAA; text-align: right; color: #AAAAAA; padding-right: 5px; padding-left: 5px; }
+td.sourceCode { padding-left: 5px; }
+pre.sourceCode span.kw { color: #007020; /* font-weight: bold; */ } 
+pre.sourceCode span.dt { color: #902000; }
+pre.sourceCode span.dv { color: #40a070; }
+pre.sourceCode span.bn { color: #40a070; }
+pre.sourceCode span.fl { color: #40a070; }
+pre.sourceCode span.ch { color: #4070a0; }
+pre.sourceCode span.st { color: #4070a0; }
+pre.sourceCode span.co { color: #60a0b0; /* font-style: italic; */ }
+pre.sourceCode span.ot { color: #007020; }
+pre.sourceCode span.al { color: red; /* font-weight: bold; */ }
+pre.sourceCode span.fu { color: #06287e; }
+pre.sourceCode span.re { }
+pre.sourceCode span.er { color: red; /* font-weight: bold; */ }
+
diff --git a/doc/guide.pandoc b/doc/guide.pandoc
new file mode 100644
index 00000000..b1636c13
--- /dev/null
+++ b/doc/guide.pandoc
@@ -0,0 +1,238 @@
+% LambdaCube 3D compiler developer's documentation
+
+The interface of the compiler
+============================
+
+Live coding system structure:
+
+           editor frame    WebGL frame
+               | ^             ^
+    source code| |errors       |
+               | | & infos     |pipeline description
+               v |             |
+             compiler----------/
+
+Try online at [http://lambdacube3d.com/editor.html](http://lambdacube3d.com/editor.html)
+
+
+Compiler structure
+==================
+
+Compiler tasks:
+
+-   find errors in source code
+-   eliminate abstractions
+    -   optimizations
+    -   reduction, partial evaluation
+    -   multistage compilation
+-   support incremental compilation
+    -   modules support
+-   code generation*
+-   support livecoding
+    -   highlight errors
+    -   inferred types in tooltips
+    -   completion
+    -   show desugared source (on a separate tab)
+    -   show optimizations (how?)
+
+Design decisions
+
+-   lexing, parsing, scope checking and desugaring is done in the same phase, sharing the same state
+
+Compilation pipline:
+
+     |
+     | source code
+     v
+    lexer + parser + scope checking + desugaring
+     |
+     | desugared source code
+     v
+    inference
+     |
+     | typed & reduced code
+     v
+    code generator
+     |
+     | pipeline description
+     v
+
+
+Modules support
+---------------
+
+Design decisions
+
+-   module header and body are parsed in two phases
+-   module status and availabe information are cached in a map
+-   modules can be loaded by filename or by module name
+-   builtin modules are always accessible by the last default path element
+
+The module header contains
+
+-   module name
+-   export list
+-   import list
+
+The driver state contains
+
+-   the paths (in a ReaderT monad transformer)
+-   map of cached modules: status + available information (in a StateT monad transformer)
+-   fatal error (in an ExceptT monad transformer)
+
+Possible module status
+
+status                                  available information
+--------------------------------------- ---------------------------------
+file cannot be found                    filename or module name (the paths are available in the driver state) 
+header cannot be parsed                 module name, filename, source code
+error with loading imported modules     module name, filename, source code, import and export lists (the successfully loaded modules and the error messages are available in the driver state)
+body cannot be parsed                   module name, filename, source code, import and export lists, parse error
+body cannot be elaborated               module name, filename, source code, import and export lists, desugared body, error message
+loaded                                  module name, filename, source code, import and export lists, desugared body, elaborated definitions
+--------------------------------------- ---------------------------------
+
+
+Lexer
+=====
+
+[Source code of the lexer](https://github.com/lambdacube3d/lambdacube-compiler/blob/master/src/LambdaCube/Compiler/Lexer.hs)
+
+Tasks
+
+-   pruning white space and comments
+-   recognise literals, reserved words and identifiers
+-   recognise indentation (case & let expressions, where blocks, ...)
+-   calculate ranges (start + end position) of tokens
+
+Design decisions
+
+-   use the `megaparsec` parser combinator library (a better parsec)
+-   Lexing and parsing is done in the same phase, sharing the same state  
+    how it works: Whitespace is consumed immediately after each "lexeme".
+-   do custom indentation sensitive parsing
+-   use custom lexer for literals
+
+Indentation sensitive parsing
+-----------------------------
+
+State used for indentation sensitive parsing:
+
+-   current position (line + column)
+-   current active indentation, in a reader monad
+
+Range calculation
+-----------------
+
+State used for calculating ranges:
+
+-   position at the end of the last lexeme
+
+
+Parsing & desugaring
+====================
+
+[Source code of the parser and desugarer](https://github.com/lambdacube3d/lambdacube-compiler/blob/master/src/LambdaCube/Compiler/Parser.hs)
+
+Parser tasks
+
+-   recognise syntactic structures
+-   compute ranges of syntactic structures
+-   handle namespaces: expression + type namespace
+
+Parser design decisions
+
+
+
+Scope checker tasks
+
+-   disambiguate names
+
+Scope checker design decisions
+
+-   scope checking is done by calculating De Bruijn indices
+
+Desugarer tasks
+
+-   handle operator precendences
+-   compile patterns and guards into function calls
+-   desugar local functions
+-   desugar node definitions
+-   desugar case expressions
+-   desugar let expressions
+-   desugar if expressions
+-   desugar list comprehensions
+-   add missing foralls in type annotations
+-   insert wildcards
+-   desugar type synonym declarations
+-   desugar type family declarations
+-   desugar type classes and instances
+
+
+Desugared source code
+---------------------
+
+~~~~~ {.haskell}
+data SExp
+    = SGlobal SIName
+    | SVar SIName Int
+    | SBind SI Binder SIName SExp SExp
+    | SApp SI Visibility SExp SExp
+    | SLet SI SIName SExp SExp
+    | SLit SI Lit
+
+data Binder
+    = BPi  Visibility
+    | BLam Visibility
+    | BMeta      -- a metavariable is like a floating hidden lambda
+
+data Stmt
+    = Let SIName (Maybe SExp) SExp
+    | Data SIName               -- name
+           [(Visibility, SExp)] -- parameters
+           SExp                 -- type
+           [(SIName, SExp)]     -- constructor names and types
+    | PrecDef SIName Fixity
+~~~~~
+
+
+Type inference
+==============
+
+Design decisions
+
+-   use a dependently typed core language
+-   use metavariables
+-   support hidden arguments
+-   use a rich environment
+-   use a zipper to move in the environment
+-   do type inference and reduction in one step
+-   use labels to remember the beginning of right hand side instead of arity information
+-   separate types and expressions
+-   erease types when possible
+
+Tasks
+
+-   find semantic errors
+-   insert metavariables 
+-   eliminate all metavariables (supersede type inference)
+-   reduce expressions (beta-reduction)
+-   avoid infinite recursion during type checking (termination checking) - this is not a priority at the moment
+
+
+Code generation
+===============
+
+Tasks
+
+-   generate pipeline
+
+Design decisions
+
+-   the type system should prevent all misuse, no error handling is needed (*)
+-   the result of inference should be very easy to deal with w.r. reduction
+-   use a custom data type which encapsulates expessions with types and De Bruijn environment
+-   use quoting to do code generation in the libraries (TODO)
+
+
+