Octaspire Dern Manual

Table of Contents

About
Building the amalgamated source release
Hello world
Values
Single and multiline comments, making script files executable
Binding names to values with define
Iteration
Comparing and changing values, predicates
Removing the last value from a vector
Branching and selection
Selecting values from collections
Accessing and manipulating command line arguments and environment variables
Formatted and regular printing
Formatted string creation
Functions with variable number of arguments
Environments
Getting help with howto
Returning from functions early
Evaluating values
Input and output ports
Converting between types
Searching and indexing
Loading libraries with require
Using custom library loader with require
Embedding in C programs
Tool support
Using the development repository

1. About

Octaspire Dern is a platform independent programming language in standard C99. It is a dialect of Lisp with influences from Scheme, Emacs Lisp and C. It runs in Amiga, Haiku, Plan9, Unix, Windows and almost anything between.

Octaspire Dern from octaspire on Vimeo.

Dern is a dynamically typed language with lexical scoping and has mark and sweep garbage collector. Functions in Dern are first class values. Somewhat unique property of Dern is that it has first class semantic version numbers built-in in the language. Unlike Lisp, that uses linked lists as its major data structure, Dern uses vectors.

Dern is written in standard C99 and depends only on a C compiler, standard library and octaspire-core library. Dern should compile cleanly without any warnings using -Wall -Wextra on any compiler supporting a subset of C99. Currently it is tested with gcc, clang, Tiny C Compiler (tcc), Portable C compiler (pcc) and Plan9's 8c.

Dern supports imperative and functional programming. It has atomic types integer, real, (utf-8) string, (utf-8) character, (utf-8) symbol, boolean, nil, (input/output) port, hash-map, list, queue, environment, first class SemVer 2.0.0, first class function, special and builtin. As non-atomic type it has vector. So, where in scheme '(1 2 3) is a list, in Dern it is a vector.

Every variable and function definition in Dern must be documented by a documentation string. Dern also makes sure that every formal function parameter is documented in the documentation of a function definition. The documentation can be accessed with the doc-function. howto is another useful function. You can give it examples of function arguments and expected result and it will tell you what functions you can call to get the expected result from the given arguments.

Dern can also be extended with functions written in C; C-functions can be registered as builtin or special. Arguments to builtins are evaluated normally, but to specials are not, so specials can be used to implement special forms like if. When defining your own builtins and specials, you can tell Dern whether the function can be tested by howto or not. Pure functions - functions without side effects - can always be tested, but if the function writes a file, or sends data through a socket, it probably should not be tested.

Dern has also a library system, that allows one to load libraries using builtin require. On most systems Dern also supports loading of binary libraries (.so, .dll or .dylib files). For the end user, it doesn't matter whether the library she loaded was binary or written in Dern; it can be used exactly the same way.

By using the amalgamated version of Dern, you need only one file. This same single source file, octaspire-dern-amalgamated.c, can be compiled into (1) stand-alone unit test runner, (2) interactive Dern-REPL and (3) used as a single header file library in programs that want to embed the Dern-language. The file contains also the source for the Octaspire Core library. The amalgamated source release is the recommended way of using Dern; there is only one file to keep track of, it is easy to use with any build system, and the resulting machine code even runs faster.

Dern is portable and is tested and known to run in Linux, FreeBSD, OpenBSD, NetBSD, OpenIndiana, DragonFly BSD, MidnightBSD, MINIX 3, Haiku, Windows, ReactOS, macOS, Termux, Plan9, AROS and Amiga. The how-to-build-directory of the amalgamated source release contains a build script for all tested platforms.

Please note, that this is the first language I have ever designed or implemented. There are some features that are not yet implemented and probably bugs that are not yet fixed. The interpreter is currently also just a tree walker, and not a faster bytecode vm.

Dern uses Semantic Versioning 2.0.0 version numbering scheme. As long as the MAJOR version number is zero anything can change at any time, even in backwards incompatible manner.

2. Building the amalgamated source release

The amalgamated source release is the recommended way of using Dern, if you don't need to modify Dern itself. To use the amalgamated release, you will need only a C compiler and C standard library supporting a subset of C99.

2.1 Linux, FreeBSD, OpenBSD, NetBSD, OpenIndiana, DragonFly BSD, MidnightBSD, MINIX 3, Haiku, macOS, Termux, AROS

curl -O octaspire.com/dern.tar.bz2
curl -O https://octaspire.io/dern.sha512
sha512sum -c dern.sha512
tar jxf dern.tar.bz2
cd dern
sh how-to-build/YOUR_PLATFORM_NAME_HERE.XX

Replace YOUR_PLATFORM_NAME_HERE.XX with FreeBSD.sh, NetBSD.sh, OpenBSD.sh, OpenIndiana.sh, DragonFlyBSD.sh, MidnightBSD.sh, linux.sh, minix3.sh, haiku.sh, macOS.sh, termux.sh or AROS.sh. More scripts for different platforms will be added later.

2.2 Plan9

hget http://octaspire.com/dern.tar.bz2 > dern.tar.bz2
tar xf dern.tar.bz2
cd dern/*
rc how-to-build/Plan9.sh

2.3 Plan9 on ARM architecture

hget http://octaspire.com/dern.tar.bz2 > dern.tar.bz2
tar xf dern.tar.bz2
cd dern/*
rc how-to-build/Plan9-arm.sh

2.4 Windows using MinGW and Git

Download and install MinGW into a directory, for example into C:\MinGW. Install the GCC compiler. Add MinGW\bin into the PATH (for example, if you installed into C:\MinGW, add C:\MinGW\bin into the PATH).

Download and install Git for Windows.

Start Git Bash and run the following commands:

git clone https://github.com/octaspire/dern.git
cd dern/release
how-to-build/windows.sh

Start Windows Command Prompt and change directory to the same release directory, as above. Run examples and programs in the Command Prompt window (NOT in the Git Bash window).

2.5 ReactOS

Use ReactOS Applications Manager to install CodeBlocks with GCC compiler, a web browser and 7-Zip. Remember the path where you installed CodeBlocks. Add CodeBlocks\MinGW\bin and CodeBlocks\MinGW into the PATH. Use a web browser to download Dern release. Extract the file two times using 7-Zip, first into dern.tar and then into dern. Go to the path where you extracted the archive and into the version-x.y.z directory. Run command how-to-build\ReactOS.bat. If you need, you can download curses and SDL2 libraries and headers from here. Extract the archive and move the contents into the version-x.y.z directory.

2.6 AmigaOS 4.x

Download and install AmigaOS SDK. Download and extract bzip2 to get bzip2_68k executable. Download the amalgamated Dern source release.

Open Shell window and run the following commands:

bzip2_68k -dk dern.tar.bz2
tar xf dern.tar
cd dern
sh how-to-build/AmigaOS41.sh

3. Hello world

Here we have a version of the classic Hello World-program in Octaspire Dern. Instead of just printing Hello, World!, it is a bit more complex to give you some feeling for the language. If you are in Unix-like system and have octaspire-dern-repl in somewhere on your PATH, you can make the script executable using the shebang. You can also run the file by octaspire-dern-repl hello-world.dern or by writing it or parts of it directly to the interactive REPL.

  #!/usr/bin/env octaspire-dern-repl
  This is a multiline comment.    !#

  ; 1. Print once 'Hello, World!' and newline
  (println [Hello, World!])
  (println)

  ; 2. Print 11 times 'Hello x World!', x is 0 .. 10
  (for i from {D+0} to {D+10}
    (println [Hello {} World!] i))
  (println)

  ; 3. Print greetings to everybody on the vector
  (define names as '(John Alice Mark) [Greetings list])
  (for i in names (println [Happy holidays, {}!] i))
  (println)

  ; 4. Add new name, 'Lola', to the names to be greeted
  (+= names 'Lola)
  (for i in names (println [Happy holidays, {}!] i))
  (println)

  ; 5. Remove one name, 'Mark'
  (-= names 'Mark)
  (for i in names (println [Happy holidays, {}!] i))
  (println)

  ; 6. Define new function to greet people and use it
  (define greeter as (fn (greeting name)
      (println [{}, {}!] greeting name))
    [My greeter function] '(greeting [the greeting]
                            name [who to greet])
                            howto-no)

  (greeter 'Hi 'Alice)

  ; 7. Redefine greeter-function with early exit
  ; using 'return'
  (define grumpy as true [is our hero grumpy, or not])

  (define greeter as (fn (greeting name)
      (if grumpy (return [I'm grumpy and don't greet]))
      (println [{}, {}!] greeting name)
      (string-format [I greeted "{}", as asked] name))
    [My greeter function] '(greeting [the greeting]
                            name [who to greet])
                            howto-no)

  (println (greeter 'Hi 'Alice))
  (= grumpy false)
  (println (greeter 'Hi 'Alice))
  (println)

  ; 8. Add names and custom greetings into a hash map
  ; and use it to greet people
  (define names as (hash-map 'John 'Hi
                             'Lola 'Hello
                             'Mike 'Bonjour)
                          [My custom greetings])

  (for i in names (greeter (ln@ i {D+1}) (ln@ i {D+0})))

  (+ 1.0.0 0.0.1) ; 1.0.1
  (< 1.0.0 1.0.1) ; true
  (> 1.0.0 1.0.1) ; false

4. Values

{D+128}              ; These are integers
{D-100}

{B+1001}             ;   9 in binary
{B-10011}            ; -19 in binary

{O+764}              ;  500 in octal
{O-764}              ; -500 in octal

{X+4B5}              ; 1205 in hexadecimal
{X-FF}               ; -255 in hexadecimal

{D+100 000 000}      ; Spaces can be used

{D+3.14}             ; These are real
{D-1.12}

0.1.2-rc.1+amd64     ; Semantic version numbers
1.0.1
2.1.0-4344b11
0.0.1+i386

[Hello]           ; These are strings (utf-8)
[Hell|6F|]        ; Hello
[Hello|newline|]  ; Hello and newline
[Я могу есть стекло, оно мне не вредит]
|a|               ; These are characters (utf-8)
|newline|         ; \n
|tab|             ; \t
|bar|             ; |
|string-start|    ; [
|string-end|      ; ]
|61|              ; a in hexadecimal notation
|7A|              ; z in hexadecimal notation
|44F|             ; я in hexadecimal notation
true              ; These are booleans
false
nil               ; Nil
'({D+1} {D+2} |a| [cat])      ; These are vectors
'()
(hash-map 'John [likes cats]  ; This is hash map
          'Lisa [likes dogs]
          'Mike '([likes numbers] {D+1} {D+2}
                                  {D+3} {D+4})
           {D+1}    |a|
           [Hi] {D+2})

The text after character ; is a single line comment. Single line comments run until the end of the line. Dern has also multiline comments that are written between #! and !#. Note that string delimiters in Dern are [ and ] and not "; this way dern code can be written inside C-programs without escaping.

Strings can be embedded in strings like this:

[string |string-start|with another inside|string-end|]

This can be useful sometimes, for example, if you need to evaluate a string as a program and need it to have strings inside.

5. Single and multiline comments, making script files executable

Below are examples of single and multiline comments:

; This is single line comment.

#! This is multiline comment.
   It can contain multiple lines...
   ... !#

Multiline comments can be used to make script files executable in UNIX-like systems:

#!/usr/bin/env octaspire-dern-repl
!#

(println [Hello World])

6. Binding names to values with define

(define pi as {D+3.14} [value for pi])
(define names as '(John Lisa Mark) [names list])
(define double as (fn (x) (* {D+2} x)) [doubles nums]
    '(x [this is doubled]) howto-ok)

Here we bind three values to a name: one real, one vector and one function taking one argument. Here is an example of using those names:

pi
names
(double {D+1})

And to see the documentation for these values:

(doc pi)
(doc names)
(doc double)

The documentation of the function contains also documentation for the parameters.

Function doc can also be used with builtins and specials defined by the standard library or user in C.

Please note that at the time of writing most of the functions in Dern's standard library are not yet documented properly. This is a work in progress.

6.1 Binding in other environments than the current one

By using an explicit environment argument as the first argument to define, we can bind names to values in other environments than the current one. Example:

(define myEnv as (env-new) [my own environment])
(define pi as {D+3.14} [value for pi] in myEnv)

pi                  ; <error>: Unbound symbol 'pi'
(eval pi myEnv)     ; 3.14

In the example above, pi is undefined in the current (global) environment, but it is defined in the myEnv-environment. We use special eval to evaluate pi in the myEnv-environment.

7. Iteration

Dern has two looping constructs: while and for. For can be used numerically, with a container (vector, string, hash-map, etc.) and with (input) ports. Below is couple of examples:

(define i as {D+0} [my counter])
(while (<= i {D+10}) (println [Going at {}] i) (++ i))

Numerical for:

(for i from {D+0} to {D+10} (println [Hello {}!] i))

Container for:

(define names as '(John Mark Lisa) [names list])
(for i in names (println [Hello {} World!] i))

Both the numerical for and container for support the use of optional step to change the way the iterator is incremented:

(for i from {D+0} to {D+10} step {D+3}
  (println [Hello {}!] i))

(define names as '(John Mark Lisa) [names list])
(for i in names step {D+2}
  (println [Hello {} World!] i))

8. Comparing and changing values, predicates

Here are few examples:

(<  {D+1} {D+2})   ; true
(<  {D+2} {D+2})   ; false
(>  {D+2} {D+1})   ; true
(<= {D+1} {D+1})   ; true
(>= {D+1} {D+1})   ; true
(== {D+3} {D+3})   ; true
(== {D+3} {D+1})   ; false
(!= {D+3} {D+1})   ; true
(+ {D+1})      ;  1
(+ {D+1} {D+1})    ;  2
(- {D+1})      ; -1
(- {D+1} {D+2} {D+3})  ; -4

(not true)     ; false

(uid +)        ; unique id of +
(=== + +)      ; compare using unique id

(len '({D+1} {D+2} {D+3}))  ; length of vector:   3
(len [abc])                 ; length of string:   3
(len (hash-map {D+1} |a|))  ; length of hash-map: 1

(define number as {D+1} [my number])
(++ number)                      ; number is 2
(-- number)                      ; number is 1
(+= number {D+2})                ; number is 3

(+ [Hello] [ ] [World.] [ Bye.]) ; Hello World. Bye.

(define greeting as [Hello] [my greeting])
(+= greeting [ World!])          ; Hello World!
(+= greeting |!|)                ; Hello World!!

(+= '({D+1} {D+2} {D+3}) '({D+4} {D+5} {D+6}))
; (1 2 3 (4 5 6))

(define capitals as (hash-map [United Kingdom]
                                [London]
                              [Spain] [Madrid])
    [country -> capital])
(+= capitals [Nepal] [Kathmandu])
(+= capitals '([Norway] [Oslo] [Poland] [Warsaw]))
(+= capitals (hash-map [Peru] [Lima]))

(-= {D+10} {D+1} {D+2} {D+3})             ; 4
(-= |x| {D+2})                            ; |v|
(-= |x| |!|)                              ; |W|
(-= [abba] |a|)                           ; [bb]

(-= (hash-map {D+1} |a| {D+2} |b|) {D+1})
; (hash-map 2 |b|)

(-= '({D+1} {D+1} {D+2} {D+2} {D+3}) {D+1} {D+2})
; (3)

(define v as '({D+1} {D+2} {D+3} {D+3}) [v])
(-= v (ln@ v {D-1}))             ; (1 2)

(define v as '({D+1} {D+2} {D+3} {D+3}) [v])
(-== v (ln@ v {D-1}))            ; (1 2 3)

Operators ++, --, +=, -=, == and != are similar to those in C. Note also that the operands need not to be numbers. You can, for example, use += to push values to the back of a vector, add characters into a string, write values into a port, etc. -== removes values from a supported collection by comparing the unique identifiers of values. It removes only values that are the same (equal values might not be the same).

Please note: all the examples above should work, but support for non-numeric types is not finished on most of the operators. Using those operators with non-numeric arguments aborts the program or returns error. Complete support for non numeric operands for the above operators should be implemented in the standard library eventually.

9. Removing the last value from a vector

Compare these two cases:

(define v as '({D+1} {D+2} {D+3} {D+3}) [v])
(-= v (ln@ v {D-1}))             ; (1 2)

(define v as '({D+1} {D+2} {D+3} {D+3}) [v])
(-== v (ln@ v {D-1}))            ; (1 2 3)

The last example removes really only the last value (compared using ===). The first example removes all the values that are equal to the last value (compared using ==).

Values can be removed this way from any position by using different indices. As with other functions, negative indices count from the end of the collection and positive from the beginning.

More efficient way of removing the last value from a collection is to use the builtin pop-back:

(define v as '({D+1} {D+2} {D+3} {D+3}) [v])
(pop-back v)                  ; (1 2 3)

10. Branching and selection

Here are some examples using if:

(if true  [Yes])         ; Yes
(if false [Yes])         ; nil
(if false [Yes] [No])    ; No

(if true  (println [Yes]) (println [No]))
; Prints Yes

; Prints Yes|newline|OK
(if true  (do (println [Yes]) (println [OK])))

Here are some examples using select:

(select true [Yes])            ; Yes

(select false [No]
        true  [Yes])           ; Yes

(select default [Yes])         ; Yes

(select false   [No]
        default [Yes])         ; Yes

(select false   [No]
        true    [Maybe]
        default [Yes])         ; Maybe

(select false [Yes])           ; nil


(define f1 as (fn () true)  [f1] '() howto-no)
(define f2 as (fn () false) [f2] '() howto-no)

(select (f1)  [Yes]
        (f2)  [No]
        false [Maybe])         ; Yes

; Prints: Sun is shining
(select (f1)  (println [Sun is shining])
        (f2)  (println [It rains])
        false [Maybe]
        false {D+2}
        false {D+3.14}
        false |a|
        false [There can be many selectors...])

11. Selecting values from collections

Values can be selected from collections using ln@ and copied with cp@. ln@ is pronounced link at and cp@ is pronounced copy at.

(++ (ln@ '({D+1} {D+2} {D+3}) {D+1}))   ; 3
(+= (cp@ [abc] {D+1}) {D+2}))           ; |d|
(ln@ (hash-map |a| [abc]) |a|   'hash)  ; [abc]
(ln@ (hash-map |a| [abc]) {D+0} 'index) ; [abc]

12. Accessing and manipulating command line arguments and environment variables

This section is not ready yet. See the example below. More information will be added later.

(host-get-command-line-arguments)
(host-get-environment-variables)

13. Formatted and regular printing

Here are few examples:

(print   [Hi])   ; Prints Hi without newline
(println [Hi])   ; Prints Hi and newline

(define name1  as 'Jim   [some name 1])
(define name2  as 'Alice [some name 2])
(define number as {D+30} [some number])

; Prints Hi Jim and Alice! It is 30 degrees outside.
(println [Hi {} and {}! It is {} degrees outside.]
    name1 name2 number)

14. Formatted string creation

Here are few examples:

(define name1  as 'Jim   [some name 1])
(define name2  as 'Alice [some name 2])
(define number as {D+30} [some number])

; Creates a sting
; [Hi Jim and Alice! It is 30 degrees outside]
(string-format
  [Hi {} and {}! It is {} degrees outside]
  name1 name2 number)

15. Functions with variable number of arguments

Here are few examples:

(define f as (fn (x ...) x) [f]
    '(x [x] ... [varargs]) howto-no)

(f {D+1} {D+2} {D+3})   ; (1 2 3)


(define f as (fn (x y ...) (println x) (println y))
    [f] '(x [x] y [rest of the args] ... [varargs])
    howto-no)

(f {D+1} {D+2} {D+3})   ; Prints 1|newline|(2 3)

16. Environments

Here are few examples:

(env-global)
(env-current)
(env-new)

17. Getting help with howto

Function howto can be used for asking howto do something. It is given first the arguments and then the expected result. It returns a vector containing a listing of forms to do the task. Not all functions support howto, but many do. Usually functions supporting howto should not have (large) side effects. When writing Dern functions, one has to decide whether those functions should support howto or not.

Here is small example:

; Enter these forms into the REPL
(howto {D+1} {D+2} {D+3})
; ((+ {D+1} {D+2}) (+ {D+2} {D+1}))

(howto [a] [b] [ab]) ; ((+ [a] [b]))

(howto '(John Mike Alice Lola) 0 'John)
; ((ln@ (quote (John Mike Alice Lola)) {D+0})
;  (cp@ (quote (John Mike Alice Lola)) {D+0}))

(howto '(John Mike Alice Lola) 'Lola '({D+3}))
; ((find (quote (John Mike Alice Lola)) (quote Lola)))


; Or print them
(println (howto {D+1} {D+2} {D+3}))
; prints ((+ {D+1} {D+2}) (+ {D+2} {D+1}))

(println (howto [a] [b] [ab])) ; prints ((+ [a] [b]))

(println (howto '(John Mike Alice Lola) 0 'John))
; prints ((ln@ (quote (John Mike Alice Lola)) {D+0})
;         (cp@ (quote (John Mike Alice Lola)) {D+0}))

(println (howto '(John Mike Alice Lola) 'Lola
                '({D+3})))
; prints ((find (quote (John Mike Alice Lola))
;               (quote Lola)))

18. Returning from functions early

The value of the last expression of function is usually the return value from that function. However, by using return one can return early and have multiple exit points from a function. Small example:

(define errorCode as {D+1} [0 means no error.])

(define start-engine as (fn ()
    (if (!= errorCode {D+0}) (return [Cannot start]))
    ; .... Start the engine here...
    [Start engine if all OK] '() howto-no))

Function return can be called with zero or one argument. If no arguments are given, then return will return the value nil. Short example:

((fn () (return nil)))   ; Evaluates into 'nil'.
((fn () (return)))       ; Evaluates into 'nil'.

19. Evaluating values

Special eval can be used to evaluate a given value. It can be called with one or two arguments. The second argument, if present, must be an environment that is used while evaluating. If no environment is given, the global environment is used instead.

Function eval is useful, for example, in situations where you build the name of the function to be called at runtime. Small example:

(define level-next as (fn ()
    (level-reset)

    (define lnum as (+ level-current-number {D+1})
        [level number])

    (if (> lnum number-of-levels) (= lnum {D+1}))

    (define name-of-fn-to-call as 'level-
        [name of the level builder function to call])
    (+= name-of-fn-to-call lnum)
    (eval ((eval name-of-fn-to-call)))) [next level]
        '() howto-no)

20. Input and output ports

Input and output can be done through ports. Ports can be created and attached to different sources and sinks of data (for example the file system).

Here is small example:

(define f as (io-file-open [/path/goes/here.xy]) [f])

(port-read f)
(port-read f {D+3})

(port-write f {D+65})
(port-write f '({D+65} {D+66} {D+67}))

Ports can be explicitly closed, but it is not required; port will close automatically when the garbage collector collects it. Some ports might also support seeking, distance measurement, length measurement and flushing. Here is another small example:

(define f as (io-file-open [/path/goes/here.xy]) [f])

(port-seek f {D-1}) ; Seek to the end
(port-write f {D+65})

(port-seek f {D+0})  ; Seek to the beginning
(port-write f {D+65})

(port-seek f {D-2}) ; Seek to one octet from the end
(port-write f {D+66})

(port-seek f {D+1})
; Seek to one octet from the beginning

(port-write f {D+65})

; Seek one octet forward  from the current position
(port-seek f  {D+1} 'from-current)

; Seek one octet backward from the current position
(port-seek f {D-1} 'from-current)

; Tell the distance (in octets) from
; the beginning of the port
(port-dist f)

(port-length f)
; Tell the size (in octets) of the port

; Buffer is flushed to disk.
; Happens also automatically on close.
(port-flush f)
; Close port. This happens also automatically.
(port-close f)

(port-length f) ; -1

Input ports can be iterated with for in similar way that containers are iterated:

(define f as (io-file-open [/path/goes/here.xy]) [f])

(for i in f (println i)) ; Print every octet

(port-seek f {D+0})      ; Seek to the beginning

; Print every other octet
(for i in f step {D+2} (println i))

(port-seek f {D+0}) ; Seek to the beginning

; Print every third octet
(for i in f step {D+3} (println i))

io-file-open will open a file for reading and writing, input-file-open will open a file only for reading and output-file-open will open file only for writing.

Below is short example about querying a port for supported operations:

(define f as (io-file-open [/path/goes/here.xy]) [f])

(port-supports-output? f)          ; true
(port-supports-input?  f)          ; true

(define f as
    (output-file-open [/path/goes/here.xy]) [f])

(port-supports-output? f)          ; true
(port-supports-input?  f)          ; false

(define f as
    (input-file-open [/path/goes/here.xy]) [f])

(port-supports-output? f)          ; false
(port-supports-input?  f)          ; true

You can use port-write and += to write to a port octets with values integer, character, string and vector of these types. Example:

(define f as (io-file-open [/path/goes/here.xy]) [f])

(+= f |a| |b| [ cat] |!|)  ; ab cat!

(port-write f '({D+65} |A| [ Hi!])) ; AA Hi!

21. Converting between types

TODO

22. Searching and indexing

TODO

(define names as '(Mike John Lola Alice Lola) [names])

(println (find names 'Mike))  ; prints ({D+0})
(println (find names 'John))  ; prints ({D+1})
(println (find names 'Lola))  ; prints ({D+2} {D+4})

 ; prints (({D+0}) ({D+1}) ({D+2} {D+4}))
(println (find names 'Mike 'John 'Lola))


(define rooms as (hash-map 'Mike {D+100}
                           'John {D+101}
                           'Lola '({D+102} {D+103})
                           'Alice {D+104})
                 [room numbers])

(println (find rooms 'Mike)) ; prints {D+100}

(println (find rooms 'Lola))
; prints ({D+102} {D+103})

(println (find rooms 'Nobody)) ; prints nil


; prints ({D+7} {D+11} {D+15})
(println (find [012345 abc abc abc] [abc]))
(println (find [012345] |3|)) ; prints ({D+3})

23. Loading libraries with require

Dern has support for loading libraries or "plugins" during run time with the builtin require. Before loading the requested library, require checks whether the library is already loaded, and loads it only if it isn't already loaded.

It first tries to find a source library (.dern file) with the given name. If it finds, it loads that. Next it tries to find a binary library (.so file in Unix) and loads that if found.

So, in the example below, require tries first to find file named mylib.dern and then, if the system is Unix, file named libmylib.so.

Here is small example:

(require 'mylib)
(mylib-say [Hello world from library])

If mylib-library is required later again, there is no need to search and load it again, because require knows that a library with that name is already loaded.

Below is a small example of a binary library for Linux, FreeBSD, NetBSD, Haiku and MINIX 3 systems.

/***
  To build this file into a shared library in Linux:

  gcc -c -fPIC mylib.c     \
      -I ../../../include  \
      -I ../../../external/octaspire_core/include
  gcc -shared -o libmylib.so mylib.o
***/
#include <stdio.h>
#include <octaspire/core/octaspire_helpers.h>
#include "octaspire/dern/octaspire_dern_vm.h"
#include "octaspire/dern/octaspire_dern_environment.h"

octaspire_dern_value_t *mylib_say(
    octaspire_dern_vm_t *vm,
    octaspire_dern_value_t *arguments,
    octaspire_dern_value_t *environment)
{
    OCTASPIRE_HELPERS_UNUSED_PARAMETER(environment);

    if (octaspire_dern_value_as_vector_get_length(
        arguments) != 1)
    {
        return
            octaspire_dern_vm_create_new_value_error_from_c_string(
                vm,
                "mylib-say expects one argument");
    }

    octaspire_dern_value_t const * const messageVal =
        octaspire_dern_value_as_vector_get_element_at_const(
            arguments,
            0);

    if (messageVal->typeTag !=
        OCTASPIRE_DERN_VALUE_TAG_STRING)
    {
        return
            octaspire_dern_vm_create_new_value_error_from_c_string(
                vm,
                "mylib-say expects string argument");
    }

    printf("%s\n",
           octaspire_dern_value_as_string_get_c_string(
               messageVal));

    return octaspire_dern_vm_create_new_value_boolean(
        vm,
        true);
}

bool mylib_init(
    octaspire_dern_vm_t * const vm,
    octaspire_dern_environment_t * const targetEnv)
{
    octaspire_helpers_verify(vm && targetEnv);

    if (!octaspire_dern_vm_create_and_register_new_builtin(
            vm,
            "mylib-say",
            mylib_say,
            1,
            "mylib says something",
            targetEnv))
    {
        return false;
    }

    return true;
}

See directory doc/examples/plugin in the source distribution for an example with Makefiles for different systems.

23.1 Building and using a binary library in Haiku

Run these commands from the build-directory of the source distribution:

make -C ../doc/examples/plugin -f Makefile.Haiku
LIBRARY_PATH=$LIBRARY_PATH:../doc/examples/plugin \
    ./octaspire-dern-repl -c

Write into the REPL:

(require 'mylib)
(mylib-say [Hello world from library])

23.2 Building and using a binary library in MINIX 3

Run these commands from the build-directory of the source distribution:

make -C ../doc/examples/plugin -f Makefile.MINIX3
LD_LIBRARY_PATH=../doc/examples/plugin \
    ./octaspire-dern-repl -c

Write into the REPL:

(require 'mylib)
(mylib-say [Hello world from library])

23.3 Building and using a binary library in Linux

Run these commands from the build-directory of the source distribution:

make -C ../doc/examples/plugin
LD_LIBRARY_PATH=../doc/examples/plugin \
    ./octaspire-dern-repl -c

Write into the REPL:

(require 'mylib)
(mylib-say [Hello world from library])

23.4 Building and using a binary library in FreeBSD

Run these commands from the build-directory of the source distribution:

make -C ../doc/examples/plugin -f Makefile.FreeBSD
LD_LIBRARY_PATH=../doc/examples/plugin \
    ./octaspire-dern-repl -c

Write into the REPL:

(require 'mylib)
(mylib-say [Hello world from library])

23.5 Building and using a binary library in NetBSD

Run these commands from the build-directory of the source distribution:

make -C ../doc/examples/plugin
LD_LIBRARY_PATH=../doc/examples/plugin \
    ./octaspire-dern-repl -c

Write into the REPL:

(require 'mylib)
(mylib-say [Hello world from library])

24. Using custom library loader with require

Sometimes you might want to override the default library searching and loading functionality and use a custom loader instead. For example, when writing a game that contains all the resources in a compressed archive or inside the executable program, or maybe the library must be first downloaded through a socket.

// ...

octaspire_input_t *my_custom_loader(
    char const * const name,
    octaspire_memory_allocator_t * const allocator)
{
    if (strcmp("test1.dern", name) == 0)
    {
        return octaspire_input_new_from_c_string(
            "(define f1 as (fn (a b) (+ a b)) [f1] "
            "'(a [a] b [b]) howto-ok)",
            allocator);
    }
    else if (strcmp("test2.dern", name) == 0)
    {
        return octaspire_input_new_from_c_string(
            "(define f2 as (fn (a b) (* a b)) [f2] "
            "'(a [a] b [b]) howto-ok)",
            allocator);
    }

    return 0;
}

int main(void)
{
    octaspire_dern_vm_config_t config =
        octaspire_dern_vm_config_default();
    config.preLoaderForRequireSrc = my_custom_loader;

    octaspire_dern_vm_t *vm =
        octaspire_dern_vm_new_with_config(
            myAllocator,
            myStdio,
            config);


    // In Dern:
    // (require 'test1)
    // (require 'test2)
    // ...
}

25. Embedding in C programs

TODO

26. Tool support

etc-directory of the source distribution contains syntax files for vim, emacs, pygments and GNU source-highlight.

27. Using the development repository

The amalgamated source release can be used without Make or other build tools; only a compiler is needed. It is the recommended way of using Dern and is available from the release directory of the git repository and from the dern.tar.bz2 archives at octaspire.io and octaspire.com. However, when there is a need to modify Dern itself or to build the documentation, then some tools are needed and the Makefile in the git repository should be used. In this case the files to be modified are found from the dev directory.

On different systems the required installation commands can vary. In any case, you should install a C compiler and git. Depending on the system, you might also need to install GNU make. If you want to build the documentation, you should also install GNU source-highlight and python. For code coverage and performance measurement you might need to install additional tools.

That's it; octaspire_core and octaspire_dotfiles are included as git submodules. The Emacs dotfiles are used to set the correct settings and styling when building the documentation (documentation is written using Emacs Org mode).

Example of using the development repository and the different targets available in the Makefile:

git clone https://github.com/octaspire/dern.git
cd dern
make submodules-init
make
make test
make codestyle
make cppcheck
make valgrind
make coverage
make coverage-show
make perf-linux

./octaspire-dern-repl

make amalgamation
release/octaspire-dern-repl

make clean

rm release/octaspire-dern-amalgamated.c
make release/octaspire-dern-amalgamated.c

rm release/documentation/dern-manual.html
make release/documentation/dern-manual.html

Running make should run make submodules-init automatically if needed (make submodules-init clones and initializes submodules), but it can be run also manually when cloning the repository for the first time. There is no need to run it again later.

If you want to update the submodules into the latest commits, you can run make submodules-pull. It does a git pull on all the submodules.

Running make builds the Dern REPL and unit test runner from the separate Dern source files and then the binary plugins using the amalgamation.

Running make amalgamation generates first the amalgamation (only, if the source files are changed). Then it detects the system and runs the correct build script. This creates the Dern REPL and unit test runner binaries and also the binary plugins on some systems.

make test runs the unit tests, make codestyle runs the C coding style checks, make cppcheck runs the cppcheck static analysis on the code, make valgrind runs the unit tests through Valgrind (dynamic analysis and memory leak detection) and make coverage generates a unit test coverage report that can be used for finding code that is not covered yet by unit tests. make coverage-show generates and shows the coverage report in a web browser. make perf-linux measures the performance in GNU/Linux using the unit test runner. make release/documentation/dern-manual.html builds the documentation.

27.1 Raspberry Pi, Debian and Ubuntu

To build Dern from the regular source distribution in Raspberry Pi (Raspbian), Debian or Ubuntu (16.04 LTS) system:

sudo apt-get install git
git clone https://github.com/octaspire/dern.git
cd dern
make submodules-init
make

27.2 Arch Linux

To build on Arch Linux (Arch Linux ARM) system:

sudo pacman -S git gcc make
git clone https://github.com/octaspire/dern.git
cd dern
make submodules-init
make

27.3 Haiku

To build on Haiku (Version Walter (Revision hrev51127) x86_gcc2):

pkgman install gcc_x86
git clone https://github.com/octaspire/dern.git
cd dern
make submodules-init
CC=gcc-x86 make

27.4 FreeBSD

To build on FreeBSD (FreeBSD-11.0-RELEASE-arm-armv6-RPI2) system:

sudo pkg install git
git clone https://github.com/octaspire/dern.git
cd dern
make submodules-init
make

27.5 NetBSD

To build on NetBSD (NetBSD-7.1-i386) system:

sudo pkgin install git
git clone git://github.com/octaspire/dern
cd dern
perl -pi -e 's/https/git/' .gitmodules
make submodules-init
make

27.6 MINIX 3

To build from the regular source distribution on MINIX 3 (minix_R3.3.0-588a35b) system:

su root
pkgin install clang binutils git-base
exit
git clone git://github.com/octaspire/dern
cd dern
perl -pi -e 's/https/git/' .gitmodules
make submodules-init
make