Announcing Godotcaml for Godot 4.2

Hello! Today I’m releasing a project on which I’ve been working, that is in an early stage of development, into the open source world. It is integration and bindings to Godot (currently just 4.2) from a new language: OCaml. It is called Godotcaml. Details below!

Why Godot?

There are many reasons to choose Godot, but the reason I’ll focus on is that it provides a full game-development IDE from which you can develop production quality 2D and 3D games. It’s suitable for small-to-medium-sized teams, quite mature, and very fun to use; I think most devs have a secret inclination to “one day” make a video game. If there is one piece of advice I could give to those devs, it’s choose a good engine that already exists, unless you want to be stuck in Vulkan hell for 6-12 months. Godot is a good engine, and happily, already exists, and is free and open source — so is a good first choice, even if you abandon it for something different later.

Why OCaml?

While my greatest loved language is Haskell, there are some specific reasons that it is somewhat unsuitable for game development. Instead of listing those, however, I will instead take a more positive approach and talk about why OCaml is an excellent language for game development.

Garbage Collected by Default: This may shock game devs used to the C++ lyfe, but will come as no surprise to people who’ve used Godot and/or Unity. Programmers are just more productive when there is a garbage collector, and programmer time is what is usually most valuable — not machine time. Now, you can write some pretty cool allocation-free OCaml code too — but that’s an optimization that you should measure your need for before you commit to it.
Functional by Default: I love functional programming, and I love the kind of code you can write in an ML-like curried functional language (such as OCaml or Haskell). So if I were to bind Godot to a new language, it would have to be a functional one. However, excellent bindings (gdext) for Rust already exist, and it can be used as a workable functional language. That being said, Rust has the borrowchecker and an aversion to garbage collection, and put simply, I don’t think it makes a particularly good game scripting language, even though it is a wonderful systems programming language. I think OCaml can one day prove to be a more efficient vehicle for experienced functional programmers to create a game in. (No hate intended! This is just how I feel.)
Eager by Default: I absolutely adore lazy APIs. I might actually be in the minority now in the Haskell community that I think that laziness was not a mistake, but was an excellent choice because of the ergonomics it provides. However, it’s definitely true that it makes it somewhat less straight-forward to reason about the runtime performance of your code, and indeed to debug it — at least without specialized knowledge that is, in my experience, rather rare to have. OCaml is eager by default, and I think that’s probably better for a soft-realtime system, unless you’re using some specialized framework (e.g. one that I don’t know whether exists or not!).
Side-effects for When You Need Them: If the world was all written in one language, Haskell would make a pretty good choice — not perfect by any stretch, but pretty good. However, we live in the real world, where a C FFI is the glue holding this pot of spaghetti we call an operating system together. Because of that, when doing FFI heavy code, a beautiful language that makes it slightly more tedious to work with side-effects is less preferable in my experience than a language that simply encourages you to think before you use them, but still easily allows you to do things like have global mutable references at the top level.
PPXes For CodeGen Help: I’m not totally sold on the pervasive use of PPXes for OCaml code, but one thing they are definitely useful for is code gen, and you would need a lot of tedious hand-written code if you wanted to interact with Godot directly by hand. PPXes lie somewhere between Rust macros and TemplateHaskell in their power, but most problems with codegen were able to be solved to my satisfaction using the wonderful PpxLib and context-free extenders. For example, here is the definition of a simple Godot class that inherits from the stock Node class, and provides a successor function for Godot ints:

module%gclass MyClass = struct
  [%%ginherits Node]

  let%gfunc succ = 
    [| ClassMethodFlags.default |]
      (module BuiltinClass0.Int)
      (module Class.Node)
      (module BuiltinClass0.Int) 
      (fun i _self -> Int64.(i + 1L))
end

Whether or not you like the use of PPXes in general, it is tough to argue that this code isn’t at least short, especially if I were to show you the amount of work you’d have to do without those %s!

More: If you’re reading this post, you probably already like OCaml already, so I’ll leave it there at “it’s a really nice pragmatic functional language, and I thought it would be a good candidate”!

What Can It Do?

This is an extremely early stage of development, but basically at this point it is possible to:

Call any builtin Godot utility function or method (static, virtual, or otherwise) from OCaml easily, and with documentation comments for the original function intact an available through your favourite OCaml LSP implementation.
Use Godot (binary) operators in a natural way from OCaml. (Unary operators are currently broken, which I will be investigating!)
Construct Godot values from OCaml easily, and from OCaml analogues if they exist (e.g. I incur a dependency on Gg for low-dimensional vector math)
Marshalling in and out of all these functions to/from the OCaml analogues. That is, a method that is in Godot on an object of type ClassyClass taking an int parameter and returning an int will appear in Godotcaml as int64 -> ClassyClass.t structure ptr -> int64, where the ClassyClass.t structure ptr is the “pointer to the Godot object”, commonly called self. (Note that this is always the last argument, to facilitate pipeline-style programming when GDScript programmers have a method-chaining interface.)
Naturally define a new Godot class in OCaml that inherits from an existing Godot-registered class. (Currently NOT tested with classes defined in GDScript and/or externally.)
Most of the code-gen for custom engines that define new stock/builtin types and classes, etc.
Simulated inheritence for stock (and easily extendable to user-defined) classes using module inclusion: That is if Derived inherits from Base, then simply include Base in the module representing Derived, and you get access to all the methods from Base without explicit casting (or in the case of Rust’s gdext, object composition).
Naturally define a new Godot method in OCaml and have it called from GDScript or another Godot-bound language. (ergonomics still WIP).

TODO (Or, What Can’t It Do):

Signals: I’m still cooking ideas for how best to do this, but user-defined signals are not currently nicely supported, and even built in ones are not nice to call or interact with right now. I’d also like them to be type-safe, so there’s that. Very WIP, but fixable with some thought and work.
Garbage Collection: Right now, if a OCaml reference is stored in a, say, GDScript variable, and contains no references in the OCaml world, then it might be collected out from under you. This is fixable because of Godot’s wonderful reference-counting hooks and OCaml finalisers, I just haven’t gotten around to it yet.
Nice Interface to Various Kinds of Methods: As of the writing of this blog post, it is only possible to define methods of arity 1 + Self from OCaml. No static methods, no virtual methods This will be fixed soon, but I wanted to get the stuff in the hands of interested enthusiasts as soon as I was able to call OCaml functions from Godot.
Real First Class Modules for Method Definitions: Taking again the example above, we can write

let%gfunc f = 
  [| ClassMethodFlags.default |]
  (module ArgumentGodotTypeModule)
  (module SelfGodotTypeModule)
  (module ReturnValueGodotTypeModule)
  (fun x self -> (* implementation here ...*) ())

making it seem like you could write, say

let m = (module SomeGodotTypeModule)

let%gfunc g = [| ClassMethodFlags.default |] m m m (fun x y -> x)

or something, but that wouldn’t work. This is due to the way I generate code, but basically you have to consider the module as part of the “syntax” for gfunc; it directly takes the packed module out of the expression and codegens using whatever the module operator is applied to (i.e. at parse time, not run time). This is pretty messed up and not ideal, but the way I justify it to myself is that let%gfunc introduces it’s own syntactic form for declaring a type signature and implementation of a method. This is, as far as I can tell, not fixable, but I’d love to hear your thoughts if you think it is. (Briefly, the problem is that if you try to use real first-class modules, their types escape the scope of the function because the implementation function contains them.)

General Clean-up: This implementation was designed sort of ad-hoc and in-the-moment, so some of the stuff doesn’t quite make sense in the module architecture. This is fixable but lower priority until I iron out the rest of the implementation details of the other features.
Better Build System Integration: I don’t know dune very well, so I got it working, but it’s not exactly nice to use and develop on. Lots of ad-hoc calls to dune exec ./gen_api.exe when something has changed, and then trying to remember to format with ocamlformat -i *.ml — that sort of thing; I’m sure dune can help with it, but I didn’t invest the time into learning properly (but I will). Fixable.
Hot-Reloading: This should be possible, as Rust somehow manages it in Godot 4.2+ but I haven’t even begun to look into it. Right now, if you change an OCaml file and recompile the extension, you probably need to restart the editor to see the effects. Fixable.
Name-mangling for Custom Operators: I just haven’t done this, but it wouldn’t be hard (and would probably make a good first issue, if you’re looking to contribute). Right now custom operators defined as gfunc methods in OCaml probably can’t be used from GDScript, or most other languages. Perhaps at all! I haven’t even tried. Fixable.
Finishing the C API: These represent a work-in-progress set of bindings that are by no means complete at the moment. That needs to change eventually. Fixable.
Embedding a TopLevel: I’d like to be able to interact with the Godot world, well, interactively, from an OCaml Toplevel. This is on my backburner, but it’s harder than it looks at first, because the shared_object you have to build must of course be native code, but Toplevel and friends are (seemingly?) only available as bytecode. Ping me if you have ideas here! Fixability unknown!
Reliably Not Segfault: This is an unsafe C api, and so it’s extremely sharp, and the interface is very rough around the edges, as I figure out what exactly each value is supposed to actually do. DO NOT TRY TO MAKE A PRODUCTION GAME IN GODOTCAML RIGHT NOW! I’m going to fix things, but they definitely aren’t ready for prime-time at the moment. This is fixable, but will take time and testing.
Testing: Speaking of testing, I have none. If you’d like to contribute here, I’d be happy to hear from you; I’m personally going to prioritize other above areas until things are a little more stable in the API, so I’m not constantly changing things and fixing broken tests (i.e. as opposed to broken code that is being tested). Fixable.
Type Safety Concerns: Right now all classes have the same object type. This makes it nice for when you’re inheriting from them, as module inclusion “just works”, but obviously have negative effects on the type safety of the system. Destructive updates of the module types during inclusion is one possible solution, but this requires you to have a module type for every Godot class from which you wish to inherit — a tall order. I believe this is fixable with some thought — and indeed, it must be fixed in my eyes, even if it means more code gen for the module signatures — but I haven’t given it much thought beyond that.
Support Multiple Native Type Sizes: Godot’s api supports multiple configurations, depending on if you want float64 or float32, and 64-bit and 32-bit systems. Right now, I’m concentrating on the float64 + 64-bit configuration, but this should be expanded at some point in the future. Fixable.

For more, check the issue page on GitHub, as that is where I’ll be doing the development.

To Be Continued

More details and a setup guide to come! If you’d like to get involved, I’d love to hear from you — best place to find me is either GitHub or the OCaml Discourse currently. Beware, the code is pretty funky at the moment, but it’ll get there!

Best,

Matt

#open-source#ocaml#godot#godotcaml#announcement