The Structure of Godotcaml (Today)

Hello again!

Welcome to (perhaps part 1) of a deep dive into the structure of Godotcaml as it stands at release. This post is likely only to be of interest if you want to use/modify Godotcaml for yourself, or if you’re looking to write bindings to Godot in another language. Check out the previous post which was an announcement of what Godotcaml is and what it can currently do.

Organization

The file structure currently looks like this:

$ ls ./ocaml/godotcaml/apis
api_builtins.ml      api_types.ml  extension_api.json  foreign_utility_functions.ml  godotcaml.ml           uses_ppx_godot.ml
api_classes_intf.ml  bootstrap.ml  foreign_api.ml      gdextension_interface.h       hello-gdextension.c
api_classes.ml       conv.ml       foreign_base.ml     gdforeign.ml                  my_ocaml_extension.ml
api_helpers.ml       dune          foreign_methods.ml  gen_api.ml                    ppx_godot.ml

These can roughly be organized as follows:

• api_*.ml: Files generated by gen_api.exe
• extension_api.json and gdextension_interface.h: Files generated by the Godot 4.2 editor that detail the stock and builtin classes in the API. extension_api.json is parsed by gen_api.exe, in particular; if you rerun gen_api.exe with dune exec ./gen_api.exe, make sure to do it from this folder so it can find this file.
• godotcaml.ml: A partial translation of gdextension_interface.h to OCaml Ctypes-compatible typs and so-on.
• gen_api.ml: Source file for gen_api.exe; used to generate all the bindings to builtin and stock Godot functionality. Also creates foreign_methods.ml currently.
• foreign_*.ml and gdforeign.ml: Files partially generated, and partially hand written. foreign_base.ml is hand-written, the others are (either now, or soon to be) generated. foreign_base.ml also contains refs that are filled in to the values passed by Godot on extension startup. gdforeign.ml is just a module that includes them all for convenience’s sake. The others contain helper-functions that the api_*.ml files use of various flavours and arities for calling Godot functions and methods
• hello-gdextension.c: This is the C file that calls OCaml’s startup routines, passes the Godot-provided required pointers and API calling functions, and contains the actual entry symbol used by the extension.
• conv.ml is hand-written, and contains routines for converting BuiltinClasses into certain native OCaml types.
• bootstrap.ml: Contains the types and values of all global enumerations in a module called GlobalEnum0. This follows a formula used a lot in the bindings, where forward declarations are found in <ModuleName>0 while the <ModuleName> is reserved for the finalized user-exposed stuff.
• ppx_godot.ml: Contains the beginnings of the PPX helpers I’m writing to make binding to OCaml objects less annoying.
• uses_ppx_godot.ml: Contains the function called by hello-gdextension.c and is the “main file” on the OCaml side of the extension.
• dune: Build file.
• my_ocaml_extension.ml: Unsused; ignore.

So, putting that together, you run dune exec ./gen_api.exe, then ocamlformat -i *.ml to make them readable. This results in the following stats:

$ cloc ocaml/godotcaml/apis
      21 text files.
      21 unique files.                              
       1 file ignored.

github.com/AlDanial/cloc v 1.90  T=0.84 s (23.8 files/s, 643619.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JSON                             1              0              0         303153
OCaml                           17          40165          24259         169570
C/C++ Header                     1            271           1818            535
C                                1              6              0             44
-------------------------------------------------------------------------------
SUM:                            20          40442          26077         473302
-------------------------------------------------------------------------------

Which suggest codegen was probably a good idea! If you need to use a more recent version of Godot, or a custom compiled engine, simply run godot4 --dump-gdextension-interface && godot4 --dump-extension-api-with-docs using your godot binary, and (hopefully!) gen_api.exe will work with the new stuff.

Gotchas

One thing I had trouble with — and this is specific to OCaml: In hello-gdextension.c I need to pass pointers to OCamlland. To do so, I allocated “custom blocks” for each of the pointers, and interpreted them on the OCaml side as nativeints which I then converted back to pointers. However, I was using ocaml_alloc_tuple at first, and this allocates a number of OCaml values, while ocaml_alloc_custom_mem works in number of bytes. So I when I switched to custom, I was having memory corruption issues during minor GC from malloc failing because I was writing an 8-byte value into a 1-byte space! This took a while to debug.

Another thing that is a little tricky — and this is general to any Godot bindings: When constructing a new object of a given custom user class, the process is this:

• Allocate an object of the base class your user class inherits from with classdb_construct_object
• Set it’s new class to the actual class of your user class using object_set_instance, with private data being stored in the opaque class_instance_pointer (currently NULL for these bindings, but probably will change).
• Set the instance bindings using object_set_instance_binding with the callbacks being whatever you need (I just pass NULL for now, and I’ll think more on it later.)

Things to Fix Still

For some reason, virtual functions aren’t properly called. I will be investigating this in the next little while. This is quite important for ergonomics, since _process is called every frame, and just isn’t for custom OCaml classes, making you have to call it in the parent-object of the scene if you want to see it happen (or do it yourself in a script). This is far from ideal.

I have some funptr leaks that Ctypes is helpfully telling me about. Will investigate those later, but for now they just print to stderr and don’t break anything.

Thanks Again!

Thanks again for reading! I hope to tackle signals next after virtual function fixes, as this is the last major missing piece before you can really interact with the engine as intended.

Best,

Matt