Abstract Modules

In the past couple sections we've worked through examples of some fairly complex Build time metaprogramming to generate Modules that share some common behaviors between them. Having those low-level Build tools in your back pocket is something that may very well come in handy during your Claro development journey. However, in general, it's worth acknowledging that there's some inherent complexity in the prior approaches. It's my hope that the community will standardize around some well-defined set of Build design patterns that are encoded into well-known, standardized interfaces (Bazel Macros/Rules) to abstract away the low-level complexity underneath.

In this section, I'll try to demonstrate what such a standardized encoding might look like for the design pattern demonstrated in the prior sections.

Limitations of the Approach in the Prior Sections

The Animal(...) Macro defined in the previous sections was extremely rigid. It encoded exactly one specific code structure. It was arguably a very useful structure, but if we wanted to create an Animal(...) that deviated even slightly from the expected structure, you'd either have to go and refactor the Macro definition itself and all usages to add support for new behaviors, or you'd just have to fall back to manually defining a Module, losing all Build level code sharing that you were trying to achieve with the standardized Animal(...) Macro.

All that said, the biggest limitation of the approach in the prior sections is that it was bespoke. While all the customizability that Build time metaprogramming gives you blows the design space wide open, it also makes it that much harder for anyone unfamiliar with the domain to follow what's going on.

Abstracting Away the "Abstract Module" Pattern Itself

Arguably, the Animal(...) macro from the previous sections could be described as an encoding of an "Abstract Module" (in a sense vaguely similar to Java's "Abstract Classes" - minus the object-orientation). "Abstract" in the sense that some portions of all "Animal" Modules are known before even knowing the "concrete Animal" Modules that you'll specifically build later on. But there's nothing about this concept itself that's unique to "Animals". All sorts of categories of similar Modules can be imagined, and they could potentially all benefit from a similar "Abstract" base encoding that later gets specialized for each concrete Module.

Largely as a draft demonstration of what a standardized encoding of this "Abstract Module" design pattern could look like, Claro provides a claro_abstract_module(...) Bazel Macro. Now, the Animal(...) Macro can be auto-generated in a few lines by simply calling the claro_abstract_module(...) Macro.

Fig 1:

# animal.bzl
load(
    "@claro-lang//stdlib/utils/abstract_modules:abstract_module.bzl",
    "claro_abstract_module",
)

Animal = \
    claro_abstract_module(
        name = "Animal",
        module_api_file = "animal.claro_module_api",
        overridable_srcs = {
            "AnimalSoundsImpl": ":default_animal_sounds_impl.claro",
            "InternalStateAndConstructor": ":default_internal_state.claro",
            "MakeNoiseImpl": ":default_make_noise_impl.claro",
        },
        default_deps = {
            "AnimalSounds": ":animal_sounds",
        },
        default_exports = ["AnimalSounds"],
    )

Override Flexibility

On top of being a standardized encoding of this design pattern, "Abstract Modules" provide an additional mechanism for various components of the Module to be override-able. In the Animal = claro_abstract_module(...) declaration above, the overridable_srcs = {...} arg lists a few different named components that have default implementations provided as .claro source files that can be optionally overridden by any concrete Animal(...) usages. For the sake of demonstration, the "Abstract Animal Module" has been decomposed into a relatively fine granularity, allowing significant customization to downstream users of the Macro.

So now the Animal(...) macro can be used very similarly as in the previous sections, but with some slightly different arguments:

Fig 2:

# BUILD
load(":animal.bzl", "Animal")

[
    Animal(
        name = animal,
        api_extensions = [":{0}_cons.claro_module_api".format(animal)],
        override = {
            "InternalStateAndConstructor": ":{0}_state_and_cons.claro".format(animal),
            "MakeNoiseImpl": ":{0}_noise.claro".format(animal),
        },
    )
    for animal in ["dog", "cat"]
]

# ...

The first notable detail is that the idea of extending Module APIs is now encoded directly into the "Abstract Module" Macros returned by claro_abstract_module(...) in the form of the api_extensions = [...] parameter. So now, we didn't need to manually concatenate api files using a Bazel genrule(...) as we did in the prior sections. Then, notice that the concrete cat and dog Animal Modules now implicitly inherit the default AnimalSoundsImpl implementation, while explicitly overriding InternalStateAndConstructor and MakeNoiseImpl with custom implementations. Now, these Module definitions can be used exactly the same as they were when defined using the approach(es) from the prior sections.

As one final motivating example, to demonstrate something that this new Animal(...) implementation can do that the prior implementation(s) couldn't, we can also define a new Animal Module that overrides the default AnimalSounds Contract implementation, by overriding AnimalSoundsImpl:

Fig 3:

# platypus_animal_sounds_impl.claro
implement AnimalSounds::AnimalSounds<State> {
  function makeNoise(state: State) -> string {
    var name = unwrap(state).name;
    if (unwrap(unwrap(state).internal).isWearingFedora) { # https://youtu.be/KFssdwb7dF8?si=Omgf1-D3qIBY6jO9
      var codename = name[0];
      return "!!!!!!!!!! REDACTED TOP SECRET MESSAGE !!!!!!!!!! - Agent {codename}";
    }
    return "Chirp Chirp - says {name}";
  }
}

Fig 4:

# BUILD
load(":animal.bzl", "Animal")

# ...

Animal(
    name = "platypus",
    api_extensions = [":platypus_cons.claro_module_api"],
    override = {
        "AnimalSoundsImpl": ":platypus_animal_sounds_impl.claro",
        "InternalStateAndConstructor": ":platypus_state_and_cons.claro",
    },
)

And now, our demo program can start use the platypus Module just as it was using the dog and cat Modules previously:

Fig 5:

# BUILD
load("@claro-lang//:rules.bzl", "claro_module", "claro_binary")

# ...

claro_binary(
    name = "animals_example",
    main_file = "animals_example.claro",
    deps = {
        "AnimalSounds": ":animal_sounds",
        "Cat": ":cat",
        "Dog": ":dog",
        "Platypus": ":platypus",
    },
)

Fig 6:

# animals_example.claro
var animals: [oneof<Cat::State, Dog::State, Platypus::State>] = [
    Dog::create("Milo", true),
    Dog::create("Fido", false),
    Cat::create("Garfield", "This is worse than Monday morning."),
    Platypus::create("Perry", false),
    Platypus::create("Perry", true)
  ];

for (animal in animals) {
  print(AnimalSounds::AnimalSounds::makeNoise(animal));
}

Output:

[0.002s][warning][perf,memops] Cannot use file /tmp/hsperfdata_runner/6 because it is locked by another process (errno = 11)
Woof! - says Milo
Grrrr... - says Fido
This is worse than Monday morning. - says Garfield
Chirp Chirp - says Perry
!!!!!!!!!! REDACTED TOP SECRET MESSAGE !!!!!!!!!! - Agent P

Additional Knobs & Implementation Details

The point of this section is really to demonstrate some possibilities available to all Claro users interested in writing Bazel Macros to encode relatively complex design patterns. And, I think we can agree that being able to hand-roll the very concept of inheritance without having to make a single change to the Claro compiler itself is a rather powerful capability!

But to say it once more, this is all meant as a demonstration, rather than encouragement of specific usage of this claro_abstract_module(...) Macro. So, we won't dive any further into the implementation details of how this prototype works, and we won't even go into the full range of capabilities that this prototype currently supports. However, if you're sufficiently interested that you really wanted to know more, feel free to check out the implementation yourself! You'll probably learn a lot about Bazel in the process of reading through it, so it could be enlightening.