Configuring methods of classes (especially: wrapped by external_configurable)

[SlapDrone]

Hey folks,

When I want to configure individual methods of a class, i.e. to write in config.gin something like:

Foo.bar.x = 10

If i'm not mistaken we need two layers of configurable decorators on our class, like so:

@gin.configurable
@dataclass
class Foo:
    x: float = 5.

    @gin.configurable(module="Foo")
    def bar(self, y: float):
        print(y)

    @classmethod
    @gin.configurable(module="Foo")
    def baz(cls, xstr: str):
        return cls(float(xstr))

This is fine, I guess the interpretation of the top-level decorator is that we only want to configure the init, and want to be explicit about which individual methods are configurable.

How do we tackle binding to method arguments on an external_configurable? In particular, I want to do something in config.gin like:

import llama_index
import mymodule.gin # inside I import ServiceContext and run gin.external_configurable on it

# ...
llama_index.ServiceContext.from_defaults.llm = %llm

Do I need to monkey patch gin-decorated versions of each method in the class from the external library?

Thanks for any advice or thoughts. Absolutely love this library btw.

[SlapDrone]

So, I've hacked something together that makes my use case work:

import logging
import inspect
from copy import deepcopy
from functools import wraps
from typing import Type, Optional, List, Union


def classmethod_from_func(func):
    @wraps(func)
    def wrapper(cls, *args, **kwargs):
        return func(*args, **kwargs)
    return classmethod(wrapper)


def make_configurable(cls: Type, method_names: Optional[List[str]] = None, module: Optional[str] = None) -> Type:
    """
    Creates a new class that inherits from the given class and makes the specified methods configurable.

    Args:
        cls: The class to make configurable.
        method_names: A list of method names to make configurable. If None, all methods are made configurable.
        module: The module name to use for the configurable methods. If None, the name of the original class is used.

    Returns:
        A new class that inherits from the given class and has the specified methods made configurable.

    Raises:
        ValueError: If a method name in method_names does not correspond to a method in the class.
    """
    # Create a dictionary to hold the new methods
    attrs = {}

    # Get all methods of the class
    for name, method in inspect.getmembers(cls, predicate=inspect.isroutine):
        # If method_names is None, wrap all methods, otherwise only wrap methods in method_names
        if method_names is None or name in method_names:
            # Check if the method is a class method
            is_classmethod = isinstance(cls.__dict__.get(name), classmethod)

            # Create a new method that is a configurable version of the original method
            if is_classmethod:
                # If the method is a class method, we need to wrap it with gin.configurable
                # without adding 'cls' as the first argument
                new_method = gin.configurable(name, module=module or cls.__name__)(method)
                new_method = classmethod_from_func(new_method)
            else:
                new_method = gin.configurable(name, module=module or cls.__name__)(method)

            # Add the new method to the attrs dictionary
            attrs[name] = new_method

    # If method_names is not None, check that all method names correspond to methods in the class
    if method_names is not None:
        for name in method_names:
            if not hasattr(cls, name):
                raise ValueError(f"Class {cls.__name__} has no method named {name}")

    # Create a new class with the same name as the original class, inheriting from the original class,
    # and with the new methods
    Configurable = type(cls.__name__, (cls,), attrs)

    # Copy class variables from the original class to the new class
    for name, value in inspect.getmembers(cls, lambda x: not(inspect.isroutine(x))):
        if not name.startswith('__'):
            setattr(Configurable, name, deepcopy(value))

    # Note: The new class shares the class variables of the original class. 
    # If the original class has mutable class variables and you modify them in the new class, 
    # the changes will also affect the original class. 
    # If you need the new class to have completely independent state from the original class, 
    # you might need to rethink your design or use a different approach.

    return Configurable

Which allows me to do, in my library code:

ServiceContext = make_configurable(llama_index.ServiceContext)

And just import this version where I need it, rather than the original class from the external library. By default this make_configurable will wrap all the methods, but one can control which are included by providing optional arguments.

This is pretty barbaric, but it works. I'd be really interested to hear from you core devs @dhr @gmrukwa @sguada @yilei if something like this could be beaten into a shape that is consistent with the design philosophy and goals of the package. Would be happy to do a PR if so.