Defining the class dynamically in Python

The dynamic definition is a term that does not often occur on the web. The reason is in a rare usage of this programming technique in a traditional workflow. Simultaneously, it might expose Python’s internal structure, especially Data Types, and might help understand the last one.

Python’s runtime overview

Before demonstrating the mentioned technique, let consider a traditional definition:

class MyClass:
pass

We could call this kind of definitions static because the interpreter executes them during the runtime. Here are the steps are done by the interpreter during such execution in outline:

  1. The interpreter checks the syntax of the source code.
  2. Then translates it to the byte code.
  3. Finally, executes it on Python’s Virtual Machine (PVM).

As the static definition is already in the source code, — class MyClass is already defined — the interpreter executes it with all the attributes, and the class becomes accessible to other objects running in the same runtime.

There is nothing extraordinary here — this is how the interpreter runs any program.

On the dynamic definitions

Now let compare the same procedure for source code with objects defined dynamically:

  1. The interpreter checks the syntax of the source code.
  2. Then translates the source into the byte code.
  3. Executes the byte code on the PVM.
  4. Defines an objects

Now interpreter takes an additional step during an execution. At this step, it defines an object by itself instead of us statically defining them with definition keywords, like in the example above.

When the source code contains both statically and dynamically defined objects, the interpreter applies these algorithms according to the definition type.

Resources for dynamic definitions

There are several resources provided by the standard library for this technique to be applicable. Please, note that they are not used often in the traditional workflow:

  • type — the metaclass that defines Data Types.
  • Compile — the function to execute some other code from the current runtime.
  • Types — is the standard library’s module that implements a built-in type.

The type metaclass

The standard way of using the type metaclass is to pass some object to it:

h = "hello"
type(h)
<class 'str'>

It then returns a type of an object. However, there is another initialisation method signature for this metaclass that says clear enough that with all the arguments provided it will return a new type:

type(name, bases, dict) -> a new type

Initialisation parameters

  • Name — is the name of the new type.
  • Bases — parent classes for a new type.
  • dict — a dictionary containing the classes’ attributes.

A «new type» is the same class resulting from the execution of a static class definition. The type metaclass calls it «type» because it acts as a type for instances of its own, like all classes in Python, do. All the classes in Python have a type metaclass as their type:

builtins = [list, dict, tuple]
for obj in builtins:
type(obj)
<class 'type'>
<class 'type'>
<class 'type'>

As the type metaclass implements a type for all other classes, they are getting it as their type. Nevertheless, they do not inherit a type metaclass directly; instead, they inherit an object base type. The object type is a base class for the type metaclass:

type.__bases__
(<class 'object'>,)

Furthermore, the type metaclass inherits an object base type along with all other classes. While this concept might be as clear as mud, pay attention to the following diagram.

There are two arrow sequences: the red one represents an inheritance, and the blue one is for the typing. Each chain has an endpoint, which completes in a loop. Inheritance ends on the object base type. That means an object base type does not inherit any class, while still being a base class for all other classes:

object.__bases__
()

Empty tuple means there are no base classes for an object base type.

The typing chain ends on the type metaclass. That makes the type metaclass a type of itself:

type(type)
<class 'type'>

While the instances of any class have a class as their type:

builtins = [list(), dict(), tuple()]
for obj in builtins:
type(obj)
<class 'list'>
<class 'dict'>
<class 'tuple'>

Moreover, when called with a single argument signature, the type metaclass returns a reference to the specific class stored in the memory.

Outcomes

  • The type metaclass implements a type for all other classes.
  • Instances are getting the reference to the class as their type.
  • Class is a custom defined type.

Dynamic definition recipe

With the above theory in mind, let me define a class dynamically:

MyClass = type("MyClass", (object, ), dict())
MyClass
<class '__main__.MyClass'>

The new class provides the same behaviour as the one defined statically. For example, I can instantiate it:

m = MyClass()
m
<__main__.MyClass object at 0x7f8b1d69acc0>

Passing attributes

There is no much sense in having an empty class, so let me add an attribute in the same definition by passing the values to the dict parameter:

MyClass = type("MyClass", (object, ), dict(foo="bar")
m = MyClass()
m.foo
'bar'

Passing the methods

The dict parameter can also accept callables, which are objects with a __call__ special method, for example, functions. One could define the method for MyClass this way:

def foo(self):
return "bar"
MyClass = type("MyClass", (object, ), dict(foo=foo))
m = MyClass()
m.foo
'bar'

However, the code then loses a piece of dynamism because the method I want to pass to the class has to be defined statically before the class definition. Besides using the self argument out of the class looks weird and might be considered as a bad practice. Instead, one could use the compile function to define and execute this method preliminarily:

code = compile('def foo(self): return "bar", "my_method", "exec")

The compile is a standard library’s function that compiles the source code into the object inside the current runtime.

Parameters

  • The code is an object that might be a string or the reference to the module.
  • The filename is the name of the file where the compile will store the compiled code.
  • The mode parameter specifies whether compile would create a module to store the compiled code in or it will execute it like the single object of the current runtime.

The above compile call will result in the code object:

type(code)
<class 'code'>

This object is a kind of middleware in the current runtime. It contains the code of the method now, but it is not related to the class yet, and it is not a method, however — it is treated as «just a code» by the interpreter. To connect this code object with a class, one has to transform it into the method type with a types module of a standard library:

from types import FunctionType, MethodType

A couple of words to say about the transformation procedure:

  1. Transform the code object to the function type using the FunctionType class.
  2. Transform the function type to the method type with a MethodType class.

The reason of performing the transformation of the code object in the two distinct steps instead of transforming it to the method type at once is that the initialisation method of the MethodType class accepts a function type object as the argument:

|  method(function, instance)

That makes it impossible to strictly pass the code object to the initialisation method of the MethodType class. At first, one should transform it into the function type.

That makes sense with keeping in mind that the method is just a function defined in the class.

So, let me start from the first step:

function = FunctionType(code.co_consts[0], globals(), "foo")

Parameters

  • The code is a code object. Here it accepts the one I have got from the compile function call. To say, it fits perfectly here.
  • Globals is the dictionary that accepts the global variables and passes them into the resulting function.
  • Name — the custom name for the resulting function.

What about co_consts[0], is the descriptor of the code object, which is a tuple containing the constants from it. Try to think of a code object as of some module with the only function defined. 0 — is the index of this function as it is a single constant in that module.

With all these parameters provided, the call above will result in the object of a function type:

function
<function foo at 0x7fc79cb5ed90>
type(function) <class 'function'>

Now let proceed to the second step to transform this object into the method type:

MyClass.foo = MethodType(function, MyClass)

After all, this one might be so simple that it does not require an explanation. However, it finally transforms the function type to the method of the MyClass, which is theretofore, was defined empty.

Let call this method now:

m = MyClass()
m.foo()
bar

Works the same as with a static definition!

Conclusion

As mentioned at the beginning, the concept of the dynamic definition is not a kind of technique to be used on regular purposes. To be more specific, here is the link to the educational project somehow related to the topic:

Please, note that it still under construction. However, take a look at this method:

This method creates a new class with methods getting their names during the runtime. Thus makes it possible to generate an API class without having to define it manually.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store