Embedding

Kopf is designed to be embeddable into other applications that require watching Kubernetes resources (custom or built-in) and handling their changes. This can be used, for example, in desktop applications or web APIs/UIs to keep the state of the cluster and its resources in memory.

Manual execution

Since Kopf is fully asynchronous, the best way to run Kopf is to provide an event-loop in a separate thread, which is dedicated to Kopf, while running the main application in the main thread:

import asyncio
import kopf
import threading
from typing import Any

@kopf.on.create('kopfexamples')
def create_fn(**_: Any) -> None:
    pass

def kopf_thread() -> None:
    asyncio.run(kopf.operator())

def main() -> None:
    thread = threading.Thread(target=kopf_thread)
    thread.start()
    # ...
    thread.join()

In the case of kopf run, the main application is Kopf itself, so its event-loop runs in the main thread.

Note

When an asyncio task runs outside the main thread, it cannot set OS signal handlers, so the developer should implement termination themselves (cancelling an operator task is enough).

Manual orchestration

Alternatively, a developer can orchestrate the operator’s tasks and sub-tasks themselves. The example above is equivalent to the following:

def kopf_thread() -> None:
    loop = asyncio.get_event_loop_policy().get_event_loop()
    tasks = loop.run_until_complete(kopf.spawn_tasks())
    loop.run_until_complete(kopf.run_tasks(tasks, return_when=asyncio.FIRST_COMPLETED))

Or, if proper cancellation and termination are not required, of the following:

def kopf_thread() -> None:
    loop = asyncio.get_event_loop_policy().get_event_loop()
    tasks = loop.run_until_complete(kopf.spawn_tasks())
    loop.run_until_complete(asyncio.wait(tasks))

In all cases, make sure that asyncio event loops are used properly. Specifically, asyncio.run() creates and finalises a new event loop for a single call. Multiple calls cannot share the same coroutines and tasks. To make multiple calls, either create a new event loop or get the event loop of the current asyncio _context_ (by default, that of the current thread). See more on asyncio event loops and _contexts_ in Asyncio Policies.

Custom event loops

Kopf can run in any AsyncIO-compatible event loop. For example, uvloop claims to be 2x–2.5x faster than asyncio. To run Kopf in uvloop, call it this way:

import kopf
import uvloop

def main() -> None:
    loop = uvloop.EventLoopPolicy().get_event_loop()
    loop.run(kopf.operator())

Or this way:

import kopf
import uvloop

def main() -> None:
    kopf.run(loop=uvloop.EventLoopPolicy().new_event_loop())

Or this way:

import kopf
import uvloop

def main() -> None:
    asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
    kopf.run()

Or any other way the event loop prescribes in its documentation.

Kopf’s CLI (i.e. kopf run) uses uvloop by default if it is installed. To disable this implicit behavior, either uninstall uvloop from Kopf’s environment, or run Kopf explicitly from the code using the standard event loop.

For convenience, Kopf can be installed as pip install kopf[uvloop] to enable this mode automatically.

Kopf never implicitly activates custom event loops when called from the code, only from the CLI.

Multiple operators

Kopf can handle multiple resources at a time, so a single instance is sufficient for most cases. However, it may sometimes be necessary to run multiple isolated operators in the same process.

It should be safe to run multiple operators in multiple isolated event loops. Although Kopf’s routines use global state, all such global state is stored in contextvars containers with values isolated per-loop and per-task.

import asyncio
import kopf
import threading
from typing import Any

registry = kopf.OperatorRegistry()

@kopf.on.create('kopfexamples', registry=registry)
def create_fn(**_: Any) -> None:
    pass

def kopf_thread() -> None:
    asyncio.run(kopf.operator(
        registry=registry,
    ))

def main() -> None:
    thread = threading.Thread(target=kopf_thread)
    thread.start()
    # ...
    thread.join()

Warning

It is not recommended to run Kopf in the same event loop as other routines or applications: it considers all tasks in the event loop as spawned by its workers and handlers, and cancels them when it exits.

There are some basic safety measures to avoid cancelling tasks that existed before the operator’s startup, but these cannot be applied to tasks spawned later due to asyncio implementation details.