etha.comm.get_chunks#

Build chunk descriptors from routes.

Functions#

calculate_chunk_shape(→ tuple[int, Ellipsis])

m2m_to_chunks(→ list[etha.comm.ir.Chunk])

Materialize an M2MMap (topology) onto local tensors into chunks.

prewarm_broadcast_groups(→ None)

Create every broadcast group across the given maps in one canonical pass.

Module Contents#

etha.comm.get_chunks.calculate_chunk_shape(num_slicers: list[int], tensor_shape: tuple[int, Ellipsis] | None) → tuple[int, Ellipsis]#
etha.comm.get_chunks.m2m_to_chunks(m2m: etha.comm.ir.M2MMap, rank: int, source_tensor: torch.Tensor | None = None, target_tensor: torch.Tensor | None = None, transfer_dtype: torch.dtype | None = None, source_partial_groups: list[tuple[torch.distributed.ProcessGroup, str]] | None = None) → list[etha.comm.ir.Chunk]#

Materialize an M2MMap (topology) onto local tensors into chunks.

etha.comm.get_chunks.prewarm_broadcast_groups(m2m_maps: collections.abc.Iterable[etha.comm.ir.M2MMap | None]) → None#

Create every broadcast group across the given maps in one canonical pass.

dist.new_group is collective on WORLD, so every rank must issue the same sequence of calls. m2m_to_chunks creates a direction’s groups on first touch, and the two sides of a pair materialize chunks in opposite direction orders (each walks local-send before local-recv), so per-direction first-touch creation interleaves the calls differently on the two sides once both directions broadcast — silently cross-wiring the communicators. Route tables are identical on both sides (merged via all_gather_object in get_m2m_map), so this union-then-sort pass runs identically everywhere and turns the later per-direction creations into cache hits.