etha.comm.get_chunks#
Build chunk descriptors from routes.
Functions#
|
|
|
Materialize an |
|
Create every broadcast group across the given maps in one canonical pass. |
Module Contents#
- etha.comm.get_chunks.calculate_chunk_shape(num_slicers: list[int], tensor_shape: tuple[int, Ellipsis] | None) tuple[int, Ellipsis]#
- etha.comm.get_chunks.m2m_to_chunks(m2m: etha.comm.ir.M2MMap, rank: int, source_tensor: torch.Tensor | None = None, target_tensor: torch.Tensor | None = None, transfer_dtype: torch.dtype | None = None, source_partial_groups: list[tuple[torch.distributed.ProcessGroup, str]] | None = None) list[etha.comm.ir.Chunk]#
Materialize an
M2MMap(topology) onto local tensors into chunks.
- etha.comm.get_chunks.prewarm_broadcast_groups(m2m_maps: collections.abc.Iterable[etha.comm.ir.M2MMap | None]) None#
Create every broadcast group across the given maps in one canonical pass.
dist.new_groupis collective on WORLD, so every rank must issue the same sequence of calls.m2m_to_chunkscreates a direction’s groups on first touch, and the two sides of a pair materialize chunks in opposite direction orders (each walks local-send before local-recv), so per-direction first-touch creation interleaves the calls differently on the two sides once both directions broadcast — silently cross-wiring the communicators. Route tables are identical on both sides (merged viaall_gather_objectinget_m2m_map), so this union-then-sort pass runs identically everywhere and turns the later per-direction creations into cache hits.