performance-tensor-parallelism-a-when-using-tensor-parallelism-e-g-tp-4-8-with-awq--34df8bd4

THEN Pad the K dimension of input activation tensor A and weight tensors (B, B_scale, B_zp) to be a multiple of BLOCK_SIZE_K (typically 128) before calling the moe_wna16_gemm kernel. For weights, perform padding once at load time to avoid dynamic overhead. For activations, use torch.nn.functional.pad dynamically in the fused MoE kernel call.

# In fused_moe.py before invoking moe_wna16_gemm:
BLOCK_SIZE_K = 128
pad_size = (BLOCK_SIZE_K - A.shape[-1] % BLOCK_SIZE_K) % BLOCK_SIZE_K
if pad_size:
    A = torch.nn.functional.pad(A, (0, pad_size))
# Similarly pad weight tensors at load time