performance-moe-kernel-misalignm-runtimeerror-size-k-must-divisible-by-block-size-k-180862ea

THEN Pad the K dimension of input activations and weight tensors to be a multiple of BLOCK_SIZE_K before calling the MoE WNA16 GEMM kernel. For input activation tensor A, use torch.nn.functional.pad in the fused MoE kernel invocation. For weight tensors (B, B_scale, B_zp), pre-pad them offline or during model loading to avoid runtime overhead.

# Inside fused_moe.py, before calling the kernel:
# Assume A is [num_tokens, hidden_size], need K divisible by BLOCK_SIZE_K
BLOCK_SIZE_K = 64  # or appropriate value
pad_amount = (BLOCK_SIZE_K - (A.size(-1) % BLOCK_SIZE_K)) % BLOCK_SIZE_K
if pad_amount > 0:
    A = torch.nn.functional.pad(A, (0, pad_amount))