Operator Fusion

• Some DirectML operators support a concept known as fusion. Operator fusion is a way to improve performance by merging one operator (typically, an activation function) into a different operator so that they are executed together without requiring a roundtrip to memory.
• https://arxiv.org/abs/2108.13342