TikTok | AI Infra Intern | Onsite Round 1

Kernel Optimization

1. Do you have experience with kernel-level optimization, such as using CuTe DSL or writing custom CUDA kernels for operator fusion? Walk through a specific example.

2. When implementing kernel fusion, what approach or framework would you typically choose, and why?

3. Have you ever seen performance get worse after kernel fusion? What caused the slowdown?

4. How does warp specialization on NVIDIA Hopper GPUs work? How is it implemented at the hardware/software level?

5. If warp specialization is removed while keeping optimizations such as tiling and shared memory, where would the main performance loss come from?

Inference Optimization

1. Do you have hands-on experience optimizing MoE inference, especially in scenarios involving reinforcement learning (RL)?

2. How can you tell whether an MoE model has actually learned meaningful expert specialization, rather than simply behaving like a dense model split across multiple experts?

3. In RL + MoE training, have you encountered routing collapse, where the reward signal causes most tokens or requests to be routed to only a small number of experts? How would you address this?

Project & Internship Experience

1. Walk through the most relevant project experience, including the technical challenges and contributions.

2. Walk through the internship experience, especially any optimization work done on production inference services.

3. Have you experimented with using an AI Agent to automatically generate CUDA kernels? If so, what was the overall approach?