TikTok | AI Infra Intern Onsite :)

Interview topics:

• Project and inference-framework discussion.
• How to configure FP8 KV Cache.
• Whether FP8 produces a gain during prefill.
• Profiling tools and bottleneck analysis.
• GPU microarchitecture and CUDA.
• Why increasing matrix dimensions can increase latency even when SM utilization is not full.
• Other categories of GPU or serving bottlenecks.
• Number of Islands