TikTok | AI Infra Intern | Onsite Interview

Topics covered:

• Project deep dive.
• Explain Python's Global Interpreter Lock.
• Compare Python inter-process communication methods.
• Derive AllReduce communication overhead.
• Compare tensor.view with contiguous tensors.
• Compare row-parallel and column-parallel tensor parallelism.
• Implement fast exponentiation.
• Explain graph fusion.