TikTok | AI Infra Internship Onsite Round 1

Topics covered :)

Basic Knowledge

1. Explain the main idea behind DeepSpeed and what it is mainly used for.
2. Why use ZeRO Stage 3? Explain how it saves GPU memory and derive the memory-saving formula.
3. Besides DeepSpeed, what other distributed training frameworks are commonly used? What are the main differences between them?
4. What are the main differences between BLIP and BLIP-2? What architecture do most modern multimodal models use?
5. How does Q-Former work, and what is its main purpose?

Training Optimization

1. What is a typical technical approach for content understanding tasks? For a chart-to-story model, how should the training data be prepared?
   Project Experience
2. Introduce a key project and explain the main work involved.

Coding

1. Implement an integer square root function (LeetCode 69) and discuss possible ways to optimize the solution.