Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary high-efficiency serving platform. The Pre-training Research Engineer will implement, scale, and improve byte-native and multimodal foundation models while developing production-grade training code and infrastructure. The role focuses on model architecture, training objectives, optimization, scaling experiments, and analysis of model behavior and quality.
Responsibilities
- Train large byte-native and multimodal foundation models across massive, heterogeneous corpora
- Implement and evaluate new model architectures, training objectives, and optimization methods
- Develop stable pre-training recipes and run scaling experiments for novel architectures
- Conduct ablations and analyze training dynamics, model behavior, and base-model quality
- Work with data and distributed training engineers to improve training efficiency, reliability, and scalability