What Type of ML Team Are They?

In established tech companies—though not exclusively in startups—you'll typically encounter three categories of ML teams: infrastructure, applied, and research.

• Infrastructure ML Teams focus on creating services that facilitate model development, which they then provide to other teams through APIs or user interfaces. Different teams may handle various aspects such as model services, feature engineering, or inference services. Key challenges for these teams include:
  • Scalability: How can services be expanded to accommodate the full range of ML models within the organization?
  • Efficiency: What strategies can be implemented to minimize the computational costs associated with training and deploying models?
  • Integration: How do we ensure that model predictions are seamlessly integrated into the product? What procedures are in place for managing errors when model inference fails?
  • Automation and Abstraction: In what ways can we simplify and automate the model development process, including building user-friendly tools for non-technical stakeholders?
• Applied ML Teams are responsible for designing, developing, testing, and iteratively refining ML models that address specific business challenges, utilizing tools from the infrastructure teams when appropriate. Their focus areas include:
  • Framing: How can we effectively translate a business challenge into an ML problem?
  • Data and Feature Discovery: What data is necessary to solve the problem? How do we ensure the reliability of labels and determine the appropriate features?
  • Experimentation and A/B Testing: Which models yield the best results for our specific use case?
  • Continuous Improvement: How frequently should we retrain models, and how can we enhance future versions with more features, improved data, or better model architectures?
• ML Research Teams are dedicated to developing innovative algorithms or model architectures, primarily aiming to publish their research in academic journals and conferences. They often pioneer groundbreaking advancements such as the ADAM optimizer or BERT. While much of their work may not see practical application, successful innovations can lead to significant breakthroughs in performance. Their inquiries typically involve:
  • How can we outperform existing benchmarks?
  • What scaling laws govern the performance of large neural networks?
  • What are the fundamental principles behind deep learning, and what limitations does it face?

Selecting a team that aligns with your desired ML focus is crucial. Generally, infrastructure teams attract individuals with software engineering backgrounds, while applied and research teams tend to draw those with academic qualifications, often PhDs, due to the experimental nature of their roles.

Lastly, steer clear of "pin factories," where model developers merely create model artifacts and pass them to engineers for deployment. This setup can lead to communication breakdowns, slowed iteration cycles, and unclear ownership of tasks.

What Is the Skill Gap?

“I absolutely know it is hard, but we’ll learn how to do it.” — Jeff Bezos

It’s rare to find a position that perfectly matches your skill set. Typically, there will always be a discrepancy between your existing abilities and those required for the role.

This creates a trade-off: a larger skill gap may offer more learning opportunities but will take longer to bridge, whereas a smaller gap allows for quicker contributions but less personal growth. Aim for a role with a skill gap that you feel confident addressing within a reasonable time frame, while also ensuring that you are still challenged and learning. For instance, if a new applied ML team employs the same technologies you currently use, it may not represent the best career progression.

When evaluating skill gaps, adopt a ‘growth mindset’—believing that you can acquire the necessary skills as you go, rather than a ‘fixed mindset’ that limits your potential.

What Do They Own and Who Are Their Customers?

Every ML team should have ownership of a specific domain and a clear customer base. For example:

• Infrastructure ML teams own services, serving either other infrastructure or applied teams.
• Applied ML teams own models, catering to the end-users of company applications.
• Research ML teams focus on specific research areas, typically serving other research teams, or, in exceptional cases, enabling practical applications through infrastructure and applied teams.

Be wary of teams that lack clear ownership or customer relationships. For instance, if an applied ML team doesn’t directly own the models but merely proposes ideas to other teams, their influence will be limited. Similarly, if an infrastructure team has minimal internal clients, it may raise questions about the necessity of their work.

Constantly assessing "What do they own, and who are their customers?" can serve as a valuable gauge when exploring new ML teams.

Conclusion

As you search for your next (or first) ML team, keep these three pivotal questions in mind:

1. What type of ML team are they—infra, applied, or research?
2. What is the skill gap, and are you prepared to tackle it? Are there new skills to acquire?
3. What do they own, and who are their customers? If ownership is unclear, how do they create value?

Remember that just as job roles and titles evolve, so too will your personal preferences. My journey began in ML research before transitioning to applied ML, where I found the opportunity to make tangible impacts more appealing than publishing academic papers. Others may prefer the stability of infrastructure roles over the unpredictability of applied work.

Ultimately, this is your career journey; prioritize your interests and aspirations to steer yourself toward success.

The first video, "Predicting the Winning Team with Machine Learning," explores how ML can be utilized to forecast outcomes in competitive environments.

The second video, "How to Get Machine Learning Right and Make Data Work Harder," discusses strategies for optimizing ML processes and improving data utility.