Principal Applied Scientist

Security represents the most critical priorities for our customers in a world awash in digital threats, regulatory scrutiny, and estate complexity. Microsoft Security aspires to make the world a safer place for all. We want to reshape security and empower every user, customer, and developer with a security cloud that protects them with end to end, simplified solutions. The Microsoft Security organization accelerates Microsoft's mission and bold ambitions to ensure that our company and industry is securing digital technology platforms, devices, and clouds in our customers' heterogeneous environments, as well as ensuring the security of our own internal estate. Our culture is centered on embracing a growth mindset, a theme of inspiring excellence, and encouraging teams and leaders to bring their best each day. In doing so, we create life-changing innovations that impact billions of lives around the world.

Our team builds and operates the end-to-end measurement stack for Microsoft Security's AI agents. We build and collaborate with industry on security domain benchmarks for foundation model comparisons, empowering product teams to build at the frontier edge of AI while driving progress in model research and development. We build rigorous evaluations of complete agentic systems, ensuring only the best solutions reach our customers. As a Principal Applied Scientist, you will define and drive the evaluation strategy for AI security agents across multiple products and orchestration frameworks. You will help set technical direction for how we measure agent capabilities at scale. You will also influence product roadmaps through evaluation insights, partner with product and engineering to translate results into readiness gates, model selection criteria, and go/no-go decisions that enable teams to ship, safe, reliable security agents.

Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

You will be part of an Applied Science evaluations team working on high-impact AI initiatives that define how security AI agents are measured, trusted, and shipped. You will lead technical direction for evaluation strategy, set standards adopted across teams, and mentor others in evaluation methodology.

Responsibilities include:

• Technical Leadership & Ownership: Define and drive technical strategy for security agent evaluations across Microsoft Security.
• Lead assessments of full agent systems, set evaluation standards adopted by multiple product groups, and translate findings into exec-ready recommendations and deployment quality gates
• Agentic System Evaluation & Measurement: Define evaluation methodology for agent actions in realistic security scenarios covering planning, tool use, reasoning, robustness, and failure modes.
• Establish metrics that directly influence product readiness and release decisions.
• Evaluation Infrastructure & Harnesses: Architect reusable evaluation platform and harnesses across orchestration frameworks.
• Enable controlled testing on production-representative data with appropriate privacy controls.
• Collect trajectories, telemetry, and results at scale
• Data & Signals for Evaluation: Set data standards and governance for agent evaluation datasets and grading rubrics.
• Ensure reproducibility, privacy-preserving data practices, and long-term reuse across evaluation scenarios.
• Research & Innovation: Collaborate with cross-functional teams to advance evaluation research, prototype novel methods, and land them into the evaluation platform used by product teams.
• Contribute to publications and external partnerships where appropriate.

Required/minimum qualifications

Applied Sciences IC5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.