When AI makes building easy, product judgment becomes the work

Code is easier to write. Prototypes are easier to build. A team can still struggle to answer a more consequential question: what should it make next?

In his conversation with Tulsee Doshi, Lenny Rachitsky observes that cheaper execution has also produced more things people do not need. Identifying worthwhile problems, defining a good outcome, and aligning a team around it are receiving renewed attention.

Doshi, Google's head of product for frontier AI models, agrees while acknowledging her own bias as a product manager. Her argument is that as execution gets easier, choosing the work, setting its quality bar, and building agreement account for more of the value.

That is a product leader's observation, not evidence of changing employment or salaries. The practices behind it are concrete: use models directly, turn quality judgments into evaluations, give requirements substance through prototypes and data, and leave room for capabilities that are still developing.

The model itself has a user experience

Traditional product planning can encourage a straightforward expectation: define the goals, then have an engineering team implement them. Model development adds another layer of uncertainty.

Doshi compares it to Snape's potions class. Teams choose training ingredients deliberately, set capability goals, and monitor runs and metrics. The resulting combination of abilities can still reveal unexpected strengths.

A model PM therefore has work on both sides of training. The role involves helping determine the direction, then exploring what the resulting model actually does well. A higher capability score still needs to be connected to a useful product experience.

That experience includes behavior. How clearly does the model explain its actions? When does it ask questions? How does it communicate uncertainty? Does its response suit the task? Those everyday interactions are product-design decisions too.

In the process Doshi describes, user feedback informs behavior specifications, which become evaluation rubrics, training data, and inputs to reward modeling. Product judgment has to reach training and evaluation to shape the experience consistently.

Google wants frontier capabilities to serve different users

Doshi is explicit about Google's ambition to compete at the frontier. She points to Gemini 4 Argon as an example of that direction. Whether a model leads still depends on the task and evidence; a vendor's positioning does not establish it.

She also describes Google's reach across hardware, models, and applications. Search users, software engineers, and cyber defenders need different combinations of capability, cost, and interaction. The product portfolio is intended to serve that range.

Interfaces should reflect those differences. AI in search needs to be approachable for a broad audience. Antigravity can accommodate people comfortable with terminals and development environments. Flow and NotebookLM illustrate experimentation with forms suited to different tasks.

Some of that work remains unfinished. Rachitsky asks why other assistants can build compelling experiences on Gmail, calendars, and documents while Google's own personal-assistant offering still leaves room for improvement. Doshi acknowledges the work ahead and expresses enthusiasm, without giving a release date.

Her optimism about safety is conditional too: monitor model behavior, intervene when it diverges from intended actions, and use AI to strengthen existing defenses. She cites Wiz's Scan for Good, whose public report also describes hospital exposures and collaborative remediation. These are specific defensive efforts, not proof that model risk has been solved.

Start with 5 prompts and define what good means

When hiring research PMs, Doshi values people who use models themselves, maintain their own test cases, and form opinions grounded in the outputs and data.

The conversation calls this an understanding of a model's “vibes.” In practice, it comes from repeated use: choose familiar work that matters, give different models the same task, and observe where each succeeds or loses reliability.

An effortless task may reveal little difference. An impossible one may reveal little progress. Doshi recommends work where models show promise but still fall short of the evaluator's standard. Writing, visual creation, and difficult analysis can all work when the evaluator knows enough to judge the result.

Formal evaluation can start small. Write 5 prompts, rate the responses personally, and build toward roughly 20 examples that expose meaningful differences between models.

For multi-step, multi-turn work, even defining a good result can be difficult. A small set makes a PM explain the standard: what succeeded, where a failure occurred, and why the distinction matters to a user. That creates a basis for scaling the evaluation.

These counts are starting points. A handful of successes cannot establish reliability across every situation. They can help turn an indistinct preference into a standard that others can discuss and test again.

Evaluate the people represented and the whole experience

Doshi uses skin-tone evaluation to illustrate how a measurement can miss the people a product serves. During her responsible-AI work, the team found limitations in using the 6-category Fitzpatrick classification, originally developed for medical purposes, to represent a global population.

Her team worked with Harvard scholar Ellis Monk to incorporate the 10-tone Monk Skin Tone Scale into evaluations, then use those evaluations to improve experiences including Pixel photography and image generation. Google's research account documents the scale's design background.

The product implication is that an evaluation's categories shape which failures become visible. People poorly represented in the sample can experience persistent failures without moving an aggregate score much. That is an inference from the example; adopting a different scale still requires checking actual outcomes.

Other failures arise outside the model.

Is the judge reliable? Does the test include the filters and safeguards used in production? Does serving change the result? Doshi treats these as part of the evaluation problem.

Audio makes the issue tangible. Correct content can still sound unnatural. Tone, voice quality, inflection, and fluency can change in delivery. The object of evaluation is the experience a user receives. A model score is one piece of evidence within it.

Use 60/20/20 to leave room for growth

Models keep changing, so a product portfolio needs both present value and room for emerging capabilities. Doshi describes an approximate 60/20/20 split:

• 60% works now and gives the current prototype and product a functional foundation.
• 20% is imperfect but already useful enough to show a plausible path to near-term improvement.
• 20% remains exploratory, with early signs that a breakthrough could enable a compelling experience.

She introduces the numbers as adjustable. They describe a portfolio of product exploration, not a standard permitting unreliable functionality to be shipped, or a fixed allocation of team hours.

Rachitsky contrasts this with the common advice to build for models 3 months ahead. Doshi says that even from inside a model team, predicting exactly what will be possible at that point is difficult.

More concrete evidence comes from the product: what works today, what is close, and what has shown only partial success. Teams can use those distinctions to allocate effort and revisit the assumptions when a new model arrives.

The approach preserves exploration while requiring a useful foundation now. Model progress creates opportunities; the product team still has to validate them and make choices.

Revisit the system around an improving model

Teams often accumulate prompts, tools, procedures, and recovery rules around a model's limitations. As the model improves, that surrounding design needs another look.

Doshi describes a harness as the model's operating environment: system instructions, tools, skills, context, and accessible information. It determines what the model can use, what it can do, and how it acts.

An overly rigid structure can hold a new model to an older model's limits. Her advice is to keep retesting, identify the rules that still help, and find the ones that have become unnecessary complexity. Changes should follow evidence from those tests.

Task organization deserves similar attention. Doshi contributed to Stanford's Flash Teams research, published in 2014. It studied how people with different expertise could be assembled temporarily and coordinated through tasks and handoffs.

She sees principles from that work that could inform agent collaboration. The original subjects were human experts. The research offers an organizational analogy, not direct evidence for the effectiveness or safety of present-day multi-agent systems.

Make room to practice and reduce unproductive work

Buying access to a tool does not automatically give a team the ability to use it well.

Doshi describes PMs building skills for everyday work and sharing them with colleagues. Others become adept at inspecting data or longer model trajectories and can help teammates practice. Individual discoveries can accumulate into shared tools and working methods.

The constraint is time. People continually rushing toward delivery have little space to learn. At the time of recording, her team was planning a 2-day workshop in November to pull PMs out of their regular work and let them build and experiment. This was a plan, not evidence that the workshop had already happened or produced a measured efficiency gain.

Repeated tasks also create an opportunity to ask which parts a model could handle. Rachitsky describes inserting that consideration between receiving a task and starting it. A useful first success can build confidence for further experiments.

Workload still needs an honest account. Doshi acknowledges that her team is working long hours and that sustainability remains unresolved. She sees churn, chaos, and unrewarding busywork as sources of exhaustion, and hopes AI can remove some of them to make more room for thought and creativity.

That remains a managerial judgment and an aspiration. People can enjoy their work and still become exhausted. After a productivity improvement, a team still needs to decide what to do with the time it saves.

Let AI collect evidence and have PMs supply judgment

Later in the conversation, Rachitsky raises a familiar problem: documents have become easier to generate, while reading everyone else's AI-produced documents has become a burden.

Doshi proposes a practical division of work. Suppose a team wants an update 2 weeks after launch: who is using the product, how they use it, and what feedback is emerging. AI can help gather material from dashboards, metrics, and user discussions.

The PM then needs to explain its significance: which feedback warrants action, what should take priority, and why that direction makes sense. The person remains responsible for that judgment.

Prototypes, analysis, and evaluations can make the reasoning testable. A PM once might have written requirements and waited for an engineer, designer, or researcher to supply their substance. AI lets the PM go further, bringing a working prototype, actual results, or a concrete quality standard to the conversation.

The work becomes more tangible: choose a consequential problem, define a good outcome, and bring enough evidence to move the team toward it. Easier production leaves plenty of that work to be done.

Original source: Why PMs are suddenly valuable again | Tulsee Doshi (Head of Product, Gemini), Lenny’s Podcast, published 2026-11-08; YouTube duration 58:33. Host: Lenny Rachitsky. Guest: Tulsee Doshi, introduced as Google's head of product for frontier AI models.

This article synthesizes the complete English transcript publicly supplied by the publisher through the video description, retaining its timestamp references and checking names and supporting examples against the primary sources linked above. The transcript's production method is unspecified and it should not be treated as error-free verbatim documentation. Advertising and the closing personal lightning round are omitted; opinions, internal practices, and future plans retain their attribution and limitations. Episode notes and source links.