Design and Engineering, Converging

Fantasy’s Head of Design on designing inside the same code engineers are building, and what designers still do that AI can’t.

At MIT EmTech AI this spring, we showed the demo I mentioned last time: your own AI agent talking to a brand’s website, with the site adapting to you as you talked. Kathy—the engineer who built it—and I skipped the usual handoff from design to engineering and worked in the same code. A few weeks out, I told her I was still sort of bashing my way through developer tools.

I did a full vibe code pass on the transitions and content to see how far we could move the needle on the fidelity and design of the demo. Kathy was updating the same code live, so we had to merge our changes the next morning. We’d also both run some of the same updates.

Kathy’s answer was to get me assigned to the issues I was coding, so the two of us weren’t doing the same work twice. She also made a point of telling me and the other designer on the demo that she was excited to have designers in the code. My rule for teams has always been “no egos allowed,” and that’s easy to say until you’re the beginner in someone else’s code.

Even before AI came around, it’s always been a challenge getting designers and engineers to sit down, talk to each other, speak the same language and actually have handoffs that work for everyone.

In my first piece I used the word “jamming” almost in passing. I got into design through music, and it’s a musician’s word for different instruments playing one piece of music at the same time, where nobody waits for a handoff to find out what the other player meant.

Jamming still takes coordination, and on the demo, assigned issues were how we coordinated. With that in place, I was designing inside the same codebase engineering was building in, and iterating while engineering, too, was iterating, right up until the day before it went live on stage.

Steve Croll, our Head of Technology, has said repeatedly we have to stop calling it vibe coding, because it kind of dismisses what it is. If you set up the foundation and use that as the baseline, it generates code that is as good as, if not better than, most engineers.

Vibe coding means prompting your way to working code and keeping whatever runs, with no foundation underneath. Anyone who has ever tried to just vibe code an app knows it can be as hard as, if not harder than, doing it the normal way.

You’re continually interating: It’s worse, it gets better, it’s worse again. Without human taste and human quality control, you can end up with the monstrosity I warned about at the start of this series.

I called my pass a “vibe code”, but it was the other kind. It ran on the foundation Kathy had set up, and she tested a bunch of the AI work first thing each morning before it got merged. That foundation and that review are why the code holds up. 

This is, and always should be, a human-in-the-loop situation. The design craft and quality need to hit our bar, and the engineering needs to be sense-checked, especially for compliance.

Novel problem solving is what AI can’t do. It’s very good at recognizing a problem and matching it with a pattern that’s already solved. But it’s not good at coming up with novel solutions.

The day after Kathy and I merged our changes, I asked the AI tool to make a “subtle webGL transition that distorts the content as it clears.” It was perfect. Kathy’s reply was, “Of course! Training data was Harry Potter movies.” A joke, of course, but she had a point. That effect was a pattern the AI had already seen.

My main note on the demo was about hierarchy. For demo purposes, and for the page to feel more agentic and reactive, the personalization needed to lead. Instead, we were burying the lede in the UI, with the personal elements sitting under a generic marketing headline.

I asked the team why we couldn’t make the page feel like it has LLM-style elements built in. So as opposed to a marketing headline, the headline would be personalized for you, and the personalized aspect would become the focus. In a quick mock, I pulled the personal elements up to the top.

AI’s quality isn’t great right now on the design side, but its engineering quality is very high. The high-craft design is still manual, and that’s intentional. We don’t want to rely on AI for it, because right now it can’t hit the same high bar that we have.

In each of my last two pieces, I promised to write about what this way of working does to design and engineering themselves. Engineers now set up the environment first and make the decisions on what we’re using, starting with what the front end is going to be built on. As they decide on the content system, or an integration with another service, that gets plugged into the same place.

Designers work in that same place, the space the engineers will ultimately pick up. UX designers build the initial prototype right in that environment. Visual designers craft the details, like what the visual language is and how to bring the brand to life, focusing on what makes it unique. Once everyone agrees on the direction, they iterate on it inside the system too.

Engineering and design will stay two disciplines, and I don’t imagine there’s going to be one person who does both. But those disciplines need to work closer and closer together, to the point where they’re integrating and, yes, jamming on things together, exactly how we created our demo.

Tools continue to evolve. You can now have multiple engineering agents (AI that carries out multistep work on its own) building things, so effectively you can have a whole team of engineers. In the (very near) future, a single, very senior engineer could oversee a bunch of these agents as they build, and review their work.

At the start of this series, I wrote that engineers go first, which is a strange sentence for a head of design to write. They go first because they set the foundation the rest of us play on. There’s a bit of a learning curve for designers, and to me that’s the biggest gap right now. Everyone needs to have an open mind about technology and be able to at least try to use these tools.

Designers come in right behind the engineers, in the same code, to make the calls AI can’t and do what we do best. Because at the end of the day, if you remove humanity from all of this, you end up with a fundamentally flawed approach. Sitting down and being human together is still—and always needs to be—a part of what we do.