SPEAKERS

Vivian Shen Co-founder & CEO @ Juni Learning

Vivian's experience ranges from strategy development for Fortune 500 companies to building teams from scratch at startups -- and everything in between. Prior to founding Juni in 2017 to satisfy the gap in the education system, Vivian served as the Director of Product at Operator, where she launched multiple products in the US and China. She also spent two years as a Consultant in McKinsey & Company’s Silicon Valley office, working with high-growth tech companies. She began her career as a software engineer at Google.

Vivian has been featured on Forbes’ 30 Under 30, as well as in Fast Company, TechCrunch and Fortune. She holds a B.S in Computer Science from Stanford, with a minor in Creative Writing. Today, she is passionate about helping kids discover and cultivate new interests and skills, empowering them to learn through the power of community and connections.

“We knew there were a lot of different things that we could tackle to make the experience better. But how do you stack rank things like, ‘If this dashboard goes down, we can't monitor all of these classes…’

So the main thing that we did was to put a prioritization framework in place and to focus on retention and 'customer operations scaling' as the number one metric. And that really helped us narrow our focus."

- Vivian Shen

This Episode Is Brought To You By

Special thanks to our exclusive accessibility partner Mesmer!

Mesmer's AI-bots automate mobile app accessibility testing to ensure your app is always accessible to everybody.

To jump start your accessibility and inclusion initiative, visit mesmerhq.com/ELC

ELC's Peer Group Program

ELC's Peer Groups provide a virtual, curated, and ongoing peer learning opportunity to help you navigate unknowns, uncover solutions to your challenges and accelerate your learning with a small group of trusted peers.

The job is hard. But you don’t have to do it alone!

To learn more and apply - Click HERE

Links & Resources

• IDEO's Design Thinking Blog with David Kelly
• Juni Learning

Show Notes

• Juni’s Covid pivot: Why they focused on engineering operations & internal tooling (2:19)
• Making the decision to pivot (4:59)
• Prioritizing and reallocating engineering resources (8:49)
• Refocusing the team and getting buy-in (11:01)
• Dealing with resistance/friction on the company’s direction (13:50)
• Juni’s pitch process to surface new ideas (17:06)
• How to leverage end-users to motivate your teams (21:03)
• Increase ownership and creativity with “Engineering Pods” (26:13)
• Lessons learned on communicating priorities (28:21)
• When to revisit your orgs priorities while scaling (32:09)
• Shifting focus back on growth (34:52)
• Collaborating in planning meetings with analytics / biz ops & engineering to impact end users (37:10)
• Rapid Fire Questions (40:13)
• Takeaways (46:19)

Transcript

Juni’s Covid pivot: Why they focused on engineering operations & internal tooling

Patrick: So first off, Jerry, and I just want to give you a big warm welcome to the engineering leadership podcast, Vivian. Thanks for joining us!

Vivian: Thank you. It's really great to be here, I'm excited.

Patrick: Wonderful. Well, we're going to get into it a little bit. So just to sort of set the stage for our conversation, Vivian, you know, your company, Juni had to make some really interesting rapid scaling induced pivots during the first month of COVID.

So our conversation today, the intention is to explore many of the different lessons and insights that you gained from that experience. So we're really trying to dive into your mindset and how and why you made certain decisions and navigated such a challenging moment for most companies.

But really to begin, we wanted to get into the story. And so what happened this first month of COVID? Why did you have to pivot focus on something like engineering operations and internal tools? Bring us into the story! What happened?

Vivian: Yeah. I mean, it was an incredible time. I think it was very bittersweet because obviously, you know, the world was sort of falling apart outside our windows. And at the same time, we just had so many folks coming to us, asking for help you know, getting their kids set up with new programs, all that good stuff.

And a lot of instructors also came to us because their universities had closed down and they were kind of out of a home trying to move home. And it was a really tough time.

For the previous kind of years before that we had been focused a lot on scaling on the kind of growth and students side. So a lot of our, product work had been around building a code editor for students and all of these kinds of tools that were more user-facing. Well, external user facing.

And then once COVID hit, we kind of had to rethink how we prioritize things. Because up to that point, the tools that we had worked well enough that you could kind of patch it by basically doing a little bit more manual typing and clicking and all of that good stuff. And once COVID hit all those tools kind of didn't scale with how many folks we had coming in and the matching system that we needed to do between instructors and students just wasn't scaling anymore as well.

And so that was one of those pretty critical decisions because bringing in new students was less of the problem... You know, to a certain extent. And the real one was actually serving all of the classes that we needed to do. And that was really tough to do with the tool set that we had at the time.

So, it was kind of a... I think, a big priority shift for the team. And pretty easy to identify in a certain way because we actually... well, first of all, you know, when your internal stakeholders are some of the ones who are using these tools. Those voices can be much more easily heard sometimes than users.

And at the same time too, we were starting to see that we literally couldn't sign up as many people as we wanted to. And so that was kind of the red flag for us that we needed to figure out what the bottleneck was.

Making the decision to pivot

Patrick: How did you arrive at the decision? Because when I think about it, in the moment, it's sometimes seems like it's so hard to identify the priorities of how you should adjust the organization. What was that moment like where you all had to come to, to decide we're taking a new path? What was that like?

Vivian: Well, I think that part of it was that me, my co-founder, a couple of our... well, basically all employees were all hands on deck, essentially as customer support people. And that's definitely a flag where you're... you know, you have to think through obviously something is not scaling.

The main thing we saw in the numbers as well, was that the capacity that we had for instructors versus students was starting to fill up. And that even when we had instructor capacity, our onboarding times started getting delayed by a couple of days and then a week or more, which was definitely a suboptimal place to be at.

I think that folks started to feel it very viscerally because we literally had to serve as the customer support team. And at the same time, like the numbers that we were seeing and the user experience that we were... we were looking at as well. Like we started getting a couple more emails and things.

Those are the first signs that, you're not scaling well. And so that was a big flag for us to, to figure out like, "Hey, this is something that, clearly from the numbers side, we're not having issues with new people. It's really the current people that we work with or during the onboarding phase, that's really the issue.

Jerry: And the factor for preventing the team to scale to a higher volume, is that mostly due to a lack of automation? Or this also involves inventing new process, et cetera, even new tools

Vivian: A little bit of both. Some of it was definitely automation. For example, at the time our matching tool was pretty rudimentary. Like there were few enough instructors where if you essentially filtered an instructor based on their availability, you could probably figure out, you know, amongst a group of maybe 10 or so instructors who would be the best fit.

And so at the time, our team personally knew every instructor as well. So that kind of helped, you know, ease things along. But during that time period, we had to manually check a lot of availabilities on instructors calendars, which that did not scale. So a lot of the work was to pull in those availabilities, be able to match it with the student availabilities that came in as well. So that was a bit of automation.

We had to rethink what was actually the best user experience as well. For example initially we tried to promise that we could start students the next day if they wanted to. But what we were finding was that was suboptimal because then you would essentially be matched with whatever instructor happened to be available.

But some students who are female, want a female instructor. Or some students are a better personality fit with another instructor... and so we staggered out the timeline to onboard a new student because of that. And I think that was a tough decision to make, because you obviously want to onboard new customers as fast as possible. But we kind of realized that the user experience was suboptimal if we did that. And so we made the decision to push out the start date. Which also got our support team some more time to get everything together, send the onboarding materials, et cetera.

And then I think that's some of the hardest scaling work that we did at the time as well with some of the monitoring that we did for classes. And so, I talked a lot about the onboarding issue, but at the time... and we still do this by the way, we monitor that every student is online for their course. So that if they're missing from class or whatever the case is, we can send them a notification. And let them know, you know, "Hey, your instructor is waiting for you"

That system that we use to monitor the classes broke a couple of times during this process. That was like a big infrastructure lift that a couple of our engineers worked on during that time to just make sure that, we could actually monitor that both the students and instructors were actually able to get on to their session from there, too.

Prioritizing and reallocating engineering resources

Jerry: It seems like there's a period of chaos when everybody's tried to help the customer . How do you transition to a better place where you sort of have a list of priorities that can reshuffle your team, engineering team included, to tackle those items. Can you walk us through that process?

Vivian: Yeah. I would say we had a pretty good issue list I think priority wise, a lot of the top opportunities were based more on let's call it " critical failures" to say the least!

But you know, from the infrastructure side, like when we weren't able to load the dashboard for monitoring these classes, like that was clearly a critical issue and surfaced, more by the engineering team and the customer support team. On the flip side decisions like pushing out the onboarding dates or how we do instructor matching, that was more of a user product related decision.

So I think actually the biggest challenge was we knew there were a lot of different things that we could tackle to make the experience better, but how do you stack rank things like, if this dashboard goes down, we can't monitor all of these classes. But those are already, I guess, active customers to a certain extent, like, do you want to prioritize new customers?

Do you want to prioritize instructors as well? So main thing that we did to put a prioritization framework in place was to focus on retention and kind of like customer operations scaling as the number one metrics. And so we, at the time, do you prioritize a lot of new future building?

We really focused on how do we get the throughput for one customer support agent up. So what are they spending most of their time on in a day?

So we kind of ran down like of our support team, they're spending the most time on these kinds of tickets. What's the tooling for that? How do we cut that down? And then automate some of it. Or just build better tools for them.

And that really helped us narrow our focus. I think it's very hard at a time like that to say, "Yeah, I'm going to focus on setting up new systems, this code editor for kids and at the same time, like these operations things. But basically that's where it was kind of, a decision that we had to make based on the main issues that we saw that were affecting the current customer experience at that time. And then it was like a data back to prioritization problem after that.

Refocusing the team and getting buy-in

Patrick: Yeah. I want to dive into that, the leadership elements of, you know, you refocus and you're no longer focused on building new product, but rather on operations.

Because I think the interesting thing is for you as a leader, you're building this team to help build this organization. And you've spent a lot of time building a team that was oriented towards a very different priority.

Now you're asking the team to focus on a totally different priority. And that takes a lot of leadership to build one team first of all. But then to transition those priorities.

So I think I'll have you kind of recap the story a bit more, but from my understanding, you know, you had product focused engineers and then you pivoted to internal tools.

How did you get the team to refocus? Can you tell us a little bit more about that and how you got buy-in to these new priorities?

Vivian: Yeah, that's a great question because I think the great thing about the team at Juni, honestly, is that everybody pitches in to further the mission and the vision of the company. And we got a lot of folks who are just... they're kind of like all hands on deck for whatever the daily or monthly situation is.

But to a certain extent, like change is normal at a startup. And I think a lot of the folks that we have just understood that and were okay with that.

So that's one. Like in the hiring process, I think looking for people who understand that priorities shift and that they're more excited about the future of the entire company than they are... you know, maybe this week, there's something that's different than what was prioritized in the previous week.

At the same time I think it's an exercise for a leader in transparency and talking through why certain things are issues versus not. I also think, you know, engineers are very data backed when they make decisions and understand things. And so that actually helped a lot was to put up the numbers of like, "Hey, we're actually not able to get this kind of throughput from our instructors because we don't have this availability matching tool." Or "We see the downtime in these session dashboards. So this many students missed their session."

Those kinds of things are very visceral and understandable for folks to size the impact of a project.

I actually think some of the things that are much harder to a certain extent are very greenfield. Which is kind of like that new product development because you have to get folks to buy in that a direction that has no data is an important one to take. And usually that's more like, "Hey, how are we positioning ourselves in the market? What are like initial indicators that we're seeing from user feedback." Et cetera.

But to a certain extent, I had a easier problem because I had a lot of data to say, like, this is clearly an issue. We're seeing capacity constraints because of these reasons.

And so it was a lot easier actually to shift that way. If you have a team that's you know, hired in for very specialized tasks. And then you ask them to focus on something else. That's where you get into more of the issues. So kind of solve that as a hiring problem, but then also using data to kind of communicate when something would changed change

Dealing with resistance/friction on the company’s direction

Patrick: Were there any moments of resistance or friction or different expectations from the direction. you know, with some of the folks in our peer groups, one of the hardest parts to navigate is that there's a person on their team and they really want to work on a particular project or a particular technology, but the needs of the business are pulling in a different direction.

So there's this sort of tension for how do we help set that up in a way where that person still feels like they're working on things that they want to work on or aligned with the direction of the company.

And so I'm just looking for your insight and how you sort of navigated those situations and pass that along?

Vivian: Yeah, it's challenging. I think, you know, it's a constant discussion to have, because there are very real reasons why people also believe that the projects that they're championing is going to impact the business the way that you're looking for as well as, a leader.

And for example you know, if, I were to say like one of the top company priorities is call it like "new parent referrals" like having our parents refer their friends to Juni.

There's like 10 different solutions for that. Right. At least! And the, I think that the, there has not been a ton of debate around what the top company metrics are. That's actually been one thing that I think aligns everybody as well. And then, you know, on top of that, like the mission for us to bring an amazing education to students around the world, like those are not debatable.

I think it's when you double-click down again and you say, "Okay, of the ways that I can still solve for the problems that we're seeing in the business, like , which solution is the best?"

I think those are actually much more complicated to help prioritize around.

I have had discussions with folks that are essentially like, "Hey, I know that, this seems really critical right now. That's only costing the company, you know, $10,000 a month to buy the solution instead of building the solution."

And once they understand that context, it's actually a big unlock for them to understand how to prioritize certain things as well. Like for instance, we still use Zoom for our video conferencing. And every, you know, six months or so we'll say, "Hey, should we rebuild Zoom in house? And, you know, try to make our own version of it?" And when you do the cost analysis would never make sense...

So those are the kinds of things where I think being really honest and open about, "Hey, these are the real trade-offs that we're looking at..." Really helps people get buy in as well.

Frankly, also people do bring me ideas that I think I didn't think of. And they're actually much more well-reasoned than the reasons why I would say not to do something. And so we try to have an open kind of roadmap planning process where folks can also bring pitches and talk about the impact that they have research that they've done as well.

And I think a lot of those are super valuable as well, because even if you don't go with one of those pitches that helps the person understand why something would be prioritized over something else because we went through the pitch process. And so that's something that I think helps people get on the same page as well.

Patrick: I just wanted to acknowledge something Jerry really quick, because Vivian, what I really appreciate is how you started with those people have reasons and they're real reasons, especially to them. And I think that as a leader that requires a lot of humility to enter into something. If somebody believes something and no matter whether it's right or wrong, it's real for them.

And I think it's a really important thing to acknowledge there. So that was really cool. Jerry, go ahead jump into your question.

Juni's pitch process to surface new ideas

Jerry: You mentioned the pitching process to surface good ideas. How do you do that?

Vivian: Yeah, of course. So in the product teams that I've worked in the past, there's been different ways to do it. And how we do it at Juni is how we do it now. But potentially it'll change.

Essentially what we do for pitches is we have a cross-functional team where it's usually a representative from, product, engineering, design, as well as marketing operations. And anybody can bring a pitch for a given week. Usually they'll work with the PMs slightly beforehand to make sure it's in the right format. They have the right data.

Usually we look for things that are like rough user stories or data on why this is a problem. And so we try to make the pitches very problem-based. And then we also have them focus on a potential solution, but there could be multiple solutions for a given problem.

So, one of the ones that we went through recently was like on our web conversion. We weren't getting as many people clicking on one of the pages as we wanted. I think like the pricing page or something...

And there were multiple solutions. Everything from like, just naming the pricing page to something else. So that it was clear. Through to, like, we need to have CTA's to the website from everywhere. Like there is a lot of different solutions.

I think it was great is that it gave the marketing team the muscle to think about how to opportunity size that. And so when they brought in the project, it was more around, "Hey, like this is the drop-off. This is the data that I'm seeing. And then here's like a potential solutions why? Cause we looked at our full story recording on how people use the page. We also looked at user research, et cetera."

And so that gets like people who are not product people thinking that way as well, which kind of shortens the loop as well.

And then we also have engineers bringing in pitches as well. They're sometimes more technical, sometimes more product based. Like we obviously have ones that folks bring here that are like, "Hey, we need to refactor this part of the code base because there's too much debt. And like, we're all slower because of it!"

I think those are very valid. And then there's also ones that are like, "Hey, you know, when I was a kid, like I loved using these kinds of code editors or, whatever the case is. This is the research on why we could focus on something like that."

So we try to have cross-functional and then the PMs work together to prioritize amongst those projects as well. But typically we keep a really tight planning loop.

I've been on teams... and we used to do like a pretty extensive quarterly planning process, but that doesn't really work for a startup. And so we've kind of moved back into doing just two cycles forward, or two sprints forward planning. And that actually helps people understand like, "Hey, if my pitch doesn't make it into this cycle. That doesn't mean it won't make it into the next."

And that also kind of lowers the bar as well for when somebody should really really bring something because if not, it won't make it in the queue for the next three months. Now it's like, "Hey, if it didn't make it this week, it could still make it in the next two weeks."

Or whatever the case is. And that actually also creates a little more fluidity amongst the team and more collaboration because you get to see every pitch on a weekly basis and understand that there's space. But it has to be THE most important thing to work on this cycle.

Jerry: A few up questions who are doing the pitches? Is everyone in the company or just specific, like maybe the team lead? And how does the pitch look like

Vivian: yes. Technically it is in a doc. I will say maybe this is controversial... It's in Notion, but I'm not the hugest fan of Notion. And so if any listeners have other tool recommendations, I'm all ears as well. But typically, yeah, we have a doc template in Notion where folks put in, all of the good stuff, like propose problem, data-backed and then propose solution, et cetera.

And typically we do have mostly that the team leads as constituent members of the cross-functional group. But what will happen is that individual members of those teams work with their TLS to kind of go through their pitch. And then the person who actually came up with the pitch also attends that specific pitch meeting.

You know, something that they worked on with our TL should get surfaced in that session. So, I think the goal is to not have these pitch meetings be like 40 people that would probably not be very productive. But the representative who, is usually an IC will bring a project that they're pitching and their TL will be there as well as the other TLS from other teams.

So that's typically how we do it to keep it an event at multiple levels. But also a little more focused.

How to leverage end users to motivate your teams

Patrick: Vivian, you were mentioning a couple of things around the different users that are involved with Juni and how different people are proposing ideas. And so this sort of segues into one topic I wanted to make sure that we covered...

I think one of the things that we've talked about in the past, I think is really fascinating is how you anchor decisions and direction to the end user. And that end user becomes the key anchor point to motivate your teams and to help them sustain momentum through different challenging times.

And so I was wondering if you could share a little bit more about, how do you use the end user to motivate your teams? And how does this then empower your engineering teams to find different solutions?

Is there like a story you could share related to that? Or dive into an example?

Vivian: Yeah. It's a great question. Let's see. So we have, I guess, four constituent groups at Juni. You know, parents, students instructors, as well as Juni internal stakeholders. Roughly in the past, we have had folks kind of matched to those different stakeholders because it helps them get a really deep understanding of who they're building for. And then also get that buy-in.

The challenge with that is that if at a certain time you have a slightly different company goal those pods can kind of shift alongside company goals as well. So that's one of the things that has been interesting to sort out.

For example, instructor hiring and I guess instructor tools. That's been to a certain extent, easier for us to build. Because it's usually a fast follow on what we built for the student or the parent facing tools. And so that team has not been a standalone team on its own necessarily because it's not the kind of like biggest opportunity for us to improve upon right now.

And so, the folks on that team have kind of been absorbed into the more like internal stakeholder tools. At the same time, the internal stakeholder tools, if you really think about it... those folks, their end user is the parent or the student as well. Because they're building tools to actually facilitate a better experience for them.

And so after we kind of started with those four groups, but they've merged a little bit into purely more around students and parents. I think at some point in time we could peel out instructors again. But really the goal is always to make either the student or the parent's experience on the platform much better.

The parents are much more oriented, around kind of like the utility of the service. Like rescheduling their classes, seeing their student progress. That the kids are really there to increase their engagement on the platform. And spend more time.

And so that orientation, I think, helps people understand both what their goals are from a metrics perspective. But then also their users so that, you know, when we do user research or when we're saying like, "Hey, who are we really building this for?"

That is a lot more clear.

For example, like a lot of very sophisticated design UX paradigms, don't really work for kids. Like you kind of you to have a giant navbar that just has everything in it for students, because they want to access things as fast as possible. Versus for parents actually, simplicity is a lot better. Like they just want to see progress and then when the next session is and how to update their next, you know, schedule or something like that.

And so it's been interesting actually to design and build for both because you know, products like Discord works super well for kids and they don't work very well for parents for a number of reasons. And that's, I think been a really exciting way that folks can get a little more aligned with who they're building for.

We've been tinkering as well with having them focus more on specific company goals. Like think about it more of like an OKR model where the pods are structured around OKRs, but TBD on, on that front, I think it'll probably be a combination of the two.

Increase ownership and creativity with “Engineering Pods”

Patrick: Can we zoom out a little bit? Could you talk a little bit more about about the engineering pods? Cause that seems like a really interesting team structure and you'd reference it a couple of times but. I think many folks listening and may be a little bit unfamiliar with what that concept is, how that interacts within the entire organization.

So what are the engineering pods?

Vivian: Yeah. So typically how I've seen our team be the most successful is if there are folks who are focused on specific goals or end users who are cross-functional and work together. And the engineering team is split up into pods that work the most closely with certain cross-functional groups.

So for example we've had our ops team that focuses mostly on our customer support team, instructor operations. And they work with a pod of, roughly four or five engineers right now. And we also have that reflected for our growth side pod more a combination of marketing plus the engineering folks. And then our learner pod which was more for our student experience... student slash parent experience. And that's like a combination of our curriculum team and a couple of engineers as well behind that.

And so I found that that actually enables folks to understand much more clearly who their collaborative folks are. Like if I'm in engineering can go directly to a curriculum person and get feedback on something very quickly. I also know who my designer is and who my PM is.

And there's kind of like this very almost like a mini company within the company, and this is our specific goal, and this is our specific end user. That actually allows for a lot more transparency. Versus, I think what's tough is if you have an engineering manager and engineering leader, who's kind of, they're the one single point person with the rest of the team. That's obviously not going to scale very well.

So we have TLs for each of the pods that are really more than representative folks. And then they kind of collaborate together with the cross-functional pod to make sure stuff is prioritized, also communicate deadlines and that kind of thing.

So I've really liked the structure of having like mini orgs with... mini companies within the company, it also creates a lot more ownership on everybody's plate because they know who their people are and, you know, what their end goal is.

So a big fan of the pod structure. If you have it, haven't noticed.

Lessons learned on communicating priorities

Patrick: What's interesting that you mentioned Vivian about, you know, having orgs within an org. And we're going back to the whole beginning of our conversation about shifting priorities and getting the whole organization behind those decisions...

I think the understanding the reason why you're prioritizing certain things and communicating that effectively, I think is really fascinating. Especially when you're communicating the difference between why something is critical for the business.

And so I was wondering if you had other insights or other lessons that you want to share. I think especially related to like how to communicate engineering and operations critical priorities for folks. Because I think sometimes the gap for like why you should make a certain engineering decision and communicate that as a priority is a lot harder to communicate.

And so I was wondering if you had other insights or lessons to share around that.

Vivian: Yeah. It's an ongoing process. We do all hands every two weeks. And I think in the past, I've used the all hands as more of an opportunity to celebrate the team or celebrate, you know, individual students or instructors of the platform and not so much to talk about us, you know, certain priorities.

And I think that was a mistake. And so recently for our all hands I surface kind of all the top level company metrics as well as why those are the priorities and what individual people, or teams are doing to support those priorities.

I think that's actually made things a lot clearer for folks. And so we're also building a KPI dashboards that people can see that on a daily basis if they like. But usually a weekly basis.

And those kinds of things, I think just help people understand the context, but also the numbers. Which I think a lot of the time people want to know because it kind of grounds various trends that we're seeing as well. That actually did a lot to communicate with folks like, "Hey, yeah. When I look at the numbers, like this part is down and this part is up. And so it makes sense that we focus on this part that's down."

And that actually has I think unlocked a way of thinking for us that was previously different. Which is like, before I think we would make a lot of more moonshot bets that were not as much backed in, "Hey, this is the company goal or whatever the case is."

I actually hear this with a lot of companies when they're searching for growth, that's led by product. It's very hard. You basically make like a bunch of very greenfield bets and they don't always pan out. And for us, it's been a focus more on "this is the funnels that we see. These are the most important things for us to focus on. That's why this is one of the you know, biggest company goals right now."

And people can get behind that very easily.

I think there are some times where you work with engineers who are like, "Hey, I wanted to solve this very gnarly technical problem in video infrastructure."

And we usually just filter those folks out at the interviewing stage for Juni. I think like they... it's hard, but you can't always build around folks like that, who if they want to solve a problem like that they should potentially be at a different company. And that's okay. I do think sometimes that folks try to over-index a little bit too much on the squeaky wheel. But for the most part, like your engineers want to build something really great and they are super smart and they want to learn why certain things are priorities, but once they do like they're all on board.

And I think that was like a big unlock for me as well. Not at Juni but at the previous company that I was at. It was a AI e-commerce company, but like AI and e-commerce at the time, I mean, AI is still very greenfield. And so the people who came for AI had a hard time working on the e-commerce aspects of the business and like that was just kind of a reality.

I think going into Juni, we've been pretty upfront during the hiring process. Like, "Hey, we look for product oriented folks who are really here to build something great for users. And you know, if that's something you're excited about all aboard."

And then that has filtered out, I think a lot of the issues that you might see with some folks were very different on what they want to focus on.

When to revisit your orgs priorities while scaling

Patrick: So I'm going back to the moment, right? Where, you'd been focusing on certain priorities, and then first month of COVID two to three X scaling in a month. And then having to totally change priorities to then deal with that next phase of scaling.

And so now it's been about a year. And so I think what I'm curious about is, you know, this concept and this idea of, you know, at what point do you revisit your assumptions or your priorities and identify what's the next level you need to get to in terms of scalability?

Can you share a little bit about, like, what was the paradigm early on pre COVID for like building for scalability?

And then now that you've built all these internal toolings when do you revisit that thought process and then create like a new set of assumptions?

Vivian: Yeah. That's a tough one. So, you know, I think that naturally it's somewhat happens on a quarterly basis. You know, we're also somewhat of a seasonal business because we work with kids who are on an academic calendar. And so there are some natural break points where, going into the fall or going into spring or the summer, like those are points where it's always been a good time for us to kind of reset.

I think that it's important on a quarterly basis to do your metrics over again. But your roadmap is a little more fluid. And so that's why we've kind of been moving to the more two cycles out planning process instead.

But on the metrics front... I think you're always looking for leading metrics to indicate if the lagging priorities are happening or not.

In startups, frankly, growth is the number one. And so basically once you kind of see that the operations metrics have stabilized a little bit. It's probably always behooves you to move people back to growth because, to a certain extent if you have growth, you can solve for the scaling problems by hiring a lot more people, raising more money to hire more people like that kind of thing, if you have growth. But if you don't have growth, then that's a very different conversation.

And so, with a startup, I would say like, constantly thinking about "when am I going back to focus on growth?"

Us specifically, our retention is very strong and so I would also keep an eye on retention at least bi-weekly basis. Because retention is a little bit of a lagging metric, so you might not see early indications of it changing as much.

But that's the other one. Like you can't leak people that you're acquiring. And you know, that's been less of a focus for us because it's naturally been a lot better of a number for us. And so, I would say like those two, even when we're focused on something else, like every week I look at those numbers and if we see trends in those that are not good, then we put more resources behind them cause those are very early signs for things like that are definitely ones that you want to focus on.

Shifting focus back to growth

Patrick: So have you, shifted away then, from the focus on internal tooling and now have like made the decision to, to refocus on growth?

And can you bring us into that moment? Because I think as somebody who's been involved in a couple different startups, like the thing I've personally experienced and heard of is like, people want to be okay with change in startups. But there is a little bit of kind of a whiplash effect for it's like, okay, we're focusing on this, we're focusing on this.

And so I'm curious, like, how did you overcome like that sensation of whiplash? Or what was that moment like where you shifted back to growth?

Vivian: Yeah, it's a good question. I think specifically for this even when we focus more on operations, I think everybody knew that growth is still the number one. And so the shift to operations was more of the anomaly, if anything.

I think that one of the more challenging discussions is like, how much do we focus on growth versus retention or engagement, right? Because if you think about the fact that you already have thousands of people who are paying you for a product, how much time do you invest in making the experience better for them versus trying to go after a new group of people?

That one, I think is a little more contentious because you can always build really better and better things for your current active users. Like you could build a whole roadmap just looking at what your users today are telling you.

And I kind of call it the echo chamber. You have to take everything with a grain of salt as well. Like our first group of users may not be our group of users when we're four years down the line. And so I think that's like a very hard balance to take.

I will say that on the growth focus, like that's always been for me the most important thing that I talk about with people anyways, because of the fact that for a startup, that's the number one metric.

And so even when we're refocused again on, on one of the other levers or whatever the case is. Like, it's more of an exception than anything. So kind of the swing back to talking about growth, I think felt less like a swing and more like a, "Okay. Hey, the time for the code red is over like back to our number one."

And I think that was a little bit better than saying like, Hey, we have, you know, these too many goals.

And also the numbers are a big deal. Like once the operations numbers got into a better spot than it was a very clean way to say, "Hey, like let's refocus the resources and the team again."

Collaborating in planning meetings with analytics / biz ops to impact end users

Patrick: I have one more sort of tactical and mechanical question, because I think one of the things that's really stood out to me through this conversation is sort of the mental pattern to connect the internal tools that are being built to the end user experience or how it's improving our different end users in the different categories.

How do you operationalize that? Can you tell us a little more about the mental thought process to take "Okay. We're deciding on building these tools or focusing on these things"

And can you walk us through a little more about like that thought process and how that shows up in different planning meetings or prioritization meetings?

Vivian: Yeah. You mean specifically when we refocused on the internal tooling

Patrick: Yeah.

Vivian: It's a good question...

I think that every company should have like a, I don't know if it's a BizOps or analytics or some other more central function, but basically like a pulse checker across the company on how the top level lifeblood metrics of the company are doing. And that was what surfaced the internal metrics for us, that showed that we needed to focus on operations because onboarding time was starting to lag. Like the customer support response time was starting to lag. Like those kinds of things...

It's tough for a team that's in the trenches to be looking at those numbers all the time. Sometimes they don't have the tools to do that either. And so building out a Central Analytics or Biz Ops function is actually really critical.

I mean, I think we could debate org structures, but we have a central ops analytics team that serves all of the different teams. So, they're like very loosely aligned. But we don't have analytics sit under, you know, marketing for example, or like product analytics separately. They're all central and the dashboards that they build are also central.

And so that I think that helped us flag, like, "Hey, these are the leading indicators that are already showing we're going to have issues in these areas. And then we could prioritize them at a top level way for the company. Versus I think, you're doing it on a micro level, you might miss the forest for the trees. If you're doing something like that. So that really helped.

And then once we did that, it came down in every meeting that was like, "Hey, these are the metrics. Like, clearly this is the one that we need to focus on right now. So let's talk about this with the engineering pods. Can we lend an engineer to one team from another. Or more if that's really the one that's kind of on fire right now?"

But then, that kind of communication is more done centrally with the TLS and the engineering lead. And then yeah, but the team is small enough right now where we can all look at the total set of metrics and then kind of stack rank amongst the different pods.

Patrick: Wonderful. Thank you. I've really appreciated Vivian, just how much you've sort of opened up your thought process and how you think about different things and how kind of exposing the synapses between how you're communicating information, engaging, and getting information from other folks and creating dialogue and discussion and how we're prioritizing all of this.

Because I think it's really hard to see how all of those things work together to get to the end point for decisions, product, and shifting direction. So thank you sharing all of that.

Rapid Fire Questions

Patrick: We have some rapid fire questions to sort of wrap up some things at the end. If you are ready. five questions

Vivian: Go for it.

Patrick: All right. So the first one, what are you reading or listening to right now?

Vivian: I'm in a book club and we're reading The Left Hand of Darkness by Ursula Le Guin I read a ton of fiction and Sci-fi and so this one has been pretty exciting. Recently read some NK Jemisin stuff as well.

I find fiction is great for my storytelling. Maybe not as practical in some ways, but it does open my mind.

Patrick: Was it the NK Jemisin book was it the Broken Earth trilogy?

Vivian: Yeah, that's good.

Patrick: Great. That was my, a couple of months ago. That was I dove into all three of them. They were awesome.

and the left hand of darkness was on like my I think one of my dressers for a long time. So for this love fiction, I'm a huge scifi person. So that's like my decompression.

These are supposed to be rapid fire questions...

okay. Question number two, what tool or methodology has had a big impact on you?

Vivian: it might be a little trite, but design thinking. I actually my specialization in undergrad in computer science was human computer interaction. So we just did a ton of like paper prototyping, user interviews, stuff like that. That I kind of took for granted.

But I find myself, you know, all these years later, that's still a huge part of keeping innovation alive, keeping people really excited about what they're able to individually impact and give us solutions as well.

I think that the search for perfection in a startup is, futile. And so, design thinking, I think really helps people understand how to prioritize things. But also that it's okay if you, work on something that maybe isn't the best solution or you launch the thing it's just the first version and you're always iterating on it.

That's always stuck with me. And I think it's really important for folks to learn more about the methodology and process.

Patrick: Do you have a primer or a specific resource that you might recommend people to sort of dip their toes into learning more about design thinking?

Vivian: Yeah. I think all of the, like David Kelly, like the IDEO design thinking folks, they have a lot of really good essays as well.

But they also have some exercises that are just like how would you design a wallet for somebody if you didn't know anything and kind of this concept of like focusing on the problem, focusing on the user, and then forgetting about solutions until later. Like it's very helpful for brainstorming as well, but there's a ton of cool stuff. I think on the Stanford design school website.

Patrick: Awesome. Well, we'll dig up some stuff.

So question number three... what is a trend that you're seeing or following that's really interesting or it hasn't hit the mainstream?

Vivian: Man... Well, I will say that three years ago it was teaching on Zoom! And I was way ahead of everybody on that one... recently I'm not as hip anymore. I

I think there's a lot of really cool stuff going on with kind of like individual people finding new educational paths... and that's both for kids and adults as well.

I don't know, you know how it'll play out, long-term in the efficacy. But I think we're seeing a lot around, certain universities getting critical mass, the lion's share of applicants. And then the rest of, you know, rest of the universities who knows how useful they are. And so I think it's been really cool to see how many individual creators are able to create new educational experiences.

I do think maybe this is a bit controversial... but I'm not a huge believer in the quality of a bunch of them. Obviously like, because of the way we built Juni, we still think there needs to be some kind of central development process or some kind of central thinking on how thorough the curriculum is.

But I think to inspire people, like there was a lot of really great stuff out there. And so, yeah, that, that was probably a little more mainstream now, but it's been happening for a little bit.

Patrick: Awesome. Okay. Final two. What's your favorite or most powerful question to ask or be asked?

Vivian: I always find asking people about like a time where they disagreed with someone to be really telling. Cause I think disagreeing is one of the hardest things to do. Or it weeds out people who you know, are very controversial very quickly.

But I think it's always telling to me both like the magnitude of what somebody thinks a disagreement is. Like, is it just one thing in like one product review where you disagree about like a potential solution to something?

Or is it like, "I disagreed with my manager on this focus area and like, this is what I did about it." Kind of thing...

So I feel like those really show me how people communicate, but also what they view as like a true disagreement or one that would surface in their mind as an important one.

So, that one, I find very telling yeah.

Patrick: Fantastic. Last question. Is there a quote or a mantra that you live by or a quote that's really been resonating with you right now?

Vivian: We're rethinking our core values right now, actually. Because the ones right now are a little old and I think a little generic actually, too, but they've served us well. But I've just been rethinking them.

And one of our PMs brought up this quote from, so us chief digital officer? But it was, "Dream in years, plan in months, ship daily."

I've been thinking about that one a lot, because I think it encompasses the fact that you do have to be fast on a daily basis. But you're a long-term optimist and you're always thinking about what the next few years would look like as well and how those stack up.

I really liked that one. It's so relevant for a startup. Like you're really out here thinking about how the next few years are gonna look but out of the daily, you kind of, you have to make an impact. And so, that, one's great.

Patrick: That one definitely hit in the chest and it felt really good.

Jerry: Yeah, I'll write that down for me too to remind me on a daily basis.

Patrick: It's going to be the new quote poster above Jerry's desk. That's fine. Fantastic.

Vivian, thank you so much. This was an absolute blast. And those are all the questions that we have. But so many more follow up questions to, to dive into. So thank you. This was incredible.

Vivian: Thank you so much for having me. It was great hanging out with you both!

Takeaways

Patrick: Vivian shared tons of great insights about how she thinks about prioritization, pivoting and gaining buy-in from engineering teams.

I wanted to quickly recap two of the practices. She shared their pitch process to surface innovative ideas and how they organize their teams and the engineering pods.

So to surface new innovative ideas continuously, one process that you could introduce are the team pitches that Juni includes in their cross-functional meetings.

These pitches come from anywhere in the company with tech leads and product managers providing support and an initial level of filtering to create those pitches.

Here's the general pitch framework. The first part is to define the specific problem. And then once you've defined the problem, then the next objective is to provide user stories or data that illustrate the impact and why this is a problem for the company.

The next then is to present your solution or the multiple solutions to address that problem. And then from there, product managers can work together to prioritize all of the different pitches that are approved by the group and to figure out a path forward and how those integrate into the roadmap.

Another interesting practice was how Juni organizes their teams into engineering pods, essentially creating many organizations within the company focused on specific end users.

And so to share a little bit about how those look... the engineering team is split up into small groups that work closely with specific cross functional parts of the organization. So think like marketing operations or customer support.

And then these small groups or pods also have a tech lead that helps support with prioritization and managing deadlines.

And so this structure has helped clarify who the key collaborators are, where to get feedback from and better connect engineering work to its impact on specific users.

Outro

Patrick: We'd like to give a special thanks to Mesmer, the exclusive accessibility partner of the Engineering Leadership Podcast.

Mesmer's AI bots automate mobile app accessibility testing to ensure your app is always accessible to everybody.

To jumpstart your accessibility and inclusion initiative, visit Mesmer Hq.com forward slash ELC. You can also follow the link in our show notes. That's Mesmer hq.com forward slash ELC.

If you enjoyed the episode, make sure that you click subscribe if you're listening on Apple podcasts... or follow, if you're listening on Spotify...

And if you love the show, we also have a ton of other ways to stay involved with the Engineering Leadership Community.

To stay up to date and learn more about all of our upcoming events, our peer groups, and other programs that are going on head to SFELC dot com that's SFELC dot com.

Or you can also follow the link in our show notes.

See you next time on the Engineering Leadership Podcast.