How to Build and Scale Your Engineering Team

Panelist Bios

Jessica McKellar is a founder and the CTO of Pilot, a bookkeeping firm powered by software. Previously, she was a founder and the VP of Engineering for a real-time collaboration startup acquired by Dropbox, where she then served as a Director of Engineering. Jessica is also a former Director for the Python Software Foundation and PyCon North America’s Diversity Outreach Chair. For her outreach efforts in the Python community, she was awarded the O’Reilly Open Source Award in 2013.

Eric Koslow is the co-founder & CTO of Lattice, where he manages the engineering, product, and design teams. Eric has been on both sides of the engineering equation. Both as an IC being recruited, and as a hiring manager doing the recruiting. Eric is passionate about engineering, HR, and watching Youtube.

Ammon Bartram is a co-founder of Triplebyte, where he runs their interview process. He’s interviewed over 1,000 engineers. Before Triplebyte, Ammon co-founded Socialcam, which was acquired by Autodesk in 2012 for $60m.

Dan Robinson is the CTO of Heap, where he built the first version of the company’s data infrastructure and designed a new kind of interview process around what actually makes software engineers effective. He lives to hike.

How do you hire your first engineer? [5:42]

[Eric] For the first 8 months at Lattice, it was just me and my co-founder Jake. I wrote all the code and Jack did all the selling. We went with the startup advice of “don’t hire for as long as possible.” Throughout that 8 months, previous co-workers had reached out, asking what we were working on. So when we were ready to hire, it was easy as we kept warm leads. Nurturing hiring leads like you nurture sales leads is really important because you never know when you’ll suddenly have an offer, or they’ll have a different life experience that allows them to work with you.

[Dan] Our answer is completely different. We had a lot of trouble with nurturing people we had worked with to join our company. A lot of our early engineers came from direct inbounds. Whether they send us an email with their resume, or we put blog posts about the engineering work we do on hacker news and that got a lot of reach.

Another fact that made it at all tractable was we were remote early on. The first person who joined after me was remote. Personally I was very pessimistic about it, but it was one of the best decisions we’ve ever made. One common piece of advice is to make sure at least one of your early hires is diverse along any of the axes of diversity. One thing I wish I knew back then is if you can find a person in your early set of hires who is like a spark plug, who makes the room fun, that absolutely changes the character of the office.

[Ammon] I think we should just take the general advice that is: It almost always makes sense for the first new hires to come from your network. Especially because at an early stage you don’t have native advantages of people knowing about your or your product. At Triplebyte, our product is a pirating platform. So we actually wanted to make our first hire through the platform as a test of the product. But the next few hires were people that I’d worked with previously.

[Jessica] Pilot’s first engineering hire was a friend of our CEOs from way back in high school, which is in your network, but in a really old-school way. Something worthwhile to keep in mind when making the first hires, is that these engineers will reinforce the culture you want to establish for your team. This person is going to have a disproportionate impact on culture, and also be very involved in evaluating future candidates.

How do you attract and close candidates at a small, unknown company when competing with companies like Google or Facebook? [13:07]

[Eric] What was useful for Lattice was, we’re solving a problem that almost all of our candidates have dealt with in the past, which is bad company cultures. Obviously we use our own product and really care about this. And that’s attractive to candidates because almost everyone has worked at a place that they didn’t feel appreciated, or they didn’t feel like they’re getting the feedback that they needed. I think joining a start-up means you get to participate in the startup experience of growing and seeing the numbers, and that makes them feel like they’re going to be a part of this thing.

[Dan] The tried-and-true answer is you have to sell the smallness of it. If you go to someplace like Google, you’ll make 50 or 100k more a year. But 10 years later, you’ll still be a mediocre engineer. But if you join this startup, you’ll be in way over your head in a way that Google would never do, and your growth is compounding it. There are other ways you can lean on your smallness, like  give people a lot more attention than they’re ever gonna get at a large company. One thing we do is we give hires a one-page description of what we liked about their interview.

[Jessica] It’s super important for early-stage companies to have clarity around who they want their competition to be when closing candidates and to base that in what your true needs as a company are. It’s mathematically impossible for every tech company to employ the 99th percentile engineers. It is also self-evident that it doesn’t require 99th percentile engineers to do all kinds of engineering. You can choose to compete against Google or Facebook, or you can also select yourself into a different competitive pool that has good engineers in it, but are perhaps not 99th percentile engineers.

Then the second thing is your wedge, or the thing that makes your company stand out in competition with other companies. Your wedge can be that the mission is really important, and then you will attract people who are attracted to that mission. Or you can just pay people more money. Or you can build up a narrative around the quality of the engineering team or the quality of the culture and lean into that. They’re all valid strategies, but you should have clarity about which one you’re trying to implement. Because if you’re not really any of them, you end up losing to other companies.

[Ammon] What’s worked the best for us is not compromising on the quality of people we’re hiring, but finding and going after undervalued engineers. Google can be very selective on things that only roughly correlate with skill. They can afford to say we’re going to only talk to people from other well-known companies, or went to a top ten CS school. The pool of people who don’t have those credentials is so enormous that there is still a vast number of top skilled engineers who learn their skills in some other way. We’ve had a lot of luck dropping some of the preconceptions about what skill engineers look like and finding people who are strong engineers that have surprising backgrounds. Google’s not making that person an offer, so that person is extremely excited and grateful to have the opportunity.

How do you build a diverse engineering team? And what trade-offs are you willing to make? [23:36]

[Ammon] The first is just caring and realizing that it is particularly important. It’s hard to talk immediately about ratios, so I don’t think from a statistics perspective you should feel bad if your first three employees aren’t diverse. And yet, you are making it part of the future. People reasonably want to join a team where they can see people like themselves being successful there. So, in spite of the fact that it’s not a statistical argument there, companies should think about that from the beginning. And to a certain extent counters the advice of hiring from your network. Maybe slow down a bit and wait a little longer to make the first hires in order to search more broadly.

[Jessica] I agree with slowing down. Working at Dropbox was really fascinating as there’s a real test of the commitment to the ideals when you want to be growing quickly or when you’re doing this at scale. At Dropbox, there was a strong belief in the values of having a diverse team. However, there was a perennial inability to actually move the needle on the diversity of teams. And what it always came down to was, Dropbox wanted to grow really quickly. When it came to needing to make a trade-off between hitting your volume or hitting your diversity goals, volume always won. This was not a resourcing problem, it was entirely a prioritization problem. That requires a commitment to potentially growing slower, and the founding team and managers have to have that buy-in and be willing to make the trade-off.

At Pilot, I’m allowed to control what recruiting channels we’re focusing on to promote a diverse top of the funnel. Even if that means not paying attention to inbound interest, or growing slower. It might mean focusing on building relationship with organizations that attract diverse talent pools, like Recurse Center or Hackbright Academy.

[Dan] It’s hard to be committed to. Especially if you’re a four-person team and you need to get one feature out the door, or you’re in a foot race with a competitor. I think the phrase “top of the funnel” is incredibly prescient here. We paid a lot of attention to ways within the funnel that there might be bias. The top of the funnel is incredibly screwed, not just for network reasons, but in our case just due to the people emailing us. One of the most fruitful things that you can do here is definitely building relationships with organizations that already do this.

What does a bad hire look like, and what do you do? [32:25]

[Eric] You will know you’ve made a bad hire very quickly. We always try and hire two people around the same time for similar roles. That way you can compare and contrast to see if it’s a systemic failure of the company, or if it’s an individual issue. Maybe the onboarding program is busted. And if you fire them, almost assuredly your team’s gonna feel better, you’re going to feel better, they’re going to feel better because normally that means that they’re in a really bad situation. Learn what signal did you not grab during the interview process. The key metric we look at is, how are they perceived by their peers when it comes to the work, not in their personality? Are they someone that their peers can trust to get the job done or that I want them on my team?

[Dan] Yeah, you’ll know very quickly. To put numbers on it, you’ll probably know within two weeks there’s something off. We used to do a lot of trial periods where they would work with us for a month before bringing them on board. And if you can do it, I highly encourage it. You also probably waited too long to be frank with them about the fact that things aren’t going well. It’s hard to turn it into objective factors. But if you don’t, you’re not doing anyone any favors by pretending that things are fine, especially the other people on the team to whom I think you owe teammates.

You should absolutely make adjustments to your interview pipeline. As your organization scales it is a really interesting metric to track how long it takes to determine that someone isn’t working out. If it takes you six months to realize a new hire is not a fit, that says something about the structure of your engineering team and what its priorities are. Likewise, on the flip side, how long does it take for the executive team to be aware? There’s a million signals. If you noticed that someone on the team feels the need to always check that person’s work more carefully. Or you go to a sprint planning meeting, and you notice that the team member does the easiest thing.

[Jessica] Slightly different, I find people that are grossly underperforming to be the minority case. Usually the hard case is for folks where you had an expectation, and they’re not hitting that mark. There’s still productivity, but not as high as you want. I don’t think I’ve ever been in an organization that actually pulls the trigger on firing folks who are not meeting expectations but also aren’t grossly underperforming. It’s tough on the team if there’s not clear underperformance, but they’re just not growing. Maybe in some cases you do fire people that plateau earlier. Or maybe your company has a philosophy about where it’s acceptable to take action. Then it’s about, how do you best leverage this person’s strengths, knowing that maybe their ceiling is lower than you were expecting?

What kind of engineer do you need early on? And how do you handle that as the company matures? [42:11]

[Eric] The people that are with you early on, don’t need to be the people that are with you forever. Companies change and it doesn’t necessarily mean that anyone failed, it just means that they drifted apart. Companies when they’re three people are very different than when they’re a hundred, and that person may not be a good fit at a hundred scale. Their skills don’t match or they’re not interested in it because they don’t like the bureaucracy. And it’s totally fine for you to go your own separate ways at that point.

[Dan] There’s some startups where you finish the product and that’s it. But I think if you’re working on something with a high technical ceiling, or something where there’s an endless summit, there’s always a frontier. People who like that are scrappy and take good initiative and have good common sense about what to do. I would hope that there would always be something that you can have them do. And the advantage they bring isn’t just those skills, it’s also years of context about the company. They’ve been around forever, they know a million decisions and why everything is the way it is.

Have you used short term contracts in your hiring process? [44:41]

[Dan] We’ve leaned on it pretty heavily. I highly recommend it. There’s no comparison between an interview process and actually working with someone to determine what it’s like to work with someone. But it’s limiting in terms of who can do that. It’s really hard to recruit someone who has a family or needs any degree of stability in their life.

[Ammon] For a first hire I think it’s just not going to work because early stage startups are  struggling to get applicants. They have a top of the funnel problem. If you’re a huge company and everyone’s applying and you have a great brand, you can afford to trade some drop-off for a more accurate evaluation process. If you can find someone willing, that’s great. It really is better for for you, and happier probably for them to it because you know they can see if they fit in. But at a very early stage for your first few hires, it’s improbable to pull that off.

How do you evaluate candidates on skills that you don’t have, but you need in a new hire? [46:14]

[Ammon] If you can call in investor’s friends, or people you know from a startup, and have them run a portion of your interview, I think makes a lot of sense. A fallback approach that actually works pretty well across everything, is to ask them a question in a technical area. They’ll give an answer and you won’t understand it. Ask them to explain it to you. And if they truly are a specialist in this area, they should be able to explain it in full depth, in terms that you understand. And someone not be able to do that is a pretty strong indicator that they aren’t actually strong in that area.

How do you rate the performance of an in-house engineer versus a remote engineer? [47:53]

[Eric] At Lattice, two of the ten engineers are remote, which is a really bad environment for them because there’s HQ, where everyone speaks and talks around the water cooler, and then you have them. They’re both phenomenal engineers, and they have to be able to produce in a bad environment.

If you want to encourage a remote environment, you have to set them up for success. You have to change the way you communicate, change the way that your dev tools work, and set up systems in place so that they can’t help but be successful. Also, make sure to determine if your business is a business that everyone needs to be in the same place. Even different roles have different requirements around this. A support role or a team that is working on a product that requires instantaneous turning to your left and talking to a person, you probably can’t do distributed.

[Dan] Heap has been remote since early on. Right now it’s around 40% remote. I would generally expect a person working without the distractions of your office to get more done. One of the biggest signals you should look for when hiring remote is the degree of structure in their life. If they don’t, things will fall apart and they’ll go crazy in six months. It’s really only at scale that you start to see a lot of the problems around keeping them in the loop, or keeping them feeling a part of the team.

[Jessica] Pilot has no distributed staff, but our first startup had a highly distributed team across many time zones. Working something out in person is a higher bandwidth way of doing it than being on a conference call, or being on a phone call, or being on Slack. Weigh the pros and cons of being able to hire for a distributed team. The talent pool is augmented by that fact. But the downsides of not having that in-person channel might be more important.

For Pilot in particular, given that for Pilot’s engineering team our customer is really our Product Specialist team (the folks who do the bookkeeping who are in the office with us), we would not choose to make that trade-off with this company with this setup.

In what context have you seen contracting work well? [53:17]

[Eric] We used contract work from a group in Argentina when I was with a previous company. We didn’t use them in the beginning — we started using them when the engineering team was around 25 people, and we used them for non-core projects. Things like integrations, where there was no proprietary technology. Integration between SFTP servers is going to be the same, no matter who programs it. That was a really great use in the beginning. It freed up our engineering resources to work on things that needed more context around the business.

[Ammon] Triplebyte employs 40 remote contract engineers. When we launched the in-house engineering team, there was a lot of time spent doing the interviews. At some point it became hard to hire for that role. Essentially we went to remote contractors to take advantage of the supply of people to want to work remotely. We offer it structured as contracting, although many of them are working 40 hours a week. This allows them to set their own schedule and work from anywhere in the United States. It also let us tap into a much larger supply of labor, and still have us hire strong engineers.

Jessica McKellar, Eric Koslow, Ammon Bartram, and Dan Robinson — all of them are CTOs at successful startups. They’ve been there, done that when it comes to hiring and expanding engineering teams.