Case Studies in Apprenticeship Vol. 2 — Blake Thomas

Time to Read:

Joe: Hey Blake, good to talk to you as always. Can you start off by telling us a bit about yourself and what you do?

Blake: Well, it’s great to talk about apprenticeship, it’s something I’m pretty passionate about. And anything for you Joe.

So, I’m a senior manager, which is one of those titles that could mean anything, and changes from company to company. For me it means I manage several teams of engineers and at the same time I drive a lot of initiatives in our department and in partnership with others. That’s a lot to say, so I usually just describe it as tackling challenges with non-obvious solutions. If they were easy, they would’ve been solved before they got to me. Actually, that’s part of how I got involved with our apprenticeship program.

Joe: Oh man, you’re good with a segue. How did you get involved with Enova’s apprenticeship program?

Blake: Right, well… as a hiring manager, every once and awhile you come across a resume or a candidate where you think “Yes. Yes… Oh.” It’s one of those things where you see the right behavior, the passion, everything, and then you look at their skills or job history and you realize that if you were to hire this exceptional person you’d be setting them up for failure because they just don’t have the necessary knowledge or experience to do the job they’d want to do. It’s always a bit heartbreaking.

Well, I had one of those. But this was a unique situation because the person was an internal candidate — they worked somewhere else in the company but wanted to be a software developer. In this case they worked in our call center, which is a solid job and pays well but it’s also hourly pay and a set schedule, which you’ll see is important in a second.

Anyway, I’m looking at this story, this person who taught themselves ruby using tutorials, pushed code to github, was going to meetups… I couldn’t just leave it at that I guess. So I reached out and started to make some arrangements, got them access to our floor of the building so they could go to our weekly tech talks and use our library, encouraged one of my senior engineers to do some pair programming… just generally tried to make resources available, as much as I could.

So, about five or six weeks into this we realized that this person had been coming into the office on one of their days off for the tech talks. That pretty much did it right there. Some brave person, who I wish was me for the record, spoke up and said we’re not doing enough, this is the kind of person we want to hire and we can teach them the rest. To my credit I guess I immediately saw the truth in it, so we put together an apprenticeship program that used our existing training program, but lengthened out the timetable and added a few things. Over time we built in mentorship and some other key pieces, but really it was a learning process for us as much as our apprentices. I use the plural there because not too long after the first we hired another, this time from an external source. Both of them have since been promoted, and we can really see the difference in terms of engagement and behavior. We’re planning on doing another round of two immediately after our current class of interns finish up, and we’ll scale the program from there.

Joe: You used the same curriculum as onboarding, but longer. How long? And how did the apprentices do with material that had been originally designed for more experienced developers?

Blake: Right, so I’ll answer the easy question first. We had no idea ahead of time how long it was going to take. We broadly estimated six months; we felt like we would at least know enough to call it a success or an eventual success by that point, or alternatively to say “it’s time to pack it in here.” In the end it took them right on about I think four and a half months, four months to get through the material for the most part. We’ve looked at the curriculum and we feel like with a little bit of reorganization that incoming apprentices are going to be able to do it in three, maybe a little over three.

The more interesting and kinda nuanced question is how the apprentices did with this material. Because you’re right, because of the nature of the curriculum, it’s designed to be this kind of self-directed curriculum with resources and pointers, and y’know, some coaching built in (or at least some opportunities for coaching). It was designed at least originally to be a kind of leveling and refresher course for more experienced engineers, which, y’know, the folks we’re talking about have a very wide separation in terms of experience, from folks who have gone through a traditional CS program.

In interesting ways, though. You might think, “oh, here’s a person with a degree in computer science. They clearly know more than this person who doesn’t have that”. But the truth is that they know different things. The folks who go through a traditional route have a very strong grounding in some of the theoretical stuff. They have a fairly good level of experience with some of the concepts as exhibited in more traditional, or older really, languages (C, C++, those kinds of things). What they don’t have, generally, is any kind of practical experience with shipping something. They may have used some form of source control, or maybe not. Depends on the program. They may have sort of stood up their own website, but that was completely orthogonal to their experience and curriculum usually.

And so it’s a very different set of experience, whereas the folks who are self taught, or go through a code school, and that’s what we’re talking about with regards to Stacy and Jill (

Jill was going through a code school, and so she was getting a kind of crash course in “here’s how you design for the web, here’s how you build a web application.” And it’s very much more — in both cases, in their own way — geared towards shipping.

Joe: Is that how you measure, then, that they’re finished? How do you decide that they’re “done” with this course of study. Especially if people who have ten years of experience don’t know everything in this training course.

Blake: I don’t think there’s one good measure. I think that we don’t live in a cookie cutter situation, otherwise we’d just hire from one source. And I think that sort of monoculture is what’s gotten a lot of companies into a mess. So I think there are a number of ways that we can assess their readiness to take on a more traditional engineering role, or be responsible for product delivery.

One of which, maybe the least obvious to a lot of folks, is conversation. You just gain a sense of what they know. Talk to them, in the same way you might interview someone, but maybe a little bit less formal. A little bit longer, a little bit more engaged. Talk to them about what they know, and what they’re comfortable with. What they’re less comfortable with.

Joe: Doesn’t that possibly introduce a lot of unconscious bias? The same kind of confirmation bias…

Blake: Absolutely. It absolutely does. I think that you would be really in a bad spot if you tried to claim that you were unbiased in this process. I think you would be really doing a disservice to everyone involved. I think the trick is recognizing the potential for bias and pursuing in as pragmatic and scientific a way possible.

So you talk to them about “what are you comfortable using. What did you struggle with. What did you have problems with.” You almost debrief them on the program and get a sense of how they felt about what they were learning. And then you try to line that up with their performance on some of the exercises.

You say, “well I can see here that you had some issues with branching, with source control. So let’s talk about source control. Where did you struggle with that? What was your experience with that? What was easy?”

Or you talk about, “look, I can see that you’re shying away from composition and kinda sticking to inheritance here. How do you feel about various object models? Do you feel uncomfortable about the idea of composition, or alternatively inheritance. Or do you feel comfortable with that?” If you have the data that you derive from their performance in these exercises, and you have the kind of empathetic discussion of how they felt with these exercises. You can figure out what they’re going to be ready to jump into, and what they’re set up for success for. In terms of their skills and their comfort level. And what are they going to need some more work on.

And at some point you just draw a line, because every engineer is going to be a little bit different. And you say “look, here are some areas where they could improve. But on the whole, they meet the criteria that we’re looking for in an engineer, based on this information.”

Joe: It sounds like there’s more of a continuum than there is a line where you’ve ticked one hundred boxes and now you’re a junior engineer.

Blake: Absolutely, and honestly, I said draw a line, but what I really should have said is, you take a notion of their holistic experience and you draw a line for them individually. And the only importance that line has is as a milestone for, “we’ve called you an apprentice until this point. Now we’re going to call you a software engineer.” And that’s the big change.

Joe: Explain to me how that bears out, given it’s a training program. So, I know they’re not working on production websites from day one. How do you make that transition?

Blake: Right, so what we did this time was a little bit different than I think we’ll do it next time. What we realized was, because of the nature of this exercise, wherein we were learning as much about how to run an apprenticeship as the apprentices were learning about how to apprentice.

We realized towards the end of it that, yeah, “we had you work on this capstone project. For the record you did a killer job. But there are still some areas we want to see you spend some time shoring up.” And so what I did, is I talked to both of them, and I did that debrief. I did a skills assessment, and I said, “okay, here are some areas that I want you to spend the next couple of weeks (literally, two weeks as it turned out) just doing your own kind of directed study.” And I gave them some resources for specific areas, and then had a conversation at the end of it about what they found, what they learned. What changed for them.

Once that had been completed, and we can talk about the formal process of them moving out of this apprentice role into a software engineer role, but once that had been completed they were, I will say, magically software engineers. There’s no magic about it, but they fulfilled what we asked them to do, and the whole social contract here was, “look, we’re on the hook for training you. You’re on the hook for bringing the A-effort. And when we’re done with this, you’re either going to be full-time, gainfully employed software engineers, or we’ll know the reason why not.” And that’s how it turned out.

Joe: How do you plan to do it next time? What are you planning to do differently?

Blake: One of the things we want to avoid is an open-ended conclusion of the program. That just has to do with the cost-conscious nature of product development these days. You’re always asking “when can I get more engineers. Aren’t those people engineers? Can’t they engineer something? Can’t we send them to make things?” So we want to bracket it. This time we’re allowing for a little experimentation, but the ultimate goal is to be able to finish this program in six months, consistently.

This next round we’re going to do something similar to what I did at the end of our first iteration. We’re going to do an assessment where we look at their performance on various exercises. We evaluate where they were good, where they struggled, do a debrief. And at that point, we’re either going to make the call to, “you know what, yes there are things you could work on, but every engineer has areas of skill where they could improve. Guess what, you’re a software engineer!”

Or, we’re going to specify some amount of time not to exceed three months where they can continue learning in a directed fashion to shore up those areas of development. The only way that we would elect to do that is if we had a really solid belief that, “here’s a person who is going to be successful, based on the arc of their learning.”

Joe: And importantly they’re getting decently paid during that time.

Blake: Yes, of course. Absolutely.

The difference is that as an apprentice, your compensation is going to be a little bit different than it is as a software engineer. There’s no two ways around that. One thing that I’ve personally discovered — and this is a really pragmatic matter, and every company is going to be a bit different. I’ll be honest with you, I think it’s a mistake to pay these folks hourly.

A lot of intern programs are very correctly hourly, because you don’t want to encourage these folks to overwork themselves and burn themselves out. It’s a little bit crazy. What I’ve found is that for folks that are working for months (and not weeks) that the distraction of thinking about hourly compensation — literally, wages and that kind of thing — it takes away from learning just a little bit. And I want these folks focused on just learning as much as they possibly can. And so, I think that this is something we learned about administering this program, that we intend to carry forward. Rather than being an hourly position, we looked at the concerns about it originally and we’ve made the call to make it a salaried position.

Joe: You already mentioned a couple things you wanted to change. Make it a little bit shorter, the capstone, et cetera. Of course a big problem with a lot of programs is that we’re talking months of “unproductive time”. Can you name a couple things that weren’t working well, that you can change to make the program more effective that way?

Blake: We really did take an Agile approach to this in a lot of ways. We had a training program that had the content we knew we wanted them to learn. But when we went into it, we had no idea what the end of this program was going to look like other than that it would be some sort of vague but bounded amount of time in the future.

And we realized as we got towards it, that we were driving a car that we hadn’t yet built the brakes on. And so we had to figure out how we want to wrap it up. And one of the things we decided based on some conversations with leadership, was that we wanted our apprentices to have an opportunity to show that they were delivering value, to address this question of value.

I understand that’s sort of a hot button topic for similar programs. We wanted to show that they were able to deliver value in the role of an engineer. What that meant for us, is can they take a project and deliver that project when it has very real implications for our business? Take it to production, literally deploy the application?

So we identified an opportunity to do that. It was a green field project. That’s not necessarily a requirement, but green field projects starting from scratch, there are certain characteristics that are nice for someone who is new to the field. We saw that opportunity and we capitalized on it. We said, “here’s this project, here are some stakeholders. We want you to work with them” both technical stakeholders and business-focused stakeholders, and they did.

Now the problem with that was that, being apprentices, they didn’t think to push back when someone in a senior role tasked with advising them started making decisions for them. By the time we were began to evaluate their work it was hard to discern how much they were able to execute on their own. So we had ask everyone to step back and let our apprentices figure out how to deliver features with a lot less intervention.

It’s true to say we don’t work in a vacuum, you always have access to senior engineers. But it’s hard to assess your skill level when you have so much outside involvement. So what we had to do ultimately was set up a period of time where, “Okay, you’ve worked on this. You’re a subject matter expert on this new application you’ve built. We’re going to take away the training wheels as it were, and let you ride on your own for a couple of weeks and see if you’re able to deliver the milestones that we spec’d out for you.” In the end they absolutely were. It was fantastic.

I wish that we’d done that from the get-go. I understand why it happened in the first place. We were trying to exercise a level of guidance and oversight that was both protective of the project and protective of the engineers. In the end, it was more of a hindrance than it was a help.

Joe: Do you feel like the scope is in a good spot now?

Blake: I think we’re mostly in a good spot. I think there are some efficiencies we can gain with the curriculum a little bit. There are some practical things that the curriculum doesn’t have built in right now that we want to encourage.

One big area there is pair programming. We’re very big on that as a concept. We want to encourage that and treat that as a way of self course correcting. I want to encourage them to both do pair programming amongst themselves — apprentice and apprentice — but also to work with more senior engineers to get a sense of what it’s like to work with folks like that, what style of work they have, what expectations they have when you sit down with them. So that’s a component we’re going to build in.

There are other elements that we want to move around a little bit and emphasize along with the curriculum. Engineering principle and practices that we want to integrate into some of the larger exercises. We want to — you know, I have a little bit of experience with curriculum design and one of the things that we’re approaching some of our material with and retooling individual lessons with a notion of “understanding by design”. Which sounds really common sense when you lay it out. The idea is to articulate, “what do I want them to come away from the lesson with?” and work backwards. Some people will call it “backwards mapping”, but it is in fact a good way to make sure that you are not expending extraneous effort in a competing a lesson.

One example of what we’re looking at doing (we haven’t decided one way or the other for sure), but at one point we have them build an entire Rails application. Which teaches them lots of very useful things in a kind of drive-by fashion. The scope is larger than it may need to be, though, for the things that we want to teach them. What we we’re talking about doing was, rather than having a start to finish full Rails app that has certain characteristics that we build into the exercise. What if instead we give them a set Rails app,with an existing codebase, and ask them to add a model and a controller that meet some specs and include behavioral testing, interacts with an API, those kinds of things.

That’s just an example. The point of doing that is, a lot of the Rails application stuff that they learn in building it from scratch, is interesting and it’s good for them to understand those underpinnings. But there are potentially more efficient ways to teach that that don’t necessarily require this end to end experience. You can think of it — as an engineer — you can almost think of it as “why do I need to do end to end testing, when I can have good unit testing for each piece, and then compose them in a way that’s sensible. And then have some level of integration or behavioral testing.”

Joe: So, changing gears a little bit. You mentioned that as the apprentices have moved into being full time engineers, you’d already noticed a difference in their engagement level, and also their behavior. Tell me a little bit about how that’s impacted the other people in the department as well.

Blake: What I have noticed is that our apprentices have become a kind of rallying point. And what I mean by that is, however you want to cast them, as socially awkward or whatever, engineers are incredibly social. This is a common misnomer. They’re quite social, they’re quite motivated to be helpful and informative. And they just happen to be, for whatever reason, a very opinionated bunch in a lot of cases.

Well, as it happens, they love to answer questions. Whether they know it or not. Sometimes they swear off “oh I don’t want to help anyone”. But the truth is that they actually do. I don’t know, it’s an odd phenomenon that I could talk about for hours.

The point that I’m getting at is, having folks around who are eager to learn and eager to absorb information, causes this weird shift in culture. I’m not even going to call it weird, it’s actually pretty obvious when you look at it. It’s kind of a human characteristic.

We start to become more open, because all of a sudden there are people who want the information we have. And that opens us up to sharing it. And curiously enough, not just with them, but with everyone. And then we start talking about the things that we’re sharing, and the things that are important.

Joe: How has this behavior played out for you? I know you do tech talks and so on. Is this something where people do more tech talks, or more informal information sharing?

Blake: All of those things. People are eagerly getting involved in these kinds of opportunities, both with the apprentices and in general. Even down to conversations about technology that are happening on a more common basis. It’s a more common occurrence to hear people talking about, “you know I was looking at the updated documentation for such and such”.

Because, it’s interesting, and it’s interesting to think about and talk about. You know what it is, maybe, is their presence. The presence of learning, and let’s just call it learning in general, because that’s what it is. The presence of learning is just enough to cause people to look up long enough to realize and remember that they’re curious.

Joe: That’s great. That’s pretty much the definition of engagement, right?

Blake: Exactly.

Joe: I have basically one other question for you. What is the future? It’s obvious that you’re treating this as a pervasive part of the department, not just a few people in one room.

Blake: Yeah! You know, it absolutely is. If it were up to me, I would hire about seventeen of these people, and find a way to onboard them if I could. I think I mentioned this before, but if I didn’t I’ll talk about it briefly.

When you’re no longer constrained to hire for skill and experience, you can hire people that have the attitude and the behaviors and the characteristics that you’re really looking for. And the thing that’s interesting about these kinds of folks is that the truth is the things that make you successful in college and as a student aren’t always the things that make you successful in a professional context. There are commonalities, but this gives us a way to get a diverse set of attitudes and ideas, while hiring — instead of for the skills, and the specific technologies, and the experience.

Joe: Which are impossible in this market anyway.

Blake: It gives us a way to say, not only is this person motivated, but they have a curious mind, they’re sharp. They’re ready to solve problems, they’re ready to work hard. I want to hire them even if they don’t know ruby just yet. Even if they haven’t written a dozen Rails apps, or maintained a dozen Rails 3 apps.

I know everyone’s on Rails 4 now…

Joe: I was going to say!

Blake: No, but I’m making a very specific point. We need someone who has experience with Rails 3, and all these people know Rails 4. We can teach the skills and hire for the behavior and the personality, and the strength of character, and the human characteristics.

And that’s fantastic. So that’s one of the things that it changes, and I just wanted to comment on that. But your question was a little different than that. You were asking, “where are we going with this?”

So like I said, if I could I’d hire seventeen, but we need to prove that this is successful, not just for the two people we’ve taken through the program, but for two more. And then three more. And then five more. And that we can execute on this consistently, because the truth is we have a track record of hiring experienced engineers, and being successful. And we have a track record of hiring engineers out of school, and being successful. We have a track record of bringing in interns, and converting them to engineers, and being successful. This is a new way to hire, and diversify the culture, literally, in a way that avoids this sort of monoculture of “oh we only hire engineers from this school in this program”.

So, we’re going to continue the program with an eye towards expansion. But for the moment, we’re going to focus on reproducibility. We’re going to go through a second iteration and try to tighten up our constraints and our execution a little bit in terms of the length and content of the program. Learn a little bit more, and once we’ve done that we’ll probably expand it modestly. So instead of doing two we’ll do three. Or if I can really convince someone we’ll do four or something like that, we’ll see.

We want to continue having this be something that changes our culture, that elevates everyone to this attitude of, not “how do I get out of the office today”, but “how do I learn something new and use it today”. And that’s maybe the biggest transformative effect, because when you see that happening you can’t help but want to be a part of it. And that’s really what we’re looking for, in software engineering, but really in the whole company if I’m being honest, at least for me.

Joe: Alright, cool. Thank you for taking the time out.

Blake: Absolutely.