alex_aldo - Fotolia
Leslie Carr has managed networks for some of the biggest names on the Web. She used to be a senior network operations engineer at the Wikimedia Foundation, and before that, she was a network engineer at Twitter, Craigslist and Google.
These days, Carr, who chats with us for this edition of "The Subnet," is a self-described "automation guru" at Cumulus Networks, a Silicon Valley technology company that develops Linux-based networking software for bare-metal switches. Carr spends her days elbow-deep in batch scripts and programming languages like Ansible, Python and Puppet, coming up with ways to automate configuration changes in Linux-based switches, just as sysadmins have been able to do with servers for years.
What are you working on lately?
I'm doing all these demonstrations -- it's demonstration craziness -- for our big 2.5 release. We're putting in new features, so I've been testing the features in a more real-world environment. I've got a lab with three racks of switches, so I get to play with all those, which makes me super happy. I've been doing a lot of testing and a lot of automation for demos to show off all the new features.
How did you make that transition from being a "pure" network engineer to one that dabbles heavily in programming?
When I started in IT, I did a lot of tech support for little companies, so I think I'm lucky in that I had some exposure to the languages. It happened socially -- just looking over people's shoulders and asking, "Hey, what are you doing? Oh, cool, let me see that." And then when I started at Google, I started in the data centers, and that's where I learned batch scripting because there were a lot of processes we'd do over and over again. I always think the best sysadmin is a lazy sysadmin because it's like, "I don't want to have to keep doing this over and over. Hey, if I use batch scripting, I can spend twice as long doing it once and then five minutes doing it every other time."
Wikimedia is where I really got into it, and what really helped was having a very supportive team. My co-workers were always really happy to help and review my code, which was really awful at first and became less awful as time went on. They'd critique in a nice way. It can be hard to find a great group of co-workers who have no ego because some people will just say, "Oh, I can do this better," instead of actually helping. But they said, "Hey, I see why you're doing it this way, but if you do it this other way, it's way better."
And also being able to Google questions on the Internet. I mean, seriously! Without Stack Overflow, I don't think I'd get half of my work done.
You've worked on networks for some of the largest Internet companies. What was that like?
When I started at Google in the data centers, we were replacing our core switches, so they had the vendor come out and give us this three-hour talk about how everything worked inside and I thought, "Wow, this is amazing." I think they had ten 24-port Gig-E blades, which back in 2004 was incredible. It was like, "Who's going to use all that bandwidth?!" Just the way all the traffic flowed in between the chips was so mind-blowing. Then, not long after, we upgraded our entire network to 10 gig, and then we thought, "We're doing this, so now they've got plenty of breathing room."
Then video started to become popular, and this was before [Google] bought YouTube. I still remember when we finally got 100 gigs at the edge, which we thought was amazing and huge at the time and which is still pretty huge for a lot of companies. We got little weights that were actually 100 grams [to represent 100 Gigabit Ethernet] and painted them gold because we were thinking, "This is amazing, and it'll take us so long to get beyond this."
Four months later, we got 250 gigs at the edge. Then about three months after that, we got 500 gigs at the edge, and they were saying, "All right, this is the last one." How video took off was insane. We thought, "Finally, the growth is going to slow down." Of course, it turned out that it grew exponentially, and now, Google doesn't even think in terms of gigabits anymore.
Then I went to Craigslist, which is very popular but is very text-heavy and very small-image-heavy -- no video -- so having to rescale my prefixes from terabits down to gigabits was a huge leap.
How did you get into IT and, specifically, networking?
My mom loves to tell this story: I was in preschool, and our preschool got an Apple II. It was new and shiny, and I got sent to time-out every other day because after I started playing with it, I didn't want to stop. I would push the other kids off the chair, and then I would lie and tell my mom I didn't get to use the computer at all. The teacher would tell her, "Actually, she was sent to time-out because she did not want to share." Then in high school, I was always good at science and math, and people would say, "You should become an engineer!" I liked chemistry, so I went to college to become a chemical engineer.
Leslie CarrCumulus Networks
I didn't have a computer in college, so I had to use the computer labs. I know I sound like an old lady -- I'm sure young people today all have laptops, but "back in the '90s," we had to share computers in a lab. The Windows and Mac computer labs always had long lines, but there also was this Linux computer lab and not only was it always empty, but the monitors were one inch bigger. So I was thinking, "I'd better learn how to use this Linux thing if I want to be able to get into the computer lab." I just sat down and was very confused and started asking people around me, looking over their shoulder and seeing what they were doing. I finally learned enough so I could explore on my own. And, honestly, since it wasn't my computer, I didn't have a fear of breaking it because I could just go sit at the next one.
While I was in college, I changed majors a lot and found out that chemical engineering was the least-exciting parts of chemistry for me. But I also needed a job, so I had several awful retail jobs. One of my friends needed tech support for a Web hosting/ISP company his friend was starting to broaden, so I was thinking, "OK, I think I can do that." I got to say a lot of times, "Are you sure your computer's plugged in? Are you sure the modem's plugged in?"
But that really got me started in IT, so I started doing lots of odd jobs. I built computers for six-packs of beer and things like that.
Then I had gotten laid off from my job in Pittsburgh during the economic downturn in the early 2000s. There were no jobs in Pittsburgh, which sucked, so I thought, "This is going to be my low point. I am going to apply for a job at McDonald's." I loved Pittsburgh and wanted to stay there, and then I got turned down from McDonald's for "not having enough retail experience." I was like, no, this is my low point. My unemployment was close to running out, and some of my friends who had moved to D.C. were saying, "There are jobs here." I looked online and put my application in at a few places, and one of the places that wrote back was Google. I got a couple of other offers, which was amazing because I went from "turned down from McDonald's" to getting several offers for technical, entry-level positions. And back in 2004, it was like, yes, I want to work for Google!
Back then, Google was smaller, so [after I got my first job there in the data centers], my manager called up the director of network engineering and said, "Hey, I've got a woman who really wants to get into your field. What should we do?" So the director of networking at the time, Cathy Chen, who is awesome, said, "Well, we have some deployments going on … so how about she just shadows and see how we work together? She can learn what's going on."
I started doing that and shadowing more and more. And I realized I was spending all of my time on the network and none of my time on the data center, so then I moved over to the networking team.
OK, one more geeky question: What's your favorite video game?
It's actually probably still Baldur's Gate or Baldur's Gate 2. It's a '90s D&D [Dungeons & Dragons] game, and they just rereleased it last year with a lot of bug fixes and upped resolution. It runs on my Mac laptop, and it's still so much fun. Even though the graphics have obviously not aged as well, I still think good storyline and good gameplay make for an awesome game. I have replayed it twice since it rereleased, and it's still a great game.
Network automation: A tutorial
Should you build or buy network automation tools?
Can you overdo network automation?