About me

Hello. If you’re reading this, it’s likely that you want to know who I am from a professional standpoint. I’ve worked on 3 teams over 8 years as a programmer at Amazon, and I currently lead the integrations team at a startup called WhyLabs. I’ve led various platform teams, designed build/release processes and internal tools/CLIs. I’ve created integrations with a wide range of technologies for whylogs (WhyLabs’ open source library), including Kafka, BigQuery, Snowflake, Dataflow, Docker, and more. I prefer to work on developer tools. I default to functional principles when I’m programming (immutability, composability, functions). I default to solving concurrency problems with actors. I care a lot about writing idiomatic, quality code that a domain expert could admire. I spend a lot of time on things others might consider silly, like making sure my Python class’ method types are properly inferred in a subclass, and I believe it’s an asset.

In general, I have a lot of strong opinions around programming and design. On this page I’ll try to help you understand how I ended up believing these things. The good news is, if you’re here for some gaming or bodybuilding related reason then you can skip all of this and check out the gaming or bodybuilding pages. I apologize in advance to everyone else.

2013 - Frying Pan

I had just graduated from UMaine with a degree in Computer Science. I was originally going to be a Kinesiology major, which I arbitrarily picked because I liked bodybuilding. I happened to meet one of my best friends the first year in York Hall, saw him programming in Python, begged him to teach me, and then transferred into Computer Science the next semester. I instantly knew I’d rather be doing this than anything else I’d seen so far, and I consider myself lucky to this day that I met him. I was terrified of failing, especially because of the math requirements, but I was stubborn enough that I went ahead anyway.

I was incredibly obsessed. Looking back, I’m surprised how much time I spent on programming. I discovered Star Trek TNG the summer after my first semester in Computer Science because I would program until 4am every day and see the latest rerun on channel 11. That’s how I knew it was time for bed. I recall being intimidated at the prospect of having to memorize public static void main(String[] args) for our first Java course (which I did). I met an important mentor who served as my muse for improvement and I learned it’s great having someone in your life who’s impossible to impress and far beyond your skill level. By the second half of my third year I was starting to feel very confident in my abilities, excluding discrete math. In my final year, I spent 6 months studying with “Cracking the Coding Interview”, which prepared me well for my surprise Amazon recruiter call. I did well on the two phone screens, was flown out for 5 more interviews, did well on all but one, and received an offer for $100,000 with a $20,000 sign on bonus. That was probably the happiest day of my life so far since it secured my future and represented a real correction in my life trajectory.

I was thrown into the frying pan in the Display Advertising org and almost immediately faced with making a choice about whether or not to move to a greenfield project to create a self service advertising website with one other developer, or remain on the current team to maintain the legacy system. I met with a few people for advice and the unanimous feedback was to switch to the new team. Not only because it was a new project, but also because of some political concerns between the two teams. My colleague would be in charge of the back-end services and I would be in charge of the build and deploy system, and front end libraries. We were building a platform that dozens of other teams were going to use to create pages on the website for setting up advertising campaigns, configuring spend, targeting different ad slots, etc. This was truly a sink or swim situation. I was surprised that they would just leave the fate of their future platform up to a random 23 year old.

Overall, this was a fun and exciting period of my life. I met a lot of people talented piers and mentors. I got to experience the full evolution of JavaScript from a series of files depending on jQuery to a full-fledged, transpiled programming languages with an elaborate build process. I had to learn how to deal with delicate interpersonal situations and I had to perform to meet deadlines. I learned a lot about how digital advertising works. Importantly, I learned that I like my work to be excellent and I gravitated towards others who worked the same way, while avoiding people I felt looked for excuses to finish as fast as possible. This was period of learning the “basics”, both professionally and technically.

2016 - Exodus

It had been three years since joining Amazon. We had launched the website and advertising on Amazon was doing very well. I recall being astounded at the figures being thrown around in our all hands meetings, but I was still impressed and intimidated at the thought of a four figure number, let alone seven. Our platform had evolved a lot since its creation. It went from a series of Backbone scripts that mutate the DOM into a React/ES6 based application built in Webpack, with tangents along the way. The front end was universally hated by everyone besides front end specialists and I was used to having a long leash because I actually liked the space. I found all of the instability in the JavaScript world to be exciting, but I can imagine how people would have been deterred by it now.

There wasn’t anything that I needed to do for my job that I wasn’t able to do. There were a lot of things I couldn’t do that I was insecure about though. I had mostly split my time between the front end and the “back of the front end”, which was the thin layer of CRUD services that directly supported the front end. I didn’t do anything that I needed my fancy degree for. The closest thing to me that I considered “hard” computer science was the real time forecasting system, which I never touched.

There were a lot of non technical concerns at this time though. The developers, myself included, were growing more disgruntled by the day for not being given the time to correct various parts of the platform that were aging poorly. This boiled over in a mass exodus of developers from the platform. I wasn’t happy about this because I really liked my teammates, but I was also over the internal drama. I learned that it was important to trust the leadership on your team after this. We didn’t trust leadership because we were too far removed from any of the choices they were making, and what we thought was important wasn’t what they thought was important.

My mentor was moving to another org as an SDM and I decided to follow him there because I trusted him. My negative sentiments may have been compounded by my irritability from starving at the time. I was competing in my first real bodybuilding show (which I won) and I had to deal with walking down to the secret kitchen every three hours to microwave fish so no one would yell at me.

2017 - Coloring Books

I followed an ambitious friend and mentor, who became a manager, to my next team on Consumer Engagement. I had a feeling about what went wrong from my first team but I had no idea what to do differently on my next team. My theory was that the most important part of choosing a team must be trusting the people in charge — the rest would work itself out. I felt confident in my technical ability within the scope that I tended to work so I wasn’t worried about being able to perform. I was still happy to work in the front half of the application too. I had a lot of niche skills around internal deployment tools and build infrastructure that most developers tended to avoid, which made me attractive and allowed me to have full authority on the things I worked on since no one else wanted to touch them.

I was really coming into my own technically. I had always liked functional programming and I had recently discovered the Elm programming language. It took me over a week to get anything compiling but I believed what I was learning would be a valuable investment. I would primarily look at what my mentors spent their time doing when deciding what I should be doing myself. The people I respected the most in college implicitly preferred functional programming and Linux, among other tools. I hadn’t forced them to articular how they arrived there but I assumed they had good reasons. I started to use several of the niche functional transpiled front end languages around this time as well for personal projects, like ClojureScript and PureScript. I absorbed several default technical stances that I still apply today, including immutability, composition over inheritance, side effect management, message passing for concurrency, and preference for coroutines over threads. I was really excited about the things I was learning.

I was learning so much in part because my new role was so boring though. I thought most of our org could disappear one night and customers wouldn’t even notice. We were building a chat application that no one asked for and only used because we stuck it on the product detail page. It wasn’t even clear what it meant to succeed or fail. There was a point where there were coloring books in the hallways for people on the floor to play with and I felt that I would slowly wither and fade away if I stayed there. Though I would continue to improve considerably at super smash bros, our lunch time activity. I learned here that trusting the management isn’t sufficient. My next theory was that you had to like the product that you were building and it had to be able to fail.

2018 - Finale

My manager friend was moving to a secret team in Amazon later known Amazon Halo (a fitness band), now closed. We worked well together and I was ready to leave the current project. I’m into bodybuilding and fitness and I had already been wearing a Fitbit for years. To me, this was the perfect opportunity to see if working on a product you would use yourself would make a difference.

I felt I was peaking in my technical abilities, which meant I had two options: drastically change the type work I was doing or move to a team where I could apply everything I’ve learned. Halo was perfect for the latter. I was in charge of the entire mobile application, which meant picking everything from frameworks, to build systems, to deployment methods. I was confident in my ability to do all of that. I ended up picking React Native as the application framework. I was already partial to react from my web experience but I gave the native SDKs a fair chance, creating a toy podcast app for Android/iOS. That application experience convinced me that I didn’t want to have a platform team with the maintenance burden of two native platform frameworks, and I wasn’t impressed with the MVC style development process. I really liked the functional approach that react took with the virtual DOM and there wasn’t anything that our designers were creating that we couldn’t do in react native. The momentum of the wider front end community was behind react as well. I crated a thin, Makefile-based wrapper around internal npm tools and Brazil (Amazon’s internal build system at the time) and set up a deployment pipeline that teams could integrate with to generate installable applications for android and iOS, independent of each other. I was satisfied with these choices throughout my tenure there.

There were some regrets though. Using React Native for the component of the application that we called the “data pipeline” ended up being a mistake. This was the code that interacted with our hardware device to filter and aggregate health data packets and upload them to our back-end. I wanted to make sure that the implementation was the same between iOS/Android and the performance benchmarks I was periodically doing at each stage confirmed that speed wouldn’t be an issue. The real issue ended up being reliability though, particularly on iOS. Sometimes React Native would get killed, or the app would start up in the background without initializing the React Native portion. This flakiness led to ballooning metrics around total sync time since it effectively only reliably worked while foregrounded on iOS. Fortunately we had those metrics, but the pipeline ended up having to be rewritten in native code for launch, which was a hard lesson learned.

The other lesson I learned at this time was that I really felt uncomfortable with how little every failure seemed to matter. We were told that we were a sort of startup within Amazon, but we were funded with tens of millions of dollars. A lot of our leadership was fresh off of the fire phone failure and it looked like Halo was bound to meet a similar end. We would make hardware mistakes that would cost tens of thousands of dollars to correct but it didn’t really matter because there was always money. Nothing felt anchored to reality and it felt like no failure would matter to anyone who mattered. My theory at this point was that it wasn’t enough to trust your direct leadership because you’d most likely be re-orged away from them anyway. It wasn’t enough to be a real user of the product you were making if you weren’t in control of how it gets created because you can’t have pride in something that you don’t respect. You can’t work in an environment without real accountability for failures if you want your work to be responsive to reality. Halo ended up closing down a few years later and most of the leadership transferred to different orgs within Amazon, having multiple large, failed products under their belt. Many of the SDEs, TPMs, and SDMs were placed onto the “sunset” team, doomed to be fired after the product was closed down in 6 months, shouldering the bulk of responsibility for the product failure.

2020 - Startup

Ultimately, I decided to leave at the start of the pandemic to go to a startup, WhyLabs. I planned to stay until our product launch but delays made it unclear when that would actually be. I knew I would matter a lot at WhyLabs, being one of less than ten people. I had connections (direct and indirect) with people who had started the company so I was confident I could trust them. Startups have to earn their funding so I was confident that reality and resources would influence the product and failures would matter. It was outside of Amazon so I would be forced to learn whichever technologies the rest of the rest of the world was using.

However, I had mixed confidence in my capabilities for the first time in years. I remained confident in my front end abilities, and I did contribute there at first, but I was still very specialized in front end technologies. One of my first projects at WhyLabs involved expanding one of our back-end services that used DynamoDB for storage. This was the sort of “back-end” technology that I never got my hands dirty with at Amazon. I had to learn about eventual consistency the hard way, even when I thought I understood it. All in all, it went smoother than I had anticipated, but it was still challenging.

My next project involved enabling our containerized whylogs REST API to upload what we call “dataset profiles” to WhyLabs. These are statistical summaries of data that are a fraction of the size of the original data. The container was basically a hack project at this point with a few files and this was the first time I touched anything related to Docker. The container ended up being a great stepping stone because it shared a lot of problems with the data pipeline I built at Halo and I ended up solving those problems with similar solutions inspired by my experience years earlier with the actor model. From there, every project was a new technology. Instead of going deep on a single thing for years, I had to jump back and forth between multiple new frameworks and languages every few weeks. I got really good at getting 85% of proficient in the span of 2–4 weeks for each new project — a skill I value greatly now. The foundation I built at Amazon through a combination of independent study and sink-or-swim project leadership made the transition outside of Amazon smooth. Despite the drama, I’m absolutely grateful for having worked there.

At this point I’ve confirmed the lessons I’ve learned in the past. Trusting the leadership of the company you work for is essential. You need to matter a lot in both technical direction and product development. You need to be in an atmosphere where there’s a real and obvious difference between success and failure. Surprisingly, being part of the target user of a product has been least important for me so far. I’ve learned that my strengths have always been my curiosity and my willingness to spend an unreasonable amount of time to do what’s “right”. I’ve learned that I love making tools and languages, as well as becoming as expert as I can manage at the tools and languages that others use. These experiences anchor my decision making now and documents like this one help me explain how I ended up here. I’ve learned to be alarmed when a team starts to make fun of its own product and I still don’t know if it’s too late once it reaches that point.

2023 - Now

Based on my experiences, I have a list of things that I think are important to me when deciding to work for a company.

• Trusting leadership is necessary but not sufficient. I need the ability to influence the project and technical direction, otherwise, I’ll be at the whim of choices beyond my control.
• Being a user of what I’m building is nice, but not necessary, and definitely not sufficient.
• Being able to fail and needing to earn success are necessary, and maybe sufficient. Without this, no choices matter, no one is accountable, and no one can say that anything ended up better than it would have been otherwise.

I have default technical stances that I take, with a willingness to abandon them for specific use cases.

• Everything should be immutable and side effect free. I should be able to reason about every piece of code knowing these principles hold. I get a little grump when they don’t.
• Using actors (message passing) to resolve concurrency issues instead of locks. Performance is usually the thing that makes me reconsider this, but it’s excellent for most typical use cases in service oriented architectures, and even does fairly well in plenty of performance critical scenarios (like the WhyLabs whylogs container).
• Everything should be simple and flat. I’ll forgo using a feature rich build tool (like grunt/gulp in the past) in favor of a plain Makefile that I understand from top to bottom. Abstractions should be as flat as possible with small surface areas.
• Strong type checking. This is truer now than ever with amazing type checking tools like Pyright and TypeScript.

I know what I like to work on: developer tools, concurrency problems, and DSLs\programming languages.

Here are some career highlights.

WhyLabs

• Lead Integrations engineer, responsible for creating integrations between our open source library (whylogs, langkit) and popular services like Snowflake, BigQuery, Dataflow, Kafka, and more.
• A lot of time went into developing a container based integration solution that served as a fallback integration for anyone who couldn’t use one of our more specific integrations. The container was originally written in Java but was rewritten to be a highly performant Python based REST service for hosting whylogs. A lot of optimizing went into this project because whylogs is CPU bound and the Python GIL exists. In the end, the container has a throughput of 2000tps on an AWS m5.large host, which is quite respectable.
• Actor model inspired concurrency solution to work around CPU bound bottlenecks in Python. This is the foundation of the whylogs process rolling logger and the core of the whylogs container — lock free, multiprocessing.

Amazon

• Led the platform team for Amazon’s burgeoning self service display advertising business. The team was brand new at the time and I was one of two developers on the project. My focus was on creating deployment tools and the front end framework for the various teams who created experiences on the display advertising website.
• Worked in Consumer Engagement to create a chat application on the retail website that users would use to ask and answer questions.
• Led the platform team for the Amazon Halo (fitness band) mobile application. I created all of the initial build and deploy tools and processes for a dozen other teams in the app and was tech lead for the team that maintained and iterated on those tools. I also worked on critical parts of the application, most notably the data pipeline that managed downloading, aggregating, and periodically uploading data from the Halo band.

I don’t have all the answers yet. I have no idea what I should do next. Things are going a lot better than I thought they would and I feel very fortunate that I started my career in the largest growth decade in history, working in physical proximity to a lot of excellent people. So I’m playing it by ear and updating these principles along the way.

See also

Some relevant social media/online profiles.

• GitHub
• LinkedIn
• Twitter