Software Sessions

Paul Smith is a Software Engineer at GitHub and the creator of the Lucky web framework. He previously worked at heroku and thoughtbot and has experience building applications using Rails and Phoenix. He's also the creator of the Bamboo e-mail package and the co-creator of the ExMachina test data package for Elixir.

We discuss:

• The tradeoffs of object oriented and functional programming
• How a lack of compile time guarantees slow down ruby and elixir development
• Creating conversational error messages
• Ways fast languages can change how you write applications
• Writing templates with Crystal instead of HTML
• Choosing what to include in a web framework
• The Crystal community and ecosystem

Related Links:

• @paulcsmith
• The Crystal programming language
• The Lucky web framework
• HTML2Lucky
• AlpineJS

This episode originally aired on Software Engineering Radio.

Transcript:

You can help edit this transcript on GitHub.

Jeremy: Today I'm talking with Paul Smith.

Paul is the creator of the lucky web framework and he currently works at GitHub. Today, we're going to talk about the crystal programming language and the lucky web framework. Paul, welcome to software engineering radio.

Paul: Thank you so much. Happy to be here.

Jeremy: There are a lot of languages for software developers to choose from. What excited you about crystal?

Paul: Yeah, that's really interesting because when I first saw Crystal, I actually was not interested at all. it basically looked like Ruby to me. And so I just think, okay, so it's a faster Ruby. And typically if I want to learn a new language and want something that feels really different, that pushes the boundaries on things.

I started getting more interested in compile time guarantees. I worked at thoughtbot previous to github and previous to Heroku and people were starting to get really into typed languages. Um, some people were starting to get into Haskell, which is like, you know, the, the big one that, I guess is probably one of the more type safe, but also hard to use languages.

Um, but also Elm, which has a good focus on developer happiness and productivity and explaining what's going on. And as they were talking about, how they were writing fewer tests and it was easier to refactor, uh, it started becoming clear to me that that's something I want. Um, one of the things somebody said was, if the computer can check the code for you let the computer do that rather than you, or rather than a test. so I started to get really interested in that. I was also interested in elixir, um, which is another fantastic language. I did a lot of work with elixir. I built a library called bamboo, which is an email library. And another called ex machina, which is what a lot of people use for creating test data. Um, so I was really into it for awhile.

And at first I'm like, wow, I love functional. And then I realized like. I can do a lot of, like a lot of the stuff I like about this I can do with objects. I just need to rethink things so that it uses objects rather than whatever random DSL

Jeremy: Cause I mean, when you think about functions, right? Like you've got this big bucket of functions and you got to pass in all the parameters right? Whereas, you know, in a lot of cases, I feel like if you have those instance variables available in the object, then the actual functions can be a lot simpler in some ways.

Yeah.

Paul: Totally. That's like a huge focus and making the object small so that it. It doesn't have too much, but that's how I began to feel with elixir is that I'm like, I just have 50 args and most of them I don't care about. Like I want to look at what's important to this method, to this method.

It's, you know, this argument, but with functions you're like, which things important. Is the first thing? Probably not. That's probably just the thing I'm passing everywhere. And so I liked that ability to kind of focus in and know like, this object has these two instance variables everywhere.

Jeremy: Yeah. It's kind of interesting to get your perspective because, it seemed like you were pretty deep into elixir if you had created, bamboo and ex machina and stuff like that, so it's kind of

Paul: Yeah. I was like way gung ho and, and then I started missing objects. And luckily with crystal and ruby, you still get a lot of the functional stuff. Like you can pass blocks around. Um, that's functions. You can use functions. But it's not the other way in Elixir, you can't use objects. It just doesn't exist.

And then the type safety. I'm just like, I still run into so many errors and it was so frustrating. I don't want to do that.

The main benefit I got out of elixir compared to rails, um, which is what I had been using and still use a lot of, was speed. That was really big. Um, in terms of bugs caught about the same, mostly because it's still for the most part dynamically typed language with very few compile time guarantees. Um, so I'd still get the nil errors. I'd still mess up calls to different functions and things like that. And so that's where I ran into crystal. It has the nice syntax. I like from elixir and Ruby. It's also very, very fast. Faster than go in some benchmarks.

So it's quick. Plenty fast for what I need.

And it has those compile time guarantees, like checking for nils. That's a huge one. and it also makes the type system very friendly. So it does a lot of type inference. And very powerful macros so you can reduce some of the boiler plate.

And so that's when I kind of started getting into crystal was seeing Elixir I still got a lot of these bugs that I was running into with rails, but I liked the speed but I don't want to use Haskell and Elm doesn't exist on the backend. so I started looking at crystal.

Jeremy: And so it sort of sounds like there's this spectrum, right? You have Ruby and you have, elixir, where you don't necessarily specify your types so the compiler can't help you as much. And then you've got Haskell, which is very strict, right? You have a compiler that helps you a lot. Um, and then there's kind of languages inbetween Like. For example, Java and C and things like that. They've been around for quite some time. how does crystal sort of compare to languages like those ?

Paul: Yeah, that's a great question cause I did look at some of those other ones. TypeScript for examples is huge. Kotlin was another one that I had looked at because it's Java but better basically. That's the way it's pitched. And so far everyone that's used it has basically said that. And also looking at rust, what it came down to was how powerful was the type system. So crystal has union types, which can be extremely helpful, um, and it catches nil. Java does not have a good way to do that. Um, Kotlin does. But also boiler plate and the macro system crystal's is extremely powerful. Elixir also has a very powerful macro system.

But crystal's is type safe, which is even more fantastic. So basically what that let me do with lucky, it was build even more powerful type safe programs. And we can kind of get into that once we, we talk about lucky and how that was designed. Um, but basically with these other languages, a lot of what we do in lucky just simply wouldn't be possible or wouldn't be possible without a significant amount of work and duplication.

Jeremy: You covered a few things there. One of the things was, macros, what are are macros?

Paul: Yeah. This is like a confusing thing. It took me a while to, to get, um, what it is. But, uh, in Ruby, for example, they have ways of, of metaprogramming. That are not done at compile time for most compile time languages, compiled languages, I should say. You need macros to de-duplicate thing, and basically what a macro does is it generates code for you.

The way I think about it is basically you've got a method or a macro, but it looks like a method. It has code inside of it. And it's like you're copy pasting, whatever's inside of that macro into wherever you called it from. So in other words, rails has a, has many, like has many users, has many tasks that's generating a ton of code for you.

So that's how Ruby does it. Um, and crystal has many would be a macro and it would literally generate a ton of code. And copy paste that into wherever you called it. Um, so it's just a way to reduce boilerplate.

Jeremy: So in the case of dynamic languages, like Ruby, when you talk about Metaprogramming, that's having I guess, a function that is generating code at runtime, right? And the macro is sort of doing something similar except it's generating that code at compile time. Is that kind of the distinction?

Paul: That's the way I look at it. there are people much smarter than me that probably have a more specific answer about what the differences are, but in my mind and in practical usage, that's what it comes down to in my mind.

Jeremy: Let's say there's a problem in that code, what do you get shown in the debugger?

Paul: Debugging macros is definitely harder than debugging your regular code for that exact reason. it is generating a code. So what crystal does, uh, there's different ways of doing this, but I like Crystal's approach. I

Henri is a frequent conference speaker and organizer of the Toronto Web Performance and JAMStack meetups.

We discuss:

• Managing images with features like lazy loading and the picture tag
• Handling varying network conditions on mobile
• Making designers a part of the performance conversation
• The WebP image format that could replace JPEG and PNG
• Ways the GIF can be an MP4 in disguise
• How lighthouse has given websites a visible target for performance
• What we can learn from "lite" news sites

Conference Talks

• A Decade of Disciplined Delivery
• Shape Of The Web
• Moving Pictures: A Snapshot At the Future of Web Media

Related Links

• @HenriHelvetica
• Cloudinary
• Picture tag
• Network Information API
• WebP
• Responsive images done right: a guide to picture and srcset
• Use srcset to automatically choose the right image
• Mozilla: Improving JPEG Image Encoding (Mozilla explains why they want to stick with JPEG in 2014)
• The Great JPEG 2000 Debate: Analyzing the Pros and Cons to Widespread Adoption
• How JPEG XL compares to other image codecs
• Serve images in next-gen formats
• JPEG XR
• High Efficiency Image File Format
• Using HEIF or HEVC media on Apple devices
• AVIF for Next-Generation Image Coding
• AV1 & Media Codecs
• WebP is now supported on Safari 10 (WebP support was added in a Safari beta but then removed)
• Safari 14 Beta Release Notes (4 years later, WebP is officially added to Safari)
• HTTP Archive Almanac: Image format usage (Shows the relatively small footprint of WebP)
• Native image lazy-loading for the web
• How To Defer, Lazy-Load And Act With IntersectionObserver
• How Medium does progressive image loading
• Above the fold in web design
• An update on mobile CPUs and the Performance Inequality Gap
• Web Page Test
• Lighthouse
• CNN lite
• NPR text only
• User Timing API - Measuring User Experience Performance

People mentioned during this episode

• @burkeholland
• @kornelski
• @andydavies
• @patmeenan
• @slightlylate
• @souders

Music by Crystal Cola: 12:30 AM / Orion

Transcript

You can help edit this transcript on GitHub.

Jeremy: [00:00:00] Hey, this is Jeremy Jung and you are listening to software sessions. This episode, I'm talking to Henri Helvetica. He's a freelance developer with a focus on performance engineering. He's also involved in the Toronto web performance and JAMstack meetup groups. And we discuss why images and performance are so tightly tied together.

We also went deep into what life after JPEG might look like with the introduction of formats, like web P. And we talk about tools that can help you during your web performance journey.

Henri is a big runner so i asked him if he started his day with a run.

Henri: [00:00:36] Thank you for the introduction. And good morning. And with regards to a run, I wanted to first thing in the morning, and as we were talking about, getting up early just moments ago, I have my alarm set for 6:30.

I tend to sort of open my eyes up around quarter to six, and figure out like how this run is going to go, but it was raining this morning. so I was a little upset, went and looked outside. The rain stopped by 7:00 AM. I was thinking, okay, maybe I should head out now. And as I was getting ready to head out, the rain started again and I thought to myself, okay, it's not going to happen simply because I knew I was doing this podcast.

So I want to be back in time and fresh. And, and afterwards, I think I'm going to watch, this, Microsoft event they have online, MS create. yeah, run does not look good today.

And it's funny. I was speaking to Burke, Holland from Microsoft. He said that he sent the clouds my way, knowing that it would force me to stay in.

Jeremy: [00:01:34] They are a big cloud company, right?

Henri: [00:01:36] That's a good one. I like that.

That was good. I'm going to have to keep that one.

Jeremy: [00:01:42] You're, you're pretty deep into the web performance space. What are some of the biggest mistakes you see people making on the front end?

Henri: [00:01:52] I mean, web performance I, I consider it a bit of a dark art. There's lots involved and, and much of it may not seem, very clear to the sort of like average developer at times. but with any auditing that takes place, whether it be web performance or accessibility or, UX, overall, you're always going to have some low hanging fruit, and, One of those fruits is, image management. and I think that, you tend to find a lot of people sort of disregarding the importance of making sure that images are set properly, as a resource loading on your page. and. It's important for a number of reasons. most notably is the fact that it's always, absolutely going to be the heaviest resource on your page. Okay. Barring video. and you know, video in the last say couple of years, specially, especially this year has become a lot more prominent. So I mean, that's a bit of a different conversation, because, you know, you could quite often find pages with no videos. So I didn't want to go too deeply into that topic, but, you know, you will find images 99.9% of the time and images are challenging. Image management has become a lot more complicated, for a number of reasons. Retina screens brought in a particular challenge with regards to how to select the right image. and then you also have more than ever people are really paying attention to, connectivity, understanding that, connectivity may vary along like a five minute period, what was 4G at the start of your walk might suddenly downgrade to like very poor 4G or even like moderate 3g. Then you might go into your home and back out. And so you have varying conductivity that, ultimately the site doesn't care about. It's like, Hey, just load this image.

You have these things to take in consideration and you luckily have some very brilliant engineers out there that are trying to make these accommodations. So, I would certainly say, images, Have been, are, and potentially will continue to be, one of the bigger challenges in terms of a low hanging fruit.

Jeremy: [00:04:15] I want to go a little more into

Pete is the senior security researcher at Brave Software and the co-chair of the W3C Privacy Interest Group.

We discuss:

• The differences between academic research and product development
• How websites track us with cookies, fingerprinting, and other techniques
• The surprising amount of data your browser gives up without a permission prompt
• What features should go in the browser vs being native only
• The confusion behind what incognito or private modes do
• The role of EasyList and the underappreciated people behind it
• Building tools like PageGraph to identify and rewrite tracking code
• Replacing resources at page load to preserve privacy without breaking websites
• Developing web standards at the W3C while preserving privacy
• Brave's plan to fund websites with ads instead of paywalls while preserving user privacy
• Getting involved in privacy and web standards

Related Links

• @pes10k
• Personal Site
• Privacy Interest Group (PING)
• W3C Privacy Community Group
• Research at Brave
• SpeedReader: Fast and Private Reader Mode for the Web
• Detecting Filter List Evasion With Event-Loop-TurnGranularity JavaScript Signatures
• EasyList
• uBlock Origin
• Panopticlick
• Protecting Against HSTS Abuse (WebKit)
• What’s the Difference Between First-Party and Third-Party Cookies?
• Understanding Redirection-Based Tracking
• MediaDevices.enumerateDevices() (Specifications give server all devices and labels without asking for permission)
• CSS difficulty with adoption (CSS2 spec was written without an implementation causing problems)
• What Are CSS Vendor or Browser Prefixes?
• FPRandom: Randomizing core browser objects to breakadvanced device fingerprinting techniques
• Brave Fingerprinting Protections v2: Farbling for Greater Good
• PageGraph
• Puppeteer
• Target: You Can’t Hide That Baby Bump From Us
• Brave Ads FAQ
• LUMAscape

Music by Crystal Cola: 12:30 AM / Orion

Transcript

You can help edit this transcript on GitHub.

Jeremy: [00:00:00] In this episode of software sessions, I'm talking to Pete Snyder about the many ways websites track us. How ad blockers like uBlock Origin work. And the process of developing web standards with privacy in mind. We start by discussing his role as a senior privacy researcher at Brave software

Pete: [00:00:18] Brave is kind of interesting or unique as a startup in that we have a proper research lab. I think our research team is seven or eight people right now. Those are people who do research in the form of published publications but also doing research that ties back into product in some way.

My research responsibilities are to figure out new ways that you can improve browser privacy, address tracking on the web, and solve the kinds of problems that Brave is interested in solving. I have one foot in engineering world and one foot in publishing world.

Jeremy: [00:00:48] Why is academic research important in this space?

Pete: [00:00:52] My gut feeling is that what's useful about academic research is that it changes the incentives and it gives you a chance to do things that are more novel and particularly things that are less tied to a short term ROI cycle. That is particularly useful for things that have watchdog functions over industry, things that are more difficult to monetize but more useful to average web users.

That's not to say there aren't people who try to build businesses around privacy or responsible computing but the incentives don't always work that way. What's really neat about doing a research focused computing career is you can do things that don't have to make somebody money in the short term. You can pick more oddball projects. The things that might not come to fruition right away.

Jeremy: [00:01:36] And is there a key difference in how you approach a problem when you're doing it in an academic context versus as a product for a company?

Pete: [00:01:46] Sure. So they go both ways. If I'm working for something at Brave the emphasis is on correctness and certainty. And knowing that when we ship it to 10 million people or whatever that it's not going to break and it's going to do what it says on the tin and that it's going to be a material improvement over the state of things before we ship that feature.

And that's really different than if you're trying to come up with a research project where.. sometimes good, sometimes bad, but the emphasis is not necessarily on a hundred percent correctness but is on novelty and doing something or figuring out some way to solve a problem in a way that it hasn't been tackled before.

And so you'll read research papers that say it works 95% of the time and that'll be sufficient or compelling for a research paper. But you wouldn't want to ship something that breaks 1 out of 20 websites if you're actually making a product. The goals are different, but also the success criteria are different.

Jeremy: [00:02:39] So it sounds like you can tackle things where it wouldn't be good enough for a product yet. But it's something that if you were working on it within the context of a company, they might say: Oh, we're not going to do that because it just doesn't seem like it's going to work.

Pete: [00:02:54] Yeah, exactly. So, maybe because certainty of success isn't there, or there isn't a one or two step obvious path to being a product. Maybe it conflicts with the current business goal or whatever else. But yeah, you have much more latitude in terms of products you can choose and kind of problems you want to tackle.

If you're writing research papers and not that I'm some incredible researcher or anything, but if you try to do successful research it doesn't reward you to solve that final 5% of the problem.

There's no benefit to getting no, not none, but there's a small benefit of going from 95% to 99% success or accuracy. On product you have to grind out as close to a hundred as you can get.

Jeremy: [00:03:37] And do you have examples of things where you worked on it in a research context and it actually became a part of a product?

Pete: [00:03:46] Sure. Yeah. So a couple of things. One is that.. so there's a research paper that we wrote at Brave called Speed Reader. Speed Reader is a different way of doing a reader mode in a browser. Right now, if you use any of the reader modes in popular browsers, you download the page, you render some subset of it.

You throw some JavaScript at it and it extracts sections that it thinks are useful, then it presents a new page to you. That's not a hundred percent correct. Chrome's DOM distiller does something slightly different, but to approximation you render the page and then you extract stuff out of it. Brave speed reader does something different.

It intercepts it at a network layer. It examines the text HTML. Does the analysis there and then feeds that back to the rendering engine. And so there's a bunch of nice benefits there. There's a privacy improvement in that you're executing less code talking with less third parties. There's an performance improvement as well in that you don't have to do the initial displaying and tear all that stuff down and build it back up. So that was the research paper that we published at. WWW 2018 2019. I don't remember, but either a year or two ago. And it's now in beta in Brave. That's maybe the oldest one. The most rece

Vlad is a Pluralsight course creator and the author of Unit Testing: Principles, Practices, and Patterns.

We discuss:

• Immutability
• Error handling
• Avoiding null
• Preventing invalid state
• Updating existing applications

This episode originally aired on Software Engineering Radio.

Related Links

• @vkhorikov
• Enterprise Craftsmanship
• Is Entity the same as Value Object?
• Combining ASP.NET Core validation attributes with Value Objects
• Error handling: Exception or Result?
• Applying Functional Principles in C#
• F# for Fun and Profit

Transcript

You can help edit this transcript on GitHub.

Jeremy: [00:00:05] Hey, this is Jeremy Jung for Software Engineering Radio. Today. I'm talking with Vladimir Khorikov. Vladimir is the author of the book Unit Testing: Principles, Practices and Patterns he's a Microsoft MVP, and he's the author of many Pluralsight courses including Applying Functional Principles in C#, and today we're going to be talking about functional programming in enterprise applications. Vladimir welcome to Software Engineering Radio.

Vladimir: [00:00:28] Thank you for having me.

Jeremy: [00:00:29] The first thing I want to talk about is sort of what functional programming means to you, because it means different things to different people. To you, what are the core principles of functional programming?

Vladimir: [00:00:41] If I were to describe functional programming in just a couple of words, I would say that functional programming is programming without hidden inputs and outputs. And that's basically it. And what I mean by hidden inputs and outputs is there are several examples of those. So the most, prevalent example of a hidden output is immutability.

So let's say that you have a method that takes in some integer and then increments that integer by one. And what it can do is it can return that incremented integer back as the return value, but. It can also mutate some global state with that integer. And by the way, by hidden, I mean that this information is not present in the method's signature.

So it's not present in the method arguments, and it is not present in their methods, return value. So in this example, when you, when you have this increment method if it, returns. A value, then it communicates what it does, pretty clearly. So it is honest about what it does. But if it instead mutates, global state, then this would be an example of a hidden output because that output is not present in the map at signature.

And to understand what this method does, you have to drill down to that method and to see what. What's actually going on, because it can this information is not present, in the, in the signature itself. So that would be an example of a hidden output. The hidden input is a similar to that. So instead of taking that integer as a parameter, as an argument. This method can also refer to some global state. So, for example, some static property or field, or it can refer to some external service to request that in integer and then incremented and then put in to some global state. So that would be an example of a hidden output.

Also reaching out to external systems such as the database or APIs would be that and also, the simple DateTime.Now would also be an example of hidden input because that input always changes. It basically refers to the systems low level, API, your, for example, windows API or Linux API to, to get, this input value so that's another example of, of a hidden input. Another example of a hidden output would be something like exceptions. So exceptions are hidden output because, when you throw an exception, and that exception is caught somewhere upper, the call stack. This exemption also is not present in the method's signature.

And, you basically introduce another, Hidden pathway, for your, program flow that is not present in the method signature. So these are common examples of hidden inputs and outputs and functional programming is about avoiding those hidden inputs and outputs. If your mathematical function, your pure function would be something that, accepts a value, as an argument and returns a value and doesn't do anything other than that.

Jeremy: [00:04:06] Okay, so let's sort of break down a few of those. So. The first thing is hidden outputs in terms of if you pass something into a function and the thing that you pass in becomes changed, then that in effect is a, hidden output because, there is no way of telling just from the method signature whether that behavior is possible.

Vladimir: [00:04:30] Correct. Yes.

Jeremy: [00:04:31] And so you're saying that the alternative to that is to make sure that the function is pure so that when you pass something in, if you are going to make a change, it would not be to what you passed in, but it would be something that you're returning back.

Vladimir: [00:04:50] Yeah. So instead of mutating the state of some existing object, what you need to do instead, in functional programming is you need to create a new object with the required properties. So instead of mutating the object that is passed in, you need to create a new object with new properties and return it back.

And with example, with a number increment that that's basically it, because when you increment the number by one and return it back, you're not, Changing the input parameter because it's a constant, you cannot change it. What you do instead is you create another number and return it back.

Jeremy: [00:05:26] And when we think about objects that we pass in, or we think about collections. a lot of times the objects and collections we work with are mutable, right? Like, we can have a list type in C sharp, for example, and we may want to add something to the list or remove something to the list. If we are instead creating a new list every time we want to change something, what are the performance implications of that?

Vladimir: [00:05:55] Well, yeah, definitely. So there are trade offs to functional programming. And one of the most common tradeoffs to, any immutability is this trade off of, Always creating new objects instead of mutating new ones. And, that's actually the reason why object oriented programming, has become so, so popular in the past.

Because if, you know, functional programming actually was introduced before object oriented programming. but why it didn't take off is because, computers back then were not as powerful as now. And so it was very costly to do functional programming with all those memory allocations, with all those new object creation.

So it was very costly to do so. And what we had had instead is we started to operate at the lower right level, of our programs. And we started to think of, , in terms of, memory allocations. So, but now we're kind of getting back to the roots and starting to, to do more of what we did back then.

There are definitely trade offs here. And if your performance requirements for your application are strict, then there are some restrictions. So there are some limitations and probably you will not be able to implement some functional programming approaches.

But in most business line applications, that's not something you need to worry about. So if you write some framework, for example, an ASP.NET Application not application, but the server itself, Kestrel, then you do need to worry about that. But in most enterprise level applications you don't, so performance is not the biggest concern.

The biggest concern is usually the complexity and the uncontrollable growth of complexity. And, what functional programming allows you to do is it allows you to reduce that complexity at the expense of maybe not as performant code as you could have otherwise.

Jeremy: [00:07:56] So would you say that in the average application that the developer should default to making things immutable, is that a reasonable default for most developers?

Vladimir: [00:08:09] I would say so, yes. If it is possible, then you definitely need to default to creating immutable classes, immutable objects by default, it is not always possible and one of the limitations why is in object oriented languages it's pretty hard to create new objects based on the existing objects. So, for example, if you take F#, there is a really nice language feature where you, where you can take data structure or an object, and, create a new object based on that, existing object, but with the mutation, of, with the addition of new properties to that object. So you can say, for example, old object with, some property equals a new value and some other proper property equals other new value.

And what it will do it will not mutate the existing object, but it will create a new object with those two properties changed and then, but all the old

Brian is a Senior Developer Advocate at GitHub and was previously a Developer Advocate at Netlify.

We discuss:

• Unintentional gatekeeping
• Formal onboarding for your projects
• The value of discord communities for newcomers
• Streaming issue triage and programming on twitch
• How Open Sauced helps developers get involved with open source

Related Links

• @bdougieYO
• Open Sauced
• ExpressJS Triager Guide
• GraphiQL
• Babel's contributing guide (Gives suggestions on familiarizing yourself with codebase)
• Webpack's funding page
• SE Unlocked episode with Dan Abramov
• Explore GitHub
• Changelog Nightly
• Code Triage
• Figma
• Discord

Twitch Streams

• bdougieYO
• jlengstorf
• Noopkat

Music by Crystal Cola: 12:30 AM / Orion

You can help edit this transcript on GitHub.

Jeremy: [00:00:00] This is Jeremy Jung and today I'm talking to Brian Douglas, he's a senior developer advocate at GitHub, the host of JAMstack radio and the creator of open sauced, an application to help new contributors to open source.

Brian welcome to Software Sessions.

Brian: [00:00:14] Hey Jeremy, thanks for having me on.

Jeremy: [00:00:16] The first thing I want to get into is. What's the biggest barrier for people getting into open source?

Brian: [00:00:23] Yeah, that's a good question. I think the barrier for open source is something I found or discovered right off the bat. I've been developing for over seven years now, seven years ish. And getting into open source can be daunting, especially if you don't know where to get started.

So I think the biggest barrier is actually onboarding and it's just knowing is a CONTRIBUTING.md, the proper place to go to or is there some other secret channel somewhere or a Slack group or something else where you could actually get a relationship with the project? I think a lot of us leverage a lot of these tools that are open source, and go years of leveraging them without even knowing who's contributing to them, who's powering it.

What community is involved in the project. So just knowing where to start is usually the hardest part. I think that we do a good job as a developer community. There are guides on how to contribute, like open a pull request, manage your commits and stuff like that.

But there's no guide of how to say hello when it comes to giving your first open source contribution.

Jeremy: [00:01:27] Not knowing where to start, even if there is a CONTRIBUTING.md and there's issues out there. People are like, I don't know which one to pick up. I don't know who to talk to first. It's just awkward, I guess.

Brian: [00:01:38] Yeah. Each project is different. So there's no centralized CONTRIBUTING.md file everybody is sourcing from so where one project can be, could say, okay, CONTRIBUTING.md, git clone, git, check out a new branch, git push origin. And then that's it. And some of them don't even have contributing MDs.

Some of them are just READMEs. Then you go to the README and there could be missing information. Some projects don't have READMEs. Some projects have READMEs and websites and documentation and Slack groups. So not knowing the balance of how to actually get involved in the project.

And I think what it really comes down to is if I started a new job the first thing I'm gonna get is a step by step okay. Here's your laptop. Here's how to do this thing. Here's how to clone the repo. Here's who to talk to. A lot of projects don't have that. Like they don't have like area owners, plugin owners, who's on the review team. Who's on the triage team. How big is the contributing group?

You can go into the GitHub repo and discover it all. But it'd be nice if someone just gave you a piece of paper or one file to get all that information. I think we've sort of grown out of the CONTRIBUTING.md and we need something else.

Jeremy: [00:02:49] When you are looking at an open source project, there's all these different issues whether it's bugs or feature requests. And it can be hard to know which things are suited for your skill level. And what do you think is the solution for that for somebody trying to pick out that issue that would work for them.

Brian: [00:03:08] Yeah. And I mean, the easy answer is they have labels like good first issue and like documentation. If some people don't know this, if you go to any GitHub repo, so like github.com/nodeJS/node. Or node/node, I think is what it is /contribute. So if you add /contribute on the URL you can see all the issues that are available for up for grabs and you can leverage them and jump into.

I don't actually recommend doing all that first and going to labels. I think the very first step is actually talking to a person. So the quickest place that you can find communication synchronous of like, Hey, I'm looking to contribute. I've been using this thing for six months on this project. I just want to give back. I had this idea for a feature like open issue, like ask questions on the issue, or even like now we have a feature at GitHub called discussions. In addition to that go into the discussion, but also limit the amount of back and forth you have to do asynchronous. And just go directly to the source, which is the person who is on-call already to chat with you.

A lot of projects have discords now. So find the discord link and then jump in there and say hello, because your experience is going to be completely different when you're actually talking to somebody and asking questions synchronously in discord.

The chat scrolls so it doesn't matter if you say a random question or you ask a question that's been asked a hundred times. Someone will give you a link, but it's better to do that than to be the person on the issue asking the same question for the fourth time. Or asking the wrong question at the wrong time.

I think that's a little daunting as well. If you don't know how the project, the underlying secret sauce of the project is actually laid out for you.

Jeremy: [00:04:45] When I think about open source, I think about communication being asynchronous, right? Going through issues, emails, mailing lists, things like that. And you're saying if you're first starting out, actually the best thing would be to, to find more synchronous communication, find that discord room or gittir, or, whatever it is, where you can actually have a conversation with someone.

Brian: [00:05:10] Yeah. And to be fair I'm catering this more towards a beginner opensource contributor. If you're experienced, do the regular thing, reference the issue, open up the PR, and no need to look for a synchronous communication if you know how to solve the problem.

GitHub itself is like 50 million developers worldwide. There's not 50 million developers doing open source. Let's just be clear on that. So there's a big difference between the users on GitHub who are just shipping code, like normal building websites for their companies or mobile apps or whatever it is to the people actually contributing the code that's powering all that stuff and powering GitHub as well.

I'm using this term called unintentional gatekeeping. I've been thinking about this a lot and I want to write a blog post on this because it's around the flow of information. So if I happen to be in the right Slack channel or the right discord, I have more
information than the person who's not there.

Because there's more information flowing through there than there are publicly on issues, because issues are treated as like a statement of work. You're declaring that this is the way it's working or declaring that it's broken and next steps to reproduce it or whatever it is.

And same thing with PRs. Like you're declaring this is the work I've done. This is the next steps. You review it. It's very robotic. But when you have that relationship that's built in a Slack channel and like, this is similar if you go to meetups or if you happen to know somebody fro

Lauren is a Software Engineer on the React Organization's Web Core team at Facebook and was previously an Engineering Manager at Netflix.

We discuss:

• Being empowered to say "no" as a senior engineer
• Straddling the line between engineer and manager
• The programmer's midlife crisis
• Resisting the urge to use clever abstractions

If you enjoyed this discussion with Lauren, be sure to check out her episode on the Changelog.

Music by Crystal Cola: 12:30 AM / Orion

Related Links

• @sugarpirate_
• Personal Site
• The Engineer / Manager Pendulum
• Should I Become a Tech Lead?
• Does it Spark Joy?
• Changelog Episode - Engineer to Manager and Back Again
• Dan Abramov's Redux twitter post

Transcript

You can help edit this transcript on GitHub.

Jeremy: [00:00:00] Hey, this is Jeremy. Usually when software developers are talking career progression, it moves in the direction from being a software engineer to becoming an engineering manager. And today I'm talking to Lauren Tan who moved in the opposite direction. She was an engineering manager at Netflix, and she recently made the decision to become a software engineer at Facebook.

We discuss why she made that decision and the differences between being a software engineer, a technical lead and an engineering manager. We also discuss what it means to be a senior software engineer and the ways that you can increase your impact and your influence in a software engineering role. I really enjoyed the conversation with Lauren and I hope you do as well.

Hey Lauren, thanks for joining me today.

Lauren: [00:00:42] Hi Jeremy. It's such a pleasure to be here. Thanks for having me.

Jeremy: [00:00:45] If we look back at 2015, you're moving from Australia to Boston, you're starting your first senior developer role at the dockyard consultancy. How did you get into this position where you decided that I'm going to leave this country I live in and I'm going to start this senior developer role in Boston?

Lauren: [00:01:03] A long time ago. I never really planned to leave Australia, let alone come to America. And I kind of traced his back to essentially how I got my career started in technology where really what started as a hobby creating silly applications then.

In fact, one of my earliest introductions to programming was through excel and making elaborate spreadsheets and writing Visual Basic or VBA. It was something that I never really planned to do. The long story short is after college, I started a startup called The Price Geek with one of my classmates.

And at the time I was getting really interested in essentially exploring more of this hobby that I had of programming and potentially exploring the idea of turning that into a career. So the year or two that I worked on that startup was really fun. We learned a lot about product development, about the business side of things, how to manage your money and how to get funding and financing.

That was all really interesting. And near the end of the startup when we were basically throwing in the towel I realized that I enjoyed it so much and despite the fact that my degree was in finance and not computer science, I enjoyed it so much that I thought to myself, wow, it would be amazing if I could keep programming as a career.

So I was very fortunate to get a first job in Australia as a software engineer. And I had started writing a bunch of blog posts and started sharing them on Twitter and on medium. And slowly but surely, I got people reading it. And there was a point where one of the creators of that JavaScript framework that I was writing about got in touch with me to say: Hey, would you be interested in coming to speak at one of our conferences? And of course, I was totally taken aback because first of all, I had never even been to a tech conference at that point, let alone speak at one. So I had totally no idea what I was doing. But I was convinced by them to apply. So I did and I'm very grateful that they did that.

And doing all of this essentially, I started to get the attention of some of the people working in America. The CEO of that consultancy DockYard reached out to me and asked if I would be interested in working there. And at the time they were pretty well known in the field of building Ember applications, Ruby on Rails applications. And so I thought it would be pretty interesting to go and work there and learn from some of the people that I really looked up to in that community. And that was the start of my career in Boston. And really it was a difficult decision to move. I think moving anywhere it's difficult but the move from Melbourne to Boston was exceptionally hard because it's a totally different country. It's so far away. My family and my friends would be not even in the same time zone anymore in opposite ends of the world really. So that was particularly difficult. And of course the Boston weather, it's terrible. And part of the reason why I was like, I need to maybe live somewhere else is because of the terrible Boston winter that I experienced in 2015.

Jeremy: [00:04:33] That makes a lot of sense how you ended up in California.

Lauren: [00:04:36] Right. I was like-- I need to go somewhere warm.

Jeremy: [00:04:40] One of the other guests I'm going to have on, Swyx-- he often talks about learning in public, which you were doing with your blog posts which got you noticed.

So I think that's good advice for software developers in general that putting yourself out there and sharing knowledge can really make these opportunities come to you.

Lauren: [00:05:01] I think it can, but I also want to say that I think that developers that learned their craft during the time that I started I think we were very fortunate in the sense that the web was a bit of a simpler place back then. People would build applications just literally using HTML, CSS, and vanilla JavaScript back then. You might just consider using jQuery or Backbone, or MooTools even.

A single page application really wasn't the norm. I think today is a very different world because software development-- I don't know if it's gotten more complex, but I think at least in the world of front end development it's gotten much more difficult to just get started.

Not saying that you can't build an app with just HTML, CSS, and vanilla JavaScript. But if you want to get a job doing it, then there is a bit of a higher bar I think. So I will say learning in public can be very helpful. But I also don't want to lie and disguise the fact that the environment has changed.

Times have changed and things are getting slightly more complicated and complex to build and that just means that there's a bit of a higher hill to climb.

Jeremy: [00:06:18] If you are going to make a site, you have so many options you have React, Vue, Ember, Svelte. There's all these different frameworks and do I use Javascript? Do I use TypeScript? It's definitely a lot more-- I don't know how you'd describe it. Intimidating, I guess.

Lauren: [00:06:39] It shows the evolution of how front end development has improved in a way. It's like, it's a mindset shift I think in the industry where previously, like 10 years ago it was still okay to just build what people might call enriched documents. Really documents sprinkled with some interactivity. But these days you're often building interactive applications that warrant a framework like react or angular or svelte or Vue. So I think maybe the problems that we're trying to solve have also changed that warrant more complex solutions.

Because I don't think the answer is to say like: Oh, we just need to get rid of all the complexity. The complexity exists for a reason. I think if I had advice for someone who was coming up in the industry, I would say, don't get intimidated by all these different technologies. And honestly, it probably doesn't really matter in the grand scheme of things which one you pick as long as you pick one and then you don't shut yourself off to learning from the others as well.

Because frameworks will come and go, but the knowledge that you acquire from using these frameworks will hopefully stay with you for a long time. And so those are much more transferable than knowing every single detail about the React API or something like that.

Jeremy: [00:07:56] Yeah, I think that's good advice. And I also wonder, when you started-- you had experience building applications in things like Rails. There are a number of frameworks where you can build a front end using primarily server side code, not necessarily build a single page application. People starting out, is that still something they should look at or do you think they should jump straight to single page ap

Swyx is a senior developer advocate at AWS, an instructor at Egghead, and the author of The Coding Career Playbook.

We discuss:

• Getting help without having a big following
• Remixing and summarizing what others create
• Creating Friendcatchers
• Betting on technologies
• His new book "The Coding Career Playbook"

Music by Crystal Cola: 12:30 AM / Orion

Related Links

• @swyx
• Personal Site
• The Coding Career Playbook
• 1% rule
• Lindy Effect
• Learn in Public
• The Ultimate Hack for Learning in Public
• The Day I Became a Software Engineer
• Developer's Guide to Tech Strategy
• Every Public Engineering Career Ladder
• Friendcatchers
• React Typescript Cheatsheet
• Steve Yegge's Google Platforms Rant
• DynamoDB Book
• Egghead
• Frontend Masters
• How I Write Backends

Transcript

You can help edit this transcript on GitHub.

Jeremy: I did a computer science bachelor's.

Swyx: [00:00:45] Nice.

Jeremy: [00:00:46] It's interesting seeing how you learned because when I went through school, I wasn't super passionate I think particularly because when I was going through school a lot of it was, data structures and algorithms and stuff like that, and it was a little bit disconnected from when I first started where I was like I'm going to make games.

I'm going to make cool GUIs and like when I get to school, it's like there's none of that. It's really on me I should have been seeking that stuff out on my own. It wasn't until a few years later after I had started working where I really started, enjoying the process, enjoying learning about the technologies and building stuff. Looking at what you were doing I definitely should have been doing that when I was going through school

Swyx: [00:01:34] Well, I mean, you're still figuring out what you want when you're still in college, Yeah. I went to school for finance and I no longer do that (laughs). But, yeah, I don't, I mean, don't live life with too many regrets it's not worth it.

I fell into this way of learning because of other people. All I'm doing is trying to spread the message and there will be more beneficiaries of this than me. I'm definitely lacking a lot of things that you learned in college.

I'm trying to make up for it. I really want to take an OS course. I want someone to force me to do a basic operating systems course. And I don't know what a syscall is, and I don't know the details on memory allocation and all that, like, but on some level, it doesn't matter because it depends on what part of the stack you want to work in. But I just don't have the option available to me if I wanted to go further down the stack. I just don't.

I still personally do wish that I did a CS degree. I'm just saying I definitely did not catch up with what you already know just from my bullshit web dev stuff. But, it's enough to get a job, which is absurd. This is the only career where, it's high paid and you can get up there in like three months ish. Maybe you won't be amazing. You're not going to be Jeff Dean or something at Google. But you can get by decently and you get paid the same as a doctor or a lawyer. And that's ridiculous.

Jeremy: [00:03:01] I found that to be pretty pretty insane. Though I will say, when you were talking about you can get a job in just three months or whatever. But your background it's really not the three month boot camp right? You had a much longer tail in terms of all the things that you've learned at your previous jobs. You said you had used Haskell right? That's before you went to the bootcamp

Swyx: [00:03:27] Yeah, I mean, I'm definitely not speaking for myself in terms of the three month thing. But I have seen my fellow bootcamp people get good jobs. Obviously there's a failure rate as some people don't make it. But it doesn't happen for medicine or law.

Jeremy: [00:03:39] Yes. Instead, it's six years, eight years, and then like you've talked before, when people learn something, it's normal for them to share what they've learned.

Swyx: [00:03:48] Also we'll promote you for it if you do a great job. It's just fun.

Jeremy: [00:03:54] I think when I first heard about you is-- I read hacker news relatively regularly and I remember you had made a post and you were saying: I used to be in finance, I'm going to do this boot camp. And so I'm doing this podcast where I interview people in my class and just a few years later, now you're like all over the place. You've got all these blog posts. You're moderating the React subreddit, you were recently, a senior dev at Netlify and I was like, this is crazy. This guy who was talking about going into a bootcamp just a few years ago he's doing so many things. I think there's a lot of lessons that people can learn from your experience starting their careers, learning how to learn, and deciding how to progress in general.

Swyx: [00:04:45] Thank you. I don't know what to say that. Yeah. I'm trying to write it down. Basically I have a lull between jobs. I'm going to join Amazon by the time this comes out. This is the only time I feel like I can write a career advice book because after this I want to focus on other things.

I guess I had a relatively fast trajectory. I finished my boot camp at the end of 2017. Started my first dev job in January of 2018 and then just got hired at Amazon at an L6 level in March of 2020. That's a relatively fast trajectory for anyone.

I'm not completely new to programming. I have done some code before, but then also I do attribute a lot of that to the ability to just learn very quickly in public. And a lot of people I think it's an alien concept. They vaguely know it's a good idea. I think they don't know how good. Just the relative rarity of people doing it means that being part of that population makes you stand out. And that's very beneficial for careers even before the current situation we are in, which is now everything's online.

So your professional profile doesn't have a physical presence anymore. You don't have to become a celebrity, a lot of people are like, ah, you have to be an influencer. No. It's more about just like having a place that you call home online. And you as a developer have an abnormal amount of control over that and you should exploit that as much as possible.

Jeremy: [00:06:25] Yeah. When you talk about learning in public you talk about exploiting that, right? It almost makes it sound like the fact that you are learning in public is a big benefit to you. You're getting things from people, whereas, I think a lot of people when they think about, Oh, I'm going to write a blog post, or I'm going to make this tweet or whatever. It's like I'm going to be helping other people, but maybe a big part of it is the opposite as well.

Swyx: [00:06:52] Yeah, I mean, look it's a plus if it helps other people, but it's completely self centered (laughs). I think that's good that means that you have the motivation to stick in this thing for the long haul. I think a lot of people get started blogging or whatever, and they don't see much immediate result. And then they get discouraged. And that's because they base their self validation on others. It's not worth anything if no one else reads the blog or likes or shares it or whatever. And that's not very healthy in terms of the way that you should approach your learning. So you should learn for learning's sake. And then if other people benefit, that's a plus. It's not an act of altruism. It genuinely is the fastest way to learn. And you're growing in your knowledge but also your network. And it turns out that your network is also super important for your career. So it comes hand in hand and I don't have to separate them so I just do them together.

Jeremy: [00:07:45] I think one of the things about this concept of learning in public, a lot of people are not really sure where to start. They start with

Philip Kiely is the author of Writing for Software Developers and has written for companies like Smashing Magazine, CSS-Tricks, and Twilio.

We discuss:

• Doing research beyond the first page of search results
• Writing e-mails people will read
• The "get it done" and "learning" modes of readers
• Making articles easily skimmable
• Long form articles vs Stack Overflow
• Promoting and launching a book
• Borrowing the reputation of others

Related Links:

• Writing for Software Developers
• Personal Site
• @philip_kiely
• Stripe documentation example (Complete code sample with switchable frontends, backends, and walkthrough as you scroll)
• Go by Example (Straightforward samples readers can copy/paste)
• Tailwind UI (Sustains Tailwind CSS open source project by providing prebuilt components)

Music by Crystal Cola: 12:30 AM / Orion

Transcript

You can help edit this transcript on GitHub.

Jeremy: [00:00:35] This episode, I'm talking to Phillip Kiely. He's the author of the book "Writing for Software Developers." We usually talk about code on this show, but knowing how to communicate with developers is just as important. We discuss topics like making articles easily skimmable and writing with a specific audience in mind as well as crafting emails that people will actually read.

We also talk about his experience creating, promoting, and launching his book. And what he learned from his many interviews with people like Cassidy Williams and Patrick McKenzie. If you want to get better at writing or you've ever considered launching a product, I think you'll enjoy this conversation with Philip.

Phillip, you recently graduated with a computer science bachelor's from Grinnell College and I'm sure you did a lot of research. When you have something new that you're trying to learn how do you get started?

Philip: [00:00:48] As a developer, I spend a lot of time getting error messages while I'm coding. The first thing I do whenever I get an error message is I just plop it into Google, see what comes up, see what people are saying on Stack Overflow, GitHub issues, that sort of stuff.

I also have a background in journalism. Having worked at the Grinnell College Scarlet and Black newspaper I have a lot of experience going and just talking to people doing interviews. And that's why that felt like a very natural thing to include in the book.

It's also something I've been fortunate to do in articles. For example, I wrote an article in Smashing Magazine about remote part time teams using agile development methodologies. As part of that, I did a quick email interview with one of my professors at the time as well as a different professor who had written the book that we were using as the textbook in this class about software development.

Bringing in those outside perspectives allowed me to write an article that was so much better than anything I could come up with from my own experience. So really when you're looking for things to use in your own writing look broadly and don't just look at the first page of Google results.

Ask specific questions, whether it be of a search engine or of an experienced person in the field. And then cross check that information against each other to make sure that it all matches up and presents the information in a way that's orderly and meaningful to the reader.

Jeremy: [00:02:19] The people you're reaching out to most likely don't know you and you're asking them for advice or for an interview. How do you get these people to say yes and agree to help you out?

Philip: [00:02:31] It depends on what I'm asking for. In the case of the article that I was mentioning earlier, I was asking for maybe 5-10 minutes of their time to do a quick quoted answer to one or two very specific questions that I knew they had off the top of their head because I had just read a whole book about it and that is actually a pretty easy ask. Most people who are in the public do receive a lot of email but don't necessarily receive a ton of these sort of requests on a daily basis. And if you make yours compelling and specific they are pretty likely to respond to you.

I'm nowhere near the public profile of the people I've been reaching out to. But over the past week with this book launch, I've gotten a ton of inbound emails so it's been really interesting to consider this question from the other side. Considering which of these emails I'm taking the time to write very detailed responses to. Which ones I'm shooting back a real quick one liner. And which ones I'm just pretending I never saw and sticking it in the trash.

I think that making my emails to these people who I'm trying to get interviews from very short, very specific, very relevant to their interests. Showing that I'm not just reaching out to them because they're a big name or because I had an idle inclination but because I'm very invested in a project that I need help with.

In terms of finding interviews for the book. It was the same process just magnified because instead of asking for a quick email exchange, I'm asking for 30-60 minutes of their time. I'm asking to ask them some really tough questions about their work and have them think through it and give answers that are going to get published and have a bunch of people read them.

I'm also implicitly asking them to endorse the work that I'm doing and the product itself with their presence in it. I have the picture of everyone who I interviewed on the sales page, for example. And with that the pitch is a little different. It's about investing credibility. The people who I interviewed for this book were kind enough to invest some of their credibility in this project and if the book was total garbage they would have gotten a negative return on that investment and future projects that they're involved with.

On the other hand, if I put in a lot of work editing and producing a great book then they're going to get a positive return on investment and to some small degree be even more credible for their involvement with this project.

I never state that explicitly, but throughout the process both the first short cold email, the scheduling, the interview itself, and then keeping them up to date on the progress of the project after the interview... I just have to deliver to them professional correspondence that gives them confidence that this project is going to have a positive return on the credibility that they invest in it.

Regarding the interview itself... I want to make sure that I'm asking them questions that people haven't asked them before or at least asking them these questions in new ways that allow them to say new things that they haven't said publicly in the past. Both because it's more interesting for them and it allows me to create a better product in doing so.

I'm going to inform them that I've read their work and not just by saying, "Hey, I've read your books" or "I've read your articles", "I've listened to your podcast". I'm going to say something specific that I liked about a previous thing that they've done. Or I'm going to reference that I just interviewed someone for the book that they know from a previous interaction. By establishing that credibility and proof that I'm going to be very invested in the project I can make them more likely to respond.

The final thing is that it really is a volume game as well. I sent almost a hundred pitches to the sort of people who you would expect to see on the list next to the 11 who did say yes. A number of people were just too busy to respond, have a policy against doing interviews or for any number of reasons didn't want to appear in the book, and that's totally fine that's their prerogative.

What I would encourage people to say is, okay, if I need 10 interviews for the thing that I'm trying to do I'm going to send a hundred emails. If I'm writing an article and I only need one or two interviews I'm going to send 10 emails. If you get more interviews than you're looking for, that's great. I was only planning on putting eight in this book and I got 11. And they all add a ton to the product. It takes a lot of time because you are sending specific targeted emails at scale but it was also worth it.

Jeremy: [00:07:18] So basically when you want to talk to someone... do your research, read their work, learn a little bit about their background and show them that in your email so it'll be more personal and it will make it clear that you're not just shooting emails out to everybody and hoping for a response.

Philip: [00:07:32] Then the other approach that you can take is more of the approach that I used pitching you to come on this show. I think I wrote you about a three sentence email and the first sentence was: "Hey, I just graduated from college and made $20,000 selling a book." The second sentence was: "I'm doing a podcast book tour." The third sentence was: here's how you can contact me.

When you're doing cold email like that it's almost li

Sara Leen is a localization programmer for XSEED Games on titles like Ys, Trails in the Sky, and Corpse Party. She got her start reverse engineering and fan translating games.

We discuss:

• What makes games different
• How games store and encode text
• Rewriting Corpse Party (and not being able to run it for a year!)
• Porting and modernizing games
• Reverse engineering raw assembly

This episode originally aired on Software Engineering Radio.

Related Links:

• @SaraJLeen
• Personal Site
• Corpse Party PC Programming Blog
• Trails in the Sky Second Chapter Localization Blog 5
• Mojibake (Garbled text when text is decoded using the wrong encoding)
• Shift JIS (Japanese character encoding used in older games)
• Hot Soup Processor (Obscure programming language used for Corpse Party)
• XSEED Games

Music by Crystal Cola: 12:30 AM / VHS Skyline

Transcript

You can help edit the transcript on GitHub.

Jeremy: Hey, this is Jeremy Jung. This episode, I'm talking to Sara Leen, a localization programmer for XSEED Games. Sara has over a decade of experience localizing games from Japanese to English. And we talk about how games are different than other types of software, storing and extracting text, porting games to different platforms, and rewriting a game from scratch.

A game written in an obscure programming language called... can you guess it? Hot Soup Processor.

Pretty good right? This was a fun episode and I hope you enjoy it.

Jeremy: [00:00:35] The first thing I want to start with is you have experience with building other types of software. I was kind of taking a look at your resume. What makes games different than working on a typical software application?

Sara: [00:00:48] With games, there's a lot of things that you have to worry about that you otherwise wouldn't like the frame rate of the application becomes very important. For example, you need to make sure that the timers of the application are coded just so that the game will always run at a steady pace. And there are various applications of timers like delta time that are usually used to ensure this goes perfectly smooth.

Jeremy: [00:01:17] When you talk about timers, I've heard of this concept called a game loop. Is that what you're referring to?

Sara: [00:01:27] Essentially a game is made of graphics, audio, and input. But all of these have to be checked every frame. And so you have your timer going so that it is basically looping through the game 60 times a second or whatever other frame rate your game is running at a lot run at 30. There will be various steps in this process, like game logic and drawing the various models. And of course you have to keep the sound system updating.

Jeremy: [00:02:01] As opposed to a web application where a lot of things are in response to something, right? Somebody's going to a webpage, clicking a button. With a game it sounds like there is this consistent loop that's tied to the frame rate. Is that correct?

Sara: [00:02:20] Right. And instead of reacting to things, you are occasionally checking the state of course so many times per frame and you will still have to respond to the user input but it's all going to be one part of this large set of functions.

Jeremy: [00:02:38] So you're in this loop and let's say somebody was pressing a button on their controller or moving their mouse that would affect the state. And when you're in that loop it's going to look at that state to determine what to do next in the loop?

Sara: [00:02:58] Yes, exactly. For example, in your input system you are usually going to be saving every frame, the state of the input system at the moment, like is this button being pushed? Is this button being pushed? Was this button pushed last frame and by comparing it against the previous frame. You know what the user is doing right now, and then many parts of the game logic will have various branches that check exactly what the user is inputting and what the current context is.

It's certainly more complex than the idea of I push a button and it calls this function.

Jeremy: [00:03:34] That sounds like like an entirely different model of thinking. The structure of the application has to be almost completely different.

Sara: [00:03:42] Absolutely.

Jeremy: [00:03:43] In terms of skillsets working on a game versus working on a regular application. What parts are really different and which parts are kind of the same?

Sara: [00:03:54] Regardless of what you're working on you're going to be working with a lot of external APIs, but these are probably going to be entirely different for a game versus other software.

For example, you need to be familiar with APIs like DirectX and OpenGL and all of these typically have different ways of handling graphics, handling audio, handling input.

There's also SDL of course that you could use and depending highly on the programming language as well there may be different ways of handling certain things that would be different from handling a normal application.

But most importantly you need to be familiar with graphics and audio APIs above all. I think those are typically less important when you're working on normal software because you're more concerned about how to build your window and how to get all of the elements displaying in the proper positions.

Jeremy: [00:04:51] With a typical application the actual act of rendering to the screen is abstracted away from you so you don't have to worry about how to draw to this screen. It's more like, I want a button or I want a window and something else is handling that for you.

Sara: [00:05:08] Yes, that's pretty much it. And when it comes to video game graphics, you have to consider a lot of different things like differences between graphics cards. Can this graphics card handle square textures or does it need a different shape? As well, of course, as being concerned with the quality of how it's drawing. Like is this texture supposed to be filtered? Will this graphics card filter it correctly? The APIs simplify that, but you're working at a lower level.

Jeremy: [00:05:36] And when you were giving examples of APIs you were talking about DirectX and OpenGL and SDL. How low a level are we talking? If I wanted to draw something to the screen at the most base level what kind of work am I looking at?

Sara: [00:05:55] Well, for SDL in particular this is somewhat of a simplified process because it handles all of the initialization of devices and such for you.

But with DirectX and OpenGL, you have to actually get the list of devices you have to initialize each process that you need. Like you need Direct3D, you need DirectInput, you need DirectSound, and then you have to make sure that you have all the device information that you need, including stuff like the current resolution or the capabilities of the device. Device capabilities is a big thing when it comes to that.

And then to actually draw the image, you are going to need to create a texture buffer. You're going to need a back buffer for the screen, and then you will have to load the texture and you will have to draw the texture to the buffer and then present that to the screen.

Jeremy: [00:06:53] When you refer to the buffer, at kind of a high level, can you kind of explain what that's doing?

Sara: [00:06:58] Essentially you have the screen that you're drawing to, and you usually don't want every single draw command to go directly to the screen. It's better to have a buffer that you can operate with so that you can display things in the correct order at all times. You can have them on the correct Z level, and of course, any sort of processing you need to do, you can do before it's drawn to the screen. And so everything goes onto the buffer after it's ready more or less.

Jeremy: [00:07:29] And when something is in the buffer is that what you plan to actually have the video card render? Or is that after it's already been rendered? Where is that in the stage?

Sara: [00:07:43] Well, rendering is basically the last step of presentation, but it depends. If you are drawing something 3D naturally you are going to have to do some processing on that via the graphics card. And the APIs allow for things like that, but typically rendering to the screen itself is always going to be the very last step of the frame.

Jeremy: [00:08:06] And then elsewhere in your application even though you're putting things into the buffer, you're d

Spencer Dixon is the CTO and cofounder of Tuple, a pair programming application for remote developers.

We discuss:

• How WebRTC works
• Capturing audio and video
• Choosing a video codec
• NAT traversal
• Choosing a native application over Electron
• Diving into C++ and MacOS development
• Getting expert help

This episode was originally on Software Engineering Radio.

Related Links:

• @spencercdixon
• Tuple
• WebRTC
• Introduction to WebRTC Protocols
• WebRTC Mozilla Docs
• WebRTC Hacks
• Discuss WebRTC Google Group
• WebRTC by Dr Alex
• Cosmo Software
• High Performance Browser Networking