FounderQuest
FounderQuest
Screw Dependencies! Learn How We Are Fighting Our Abandonware Problem.
Full Transcription:
Starr: So, I learned something amazing. Just before coming on here I learned something amazing. Okay, Beto O'Rourke, the presidential candidate, used to be a member of the Cult of the Dead Cow, hacking group. In, yeah the 90's.
Josh: Wow.
Ben: Awesome.
Starr: I know!
Josh: Sold.
Announcer: Three developers. One mission. Build a business to nurture personal fulfillment. It's not stupid, it's Founder Quest.
Ben: It might have to go look for his byline, what the frack?
Starr: I know, I know, right? I used to read their reports from Def Con and all the conferences and stuff. I used to be like, man these guys are so cool, they're so much more mature than I am. I don't know how they got all the money and go to Las Vegas and do things when I'm 17 and have no job.
Josh: That's kinda how you feel about the guy who's running for president now probably.
Starr: Yeah, probably.
Josh: So, I guess not much has changed.
Starr: Oh, speaking of Yaks and blockers and all this stuff, we finally have our pod cast artwork as you saw.
Josh: Yeah.
Starr: So we can now actually... This is episode number 6 of Founder Quest.
Josh: Is it? Wow.
Starr: And, yeah nobody's heard it, but me. Really.
Josh: I kinda like it like this, there's no pressure.
Starr: Yeah. Should we just keep saving them to drop box.
Josh: Yeah.
Josh: And that's it? I mean like... yeah that's cool with me.
Josh: Yeah.
Starr: That's fine they can release them after we're like dead and famous.
Josh: Right, yeah.
Starr: It's like I didn't want to set up the web site because I don't know what colors the art work has in it and you have to have ... you know, you want to have them matched colors and stuff so.
Josh: Yeah, but know we can get the first one out?
Starr: Yeah, now we can start rolling em out. See what people think of em. In slack we've been talking all morning yesterday about all these blockers we have right, because when you're doing development on any sort of bigger project, you have this idea of the real work you want get to, like the feature work, and then you have the things that are preventing you from doing the real work. Sometimes you call that yak shaving because in order to, you want a sweater, but you know you need yarn to make the sweater, then you need wool to make the yarn and you eventually end up shaving a yak. I think that's what I
Starr: Yeah that sounds about right. Is this a false distinction you guys, you think there is a such thing as the real work?
Josh: You mean like does the real work exist, or is it all just yak shaving.
Starr: Yeah, does the real work exist?
Josh: I think you could make a case that it is yak shaving to an extent like anything you would do, would be blocking something else at least.
Starr: The reason I ask is I had this bit of an epiphany when I was struggling through some random webpacked stuff where I was like man what if this is all there really is, like what if this is it guys?
Josh: Kind of like an existential crisis.
Starr: Yeah, kind of, kind of. And it was fine like I was having a good time you know, just doing my webpack updates and everything, but this idea that we have some sort of mythical real work to do.
Josh: Well web development has become, feels like to me its become a lot more complex over the years, like I don't know, that could be an illusion too you know, computers have always been hard. But it feels like the amount of things that you have to do just to do web development in the first place, has increased. I don't know, what you, how you guys feel but that's how it feels to me.
Ben: Yeah, I can definitely agree with that, I mean it's not as simple as just drawing some HTML up on the webs you now, and having people see it, right?
Josh: Yeah.
Ben: So, yeah I think when you're building on anything, right, you have to deal with all the things you're building on top of.
Josh: Kind of like when we used to write HTML and then we had to write some PHP in our HTML.
Starr: So the three of us kind of, I don't know, came of age but we really enjoyed the rise of rails, and I wonder if that was maybe some sort of golden moment in which things became simple enough you could build an entire website, state-of-the-art website, with the skills of sort of one person, right? I remember working on rails projects and feeling like, man I've got this rail stuff down, it's like I can go over here write my ruby I just gotta make a few little views in HTML got some CSS, done, like I am a ninja at this stuff. But now it feels, you know, different. It's like okay, I can work on a feature in ruby for a while, and then I'm going to have to go and redo my JavaScript tooling to make JavaScript compile, because you know something happened and there's all this context switching that maybe there didn't used to be.
Josh: Yeah, and the pace of change again, especially I think with JavaScript tooling is sped up so much and everything's changing so rapidly that I think we have to go back and re-evaluate our tooling and that stuff more often.
Starr: Yeah you're working on something right now aren't you Josh, you're re-doing some of our code on our point library for JavaScript?
Josh: Yeah, I'm working on our Honeybadger.js or our Honeybadgerjs library for a big 1.0 release finally we've been pre 1.0 all this time, but-
Starr: Oh we're still not up to 1.0 yet.
Josh: No, I'm a big fan of semantic versioning and you don't really have to do it if you're not to 1.0 yet, so, you know that's a good way around it.
Ben: Yeah I think the JavaScript world has definitely made us deal with acceleration and the dependency management world I guess, like with rails, what I would appreciate is that they've taken kind of a measured approach, it has definitely changed over they years, but the change comes pretty slow. Like rails basically you know, every couple of years there's a major version, and you have to deal with that, right? But it seems like JavaScript libraries every couple months there's a major version, or one of the five things that you're working with has a major version, so there's always a major version change happening and you gotta rebuild it.
Josh: Yeah, the thing I'm working on now is, I've been trying to, I've done a bunch of features, like a feature work on the library, but it always seems to come back to when I have to work on our build tooling, or our CI continuous integration, is where I seem to spend the majority of my time especially since I don't develop this package all the time so I'll let it sit for a while then come back to it and do a bunch of work on it so usually by the time I get back to it, things have kind of moved on in a lot of ways and there tends to be things that I need to kind of troubleshoot to bring up to modem standards again.
Starr: Yeah I'm feeling the same, the same thing. Maybe a year ago I converted all of our Coffeescript to ES6, and I'm really glad I did it and as part of that we have webpack now in our sort of build system. And in addition to webpack, we have a lot of webpack, we have a lot of dependencies, so recently we use this thing called Dependabot to see any if any of our dependencies are out of date, they have security problems with them, stuff like that. It opens a PR in GitHub and, I guess it was Ben, turned this on for JavaScript and so suddenly we have a ton of dependencies that are out of date, and I've been sort of working through those.
Starr: It's a little but rough, one thing I find that is really different between the sort of JavaScript dependencies and ruby dependencies, is that with the ruby dependencies it seems like, and you guys can correct me if I'm wrong because I haven't really done a ton of them but it seems like okay, it's a new version of the gem, you update maybe you have to change your code a little bit but that's it, but one thing I'm finding with these JavaScript dependencies is that the Dependabot way of upgrading each one individually and then opening a new PR for each upgrade, is kind of difficult because you have to upgrade all these things at once for them to work together or else you're just screwed because they're all so tightly interconnected.
Ben: I haven't dealt much with the JavaScript dependencies so I can't say the differences between them and the ruby side, but yeah, definitely I agree that the ruby dependencies are usually pretty independent of each other, right? You can upgrade this gem and not that gem.
Josh: Yeah, and there's a lot fewer of them typically.
Starr: I wonder how big our node module directory is?
Ben: I have back in the dark days where rails three, when you would have to update gems, I remember a number of times where I was like, okay just bundle update, and update everything at once and it was a mess.
Josh: Yeah.
Ben: So maybe that's the same point we're at with JavaScript right now, where really to get a good update you have to update everything at the same time-
Josh: Yeah
Ben: And that's just a mess.
Josh: Yeah, and I think it also has a lot to do with how you lock your dependencies, and, I don't know what we're doing on that, and as well for JavaScript too, but that's also a ruby thing.
Starr: And one thing I'm finding with JavaScript dependency upgrades is that, well we sort of straddle two worlds, right? With Honeybadger with our front end, because we have on one hand all this modern tooling, we have a lot of JavaScript, but when we started building this, this was before react, this was before pretty much all the more modern ways of doing things with JavaScript, and so most of our code uses a sort of older paradigm using jQuery, updating the DOM and stuff, and it works fine, we've got it organized in a way that I'm very happy with.
Starr: One thing I'm finding is that because, we use, dependencies, that are from this sort of older time we're actually having some of them becoming sort of, abandon wear, and this is a bigger problem for us right now because I recently needed to upgrade jQuery to the latest major version, but we use jQuery Pjax which is a plug in, to give us this single page like, Turbolinks style behavior on our front end, and it just doesn't work with the new jQuery because it's abandoned, pretty much. So now, in order to update jQuery we have to migrate to turbo links, or some other more modern way of doing things, so it's just a backasswards way of working that's just incredibly frustrating at times because you just want to freakin build features-
Josh: Yeah.
Starr: And ship them, but yeah.
Josh: Well I remember back when Pjax was the hotness, that was well before turbo links, and I think it was, was it, it was GitHub that initially...
Starr: It was GitHub, yeah.
Josh: And initially, kind of started that whole way of doing, making kind of normal traditional web apps faster, with the Pjax and that's basically what got picked up and rolled into turbolinks.
Starr: Yeah maybe we should explain what this is, with Pjax and turbolinks, instead of having a single page application where you have a bunch of JavaScript that creates an application in the browser, what you do is, everything is still rendered server side, just like a traditional web application, but there's a little bit of JavaScript that goes in and when you click on a link, instead of fetching a whole new page, it intercepts that, and sends and an Ajax request to your backend, and the back end renders the new page, sends it to the front end and JavaScript is used to update and refresh the page. And that may sound like, not much of a change, but it actually results in significantly improved performance times, because you don't have to reparse all the JavaScript, you don't have to reparse all the CSS you're using. So its actually a really great compromise.
Josh: Yeah, I recently threw turbolinks onto my blog. You know it was already fast because it was basically just like HTML pages, but it created a noticeable improvement in the page load times, and all that, so, that was kind of a cool little thing. It's like three lines of code you just throw it in the page and its done. So, my blog is now a singe page app.
Starr: That's.... awesome?
Ben: Yeah.
Josh: Well I mean it seems to be kind of the thing to do these days.
Starr: It is.
Josh: I know a lot of people who are converting their blogs to react and that sort of stuff, so I figured I'd have to you know, give it a try.
Starr: I published my personal blog on a xerox machine, so...
Josh: Oh, cool. You were saying kind of, how, dependencies, some dependencies tend to become abandoned ware or become outdated, and that's something that I've also been running into with this, Honeybadgerjs, the build system work because we use a build system called Grunt which is, it's a build system, it basically lets you create tasks that automate, things that you're doing when you're building the production JavaScript package.
Josh: It's kind of similar to Rake and Ruby, or Make even, but it has a bunch of packages, or plugins that have been developed for it, and I've been running into the same thing, where because Grunt is not the new hot build system anymore, there's a bunch of other ones like Gulp, and I'm sure like five others that are even more popular than that now, I keep running into packages that I want to use, but the last commit on them was like 2016, or I'm running across GitHub issues when I'm trying to figure out how, why somethings not working, and I check the date and its like 2013.
Starr: Ohhh.
Josh: So...
Starr: Those are the worst.
Josh: Yeah, so it's like do I use this stuff, because some of it actually works but its, unsupported obviously, but the only other alternative is set up a new build system, and re-do everything, and that in itself is a massive yak to shave, so I'm making yak trade offs. Like which yak do I want to shave, which one is bigger.
Starr: So they would say open source isn't free, and I guess this is one of the costs of open source.
Josh: Yeah.
Ben: Yeah I guess the alternative would being that you're stuck with whatever your vendor tries to put on you right? And you know if things are dead they're like, you know tough, buy the new version.
Josh: Yeah, that's true.
Ben: But, I guess, a lot of web developers, don't remember the bad old days of building for a platform like windows, right? Where you only have one choice and that's whatever the vendor decides to give you.
Josh: Yeah.
Ben: To give you. Like now, we have all this choice, and all these ways to be able to implement whatever we want, and so, I guess that's the cost, like Starr was saying, that's the cost of open source.
Starr: These problems we have are problems that more established products have, like we didn't have these problems when we were starting out because we didn't have this much code, we didn't have as much infrastructure, anything like that because we just built it all it was all relatively new, so I don't know I think we should maybe just ditch this one, start something new.
Josh: That sounds like a good solution.
Ben: Well you know one thing that's nice about dependabot, the thing I really enjoyed about that, is that it keeps you aware of the dependency changes, so you don't get so behind.
Ben: I think the real pain of abandonware is when you rely on something that hasn't been updated in three years, but you know if you notice, things are moving on, and you're only behind like three months, well that's not as quite as a painful update.
Starr: That's true.
Ben: And yeah, that's what I really like about dependabot, its like you should, I guess, you have to combine that with the discipline to actually take some time to go and manage those updates, like even if it's a simple ruby gem update, right, you have to take a look at the commits and maybe the maybe the change log and see what's changed, and will this break my app if I add this dependency, and that takes time.
Josh: Yeah.
Ben: Right, so I guess you just have to be disciplined about it to really save yourself the pain down the road.
Starr: The thing at this point, it might be a little bit of a stretch, but I think we could probably come close to keeping a junior hire busy simply upgrading dependencies, and doing tests to make sure that everything worked, and all that.
Josh: Yeah cross like all our repos, and everything.
Starr: Yeah, cross all the repos, I bet we could keep somebody busy.
Josh: I really like, what you said Ben, a lot of this what we're calling yak shaving here, is really managing, its managing technical debt, and not upgrading our, like pushing things off like upgrading, creates more debt down the road and so when we go and we actually need to get some work done on something and we realized oh, we haven't touched this place, or these dependencies in like a year or two we have to pay some interest before we can actually get to work, and so, I think yak shaving is kind of, you create more yaks with the more technical debt that you have, or you can also call it interest on your debt.
Starr: I'm not saying we shouldn't do all this stuff, but it's kind of like, you're on a treadmill almost, you're running just to stand still.
Josh: Yeah, well, some of these automated tools like dependabot are, I think are really going to help, and they already are helping us with that but as we integrate them more into our development flow, I think that I would expect them to help this problem a little bit because we're not going to get this far behind hopefully, and as long as we're a little bit more disciplined and these tools kind of help us, I think it'll hopefully create less work in the future and less technical debt to deal with.
Ben: On your note of "let's just build something new," I think that is part of the argument behind microservices, right? So if you have a new feature you wanna build, like that blog post you just wrote about the autocomplete.
Josh: Yeah.
Ben: And you implemented that completely outside of our main app so you could call that a microservice, if you want, but the dependencies there are much smaller, like the scope of the work is much smaller you're just focused on that one feature, and so, you don't have to worry about this dependency conflicting that dependency, and blah-da blah-da blah, and so yeah I think one of the arguments people have for going now the microservice route is we have all these small services that are all very self contained, and upgrading the dependencies on them is much easier piece by piece, but then of course you have to do that on 20 different apps rather than just one.
Starr: Yeah, that's interesting, its like there's, you're right though because you'd be doing more dependency upgrades, but there might be fewer dependencies per upgrade, so there's less chance of them clashing. Yeah, so, we talked briefly about turbolinks, and stuff, and one thing, this isn't really related to the subject, but it kind of is because I'm hoping that this new thing, this new technology will completely eliminate the need for me to ever have to set up webpacker again, or webpack again, and that is, I'm talking about phoenix live view, in case people don't know, phoenix live view is essentially going to be, turbolinks on steroids.
Starr: The thing with turbolinks is that when you request a new page, what happens is that it goes and it fetches the entire new page and then it puts it in the DOM and everything, what phoenix live view does, is because phoenix is based on elixir and elixir is very, very good at doing high concurrency things. What happens with phoenix live view, I think, is that you can actually go and fetch a little component, a part of the page, you can have the server render that and then return it, and you can have several of these requests going at the same time, so essentially you have a highly interactive front end type app, but its being entirely rendered server side, so there's no need for real any sort of custom JavaScript. I don't know if I got that right.
Josh: Yeah its kind of I think, the way it struck me when I first saw it, was as a rails developer its kinda how you can do JavaScript partials in rails, you know where you'd make a partial, like an Ajax request the server and then you can render out some sort of JavaScript code that then gets passed back to the server, and then executed on the client side, but with phoenix it's so fast that you can actually do that almost in real time basically so, you can build applications that you couldn't with the, even though rails can kinda do that you can't build a real time application, just because its not fast enough, and with phoenix live view at least the claim is, that it's actually because of the way phoenix and elixir and Erlang work with their process model, it is fast enough basically to serve a bunch of different clients like that in real time, to the point almost where you could actually, have a chat client, or even do real time animation that's server rendering your HTML which is kind of, that was an example that was crazy to me.
Starr: Yeah because phoenix has excellent support for websockets, and all that.
Josh: Yeah.
Starr: But I think it's, it's a little bit different from JavaScript partials.
Josh: Yeah.
Starr: Because, well I really hope it's different because JavaScript partials kind of suck, uh, I think with live view you're not sending back JavaScript that manipulates DOM and stuff like that. I think what happening is you're rendering a template, you return some HTML, and then the live view JavaScript library goes in and does something very similar to React, and that it, does this sort of shadow DOM thing, where it looks at the snippet of code, trying to be inserted into the DOM and then it only changes the elements, sort of as necessary in place. So you don't get that, sort of, full refresh sort of penalty.
Josh: I think that's right, and the thing that really makes it possible is the websocket, support because it does everything over websocket, which is why you can, well it's one reason why you can take advantage of more like real time capabilities, because it's not making as many round trips to the server.
Starr: I mean it's not gonna work for things like necessarily mobile, it's not gonna work for things where you have to have offline access, but you know, I'm willing to never write a mobile app if it means that I can write things in pure elixir and have these beautiful interactive, fast applications.
Josh: Yeah, I don't know, it's the most rails-y thing I've seen, that is purely phoenix and elixir, personally, it's really cool. I could see rails doing this if ruby could support, if ruby was capable of it, but because elixir has those capabilities it's one of the few things that I look at and it's like, ruby really can't handle, it couldn't. I don't see it ever being able to handle that sort of thing, and it's totally unique way to build a web application.
Starr: And I have to say, even if this thing doesn't pan out, it still gives me hope, because without hope all I see is endless JavaScript dependencies, endless node module directories, just stretching out before me and It's bleak guys, it's bleak.
Starr: I don't like it.
Starr: I don't want this future.
Josh: Yeah, it is kind of heartening, that people are still trying to build frameworks that simplify things, in that regard, and I think you talked about how rails, back in 2010 or something, was like the golden age of web development because it was basically just like simple, and you could kind of understand the full stack. For me like, what I loved about rails, was that it basically took all the stuff that I was already doing, and it organized it. Like it wasn't really doing anything new, but it was still my server side code, and my HTML, and CSS, and JavaScript, but it was organized in a way that made me a lot more efficient, so I'm hoping that people still kind of build in that way in the future, I guess like with a lot of this frontend stuff it can kind of get pretty complicated pretty quickly.
Starr: Yeah, as somebody who likes to, work in small teams, work by myself, and build products I really love solutions like that where if not just, so much of the process, and sort of technology that we have now is about sort, of being able to manage complexity and that's fine but you still have to manage it, I rather it just not be there.
Josh: A lot of these technologies really are build for, their build for much larger teams, and larger problems, than.. or you know, larger things basically than what we're even doing. Some of these, the scale of some of these applications that people are using, some of them more complex, like JavaScript frameworks for, I don't see us ever really attempting to tackle as a three person team, so.
Starr: No I don't think we could, like we couldn't build Facebook, like that's ridiculous.
Josh: So I'm wondering are we getting to the point where the tooling is like diverged, where some of the tooling is built for, managing us a massive level of complexity with a massive number of people, working on it at the same time, does a team like ours, really need to use the same tools that, Facebook needs to use for instance.
Starr: That's an excellent question. So, talking about blockers and yak shaving, I think guys, I think I'm about ready to give the mac dust off another shot.
Josh: Throwing in the towel, huh?
Starr: I don't know if I'm throwing in the towel, I love my Linux box for running docker, for doing development on, but pretty much everything that's not vim and a terminal, it's really starting to get on my nerves, and its not that you can't set it up to be nice, but it's like we talked about with all these dependencies, and managing that, you set it up to be nice, once and then over time it just slowly devolves and becomes less, and less, and things just gradually stop working, then you have to go in and figure out, well okay what changed about some new version of the thing to make this not work, and that's fine if that's my job, but I don't wanna have to do that with like, Spotify. I don't wanna do that to be able to put emojis in my tweets.
Josh: Yeah.
Starr: You know, that's just, ahh, I don't know.
Ben: Yeah that's where I was when I was using Linux on desktop after several years I was like you know what, when OS10 came out, and I saw the beauty that was the Mac UI on top of the Unix shell, I thought that was just awesome, I knew that was what I wanted to be and I decided, I had [inaudible] after that point, and had the same experiences, you know you always gotta worry about thins that are breaking, and at some point I decided, I wanna do work with my computer, rather than working on my computer.
Starr: Yeah, so I've got a suggestion just for all of us, lets just destroy our computers, and we'll buy an ice cream truck, and we'll go and sell ice cream to the little children, and we'll get to see the smiles on their beautiful faces as they, you know, eat their treats. So much nicer.
Josh: The way you described that, is, is beautiful.
Starr: I know.
Josh: I'm in, yeah, I'm in.
Starr: Alright, I think we have enough content so, should we, should we say goodbye?
Announcer: Founder Quest is a weekly podcast by the founders of Honeybadger. Zero instrumentation, 360 degree coverages of errors, outages, and service degradations for your web apps. If you have a web app, you need it. Available at Honeybadger.io. Want more from the founders? Go to Founderquestpodcast.com that's one word, you can access our huge back catalog, or sign up for our newsletter to get exclusive VIP content Founder Quest is available on iTunes, Spotify, and other purveyors of fine podcasts. We'll see ya next week.
