Ex-npm Employee Making a New Package Manager?! Vlt with Darcy Clarke

Welcome to Syntax, the podcast with the tastiest web development treats out there. We've got Darcy Clark on today.

This is a a pretty exciting episode for me because Darcy, among other things, he was in my wedding party, and, he taught me much of what I know today, so I'm really excited to talk to him about all exciting stuff that that he's working on. He's working on a New package manager, will probably bait the title as, like, ex npm employee,

Build's new package manager or something silly like that. But welcome, Darcy. Thanks for coming on. Thanks, guys. Yeah. It's, So it's great to be on. And, I don't think I've taught you that much in the last decade or so. So you've been on your own once.

I've been on my own. So If anyone's curious about the story, I when I was doing WordPress dev, I was still in, like, university, and I got brought into this agency called Jet Cooper in Toronto, which was Which was bought by Shopify.

Now they are a good good chunk of Shopify.

And Darcy was, like, the literally the only dev there, and this guy was pulling, like like, 28 hour days, cranking we're just Cranking beers, building websites. I remember we we, I I didn't really do much work on the that one, but, like, you built, like, a tweet wall Where it was a projector on, like, a, like, Holt Renfrew or something like that, and it would, like, pull tweets And I remember we had to find a list of swear words and filter them out. Oh, nice.

Yeah. It was hard to find that list, and then it was even harder to be like, Okay. Does this have everything we need in it? So you had to go through, like, this list of really naughty naughty naughty things that You didn't want in the in the feed.

I think the best part of that that project was actually how do we refresh the application.

Back in those days, there was, like, no live reload. There's no, like, a mechanism for, like, reloading entire application remotely, because we didn't have access to the store.

And so we did some interesting hacks to, like, get it to essentially refresh the entire experience.

Yeah. Like, it was It was primarily at night, and you it was, like, in, like, a locked glass building. Right? And, like, it was on Queen Street, I think, and everybody would walk by and see. Wow. But if something went Wrong.

Like, what do you do? You know? Like, the it's locked, and the it's basically just a web page on a projector,

on there. So I have an experience Or a memory of that where I was literally outside of that that that store in the middle of the night with my laptop tethered to my my iPhone 3 g, And, trying to I looked like a crazy person trying to, like, hack the store on the other side of the glass. Just you know? Yeah. Yeah. I have I have such fun memories of that time in web development. Even just I remember

you were able to make the tweets, like, Like, load in. Like, they would animate down as new ones came in. And, like, we didn't have CSS transitions back then, so that was all in probably jQuery. Right? And,

I became a JK master very early on in my career.

All that stuff, it felt like magic, Though, when you when you figured it out, I remember the 1st time I did anything with WebSockets. I was like, this is magic.

Unbelievable. Before you had to use a flash bridge, I remember in high school having to use, like Oh, yeah. A flash server to, like, do anything kinda socket based stuff. And, yeah, when Socket. IO came out, it just, like, blew my mind. Yeah. Yeah. That's wild.

But, yeah, amongst other things, we worked on, like, I had this thing called Deal Page, which was a daily deal aggregator.

So it pulled in Groupon and LivingSocial, and there was, like like, probably 50 of them in Toronto that were every day, they had a new deal out. And I would aggregate them all into 1 page, and it got pretty popular.

So me and Darcy, like, rebuilt it and Made it for, like, all of North America, and, that was, like, a really good learning experience for me because, like, I remember needing to send somebody an email every morning of the daily deals in their area. And I, like, I use Mailchimp, and I, like, programmatically created new lists Spur City. And, like, I didn't even know what transactional email was at the time. Now that I know, I'm like, I coulda just looked over the database and Incentive, but I had I had no idea.

That was a lot of fun. We did some we pulled some crazy nights to get that done. I think I was working Yeah. 3 3 different jobs. I was still at, Jack Cooper, that company that was eventually acquired by by Shopify.

And then I was building a new company, Themify, which was commercial WordPress steam company.

And then I was writing a book actually for Manning Publications, and, Never actually went through with that, but it was a a I wrote like a 100 pages of a book at the same time. And then we started that That project's or I started helping you with Deal Page. And, yeah, I was Yeah. I remember being at your place till, like, 3 in the morning, 4 in the morning. So Yeah. That was awesome. We when we built that, it was it was it code igniter Air or cake. Code code igniter. I was a big code igniter fan back in the day. Yeah. I was too. Yeah. I I was, like, I really appreciate the NBC structure.

And then once MB Star came around, I was like, what is why are people, like, throwing everything together? View controllers are, like, Interesting, like, to to just have all this logic business logic baked into, like, the the view rendering. And and I I feel like the industry has, like, Gone in and out of that trend a few different times, like putting these things together and then splitting them apart.

Yeah. Yeah. There was a time where everything we built was XPression Engine. That was, like, our stack of choice. And I I just went to their website, and It it looks like they've modernized it a little bit. I have no idea. I didn't know expression engine was still being used at all, but they have a you know, they have the classic client client's cloud of Nike and Pizza Hut and Disney and whoever. So, hey, Some people are using ExpressionEngine out there. Yeah. And the WordPress,

ecosystem is still thriving. That was another one that I wrote for a long time.

I did actually a few WordPress .com, which is sort of like their enterprise. They had a few enterprise clients like Facebook

And a couple of companies like that that I actually did work for late into, I would say, my my PHP days. Yeah. Yeah. Man, you have, Like, you've been all over the place. Like, we haven't even talked about like, you worked on PHP 7

as well, didn't you? I no. I I don't think I I Contributed back to PHP 7, but I definitely worked on the docs, of PHP. And the PHP docs were, like, to me, The quintessential like, that's how I learned so much about the language was the PHP docs were actually, like, really good back in the day.

I and so I did a lot of work. I think, initially, some of my 1st open source contributions

were to, like, docs websites, adding, like, better CSS and styling just to, like, even improve the experience that I had. And then you worked at like, I I don't wanna make this, like, a Darcy hour of work work. Like, you you you worked at so many Cool place like what was that? F I, Fantasy Interactive in New York? Yeah. So I moved to after Themify sort of,

started to grow, so we built, me and my business partner Nick Law, built Steamify after I left Jack Cooper.

And that company I ran for about 2 years before I moved to New York and worked for a company called Fantasy Interactive.

I think they're now just called Fantasy.

Yeah. And so that that company was doing really, really amazing, web experiences. Stuff that, like, you know, it it was like almost like we would get the r and d budgets from Google, Red Bull, like all these big big brands, and we got to experiment with, the latest and greatest, Chrome web applications and and some cool stuff there. So I was there for about a year before I moved back to Toronto.

Miss my family. So Yeah.

Man. So, you worked, I guess, like, the last one we'll talk about is you worked at n p At NPM, how did you end up there?

So I actually knew, Isaac Schluter from, like, back in the day. I'm sure you folks have bumped into him at conferences. I think One of the earliest times we had bumped into each other was, Jess Koff.

One of the ones in Florida. One of the like, Maybe in the 2000 2010.

20 tens. Yeah. I only know because I have the, I have the T shirt from the one in Phoenix, and that was 2009. And I base everything off of that. That's all my memories is based on that t shirt because I still have it. That was wild. Is that the one that had,

they had, like, wild west photos

taken? Yeah. Wild west photos, Weekend of the Bull.

Yeah. Those those are some good conferences. So, yeah, I had met Isaac,

who's the founder of MPM and creator of the MPM CLI, way back in the day. And and MPM in 2018, 2019 was sort of going through a really tough tough time.

And they had a bunch of amazing folks, leave, and and their open source, team. And and the team that was supporting the CLI It was it was pretty down to the bare bones. It was like a skeleton crew.

And so I came on board, actually, just before me, the a new CTO was appointed. So our friend, Ahmed Nasiri, who had worked with, at, doing some contracting, but also had worked with him, doing node school for 7, 8 years, like, we had we'd been giving back our time and and helping build up the community here in Toronto, around node school.

He actually took that position as CTO at NPM, and he said, oh, like, if we need to start staffing up again, The sort of community CLI team, open source team, like I have the right I have the perfect person in mind. I had like a quick Hour long call.

I say quick, but it was it really flowed really well, with Isaac, and and we saw whether or not we'd be able to get get along together.

And then I was hired within, you know, a week or so.

So I came on board as sort of, both a product manager and a engineering manager for For basically all the open source projects at MPM.

And so we had over oh my gosh. Like, over a definitely over a100. I think We had, like, many hundreds of open source projects, and I, over time, sort of was, became a janitor and cleaned things up.

But,

you know, IBM was really good at, like, sharing their learnings, and and obviously, it was a company that was sort of blazing Blazing ahead in in the node ecosystem early on, and and Isaac, it was a big part of that.

Until I I came on board in in July 2019, worked really hard to build up a a new team, and sort of reposition the the the product to as, You know, one that people could trust again and and care about.

And then roughly a year later, we were acquired by GitHub.

And, there's, You know, some some sentiment from GitHub as far as I know that that some of the things that we were doing were, you know, trending in the right direction.

Man, I I can't imagine. Like, we we had a story on the the recent horror story podcast of, I I can't say his name, but he said he had one of the 6 laptops in the world, that could fix NPM if it went down. And, like, Like, NPM is it's probably by far the biggest package manager out there. Right? Like, does anything else come close? Because It's front end, back end.

Like, home assistant uses it. Like, it's the package manager for so much more than just JavaScript. Right? Yeah. It's it's interesting. Like, you you have to try to

put it in a category with package managers. Sometimes there's just clients that come with them pre, like a registry prebuilt. So if you look at, like, Homebrew, like, there's, like, a a registry there, and it's curated by the contributors, and and other package managers and other ecosystems also don't necessarily have infrastructure on the other end. So they may leverage, Like GitHub or piggyback off GitHub, and sort of keep, like, an index of of known packages or known good packages or Known maintain packages that they have access to. So when I when I say that NPM is the largest or when you hear that NPM is the largest, Yeah. Registry.

I think you have to always, like, put it into you you have to kinda get into the nuance of what that actually means because there's some package managers, quote unquote, that have access to you, like, even more than just the packages on the NPM registry.

AppGM itself even will download packages from GitHub or remote tar balls. And so when you talk about the scope and the scale of the actual Package ecosystem, it's actually a lot bigger than just the numbers that you see on npmjs.com or even, GitHub. When GitHub talks about packages On their platform, it's, you know, it's only a small percentage of actually the packages that exist in the ecosystem. But, yeah, MPM is definitely the most, I would say most notable, and it I think it definitely unlocked for the ecosystem a way to contribute to that corpus of of open source without having To be, you know, part of sort of a blessed community. And and, you know, there was a there was a gate before, that kinda got unlocked by by making publishing really easy. Totally. What was what was working at npm

Like, in that era of NPM, was it insanity, or what what was it was it exciting? Like,

I I just I talked to myself about it. Yeah. It was a it was pretty,

you had to like, I had done a lot of contracting.

Wes kinda noted I've been all over the place, but I I did do consulting as well, for for almost a decade, I had, like, a a side gig that I would always lean back on if I wanted to pick up extra work.

And and what I learned through that time of my career was, you know, how to pair a troop and and come into situations where, You know, your you know, the project is on fire. How do we get it out the door? I did some work with, like, digital agencies for, like, Really pressing, you know, time, time pressing sort of projects. So, like I did projects for, like, the Grammys. I did projects for Mhmm. Other things where it's like there's these hard deadlines. And so when I came into MPM, it seemed like, oh, we've got a lot of Hard deadlines because we're a VC backed company.

We wanna show that we're still able to scale and grow, and then there's still, More opportunity in the platform. And so when I came in there, it was it was definitely like a lot of alarm bells were ringing in terms of, Okay. We need to quickly find, product market fit again to show that, you know, we aren't Sort of laggards in this ecosystem, notably, other companies had come in and sort of eaten a lot of the enterprise Space. So if you know JFrog or you know Sonotype, even GitHub. When I I first, was hired, GitHub had just now get announced GitHub packages. So you could see that there was, like, this encroachment on on sort of, you know, this this turf of of package management, When I got there, and and also some of the goodwill had left with some of the the great people that left. And so, that was another thing I had to quickly try to To show people that, you know, I really cared about the ecosystem, and I was there to to hopefully rebuild trust and, show Show the ecosystem that I cared about what, what they thought about what we were doing, and, yeah.

Mild. Yeah. Man, that's true. I I didn't even think about that because, Like, there there is the, like, hosting the registry, but that doesn't stop other tools, enterprise tools from coming along and just building the tool on top of that. And then before you know it, you're you're stuck paying the bandwidth bill For everyone's 10 PM install, which I bet is not cheap, but,

like and then everybody else gets the money. Yeah. Yeah. There's some knock doors that were knocked on, I think, by our our support and sales and and executive teams to The big companies that were definitely abusing, you know, the free bandwidth. So

I can't even imagine. We had, Ahmed Nasser at the meetup, not the last one, but the first one in in Hamilton.

And, he he was telling us some stories that I'm sure I'm not allowed to repeat on the air, but you give you give people a spot to put files, They will abuse it.

Yeah. There was always movies, torrents. If that like, our support team was actually really good at that time of knowing if there was a new, like, Marvel movie out to To be, like, watching for these keywords.

Oh, really? No joke.

Yeah.

And to be basically ready to take down those types of packages. So that's just like one example of, like, you know, the team worked really hard, and I I everybody I worked with at MPM was was amazing.

And I, unfortunately, didn't get to work with some of the amazing people that helped build the software to where it was at before I got there. Yeah.

But yeah. I was I was really excited. And so just to summarize, I guess, that time, it it was definitely chaotic.

But I I looked at it as like, Here's an opportunity to add all these features to the this product that I I use every day and that other JavaScript developers use every day. I I thought, Oh my gosh. Like, this is a dream job. Right? So Save me. Yeah. I'm I I remember around that time, I had lots to say about just the vomit

Stuff that would like, you npm install something, and, like, your screen like, I I would get the support request probably every day of someone being like, I npm installed, and this is what I got.

And it would be a 1,000 warnings and all this funding, and then and I'd be like, no. That's It's fine. That's how it works. Be. Yeah. You know, it's supposed to have 5,000,000 warnings and 80 security issues and and whatnot, But it's, yeah, it's it's much much better. And I had never strayed to using yarn or PMPM for any of my courses because I knew I was like, at some point, they're gonna change it, and it's not gonna work, and, like, it's gonna get out of I was like, I'm sticking with NPM.

I use NPM for a lot of my Personal projects, but never for my course. I gotta stick to NPM. So, maybe that's a good good segue there is What is Volt? And and why do you you need to make another both package manager and registry? Yeah. That's that's interesting. So Volt VLT,

I've already gotten hit over the head a few times for the the name because it's very similar to Volta, which is a Rust based, package, manager package manager manager. It's it's kinda similar to, like, Core pack. I I don't know. There's everything's a package manager to me at this point. Dependabot's a package manager, like, you know, all these tools that try to help you, You know, orchestrate or or maintain dependencies are are basically package managers in my mind.

But, yeah, the the last December, I decided to Quick, after three and a half years, basically working on the largest package manager in the world, MPM, and having, people like yourself, Wes, you know, be angry at me all the time for for a long output or whatever it was. We got a lot of Hate for peer depths and and things like that. But, yeah. The the re just reason I think that we Need a new package manager, and and more so, we need a new, registry.

Is when there's been a lack of innovation on the infrastructure side, of that. So clients, you you see, Bun has come out, with sort of some innovative takes on on how to install All your dependencies, PMPM, like you said, also had an innovative, take on how to reify and install your dependencies.

The clients are sort of limited to whatever the server and the APIs are gonna give them.

And the there's still a ton Left to sort of add to the infrastructure and the registry, I think in the JavaScript, package management space. So, that's why I think it's it's really important that we both provide a new client as well as the infrastructure to to sort of add some some new value there.

Yeah.

Awesome. So, what what kind of stuff is is it missing? Like, maybe go into the the the main features Of and and we should also say is that, like, this is very early days for you. Right? Like, you are in, what, sort of planning stages?

Yeah. I I'll I'll say it's it's It's super early. You can go to vault vlt.sh.

You can sign up to get early access once we we, Distribute that.

So, essentially, it'll come with a client and a registry implementation.

I think what Yeah. What's kind of, net new to the space is that we'll be providing the registry proxy with the clients.

So it's essentially you are hydrating an instance of the, infrastructure locally.

So this is a bit different than what current package manager clients do, which is that they, store, the packages at rest differently than essentially how they're stored in the registry infrastructure.

So it gives you an opportunity to essentially manage, private instances of the of this registry yourself.

If you want to, if you're an enterprise, you need to be firewalled.

It also gives you us the ability to build some really cool, APIs, and net new capabilities on on top of the existing package Ecosystem.

So, like, that's one thing that always got me is what's on GitHub is maybe Not necessarily what's on NPM. Right? And and when you look into the node modules folder, sometimes The code is there. Sometimes it's not. We had a whole show on why is my node module so big, and the the answer the answer at the end of the day is is is text. There's lots of text in there.

But so, like, are you saying that, like, you could possibly like, it's going to ship the authored

Values as well or or the bundled version or does it do the bundling? So that's what you're you're barking up the right tree. Essentially Okay. If you're able to control, the infrastructure, the one innovation, I guess, I would say I've seen in this space, has been by, like Turborepo and, NX and Loga, a project by Microsoft.

And and that one feature is remote caching.

And when you think think about remote caching, it doesn't sound that unique. It doesn't sound like it's this Crazy idea of having, shared, build artifacts available to your teams.

When you apply that to essentially the package management space, you say, how many times a day are we all installing the same packages And resolving the same dependency graphs. And how can we make that more efficient? And beyond just how do we make the resolution more efficient? Do we actually make the bundle size more efficient? Right? And, if you look at your node modules, like you were saying, Wes, like the There's a lot of stuff in there you don't use. Right? Yeah. And Almost all of it. Yeah. Yeah. Right. And so if there was a way for you to essentially Yet, a variant of a package.

So what we're calling distributions, it it would actually make the installation process Exponentially faster than on all package managers if they could essentially, you know, ask for the production version of, of a package which leaves behind its tests or docs or things like that. All this cruft that you might not need.

And by being able to do that, we'll be faster than any other package manager out there. That feels like it makes too much sense. It feels like like, Why yeah. That so many so many things could be solved with that. Like, 1 guy putting an 8 meg screenshot

Alongside his Readme is probably taking up terabytes of space sitting on some server somewhere. Bandwidth. Because Yeah. You don't even think, oh, yeah. Like, I'll put a little screenshot of of what this is in the read me, and, yeah, you forget to add it to your npm ignore, and then it it gets

It gets pushed to NPM, and now next thing we're going on is airplane trying to download all this stuff. Right. Yeah. Because your client is dumb. Like, your client isn't very smart. It's not asking for the right things. Really, the infrastructure needs to be like, oh, let's create optimized versions of all these packages, And you can, essentially request for for those. If you need the source, if you need types, if you need, the Original version or variants of that package, then you can install it. But that's in 80, You know, using 80 20 rules here, like, 80% of the use cases, you don't need the read me.

Right? So there's immediately.

Man, and we had Mark Erickson on who maintains all of the packages around Redux, and he just went through this, like, year long trying to get everything thing published to ESM, and it is it's a nightmare trying to get it all published. And that's part of why everything is so tricky because you're shipping Common JS, ESM, the source, TypeScript types, all of that stuff along with it. Do you he we asked him this question. I'll ask you as well.

Do you ever foresee a future where we literally don't pre bundle or precompile, and we were just to ship

As authored, and we could require as authored? Yeah. Totally. So I I've listened to that podcast, and I've actually had these discussions with many different, package authors that are trying to do the right thing. They're trying to give you, their software in whatever way you are going to Install it or import it. And Yeah. The the huge problem with that is that it doesn't back port to all of the existing packages that, are out there. So every common JS package out there that was published, you know, 5 years ago, that might still be good Software that might not have bugs, that might not be, you know, needing to have any updates to it. It may not be able to be compatible with your Your run time or or the way that you're, you know, importing package or modules. So the module syntax problem that we have right now, like, and actually, the dual module hazard that exists, dual package hazard that exists, are 2 things that I think can be solved by package variants, and by having a smarter, registry and infrastructure to essentially derive, you know, those variants for for you. So in the case of, let's say, a legacy, node module, we can essentially create an ESM Variants of it, and make that essentially seamless for you to install.

This is a bit different. That strategy is a bit different than the one that Von has taken, where they've Allow you to do requires an imports alongside each other, which I think is a Mhmm. Kind of dangerous, because there's a lot of magic that's happening there.

But the the hope here is that actually by including a specific variant of the the software that you want, That specifically has the the imports or exports that you need, then you're gonna be able to, essentially have, consistent packages and and consumption sort of strategy. That's awesome. And

the other thing is and and maybe you can explain how this works. Is that like how come an NPM install sometimes takes forever? And and and you can PMPM, and it's it's way faster.

Are you able to talk about explain why that is and, like, what your thoughts to that approach would be?

Yeah. I would say, like, it's always apples and oranges when you're comparing these package managers. What I noticed a lot, while I was managing MPM was people would Throw benchmarks at me, and I would say, that's that's great. Like, but, if you only care about performance, that's a race to the bottom. Like, rip out all your code, And it's gonna be super you know, your package manager is gonna run super fast if it has no features.

So, npm is is Pretty, feature rich and highly configurable, more so than any of the existing or the other legacy Package managers is what I'll call them, which makes it slower to interpret your dependency graph. And also how it lays, your your depths on disk It's a it's a slower strategy by default, where it actually copies to disk, all the all the packages. So that's That strategy is inherently gonna be slower than doing a SIM link approach, where you essentially only write to disk The one time for the package, and you don't have to copy around.

I know Jared from Bun has found a fast copy, like system call for, I think, Linux and and Mac OS. He's found a trick there to to make copying a bit faster.

That's one thing I'll I'll be looking at for sure, but I don't think it solves the problem of copying your entire, you know, dependency Yeah. Free into your node modules. What are the problems with SIM linking? Well, they're not supported by Windows.

So you have to use junctions, and it becomes a big mess, to to handle.

But also the ecosystem broadly, relied on the behavior of expecting packages to to Live in their parents. So there's actually, like, whole ecosystems. Like, I think the ESLint ecosystem sort of relies on this, Being able to access packages you never or dependencies you never defined, which is really scary. We call that, like, phantom dependencies.

And yeah. I I just went we always went through that with, the same idea with the Vercel file system is that, like, we had a package that simply just referenced, Like a string, and it would assume that the file would be there. But because it has no there's there nowhere in the The required tree was that WASM file.

It was simply just a string that it hoped would be there.

We had the hardest time trying to get it on Vercel's Mhmm. File system. And, like, that's the beauty of all these this modern JavaScript tooling is that, like, if it If it knows that it is there. You know? If you import the thing or if it can statically analyze it somehow, then a lot of this becomes

A lot easier. I was gonna say, yeah, it's a problem for some ecosystems like the React Native ecosystem.

They weren't able to adopt Or even, I think, try out Yarn's PMP implementation, which did something similar in terms of a virtual, file system and similar to PMPM's, installation strategy, I think it Breaks on, it breaks the React Native Ecosystem. So there's certain certain folks that can't even sort of,

entertain that install strategy. Mhmm. What happened to yarn? Like, it was it was popping for a long time, and then I know yarn 2 came out. And Do you have, like, a a a Kohl's notes of that whole,

story? 4 just came out. So I I I do wanna give. Wow. Yeah. Yarn 4th. I wanna give a big shout out to to Myel and and the group that does maintain yarn.

You know, I would say that competition breeds Innovation. And so without Yarn, we wouldn't have with the MPM. And and without PMPM, even at MPM, we wouldn't have, You know, I think been pushing as hard as we were in in a bunch of different places. And so, like, you know, they do deserve a lot of respect.

But you're right. The the Yarn actual market share dropped while I was at, at MPM and GitHub, and and we saw that, in all the various numbers we were tracking. Yeah. And Yarn was the first to do workspaces, was it, or was that more of Like a learn nothing. I don't know. I think they came out around the same time. I'm not sure if you guys remember which was first.

Yeah. I think Yaron was the 1st package manager to do it,

You know? Yeah. Made that so much easier. That I was that was a world of hurt when you couldn't do that. Now it's it's supported in all the tools, and

It's it's great. Yeah. The the workspace support is is interesting. Like, monorepo, tools. Like, I I tip my hat So what NX is doing, and I I think that they are, you know, in Volt, we will be likely, Hopefully, direct competitors with with tooling like NX, which, you know, is trying to make, monorepo, development

More seamless and Yeah. Easier? That's good. I'm I'm very much on board with a lot of these tools that do it all for you. You know? Like, I will buy into these tools if it's all I don't have to put together 8 different things. Even Even the ESLint thing, they they are rewriting their new config because of the you can't have a dependency of an ESLint config, and I had to install some crazy hack that Microsoft came up with to Jeez. To do that with mine, but it's coming. We had Nicholas Zakas on the on the podcast, and I can't imagine the the nights he spent thinking about how to approach this type of thing. Configuration is tough. That's a it's really interesting, like, the all in one solutions.

What I will say about Volt is that we, you know, I'm I'm very, aware of the current workflows that teams have set up.

I worked very closely with the Microsoft Office team while I was at at GitHub.

And they were working on an internal project to to, I think, bring together all The JavaScript projects into a single monorepo, and so there was, like, something like it it was Not hundreds. It was almost thousands of repo repos they were trying to bring into a single monorepo.

Really? Yeah. And so it was, like, Definitely, one of the biggest projects to, like, be mindful of as we were building out the workspace implementation in NPM. And And, there's specific tools that they they use today to sort of scaffold and build out their packages.

And you'll see actually, like, people have to use 2 or 3 tools today to even, ship a package.

You you might use Ferdaccio in your CI to To scaffold, like, a private, registry to stage packages so that you can test in your CI those packages as if they were in the registry. Like, it's it's a lot of work just to even publish a package, today, and and make sure that it's it's good.

And all that cruft could be, thrown away if you had a a client that actually can help you stage and test packages all in one. Yeah. Yeah. That's a whole another world,

These companies that have, like, thousands of engineers writing JavaScript every single day. Like, you think, oh, Microsoft Office. Like, What are they doing with JavaScript? Oh, yeah. The like, every single product has a web version. Right? And it's super complicated.

And I'm sure they have some, like, Copy paste library that needs to make sure the copy paste is consistent across all the libraries, and it has to be tested. And, Man, that's a that's a wild world when you go from, like, slinging together a little Svelte site versus, like,

Thousands of packages. You know? Photoshopped on the browser. Yeah. Yeah. Yeah.

For real.

I I'm curious about your personal thoughts on on The concept of, like, people talking about you know, because we're we're getting into config a little bit here. But, like, people saying, like, alright. We have all these massive amount of config files.

What are your personal thoughts on the solution for that?

This came up actually. I worked very closely in the the node tooling working group, space. We had a whole discussion about what to do with configs, and that I recently saw that got kind of, went viral again On on what we should do with config. And I I think note having just supported, your dot m file by default, and they were sort of pushed into that by button. I think this has become a a problem again.

I've always looked at config as as just like, we we are burdened with abundance of of great technology.

The fact that you have a 100, config files is because you are utilizing a 100 different projects that probably are providing some level of value.

And their location to me isn't that that problematic.

If you are concerned about people touching those things or or For it to be noisy, then that's like another another problem. You can easily hide those things. I think Cyndrasaurus has, like, a great Git Plug in that helps you hide them in in, Git repo. So or GitHub repo. So you don't even have to see all these noisy configs in if you're surfing GitHub.

So it's it's not like a major problem how we've evolved that ecosystem today. I think you're gonna end up with, like, an x k c d situation if you try to, like, standardize a new, you know, a new place to put all your config.

Like you know? Yeah. So for anyone, not familiar with that xkcd article, it's, why do we have all these different standards? I'm going to create A new standard, that will be the standard for all standards. Yeah. And and then also, now you have 3 competing standards, and we often we I think we even have a show on that about, like, configs. It's like, why is there not a single config out there? Like, why has nobody made the golden convict, and it's because they are all so different. Some of them need to be able to be statically analyzed. Some of them need to be able to extend And, like, deeply fold into other configs. You know? Like, if you have a a config and you're like, alright. Well, I wanna override this Level that's 8 levels deep, does that then overwrite everything, or does it fold in? Does it fold in just just that object or all the parents? And, do you need to be able to import things into it? What kind of should it support Wasm? Like, it's it's a minefield, and that's why Nobody has really figured that out. There's there's the config folder, which is great. You can you can hide some of a lot of those files, but, yeah, I I I had one of those tweets too. I took a screenshot of all the config, and people are like like, what like, we should delete this, or what happens if you delete those? Or that that was a That was the 1 person who was like, what happens if you delete those? And I was like, well, if you delete code from your code base, it will probably break. Yeah. You know? Like, it's part of your it's For your application, you kinda need that. As much as I I don't like those config files, it's better than having to remember what buttons you clicked

or settings in your your login. You know? And and also, here's a little sidebar. If you're using Versus Code, there is a feature, File nesting patterns.

And something that I've done personally is I just have all of those config files nested underneath package dot JSON. So package dot Jason looks like a folder. You could pop it open. Hides it from your your route. That's cool. Yeah. We should let's check that out right now because people always say, oh, what was that thing you talked about? It's not a plug in. It's simply it's built into Versus Code, but there's, like, a big config out there. I think it's from Ant Fu. He he does a lot of stuff with that. I'll I'll tweet the 1 or I'll at least have in the show notes the, the line that I have for,

specifically for package dot JSON stuff. Yeah. It's If you Google Antfu, a n t f u, Versus code file nesting config, or we'll have it in the show notes as well. And you basically Copy paste that sucker into your Versus code, config, and it will start nesting them for you.

I wanna ask real quick about the on prem registry. I just this is just out of sheer curiosity. Is that, like, sometimes companies need npm in their company. Right? Like, I've I've had workshops before where we could not figure out why npm install wasn't working, and it's because They have their own private registry of NPM because they can't you can't necessarily trust that one or they don't trust it. Right? Is do they literally have, like, a server with every single NPM package on it in their

in their firewall? Yeah. So there there's a couple ways that companies do this.

One is exactly what you're saying. They host, like, an internal registry, whether it's For Datchio, which is an open source, sort of backwards compatible, NPM clone ish That you can run yourself, or they use one of the enterprise, offerings like JFrog.

JFrog's Artifactory or Sonotypes Nexus, or even there's a new entrant Cloudsmith, that also has has, these capabilities for for companies. So they will literally, mirror The packages they need for their their projects in those instances, and those will be firewalled.

And in some cases, I've even seen, IT and then heard of IT, folks, locking Any access to the public registry. Like, they they wanna ensure that the way that your company is getting its packages is through its its third party Tool like like Artifactory.

There's some benefits to that because you can create policy engines, and and try to Codify heuristics about what is and isn't a good package to be, installing.

And this is sort of one of those key capabilities that NPM never built into its Platform that I think allowed for for it to sort of, for a market to grow around it.

And so, yeah, companies Today do essentially host their own caches of of packages.

One, they can speed up build times. And and again, going sort of back to some of the Quote, unquote innovations of of some of these tools now, like remote caches are are no different than, essentially, bundling or or Hosting a a cash version of of your the packages you need for your your projects.

The one dangerous thing about what those tools have done though Is oftentimes they will, allow you to fork a package and then let it, override what is in the public upstream registry. And so you've heard you might have heard of this, Stream registry. And so you've heard you might have heard of this, vulnerability in the ecosystem called dependency confusion.

And dependence confusion is where you might have misconfigured, a scope to be looking at your private registry, but he actually goes and and actually fetches Upstream because that tool actually just proxies through that request. So you expect to get one thing and you actually get another.

And it can be really dangerous if, if you happen to be an attacker that knows might have known about some, like, leaked package names companies are using.

Wow.

Awesome.

And do you know how big is if you downloaded every single NPM package, do you have any idea how big that is? I'm just So cute.

He put on a USB stick.

Yeah. You can't put it I think depending on how big USB sticks have gotten, you might be able to do it. It's, I think the last number I heard and and I haven't checked my own, because I have a running, A follower right now, so I'm I'm basically indexing the entire MPM, package registry as we speak.

But, my good friends at socket socket dot dev, Fras and I have spoken a lot.

I think the last number he gave me was 28 terabytes.

So Holy.

Yeah. So it's 28 terabytes worth of, data. Big,

but not as big as yeah. That that is a lot. I could fit that on my NAS my NAS drive. Yeah. Yeah. Like, Oh, that's exactly where my mind went. I was like, wow. That's big. I was like, you could, like, you could walk to a store and buy 28 terabytes as well.

Totally. But I wouldn't wanna get the bill for that. It's reasonable. It fits in a room. Right? You can you can sort of conceptualize how big 20 terabytes is of of,

Well, I I hope one day you'll be selling the, vault box where Oh, yeah. Feeds. Like, I saw on Marketplace the other day, Somebody was selling a Google search indexing server. Yeah. Like, it was a it was a blade server, and you you throw it in your rack, And it it would index all of the content and then send it to Google, but it would do it, like, on the thing so that you could if you had, like, like, Shopify might have a 1,000,000,000 products. Right? Like, you could you could index them on the hardware locally. That, Wes.

We had a a client at one of the agencies I worked at who ended up buying 1 of the boxes and and keeping it in their rack. Really? Because what they did is they had It was like a the site was it was a foundation. They had a lot of grants, but they had just this massive library of articles and all kinds of things. And the solution we went with was get a Google box, then we had to hire an expert with that Google box to configure it.

And it worked out for them, but, yeah, it's a a wild solution.

And where does that that was a question before you mentioned Socket is where does Socket dot dev and Snyk security, where does that fit into this whole Package ecosystem.

Yeah. So if you actually go to the vlt.sh today, I do have a link to a blog, and there's only 1 post on it from a few months ago.

We're actually talk about a security vulnerability in the ecosystem today.

So Vault's, by default wants to index all packages.

So the fact that I'm right now following the NPM registry and and indexing all of that, is just actually a fraction of what I want to index.

So imagine that Volt becomes like the Google of packages.

That's sort of the hope and goal of this.

So Today, the, FPM registry is really looked at through like a singular lens that the API, that was written, like, 10 years ago is is accurate and that it's doing the right thing. And unfortunately, a lot of The security companies legacy security companies like Snyk's, have looked at sort of metadata From the registry as if it is accurate.

And so I wrote a blog post recently called man at the, you know, the I forget what the title is, but some scary title about, like, the massive The massive bug at the heart of the npm ecosystem.

That's right. So Click clickbaity

title for sure.

And I coined essentially this term, manifest confusion, which is a play on Dependency confusion, again, hopefully, I I saw to to give you some numbers, I saw like a mill half Half a 1000000, hits to the website, like, that week after I I wrote that blog post, which is Wow. Is awesome.

But, You know, the security companies, I think, that are reading from, the NPM registry have done a bit of a disservice to the ecosystem because they've actually just Read mostly the metadata, and they haven't really been looking at the contents of the packages.

And so there's actually a difference between those 2 things.

And in the blog post, they actually outlined the very dangerous problem here, which is that the metadata can be essentially, You know, scaffolded and and and, you know, you can essentially throw whatever you want in it. And it can be completely different one from what's actually inside of the package.

And most clients today, most package managers today, actually will use both of those things, the metadata and the, package to install your dependency, but then they'll throw away the metadata on subsequent installations.

So if you have a cache Today, that has a whole bunch of packages.

What it you might find is that you get inconsistent buggy behavior when you're, either live and and pulling from the registry or, installing from caches. And it's because of this This bug of and this inconsistency in the actual APIs.

So security companies, I think, again, have inaccurate information today. So they might show you licenses that are completely inaccurate, author information, or even dependency information that is completely inaccurate.

So one of the great, you know, companies that has sprung out of this and and this massive issue though is is socket. And they they are actually doing analysis of packages themselves.

And they're taking a a new approach, I think, in this space of actually, trying to annualize Behaviors and and the actual contents of packages versus just relying on metadata to try to infer, or or using legacy sort of CVE data, which It's also been, mostly annoying and and not necessarily helpful for finding malware. Wow. Beautiful.

We should have Faras on, actually. Yeah. Totally.

He he's he's built many cool things in the day.

An app you might use, he built WebTorrent, like and The ability to download a torrent in the browser is wild. That and then, like, he just figured out how to do, like, networking and File concatenation.

So smart guy. Frost is the best. Yeah. We we work very closely together, Over the last I think he well, I I think I can say this. He tried to poach me away from GitHub, before I I decided to build Volt, and and we'll probably be working very closely together in the future.

The metadata that Volt or, Socket Is providing is information that we want you to query in the Volt dependency graph.

So that's something that I didn't mention before, but, one of the key innovations that I I sort of came up with in the last year or so is the idea of, dependency selectors.

And they're basically lifted CSS selectors that allow you to write, like, really expressive ways of of traversing your dependency graph just like CSS.

And treating basically the dependency graph just like the dom.

And, it comes really, it it it makes it really

Interesting for how I think developers can can start to introspect and and take actions on the fancy graph if they they write these selectors. So Let's move into sick pics here because you said you had a lot of them, so I wanna make sure we have enough time to chitchat about that.

What what do you got for Sick Picks today?

Well, I have to ask. Like, I have to get limited here. Because I have, like, an animal, a book, A thing, a movie, and a whole bunch of music that because I haven't talked to you in so long, West, I I just need to share everything.

Oh, yeah. I was I was actually gonna ask you about, about the music as well. We'll save that one for for the end, but let let's go through them all. We have time. What's your Yeah. You got an animal?

An animal. So yeah.

We'll start.

I got a dog last year. His name's Charlie.

And I I know you have a dog, Wes. I'm not sure. Scott, do you have a dog? Oh, I got a couple. Yeah.

Oh, you 2? The dog has changed my life for sure. He's a goldendoodle, but he's a Flat haired goldendoodle. I don't know if you've ever seen this. No. It's he looks just like a golden retriever.

Anyways, he's he's amazing. You don't have a dog around and you can, I definitely

say get it? Yeah. Shout out to dogs, For real.

Dogs. Yeah. Dogs are wicked. Gold goldendoodles are awesome. They are my sister has a Yeah. My my sister has, I'm pretty sure, a gold doodle or yeah. She has a goldendoodle, and, he's not the smartest dog in the world, but great great spirits.

You know? And, I think it depends on which how much you get because of the doodle or the poodle aspect. Poodles are very smart.

Don't know about golden retrievers. I think that golden retrievers are smart to an extent, but I think they're kind of, you know, happy dumb. Yeah. I I like it when he licks my face. I don't like it when he is knows exactly where all his toys are hidden.

Yeah.

That's great. And then, yeah, I have a book or most actually a magazine. I just got back from Spain. And and when I'm traveling, I was speaking at a conference there in Spain.

When I travel, I usually pick up a copy of Scientific American.

And I'm not sure if you folks have ever read that magazine, but it's It's great. I do. I love it. I have never read it. I don't think I've read that in a long time,

but that, Like, even just even holding that on a plane makes you look like such a baller.

You you look you look smarter than you are. I can't understand half of what's in it. I don't actually, like, I think the last app, I think the last magazine I got was called mind bending physics. I don't know what the heck they're talking about in it, but it's it's great. It's If you surround yourself with enough of that stuff, Some of it permeates into the

brain. I I hope. I really hope. They just hold it up close to you. Yeah.

What what is the thing now? The thing is Nespresso machine? Have you you guys got one of these? Oh, yeah. We got one of those. We don't have one, but I often talk about Like, I've had it many times, and I'm always just like, this is good. This is really good. You hit a button and you walk away and you come back, and it's It's it's great coffee.

I got it as a gift, so I I'm not saying like, I know they're they're super expensive. They're not they are not cheap.

It's like a it's a much fancier Keurig type of deal, though. You you have the pods, but the pods are recyclable.

You know? So it's not just, like, Committing trash to the world here.

Yeah. No. I I I like I like the Nespresso. I can't do hot coffee. Too acidic for me, but The the vibes are good. I often talk about the Nespresso as and I I know coffee people are gonna get mad at at this type of thing, and I appreciate really good coffee. But, like, There's something to be said about how technology figured out how to make it consistently good.

You know? Like, it's, It's not the the best coffee in the world, but it's really good. And it's consistent, and it's you can just pop a pod in and And go for it. So I've often wanted one of these. I I keep looking out at like a I gotta find 1 on the side of the road and fix it before I get one. But You're saying your daddy's instant coffee. Right, Wes?

Yeah. Yeah. Yeah. It looks nice. I I think you would really enjoy it. Totally. And and you get everybody has, like, their own taste. Right? So you could just get different pods. And and, I still make pots of coffee here and there because I I'm just a caffeine addict. But, yeah.

Yeah. And then I had a movie, which I think I hope you both have seen by this point. But, the Blackberry very movie? I have not. No. I'm not Canadian, though, so it hits less hard for me. To.

This is I'm sorry, Scott. You're outnumbered on on on the Canadianness.

Yeah. I did hear there's some hockey stuff in this movie though, so I'm down for that. Yeah. Jim Vasili,

like, it's just It's he tried to buy a hockey team and bring it to Hamilton, and that was a whole thing.

And, Yeah. Wes, if you haven't seen it, it's really good. And and Scott, like just even for the tech tech,

history lesson, it's really good. It's it's funny too. So Yeah. I so I follow a YouTuber, Mathias Wandel, and, his YouTube channel is called Wood Gears. And he's just like a brilliant engineer, and He applies that engineering to everything.

He even he even did a video after I asked him a question. I said, is it better to have a fan pointing into the house or out of the house on a hot day. You know, everyone argues about that. So, like, he But but, like, he's in a chair, so he he has the he backed it up with stats, and he did all the testing. Anyways, he he used to work at BlackBerry In, like, very early days, like, back when it was all, he would build, like, radios for the BlackBerrys.

And, he has a video on his YouTube channel where he goes through all the different parts of the BlackBerry movie and says, like, That was real. Jim was really like that. No. That was a bit exaggerated. I don't know who they're talking about here. So I'm gonna try it. Is it out on, like, streaming yet?

I watched it on the plane, so that's why it's fresh in my memory.

So I don't know if it's on streaming yet, but, yeah. It was it was just I have a I have a Letterboxd

pro account that tells you all that stuff. It's on iTunes, Google Play, Amazon, and Vudu.

That's hilarious. That is not in Canada yet. It's not streaming anywhere in this room. Yeah. I'll go, fire up some software from my friend, for us, and, hopefully, we can find it.

Pay for, like, A 1,000 different streaming services that can never watch what I want. It kills me.

Awesome.

And music. I know. So we're both both a bit of a a punk rocker. I remember you introduced me to Sleeping with Sirens, which is, like,

One of my favorite bands ever. So I'm curious what you got for me now. Yeah. So I'm not sure if either of you have been listening to the new Blink album. Oh, yeah. It just dropped.

Oh, so good. The specifically, Anthem part 3 and then More Than You Know, like, Those just slap. It's like

We've been having that on every single dinner. We don't we we listen to music during dinner, so we'll just have that album on every night during dinner right now. Did you get tickets to the the Toronto show? Alexa on Fire's opening for them. No. I didn't see that.

But I do have tickets to the next band that I need to shout out, which is Monine. Oh, yeah. I love Monine. Next At

history? Scott, you like Monine. Yeah. Some of those You know many shows or or some of those albums oh my gosh. I could listen to those on on repeat. I could, man. In fact, now I gotta put them all on. Yeah.

I think the first one I got was The Red Tree, and then I went Yeah. The Oh, no. Are We Really Happy with who yeah. All of all of these albums are great. I'm a big fan.

Yeah. And we're really happy with who we are right now. Yeah. And they're playing,

next week. I'm going to the show here in Toronto, at the History Music Hall.

Yeah. And then, like, a week later or 2 weeks later, protest is coming So the same spot, protest the hero.

They also just dropped new music, which I've been listening to. And it's like, Oh, it's it's it's crazy. Like, there's so many Canadian bands that I'm like, Comeback Kid also dropped new Music last year and, are also playing in Toronto in the next month or 2. So

I feel like we're in, like, a really good era of Emo, punk, post hardcore lately. Don't you don't you feel like that? Like, there's, like, it's back? Yeah.

Do you listen to Bring Me the Horizon? Oh, yeah. Yeah. So they've been also dropping a whole bunch of singles, and on repeat has been, Strangers and Lost, like, 22 ones that I've been listening to a lot. And then, just like 3 other bands, No Pressure, Which is a a small band that came out of the story so far. Mhmm. Neck Deep and the Wonder Years all have, like, new music in the last year that I've just been, like, Crush it, man. I need to you you need to get on to a couple I have a couple playlists.

I've got one. It's called retired scene store metalcore, and it's just me Me and another guy, Nathan Noller, and, like, this guy knows exactly what can go on it. Like, he he knows like, Sometimes people add stuff, and I I remove it, but never have I removed one of his. And then I also have a retired emo kids

one as well. So I'm an emo kid for sure. I was I was like a get up kids fan, dashboard

Yeah. Fan. Oh, yeah. It's It's good stuff. Sweet. I'm that Monine, that is gonna be a good show. I I was at history a couple weeks ago for the fit for a king, and Devil Wears Prada, and there was a a sign for Moneen coming. I had no idea they were They're still up there. I saw them for $5 at at the YMCA. Yeah.

That's good. I had, like, a Monique hoodie.

They're my favorite band for sure. Them and I think the last show we went together actually to, Wes, was the Full Blast.

Full Blast was also like a band that I just, like, love. I still I

have a Full Blast shirt that's a medium,