How do you security-audit external software using NPM packages?

As much as the JS ecosystem terrifies me, Node isn't really the problem here. Receiving plugins that contain minified blobs of JS is, practically, quite equivalent to receiving plugins that contain binary blobs.

If you accept receiving and using plugins that contain unauditable blobs of software, whether it's minified JS or a binary, a good-quality audit is going to be virtually impossible.

In many other ecosystems this wouldn't be normal. If a Rust crate ships binary blobs with no easy access to source code, I wouldn't ever consider depending on it.

If you can't prevent these blobs from infecting your system, you have to deal with the risk another way – locked-down containers on the server side, strict CSPs on the client side, and monitoring.

We've been using trivy [1] to audit the container builds we've been producing for a relatively security focussed project. As well as scanning for OS package level vulnerabilities it also scans for reported vulnerabilities in NPM packages. Works well for us.

But the other complementary approach is to lock down other things - so for example, if you're running in a container, make sure that container can only talk to the proxy in front of it. That way, even if there was some kind of malicious code running in one of the modules, there's no way for any data to get in or out (unless it finds a way of injecting into any web input/output, but then you need to be scanning for that too)

[1] https://github.com/aquasecurity/trivy

Independent of the language, I only use external code if it is small enough that I can manually review it. Often I refactor it into a single file during this process.

This of course excludes the majority of packages out there. But apart from security, it has another benefit: These dependency very rarely break and need updates. So compared to projects with a more complex stack, projects with a lean stack are easier to maintain.

It would be great if there was a "single small file packages" movement so that more lean open source software will be created.

While you'll probably get a good overview on the question you asked from the other comments, don't overlook that non-audit measures might give you even better bang for the buck — and that going 80% on a mix of lots of different measures will usually give you better overall effectivity than going 99.8% on just auditing.

For example, putting that external software in a hardened container with network policies, non-root user, capability drops, readonly filesystem (,...) will go a long way towards securing it, even if there's a cleverly disguised backdoor you didn't manage to find.

Dumping your database is only half the fun if the app isn't able to send all the data to Alonistan...

In practice, I don't think anyone bothers. I asked a Node developer how they ensure none of their 3000+ NPM packages would send our customers' confidential information to Somalia, and he looked at me like I'm from another planet. To me that's reason enough to not touch something like this with a hundred foot pole, and keep well away from the blast radius when this inevitably backfires.

Nexus Lifecycle / Nexus Auditor tends to be useful for this - in absence of a package.json it crawls the raw js files and finds their source. It can help figure out things like embedded jqueries etc. That being said, it has the same limitation as other tools - minification and bundling obfuscates origins and makes it harder to assign identity to the source package.

The only way that I can think of getting around this is to have a hard requirement for a source registry - or asking the premium plugin producers to produce a SBOM like cyclonedx or spdx and evaluate that in lieu.

You’re primarily talking about proactive auditing here, but if something does sneak in you’ve got problems. In the best spirit of layered security you should also build up a strong Content Security Policy and include that with your pages to make sure that there’s a whitelist of the servers the page can talk so, and that technologies you know you’re not using are locked down.

I believe one thing that can help security a little is to use full version numbers in your dependency list. This applies to all package managers.

Because the moment someone hits update on the package manager nothing will get updated and you won't receive potential dangerous updates you did not review first.

Edit: sorry this is not relevant to the question...

If any software pulls in more than a few independent npm packages, I call it a huge risk and sandbox it as if it's a ticking time bomb. After some deliberation I've come to the conclusion this is a reasonable approach with all software. It's for me a nice approach to deny every capability unless it is critical for the functioning (that you want) of the software. If that's "full network access and subprocess spawn capability", then you should probably not be using it anyway.

Sonatype Lifecycle is designed to analyze a built package and figure out what's inside it, specifically when there aren't manifest files to tell you what's -supposed- to be there. It can obviously do a lot more, but the analysis is designed to solve the exact problem you're describing.

https://blog.sonatype.com/mapping-the-javascript-genome-for-...

I am maintainer of couple of OSS projects https://github.com/ossf/scorecard https://github.com/ossf/package-analysis

These projects help with repository best practices and does some level of npm package analysis based on rules.

I'm in the process of moving to Deno (away from node+npm) due to its feature set reducing the need for and management of external code--in part by designing away a package manager (dependency 1) and instead using web platform features (ie many other dependencies--before my own project's). The docs include a section specific to external code https://deno.land/manual/linking_to_external_code and the solutions work well overall. Bundling has some bugs and there are transformations and complexities in the mix of rollup+npm that I'm still working through, specifics related to caching, subresource integrity and bundling external code at https://stackoverflow.com/a/69833335/965666 As endymi0n noted a mix of strategies is the best approach.

I've been avoiding the Node.js/NPM ecosystem for this reason among others. Unfortunately more and more packages are including Node for simple scripting functionality.

At this point I can no longer hold my breath and hope that the Node.js trend goes away.

Is there really anything better than `npm audit`, even with all its faults? Any relatively popular and well tested library will pull in dozens of dependencies.

  $ du -hs node_modules
  
  289M node_modules

Yeah no...

In terms of preventative measures, harden the underlying infrastructure. For example: prevent outbound connection initiations. If you need it, profile the connections and lock them down.

From a detection standpoint, the free options are NPM audit and GitHub’s Dependabot, which are ok. A commercial option (e.g. Snyk, WhiteSource, BlackDuck) is typically more recommended to manage exceptions and get more accurate results (e.g. is the vulnerable code used by your code).

3rd party npm libs are a massive security breach waiting to happen. The recent trend towards compiled javascript has made the problem even worse, since code can no longer be manually checked for security issues. At the moment, the main advice seems to be just keep your `dependencies` as up to date as possible (doesn't remove vulnerabilities), or lock down your run time environment (not an option for code that is run by others)

Snyk and Dependency Check are two tools I've used when performing security checks.

It's not free, we're using Whitesource which provides alerts against libraries being used in the codebase. It can scan package.json, but it can also scan individual files. It matches the hashes of those files with those from open source projects so it usually able to identify which library that file came from, or at least where it was first seen. That way the package.json isn't always needed.

I would love to know what you specifically look for in Wordpress plugins, especially on the PHP side. I've written quite a few but would appreciate some tips on security.

If you prefer a private discussion, my gmail username is the same as my HN username. Thank you!

This should be related, but I haven't tried it (my brother builds it, so I am not affiliated) https://requiresecurity.com/

A WordPress plugin may contain hundreds of interdependent npm packages all neatly bundled and minified. Without access to a package.json or package-lock.json it is quite hard to find out which individual packages have been used. Quite often there is also no public repo available of the development files.

To give an example of my process thus far:

Someone in my team wants to see if we can use plugin X. I’m downloading the plugin to have a look at the code. Luckily this plugin has included a non-minified version of the js file. I can derive the use of npm packages from this file. Using Snyk I have a look at the first package mentioned. It’s axios. The included version is vulnerable (high & medium severity) and has been for almost a year (Note: the last version of the plugin is 3 months old and does not exclude this vulnerable version in it’s package.json which I found in a Github repo later on).

Since I have no package.json nor package-lock.json (all I have is the distributed build) I can’t easily update the npm package. I have no clue as to how this package relates to the other packages and how their version might depend on each other. Even if I would update the package, all other users of this plugin are still vulnerable. I contacted the plugin author. He tells me he will update the plugin as soon as possible. The plugin is (as of today) still not updated & has not released a new version. In the meantime there have been two new versions of the axios package released.

Every user of plugin X is still vulnerable to the issues mentioned on Snyk, but is this a real problem in this specific WordPress plugin context? I’m not sure how to interpret the high & medium severity in the context of this plugin. How exploitable are these issues & what is the impact of the exploits in the context of this plugin? Do I need to be a logged in user? Is this something which can be triggered by any visitor? What am I able to do when I can exploit these vulnerabilities? I can only try to find answers to these questions if I’m willing to invest a lot more time into this, which more or less beats the purpose of using a ‘ready-made’ WordPress plugin. And this is just one package of multiple npm packages used in this plugin. Packages which also have their own dependencies as well….

At this moment I’m wondering if any WordPress plugin using npm packages can be trusted at all.

ps: The way the npm ecosystem is structured is, in my view at least, problematic. Often packages are not like libraries as I’d expect, but look more like a function call or method call. I’d prefer to write these short pieces of code myself instead of depending on external code which also includes extra risks. The very rapid release schedules makes it even harder to trust external software (like a WordPress plugin) using npm packages as it seems they cannot keep up with it.

I’m sorry if this seems like a npm rant, but I’m seriously looking for methods on how to deal with these issues so we can use external software (like WordPress plugins) built with npm packages.

Can you reverse lookup the packages using the abstract syntax tree? Most of it's probably from the top 10,000 packages.

I use `npm audit` and (and maven-dependency-check) and I trust that vulnerabilities discovered by others are enough.

I assume that if I were a sensitive institution, I would pay people to inspect those dependencies and discover vulnerabilities.

The medium term would be to publish a bug bounty, so researchers are incentivized to find vulnerabilities.