`--deny-import` is matched against the specifier's hostname before any
DNS lookup, so a module whose hostname resolves to a denied address loads
anyway. Public wildcard DNS services make this trivial to reach —
`127.0.0.1.nip.io` resolves to `127.0.0.1`, defeating `--deny-import=127.0.0.1` — but no
DNS control is needed at all, since any record pointing at a denied address
works.
The module fetcher's HTTP client now uses the permissioned connector
`fetch()` already goes through, so every resolved address is checked against the
import deny list before a socket is opened. This is the post-resolution check
`check_net_resolved` performs for `--deny-net`, applied to the import
deny list instead. The client is built from a deny-only permissions container
rather than the factory's root one, because building that resolves the main module,
which needs the file fetcher. Modules already in the HTTP cache are still
loaded without this check, since no connection is made for them.
## Summary
- keep redirect request headers as chain state instead of rebuilding
them from the original request on every hop
- ensure an `Authorization` header removed by redirect handling remains
absent on subsequent hops
- add a deterministic multi-hop test using ephemeral local listeners
## Behavior
Same-origin redirects and same-host HTTP-to-HTTPS upgrades continue
forwarding `Authorization`. Host or port changes and HTTPS-to-HTTP
transitions remove it for the remainder of the redirect chain.
## Tests
- `./tools/format.js`
- `cargo test -p deno test_http_client_redirect_does_not_restore_auth
--lib`
- `cargo test -p deno test_should_strip_auth_on_redirect --lib`
- `cargo test -p deno test_http_client_download_redirect --lib`
- `cargo check -p deno --lib`
- `cargo build -p deno --bin deno`
When fetch transparently decompressed a gzip or brotli response body,
the
client also removed the content-encoding, content-length, and
transfer-encoding headers from the response. Per the fetch spec,
handling
content codings only decodes the body and leaves the header list as
received, which is what browsers and Node.js (undici) do. Code that
inspects response headers (for example to detect that a server
compressed
the response) behaved differently in Deno than everywhere else.
The headers are now preserved as received from the server. Since the
kept
content-length describes the encoded body rather than the decoded
stream,
the CLI download progress bar no longer uses it as a total when a
content-encoding header is present. Existing tests asserting the
stripping
behavior were updated to assert the preserved headers instead.
Fixes#20548
This PR implements the missing support for `certfile` and `keyfile`
options. Now when both are set, it creates an HTTP client with the
client certificate attached.
Fixes#29869.
## Summary
When an npm registry is configured with `http://` in `.npmrc` and the
server
redirects to `https://`, the authorization header was being stripped.
This
happened because the previous origin-based check (`url.origin() !=
new_url.origin()`)
treated scheme changes as cross-origin, since `url::Origin` includes the
scheme.
This caused 401 errors for private registries like GitHub Packages when
configured with `http://` URLs.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Bypass the tower-http `Decompression` middleware for npm registry
requests and instead decompress gzip on a blocking thread. This prevents
large packument decompression from blocking other HTTP/2 streams on the
event loop.
This right now doesn't have a huge impact but after a later PR to get
more parallelism of downloads it makes a difference
Solution for: https://github.com/denoland/deno/issues/31080
Modern reqwest (0.12+) can safely share clients across tokio runtimes.
Because it uses the current runtime's executor instead of spawning
background threads.
Using a single shared client prevents resource leaks when worker threads
terminate.
## Context
It seems that we started to use one client per thread because hyper and
reqwest was not safe to share across tokio runtimes
(https://github.com/denoland/deno/pull/23699). But seems it's fixed at
latest reqwest versions
(https://github.com/seanmonstar/reqwest/issues/1148#issuecomment-3120048532).
This PR is using one client again, as we now already use the version of
reqwest with this fix.
This slightly degrades the performance of CJS export analysis on
subsequent runs because I changed it to no longer cache in the DENO_DIR
with this PR (denort now properly has no idea about the DENO_DIR). We'll
have to change it to embed this data in the binary and that will also
allow us to get rid of swc in denort (will do that in a follow-up PR).
This brings some of the HTTP/2 retry behavior from reqwest to
`ext/fetch`. It will retry very specific HTTP/2 errors once, if the body
is able to be used again.
Closes#27332
Fixes#26085.
Adds a basic retry utility with some defaults, starts off with a 100ms
wait, then 250ms, then 500ms
I've applied the retry in the http client, reusing an existing function,
so this also applies to retrying downloads of deno binaries in `upgrade`
and `compile`. I can make a separate function that doesn't retry so this
doesn't affect `upgrade` and `compile`, but it seemed desirable to have
retries there too, so I left it in.
This commit rewrites the internal `version` module that exported
various information about the current executable. Instead of exporting
several consts, we are now exporting a single const structure that
contains all the necessary information.
This is the first step towards cleaning up how we use this information
and should allow us to use SUI to be able to patch this information
in already produced binary making it easier to cut new releases.
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This commit re-implements `ext/fetch` and all dependent crates
using `hyper` and `hyper-util`, instead of `reqwest`.
The reasoning is that we want to have greater control and access
to low level `hyper` APIs when implementing `fetch` API as well
as `node:http` module.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit updates Deno to use `reqwest` at 0.12.4
and `rustls` at 0.22. Other related crates were updated
as well to match versions accepted by `reqwest` and `rustls`.
Note: we are not using the latest available `rustls` yet,
but this upgrade was non-trivial already, so a bump to
0.23 for `rustls` will be done in a separate commit.
Closes#23370
---------
Signed-off-by: Ryan Dahl <ry@tinyclouds.org>
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This commit adds initial support for ".npmrc" files.
Currently we only discover ".npmrc" files next to "package.json" files
and discovering these files in user home dir is left for a follow up.
This pass supports "_authToken" and "_auth" configuration
for providing authentication.
LSP support has been left for a follow up PR.
Towards https://github.com/denoland/deno/issues/16105
These methods are confusing because the arguments are backwards. I feel
like they should have never been added to `Option<T>` and that clippy
should suggest rewriting to
`map(...).unwrap_or(...)`/`map(...).unwrap_or_else(|| ...)`
https://github.com/rust-lang/rfcs/issues/1025
This change makes absolute urls, that contain no path like `deno install
https://my-cli.io` to follow redirects and extract the name from it.
It allows modifies `test_util` server listener on port `4550`
(`REDIRECT_ABSOLUTE_PORT`) to allow for specifying `redirect_to` query
param, that fill use that value for it's next redirect.
Fixes https://github.com/denoland/deno/issues/17409
Stop allowing clippy::derive-partial-eq-without-eq and fix warnings
about deriving PartialEq without also deriving Eq.
In one case I removed the PartialEq because it a) wasn't necessary,
and b) sketchy because it was comparing floating point numbers.
IMO, that's a good argument for enforcing the lint rule, because it
would most likely have been caught during review if it had been enabled.
Bartek Iwańczuk