Compare commits

...

No commits in common. "c8" and "c9s" have entirely different histories.
c8 ... c9s

24 changed files with 225 additions and 1328 deletions

1
.fmf/version Normal file
View File

@ -0,0 +1 @@
1

22
.gitignore vendored
View File

@ -1 +1,21 @@
SOURCES/wget-1.19.5.tar.gz
wget-1.12.tar.bz2
/wget-1.13.4.tar.bz2
/wget-1.14.tar.xz
/wget-1.15.tar.xz
/wget-1.16.tar.xz
/wget-1.16.1.tar.xz
/wget-1.16.2.tar.xz
/wget-1.16.3.tar.xz
/wget-1.17.tar.xz
/wget-1.17.1.tar.xz
/wget-1.18.tar.xz
/wget-1.19.tar.xz
/wget-1.19.1.tar.xz
/wget-1.19.2.tar.gz
/wget-1.19.4.tar.lz
/wget-1.19.4.tar.gz
/wget-1.19.5.tar.gz
/wget-1.20.tar.gz
/wget-1.20.1.tar.gz
/wget-1.20.3.tar.gz
/wget-1.21.1.tar.gz

View File

@ -1 +1 @@
43b3d09e786df9e8d7aa454095d4ea2d420ae41c SOURCES/wget-1.19.5.tar.gz
7a14aeb3871fa4ec5e2580d2718913d1665cb49b wget-1.21.1.tar.gz

View File

@ -1,110 +0,0 @@
From 2bbdfd76dab187ab29e22bed18d737f94343e629 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Tue, 4 Sep 2018 11:22:14 +0200
Subject: [PATCH] Add TLS 1.3 support for GnuTLS
* doc/wget.texi: Add "TLSv1_3" to --secure-protocol
* src/gnutls.c (set_prio_default): Use GNUTLS_TLS1_3 where needed
Wget currently allows specifying "TLSv1_3" as the parameter for
--secure-protocol option. However it is only implemented for OpenSSL
and in case wget is compiled with GnuTLS, it causes wget to abort with:
GnuTLS: unimplemented 'secure-protocol' option value 6
GnuTLS contains TLS 1.3 implementation since version 3.6.3 [1]. However
currently it must be enabled explicitly in the application of it to be
used. This will change after the draft is finalized. [2] However for
the time being, I enabled it explicitly in case "TLSv1_3" is used with
--secure-protocol.
I also fixed man page to contain "TLSv1_3" in all listings of available
parameters for --secure-protocol
[1] https://lists.gnupg.org/pipermail/gnutls-devel/2018-July/008584.html
[2] https://nikmav.blogspot.com/2018/05/gnutls-and-tls-13.html
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
doc/wget.texi | 6 +++---
src/gnutls.c | 28 ++++++++++++++++++++++++++++
2 files changed, 31 insertions(+), 3 deletions(-)
diff --git a/doc/wget.texi b/doc/wget.texi
index 38b4a245..7ae19d8e 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -1780,9 +1780,9 @@ If Wget is compiled without SSL support, none of these options are available.
@cindex SSL protocol, choose
@item --secure-protocol=@var{protocol}
Choose the secure protocol to be used. Legal values are @samp{auto},
-@samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1}, @samp{TLSv1_2}
-and @samp{PFS}. If @samp{auto} is used, the SSL library is given the
-liberty of choosing the appropriate protocol automatically, which is
+@samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1}, @samp{TLSv1_2},
+@samp{TLSv1_3} and @samp{PFS}. If @samp{auto} is used, the SSL library is
+given the liberty of choosing the appropriate protocol automatically, which is
achieved by sending a TLSv1 greeting. This is the default.
Specifying @samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1},
diff --git a/src/gnutls.c b/src/gnutls.c
index 07844c52..206d0b09 100644
--- a/src/gnutls.c
+++ b/src/gnutls.c
@@ -565,6 +565,15 @@ set_prio_default (gnutls_session_t session)
err = gnutls_priority_set_direct (session, "NORMAL:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1", NULL);
break;
+ case secure_protocol_tlsv1_3:
+#if GNUTLS_VERSION_NUMBER >= 0x030603
+ err = gnutls_priority_set_direct (session, "NORMAL:-VERS-SSL3.0:+VERS-TLS1.3:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-TLS1.2", NULL);
+ break;
+#else
+ logprintf (LOG_NOTQUIET, _("Your GnuTLS version is too old to support TLS 1.3\n"));
+ return -1;
+#endif
+
case secure_protocol_pfs:
err = gnutls_priority_set_direct (session, "PFS:-VERS-SSL3.0", NULL);
if (err != GNUTLS_E_SUCCESS)
@@ -596,19 +605,38 @@ set_prio_default (gnutls_session_t session)
allowed_protocols[0] = GNUTLS_TLS1_0;
allowed_protocols[1] = GNUTLS_TLS1_1;
allowed_protocols[2] = GNUTLS_TLS1_2;
+#if GNUTLS_VERSION_NUMBER >= 0x030603
+ allowed_protocols[3] = GNUTLS_TLS1_3;
+#endif
err = gnutls_protocol_set_priority (session, allowed_protocols);
break;
case secure_protocol_tlsv1_1:
allowed_protocols[0] = GNUTLS_TLS1_1;
allowed_protocols[1] = GNUTLS_TLS1_2;
+#if GNUTLS_VERSION_NUMBER >= 0x030603
+ allowed_protocols[2] = GNUTLS_TLS1_3;
+#endif
err = gnutls_protocol_set_priority (session, allowed_protocols);
break;
case secure_protocol_tlsv1_2:
allowed_protocols[0] = GNUTLS_TLS1_2;
+#if GNUTLS_VERSION_NUMBER >= 0x030603
+ allowed_protocols[1] = GNUTLS_TLS1_3;
+#endif
+ err = gnutls_protocol_set_priority (session, allowed_protocols);
+ break;
+
+ case secure_protocol_tlsv1_3:
+#if GNUTLS_VERSION_NUMBER >= 0x030603
+ allowed_protocols[0] = GNUTLS_TLS1_3;
err = gnutls_protocol_set_priority (session, allowed_protocols);
break;
+#else
+ logprintf (LOG_NOTQUIET, _("Your GnuTLS version is too old to support TLS 1.3\n"));
+ return -1;
+#endif
default:
logprintf (LOG_NOTQUIET, _("GnuTLS: unimplemented 'secure-protocol' option value %d\n"), opt.secure_protocol);
--
2.17.1

View File

@ -1,18 +0,0 @@
diff --git a/src/iri.c b/src/iri.c
index 7dcf3ac..1c8695c 100644
--- a/src/iri.c
+++ b/src/iri.c
@@ -189,9 +189,10 @@ do_conversion (const char *tocode, const char *fromcode, char const *in_org, siz
{
tooshort++;
done = len;
- len = outlen = done + inlen * 2;
- s = xrealloc (s, outlen + 1);
- *out = s + done;
+ len = done + inlen * 2;
+ s = xrealloc (s, len + 1);
+ *out = s + done - outlen;
+ outlen += inlen * 2;
}
else /* Weird, we got an unspecified error */
{

View File

@ -1,41 +0,0 @@
From 8990d706da3e32b12debd9b8dea7b42134631770 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Fri, 10 Aug 2018 14:32:13 +0200
Subject: [PATCH] Don't limit the test suite HTTPS server to TLSv1
In Fedora, we are implementing crypto policies, in order to enhance the
security of user systems. This is done on the system level by global
configuration. It may happen that due to the active policy, only
TLSv1.2 or higher will be available in crypto libraries. While wget as
a client will by default determine the minimal TLS version supported by
both client and server, the HTTPS server implementation in testenv/
hardcodes use of TLSv1. As a result all HTTPS related tests fail in
case a more hardened crypto policy is set on the Fedora system.
This change removes the explicit TLS version setting and leaves the
determination of the minimal supported TLS version on the server and
client.
More information about Fedora change can be found here:
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
testenv/server/http/http_server.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/testenv/server/http/http_server.py b/testenv/server/http/http_server.py
index 434666dd..6d8fc9e8 100644
--- a/testenv/server/http/http_server.py
+++ b/testenv/server/http/http_server.py
@@ -49,7 +49,6 @@ class HTTPSServer(StoppableHTTPServer):
'server-key.pem'))
self.socket = ssl.wrap_socket(
sock=socket.socket(self.address_family, self.socket_type),
- ssl_version=ssl.PROTOCOL_TLSv1,
certfile=CERTFILE,
keyfile=KEYFILE,
server_side=True
--
2.17.1

View File

@ -1,116 +0,0 @@
From 83c408842b80b4ed26a3fe8a61177846dda18c60 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Tim=20R=C3=BChsen?= <tim.ruehsen@gmx.de>
Date: Wed, 26 Dec 2018 14:38:18 +0100
Subject: [PATCH] Don't save user/pw with --xattr
---
src/ftp.c | 2 +-
src/http.c | 4 ++--
src/xattr.c | 24 ++++++++++++++++++++----
src/xattr.h | 3 ++-
4 files changed, 25 insertions(+), 8 deletions(-)
diff --git a/src/ftp.c b/src/ftp.c
index daaae93..c02ed02 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -1580,7 +1580,7 @@ Error in server response, closing control connection.\n"));
#ifdef ENABLE_XATTR
if (opt.enable_xattr)
- set_file_metadata (u->url, NULL, fp);
+ set_file_metadata (u, NULL, fp);
#endif
fd_close (local_sock);
diff --git a/src/http.c b/src/http.c
index 499a43b..18e58e9 100644
--- a/src/http.c
+++ b/src/http.c
@@ -4120,9 +4120,9 @@ gethttp (const struct url *u, struct url *original_url, struct http_stat *hs,
if (opt.enable_xattr)
{
if (original_url != u)
- set_file_metadata (u->url, original_url->url, fp);
+ set_file_metadata (u, original_url, fp);
else
- set_file_metadata (u->url, NULL, fp);
+ set_file_metadata (u, NULL, fp);
}
#endif
diff --git a/src/xattr.c b/src/xattr.c
index 6652422..0f20fad 100644
--- a/src/xattr.c
+++ b/src/xattr.c
@@ -21,6 +21,7 @@
#include <string.h>
#include "log.h"
+#include "utils.h"
#include "xattr.h"
#ifdef USE_XATTR
@@ -57,7 +58,7 @@ write_xattr_metadata (const char *name, const char *value, FILE *fp)
#endif /* USE_XATTR */
int
-set_file_metadata (const char *origin_url, const char *referrer_url, FILE *fp)
+set_file_metadata (const struct url *origin_url, const struct url *referrer_url, FILE *fp)
{
/* Save metadata about where the file came from (requested, final URLs) to
* user POSIX Extended Attributes of retrieved file.
@@ -67,13 +68,28 @@ set_file_metadata (const char *origin_url, const char *referrer_url, FILE *fp)
* [http://0pointer.de/lennart/projects/mod_mime_xattr/].
*/
int retval = -1;
+ char *value;
if (!origin_url || !fp)
return retval;
- retval = write_xattr_metadata ("user.xdg.origin.url", escnonprint_uri (origin_url), fp);
- if ((!retval) && referrer_url)
- retval = write_xattr_metadata ("user.xdg.referrer.url", escnonprint_uri (referrer_url), fp);
+ value = url_string (origin_url, URL_AUTH_HIDE);
+ retval = write_xattr_metadata ("user.xdg.origin.url", escnonprint_uri (value), fp);
+ xfree (value);
+
+ if (!retval && referrer_url)
+ {
+ struct url u;
+
+ memset(&u, 0, sizeof(u));
+ u.scheme = referrer_url->scheme;
+ u.host = referrer_url->host;
+ u.port = referrer_url->port;
+
+ value = url_string (&u, 0);
+ retval = write_xattr_metadata ("user.xdg.referrer.url", escnonprint_uri (value), fp);
+ xfree (value);
+ }
return retval;
}
diff --git a/src/xattr.h b/src/xattr.h
index 10f3ed1..40c7a8d 100644
--- a/src/xattr.h
+++ b/src/xattr.h
@@ -16,12 +16,13 @@
along with this program; if not, see <http://www.gnu.org/licenses/>. */
#include <stdio.h>
+#include <url.h>
#ifndef _XATTR_H
#define _XATTR_H
/* Store metadata name/value attributes against fp. */
-int set_file_metadata (const char *origin_url, const char *referrer_url, FILE *fp);
+int set_file_metadata (const struct url *origin_url, const struct url *referrer_url, FILE *fp);
#if defined(__linux)
/* libc on Linux has fsetxattr (5 arguments). */
--
2.17.2

View File

@ -1,62 +0,0 @@
From 0e991351c8bd3996bfc396402a67445abcf1319e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Tim=20R=C3=BChsen?= <tim.ruehsen@gmx.de>
Date: Wed, 26 Dec 2018 13:51:48 +0100
Subject: [PATCH] Don't use extended attributes (--xattr) by default
---
doc/wget.texi | 8 ++++++++
src/init.c | 4 ----
src/main.c | 2 +-
3 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/doc/wget.texi b/doc/wget.texi
index 66edab8..d672bbf 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -540,6 +540,14 @@ right NUMBER.
Set preferred location for Metalink resources. This has effect if multiple
resources with same priority are available.
+@cindex xattr
+@item --xattr
+Enable use of file system's extended attributes to save the
+original URL and the Referer HTTP header value if used.
+
+Be aware that the URL might contain private information like
+access tokens or credentials.
+
@cindex force html
@item -F
diff --git a/src/init.c b/src/init.c
index eb81ab4..800970c 100644
--- a/src/init.c
+++ b/src/init.c
@@ -509,11 +509,7 @@ defaults (void)
opt.hsts = true;
#endif
-#ifdef ENABLE_XATTR
- opt.enable_xattr = true;
-#else
opt.enable_xattr = false;
-#endif
}
/* Return the user's home directory (strdup-ed), or NULL if none is
diff --git a/src/main.c b/src/main.c
index 81db931..6ac1621 100644
--- a/src/main.c
+++ b/src/main.c
@@ -754,7 +754,7 @@ Download:\n"),
#endif
#ifdef ENABLE_XATTR
N_("\
- --no-xattr turn off storage of metadata in extended file attributes\n"),
+ --xattr turn on storage of metadata in extended file attributes\n"),
#endif
"\n",
--
2.17.2

View File

@ -1,141 +0,0 @@
From c11cc83d9ee9230f090c2400a57bbd562905d782 Mon Sep 17 00:00:00 2001
From: Nikos Mavrogiannopoulos <nmav@redhat.com>
Date: Mon, 8 Oct 2018 10:42:22 +0200
Subject: [PATCH] Enable post-handshake auth under gnutls on TLS1.3
---
src/gnutls.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 96 insertions(+)
diff --git a/src/gnutls.c b/src/gnutls.c
index 206d0b09..a2c9d1c1 100644
--- a/src/gnutls.c
+++ b/src/gnutls.c
@@ -60,6 +60,11 @@ as that of the covered work. */
static int
_do_handshake (gnutls_session_t session, int fd, double timeout);
+#if GNUTLS_VERSION_NUMBER >= 0x030604
+static int
+_do_reauth (gnutls_session_t session, int fd, double timeout);
+#endif
+
static int
key_type_to_gnutls_type (enum keyfile_type type)
{
@@ -287,6 +292,14 @@ wgnutls_read_timeout (int fd, char *buf, int bufsize, void *arg, double timeout)
if ((ret = _do_handshake (ctx->session, fd, timeout)) == 0)
ret = GNUTLS_E_AGAIN; /* restart reading */
}
+#if GNUTLS_VERSION_NUMBER >= 0x030604
+ if (!timed_out && ret == GNUTLS_E_REAUTH_REQUEST)
+ {
+ DEBUGP (("GnuTLS: *** re-authentication while reading\n"));
+ if ((ret = _do_reauth (ctx->session, fd, timeout)) == 0)
+ ret = GNUTLS_E_AGAIN; /* restart reading */
+ }
+#endif
}
}
while (ret == GNUTLS_E_INTERRUPTED || (ret == GNUTLS_E_AGAIN && !timed_out));
@@ -519,6 +532,84 @@ _do_handshake (gnutls_session_t session, int fd, double timeout)
return err;
}
+#if GNUTLS_VERSION_NUMBER >= 0x030604
+static int
+_do_reauth (gnutls_session_t session, int fd, double timeout)
+{
+#ifdef F_GETFL
+ int flags = 0;
+#endif
+ int err;
+
+ if (timeout)
+ {
+#ifdef F_GETFL
+ flags = fcntl (fd, F_GETFL, 0);
+ if (flags < 0)
+ return flags;
+ if (fcntl (fd, F_SETFL, flags | O_NONBLOCK))
+ return -1;
+#else
+ /* XXX: Assume it was blocking before. */
+ const int one = 1;
+ if (ioctl (fd, FIONBIO, &one) < 0)
+ return -1;
+#endif
+ }
+
+ /* We don't stop the handshake process for non-fatal errors */
+ do
+ {
+ err = gnutls_reauth (session, 0);
+
+ if (timeout && err == GNUTLS_E_AGAIN)
+ {
+ if (gnutls_record_get_direction (session))
+ {
+ /* wait for writeability */
+ err = select_fd (fd, timeout, WAIT_FOR_WRITE);
+ }
+ else
+ {
+ /* wait for readability */
+ err = select_fd (fd, timeout, WAIT_FOR_READ);
+ }
+
+ if (err <= 0)
+ {
+ if (err == 0)
+ {
+ errno = ETIMEDOUT;
+ err = -1;
+ }
+ break;
+ }
+
+ err = GNUTLS_E_AGAIN;
+ }
+ else if (err < 0)
+ {
+ logprintf (LOG_NOTQUIET, "GnuTLS: %s\n", gnutls_strerror (err));
+ }
+ }
+ while (err && gnutls_error_is_fatal (err) == 0);
+
+ if (timeout)
+ {
+#ifdef F_GETFL
+ if (fcntl (fd, F_SETFL, flags) < 0)
+ return -1;
+#else
+ const int zero = 0;
+ if (ioctl (fd, FIONBIO, &zero) < 0)
+ return -1;
+#endif
+ }
+
+ return err;
+}
+#endif
+
static const char *
_sni_hostname(const char *hostname)
{
@@ -655,7 +746,12 @@ ssl_connect_wget (int fd, const char *hostname, int *continue_session)
gnutls_session_t session;
int err;
+#if GNUTLS_VERSION_NUMBER >= 0x030604
+ // enable support of TLS1.3 post-handshake authentication
+ gnutls_init (&session, GNUTLS_CLIENT | GNUTLS_POST_HANDSHAKE_AUTH);
+#else
gnutls_init (&session, GNUTLS_CLIENT);
+#endif
/* We set the server name but only if it's not an IP address. */
if (! is_valid_ip_address (hostname))
--
2.17.2

View File

@ -1,52 +0,0 @@
From 706e71564cadc7192ac21efbf51b661c967f35b5 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Tue, 24 Mar 2020 13:18:40 +0100
Subject: [PATCH] Don't print message about loading crl or ca-cert files with --no-verbose
* src/gnutls.c (ssl_init): Use LOG_VERBOSE verbosity for informative
message related to loading CRL or CA certificate file.
Before change [1], wget didn't produce any output related to loading CA
certificates when --no-verbose option has been used. When --no-verbose
option is used, only error messages and basic information should get
printed. Information about loading CRL or CA certificate is probably not
a basic information. Any error when loading the CRL or CA certificate
will be still printed with --no-verbose.
Some users rely on wget not printing such information and they consider
it a regression.
Reported as https://bugzilla.redhat.com/show_bug.cgi?id=1807267
[1] http://git.savannah.gnu.org/cgit/wget.git/commit/?id=e4a8fe84e2b813b65d91aec29298eecabe4850a5
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/gnutls.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/gnutls.c b/src/gnutls.c
index e95ecea..7ab1f08 100644
--- a/src/gnutls.c
+++ b/src/gnutls.c
@@ -172,7 +172,7 @@ ssl_init (void)
else
{
ncerts += rc;
- logprintf (LOG_NOTQUIET, _ ("Loaded CA certificate '%s'\n"), opt.ca_cert);
+ logprintf (LOG_VERBOSE, _ ("Loaded CA certificate '%s'\n"), opt.ca_cert);
}
}
@@ -186,7 +186,7 @@ ssl_init (void)
return false;
}
- logprintf (LOG_NOTQUIET, _ ("Loaded CRL file '%s'\n"), opt.crl_file);
+ logprintf (LOG_VERBOSE, _ ("Loaded CRL file '%s'\n"), opt.crl_file);
}
DEBUGP (("Certificates loaded: %d\n", ncerts));
--
libgit2 0.28.2

View File

@ -1,376 +0,0 @@
From b24351183ec574f81c729cbb3286aceaee3f03c8 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Mon, 30 Jul 2018 12:20:27 +0200
Subject: [PATCH 1/6] * src/ftp.c (getftp): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/ftp.c:1493: alloc_fn: Storage is returned from allocation function "fopen".
wget-1.19.5/src/ftp.c:1493: var_assign: Assigning: "fp" = storage returned from "fopen(con->target, "wb")".
wget-1.19.5/src/ftp.c:1811: leaked_storage: Variable "fp" going out of scope leaks the storage it points to.
\# 1809| if (fp && !output_stream)
\# 1810| fclose (fp);
\# 1811|-> return err;
\# 1812| }
\# 1813|
It can happen, that "if (!output_stream || con->cmd & DO_LIST)" on line #1398 can be true, even though "output_stream != NULL". In this case a new file is opened to "fp". Later it may happen in the FTPS branch, that some error will occure and code will jump to label "exit_error". In "exit_error", the "fp" is closed only if "output_stream == NULL". However this may not be true as described earlier and "fp" leaks.
On line #1588, there is the following conditional free of "fp":
/* Close the local file. */
if (!output_stream || con->cmd & DO_LIST)
fclose (fp);
Therefore the conditional at the end of the function after "exit_error" label should be modified to:
if (fp && (!output_stream || con->cmd & DO_LIST))
fclose (fp);
This will ensure that "fp" does not leak in any case it sould be opened.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/ftp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/ftp.c b/src/ftp.c
index 69148936..daaae939 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -1806,7 +1806,7 @@ Error in server response, closing control connection.\n"));
exit_error:
/* If fp is a regular file, close and try to remove it */
- if (fp && !output_stream)
+ if (fp && (!output_stream || con->cmd & DO_LIST))
fclose (fp);
return err;
}
--
2.17.1
From b8be904ac7c25387672b0aa39f7cba699bffc48e Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Mon, 30 Jul 2018 15:38:45 +0200
Subject: [PATCH 2/6] * src/http.c (check_auth): Fix RESOURCE LEAK found by
Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2434: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/http.c:2434: var_assign: Assigning: "auth_stat" = storage returned from "xmalloc(4UL)".
wget-1.19.5/src/http.c:2446: noescape: Resource "auth_stat" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:70: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "auth_err".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "auth_stat" going out of scope leaks the storage it points to.
\# 2474| /* Creating the Authorization header went wrong */
\# 2475| }
\# 2476|-> }
\# 2477| else
\# 2478| {
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2431: alloc_fn: Storage is returned from allocation function "url_full_path".
wget-1.19.5/src/url.c:1105:19: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:1105:19: var_assign: Assigning: "full_path" = "xmalloc(length + 1)".
wget-1.19.5/src/url.c:1107:3: noescape: Resource "full_path" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:1110:3: return_alloc: Returning allocated memory "full_path".
wget-1.19.5/src/http.c:2431: var_assign: Assigning: "pth" = storage returned from "url_full_path(u)".
wget-1.19.5/src/http.c:2446: noescape: Resource "pth" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:40: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "path".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "pth" going out of scope leaks the storage it points to.
\# 2474| /* Creating the Authorization header went wrong */
\# 2475| }
\# 2476|-> }
\# 2477| else
\# 2478| {
Both "pth" and "auth_stat" are allocated in "check_auth()" function. These are used for creating the HTTP Authorization Request header via "create_authorization_line()" function. In case the creation went OK (auth_err == RETROK), then the memory previously allocated to "pth" and "auth_stat" is freed. However if the creation failed, then the memory is never freed and it leaks.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/http.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/http.c b/src/http.c
index 093be167..4e0d467a 100644
--- a/src/http.c
+++ b/src/http.c
@@ -2451,6 +2451,8 @@ check_auth (const struct url *u, char *user, char *passwd, struct response *resp
auth_stat);
auth_err = *auth_stat;
+ xfree (auth_stat);
+ xfree (pth);
if (auth_err == RETROK)
{
request_set_header (req, "Authorization", value, rel_value);
@@ -2464,8 +2466,6 @@ check_auth (const struct url *u, char *user, char *passwd, struct response *resp
register_basic_auth_host (u->host);
}
- xfree (pth);
- xfree (auth_stat);
*retry = true;
goto cleanup;
}
--
2.17.1
From dfef92bac3997b9848e86d84a843d5d7dde4fd99 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Tue, 31 Jul 2018 16:58:12 +0200
Subject: [PATCH 3/6] * src/http.c (http_loop): Fix RESOURCE LEAK found by
Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:4486: alloc_fn: Storage is returned from allocation function "url_string".
wget-1.19.5/src/url.c:2248:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "result" = "xmalloc(size)".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "p" = "result".
wget-1.19.5/src/url.c:2250:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2253:7: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2257:11: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2264:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2270:7: identity_transfer: Passing "p" as argument 1 to function "number_to_string", which returns an offset off that argument.
wget-1.19.5/src/utils.c:1776:11: var_assign_parm: Assigning: "p" = "buffer".
wget-1.19.5/src/utils.c:1847:3: return_var: Returning "p", which is a copy of a parameter.
wget-1.19.5/src/url.c:2270:7: noescape: Resource "p" is not freed or pointed-to in function "number_to_string".
wget-1.19.5/src/utils.c:1774:25: noescape: "number_to_string(char *, wgint)" does not free or save its parameter "buffer".
wget-1.19.5/src/url.c:2270:7: var_assign: Assigning: "p" = "number_to_string(p, url->port)".
wget-1.19.5/src/url.c:2273:3: noescape: Resource "p" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:2287:3: return_alloc: Returning allocated memory "result".
wget-1.19.5/src/http.c:4486: var_assign: Assigning: "hurl" = storage returned from "url_string(u, URL_AUTH_HIDE_PASSWD)".
wget-1.19.5/src/http.c:4487: noescape: Resource "hurl" is not freed or pointed-to in "logprintf".
wget-1.19.5/src/http.c:4513: leaked_storage: Variable "hurl" going out of scope leaks the storage it points to.
\# 4511| {
\# 4512| printwhat (count, opt.ntry);
\# 4513|-> continue;
\# 4514| }
\# 4515| else
There are two conditional branches, which call continue, without freeing memory potentially allocated and pointed to by"hurl" pointer. In fase "!opt.verbose" is True and some of the appropriate conditions in the following if/else if construction, in which "continue" is called, are also true, then the memory allocated to "hurl" will leak.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/http.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/http.c b/src/http.c
index 4e0d467a..46fde6f2 100644
--- a/src/http.c
+++ b/src/http.c
@@ -4492,6 +4492,7 @@ http_loop (const struct url *u, struct url *original_url, char **newloc,
&& (hstat.statcode == 500 || hstat.statcode == 501))
{
got_head = true;
+ xfree (hurl);
continue;
}
/* Maybe we should always keep track of broken links, not just in
@@ -4510,6 +4511,7 @@ Remote file does not exist -- broken link!!!\n"));
else if (check_retry_on_http_error (hstat.statcode))
{
printwhat (count, opt.ntry);
+ xfree (hurl);
continue;
}
else
--
2.17.1
From c045cdded4e3850724d8bb3a655852948e62c0df Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Thu, 2 Aug 2018 13:49:52 +0200
Subject: [PATCH 4/6] * src/utils.c (open_stat): Fix RESOURCE LEAK found by
Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/utils.c:914: open_fn: Returning handle opened by "open". [Note: The source code implementation of the function has been overridden by a user model.]
wget-1.19.5/src/utils.c:914: var_assign: Assigning: "fd" = handle returned from "open(fname, flags, mode)".
wget-1.19.5/src/utils.c:921: noescape: Resource "fd" is not freed or pointed-to in "fstat". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/utils.c:924: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
\# 922| {
\# 923| logprintf (LOG_NOTQUIET, _("Failed to stat file %s, error: %s\n"), fname, strerror(errno));
\# 924|-> return -1;
\# 925| }
\# 926| #if !(defined(WINDOWS) || defined(__VMS))
This seems to be a real issue, since the opened file descriptor in "fd"
would leak. There is also additional check below the "fstat" call, which
closes the opened "fd".
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/utils.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/utils.c b/src/utils.c
index 0cb905ad..c6258083 100644
--- a/src/utils.c
+++ b/src/utils.c
@@ -921,6 +921,7 @@ open_stat(const char *fname, int flags, mode_t mode, file_stats_t *fstats)
if (fstat (fd, &fdstats) == -1)
{
logprintf (LOG_NOTQUIET, _("Failed to stat file %s, error: %s\n"), fname, strerror(errno));
+ close (fd);
return -1;
}
#if !(defined(WINDOWS) || defined(__VMS))
--
2.17.1
From 8b451f9f21cc1b00d1a08116b542fb7bd7589405 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Fri, 3 Aug 2018 16:19:20 +0200
Subject: [PATCH 5/6] * src/warc.c (warc_write_start_record): Fix potential
RESOURCE LEAK
In warc_write_start_record() function, the reutrn value of dup() is
directly used in gzdopen() call and not stored anywhere. However the
zlib documentation says that "The duplicated descriptor should be saved
to avoid a leak, since gzdopen does not close fd if it fails." [1].
This change stores the FD in a variable and closes it in case gzopen()
fails.
[1] https://www.zlib.net/manual.html
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/warc.c:217: open_fn: Returning handle opened by "dup".
wget-1.19.5/src/warc.c:217: leaked_handle: Failing to save or close handle opened by "dup(fileno(warc_current_file))" leaks it.
\# 215|
\# 216| /* Start a new GZIP stream. */
\# 217|-> warc_current_gzfile = gzdopen (dup (fileno (warc_current_file)), "wb9");
\# 218| warc_current_gzfile_uncompressed_size = 0;
\# 219|
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/warc.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/warc.c b/src/warc.c
index 3482cf3b..5ebd04d7 100644
--- a/src/warc.c
+++ b/src/warc.c
@@ -203,6 +203,7 @@ warc_write_start_record (void)
/* Start a GZIP stream, if required. */
if (opt.warc_compression_enabled)
{
+ int dup_fd;
/* Record the starting offset of the new record. */
warc_current_gzfile_offset = ftello (warc_current_file);
@@ -214,13 +215,23 @@ warc_write_start_record (void)
fflush (warc_current_file);
/* Start a new GZIP stream. */
- warc_current_gzfile = gzdopen (dup (fileno (warc_current_file)), "wb9");
+ dup_fd = dup (fileno (warc_current_file));
+ if (dup_fd < 0)
+ {
+ logprintf (LOG_NOTQUIET,
+_("Error duplicating WARC file file descriptor.\n"));
+ warc_write_ok = false;
+ return false;
+ }
+
+ warc_current_gzfile = gzdopen (dup_fd, "wb9");
warc_current_gzfile_uncompressed_size = 0;
if (warc_current_gzfile == NULL)
{
logprintf (LOG_NOTQUIET,
_("Error opening GZIP stream to WARC file.\n"));
+ close (dup_fd);
warc_write_ok = false;
return false;
}
--
2.17.1
From 2f451dbf4e83c751f6bbba7ed26d90bf275fcbf7 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Fri, 24 Aug 2018 16:57:37 +0200
Subject: [PATCH 6/6] * src/warc.c (warc_write_cdx_record): Fix RESOURCE LEAK
found by Coverity
Error: RESOURCE_LEAK (CWE-772): - REAL ERROR
wget-1.19.5/src/warc.c:1376: alloc_fn: Storage is returned from allocation function "url_escape".
wget-1.19.5/src/url.c:284:3: alloc_fn: Storage is returned from allocation function "url_escape_1".
wget-1.19.5/src/url.c:255:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:255:3: var_assign: Assigning: "newstr" = "xmalloc(newlen + 1)".
wget-1.19.5/src/url.c:258:3: var_assign: Assigning: "p2" = "newstr".
wget-1.19.5/src/url.c:275:3: return_alloc: Returning allocated memory "newstr".
wget-1.19.5/src/url.c:284:3: return_alloc_fn: Directly returning storage allocated by "url_escape_1".
wget-1.19.5/src/warc.c:1376: var_assign: Assigning: "redirect_location" = storage returned from "url_escape(redirect_location)".
wget-1.19.5/src/warc.c:1381: noescape: Resource "redirect_location" is not freed or pointed-to in "fprintf".
wget-1.19.5/src/warc.c:1387: leaked_storage: Returning without freeing "redirect_location" leaks the storage that it points to.
\# 1385| fflush (warc_current_cdx_file);
\# 1386|
\# 1387|-> return true;
\# 1388| }
\# 1389|
url_escape() really returns a newly allocated memory and it leaks when the warc_write_cdx_record() returns. The memory returned from url_escape() is usually stored in a temporary variable in other parts of the project and then freed. I took the same approach.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/warc.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/warc.c b/src/warc.c
index 5ebd04d7..2eb74966 100644
--- a/src/warc.c
+++ b/src/warc.c
@@ -1364,6 +1364,7 @@ warc_write_cdx_record (const char *url, const char *timestamp_str,
char timestamp_str_cdx[15];
char offset_string[MAX_INT_TO_STRING_LEN(off_t)];
const char *checksum;
+ char *tmp_location = NULL;
memcpy (timestamp_str_cdx , timestamp_str , 4); /* "YYYY" "-" */
memcpy (timestamp_str_cdx + 4, timestamp_str + 5, 2); /* "mm" "-" */
@@ -1382,18 +1383,19 @@ warc_write_cdx_record (const char *url, const char *timestamp_str,
if (mime_type == NULL || strlen(mime_type) == 0)
mime_type = "-";
if (redirect_location == NULL || strlen(redirect_location) == 0)
- redirect_location = "-";
+ tmp_location = strdup ("-");
else
- redirect_location = url_escape(redirect_location);
+ tmp_location = url_escape(redirect_location);
number_to_string (offset_string, offset);
/* Print the CDX line. */
fprintf (warc_current_cdx_file, "%s %s %s %s %d %s %s - %s %s %s\n", url,
timestamp_str_cdx, url, mime_type, response_code, checksum,
- redirect_location, offset_string, warc_current_filename,
+ tmp_location, offset_string, warc_current_filename,
response_uuid);
fflush (warc_current_cdx_file);
+ free (tmp_location);
return true;
}
--
2.17.1

View File

@ -1,26 +0,0 @@
From 7ddcebd61e170fb03d361f82bf8f5550ee62a1ae Mon Sep 17 00:00:00 2001
From: Tomas Korbar <tkorbar@redhat.com>
Date: Wed, 29 Aug 2018 12:33:43 +0200
Subject: [PATCH] Avoid creating empty wget-log when using -O and -q in
background
* src/log.c (check_redirect_output): Check for quiet mode
---
src/log.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/log.c b/src/log.c
index d879dffe..e8cca2f3 100644
--- a/src/log.c
+++ b/src/log.c
@@ -974,7 +974,7 @@ check_redirect_output (void)
{
pid_t foreground_pgrp = tcgetpgrp (STDIN_FILENO);
- if (foreground_pgrp != -1 && foreground_pgrp != getpgrp ())
+ if (foreground_pgrp != -1 && foreground_pgrp != getpgrp () && !opt.quiet)
{
/* Process backgrounded */
redirect_output (true,NULL);
--
2.17.1

View File

@ -1,28 +0,0 @@
commit fd85ac9cc623847e9d94d9f9241ab34e2c146cbf
Author: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Date: Thu Oct 25 17:39:52 2018 -0300
* src/host.c (sufmatch): Fix dot-prefixed domain matching
Current sufmatch does not match when domain is dot-prefixed.
The example of no_proxy in man (.mit.edu) does use a dot-prefixed
domain.
Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
diff --git a/src/host.c b/src/host.c
index b42cd6e8..2bf848f3 100644
--- a/src/host.c
+++ b/src/host.c
@@ -1033,8 +1033,9 @@ sufmatch (const char **list, const char *what)
/* Domain or subdomain match
* k == -1: exact match
* k >= 0 && what[k] == '.': subdomain match
+ * k >= 0 && list[i][0] == '.': dot-prefixed subdomain match
*/
- if (j == -1 && (k == -1 || what[k] == '.'))
+ if (j == -1 && (k == -1 || what[k] == '.' || list[i][0] == '.'))
return true;
}

View File

@ -1,285 +0,0 @@
From dea0f6272889adcff846144fff5714c076067b16 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Thu, 7 Nov 2019 12:46:15 +0100
Subject: [PATCH 1/3] testenv: HTTPTest.begin() should return exit value
* testenv/test/http_test.py: Ensure that HTTPTest.begin() always retuns a value
Previously the HTTPTest.begin() method always returned None. However this is not consistent with the begin() implementation of the parent class (BaseTest). This change ensures that HTTPTest.begin() returns a value.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
testenv/test/http_test.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/testenv/test/http_test.py b/testenv/test/http_test.py
index fef0c2ef..462ac6e7 100644
--- a/testenv/test/http_test.py
+++ b/testenv/test/http_test.py
@@ -42,7 +42,7 @@ class HTTPTest(BaseTest):
print_green("Test Passed.")
else:
self.tests_passed = False
- super(HTTPTest, self).begin()
+ return super(HTTPTest, self).begin()
def instantiate_server_by(self, protocol):
server = {HTTP: HTTPd,
--
2.21.0
From 7fba12cf25ff7cc352f0f5df7d91670df7035823 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Thu, 7 Nov 2019 13:01:44 +0100
Subject: [PATCH 2/3] testenv: Allow definition of environment variables for
wget execuion
* testenv/README: Added description for new EnvironmentVariable hook
* testenv/conf/environment_variable.py: Added implementation of EnvironmentVariable hook
* testenv/test/base_test.py: Modified exec_wget() to enable use of EnvironmentVariable hook
Added new test hook called EnvironmentVariables, for defining environment variables when wget is executed in tests. This is handy for testing environment variables, which are accepted by wget.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
testenv/README | 3 +++
testenv/conf/environment_variables.py | 14 ++++++++++++++
testenv/test/base_test.py | 6 +++++-
3 files changed, 22 insertions(+), 1 deletion(-)
create mode 100644 testenv/conf/environment_variables.py
diff --git a/testenv/README b/testenv/README
index aca8cdda..d4fabddd 100644
--- a/testenv/README
+++ b/testenv/README
@@ -224,6 +224,9 @@ executed. The currently supported options are:
file. While all Download URL's are passed to Urls, a notable exception is
when in-url authentication is used. In such a case, the URL is specified in
the WgetCommands string.
+ * EnvironmentVariables: A dictionary with key-value items, which will be
+ defined as environment variables during the execution of wget command in
+ test.
Post-Test Hooks:
================================================================================
diff --git a/testenv/conf/environment_variables.py b/testenv/conf/environment_variables.py
new file mode 100644
index 00000000..323c051c
--- /dev/null
+++ b/testenv/conf/environment_variables.py
@@ -0,0 +1,14 @@
+from conf import hook
+
+""" Test Option: EnvironmentVariables
+This hook is used to define environment variables used for execution of wget
+command in test."""
+
+
+@hook(alias='EnvironmentVariables')
+class URLs:
+ def __init__(self, envs):
+ self.envs = envs
+
+ def __call__(self, test_obj):
+ test_obj.envs.update(**self.envs)
diff --git a/testenv/test/base_test.py b/testenv/test/base_test.py
index dbf4678f..04a6f748 100644
--- a/testenv/test/base_test.py
+++ b/testenv/test/base_test.py
@@ -51,6 +51,7 @@ class BaseTest:
self.wget_options = ''
self.urls = []
+ self.envs = dict()
self.tests_passed = True
self.ready = False
@@ -97,12 +98,15 @@ class BaseTest:
cmd_line = self.gen_cmd_line()
params = shlex.split(cmd_line)
print(params)
+ envs = {"HOME": os.getcwd()}
+ envs.update(**self.envs)
+ print(envs)
if os.getenv("SERVER_WAIT"):
time.sleep(float(os.getenv("SERVER_WAIT")))
try:
- ret_code = call(params, env={"HOME": os.getcwd()})
+ ret_code = call(params, env=envs)
except FileNotFoundError:
raise TestFailed("The Wget Executable does not exist at the "
"expected path.")
--
2.21.0
From 0d50becc19ba07f34157b2842ca97675cc95fc1a Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Thu, 7 Nov 2019 13:11:30 +0100
Subject: [PATCH 3/3] testenv: Add test for handling of no_proxy environment
variable
* testenv/Test-no_proxy-env.py: Added new test for no_proxy env
Added new test with 5 cases, which are testing various combinations of no_proxy environment variable definition and requested URLs
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
testenv/Test-no_proxy-env.py | 142 +++++++++++++++++++++++++++++++++++
1 file changed, 142 insertions(+)
create mode 100755 testenv/Test-no_proxy-env.py
diff --git a/testenv/Test-no_proxy-env.py b/testenv/Test-no_proxy-env.py
new file mode 100755
index 00000000..ea7f38c4
--- /dev/null
+++ b/testenv/Test-no_proxy-env.py
@@ -0,0 +1,142 @@
+#!/usr/bin/env python3
+from sys import exit
+from test.http_test import HTTPTest
+from test.base_test import HTTP
+from misc.wget_file import WgetFile
+
+"""
+ This test ensures, that domains with and without leftmost dot defined in
+ no_proxy environment variable are accepted by wget. The idea is to use
+ non-existing proxy server address and detect whether files are downloaded
+ when proxy settings are omitted based on no_proxy environment variable
+ value.
+
+ current wget's behavior:
+ - "no_proxy=.mit.edu"
+ - will match the domain and subdomains e.g. "www.mit.edu" or "www.subdomain.mit.edu" (Case #4)
+ - will NOT match the host "mit.edu" (Case #3)
+ - "no_proxy=mit.edu"
+ - will match the domain and subdomains e.g. "www.mit.edu" or "www.subdomain.mit.edu" (Case #2)
+ - will match the host "mit.edu" (Case #1)
+ - downside: can not match only the host
+"""
+# File Definitions
+File1 = "Would you like some Tea?"
+File2 = "With lemon or cream?"
+
+A_File = WgetFile ("File1", File1)
+B_File = WgetFile ("File2", File2)
+
+WGET_URLS = [["File1", "File2"]]
+WGET_ENVS = {
+ "http_proxy": "nonexisting.localhost:8080",
+ "no_proxy": "working1.localhost,.working2.localhost"
+}
+
+Servers = [HTTP]
+Files = [[A_File, B_File]]
+
+ExpectedReturnCodeWorking = 0
+ExpectedReturnCodeNotWorking = 4 # network error (non-existing proxy address)
+
+ExpectedDownloadedFilesWorking = [A_File, B_File]
+
+# Pre and Post Test Hooks
+test_options = {
+ "Urls" : WGET_URLS,
+ "EnvironmentVariables": WGET_ENVS
+}
+post_test_working = {
+ "ExpectedFiles" : ExpectedDownloadedFilesWorking,
+ "ExpectedRetcode" : ExpectedReturnCodeWorking
+}
+post_test_not_working = {
+ "ExpectedRetcode" : ExpectedReturnCodeNotWorking
+}
+
+# Case #1:
+# - Requested domain matches exactly the domain definition in no_proxy.
+# - Domain definition in no_proxy is NOT dot-prefixed
+# Expected result: proxy settings don't apply and files are downloaded.
+pre_case_1 = {
+ "ServerFiles" : Files,
+ "Domains" : ["working1.localhost"]
+}
+
+err_case_1 = HTTPTest (
+ pre_hook=pre_case_1,
+ test_params=test_options,
+ post_hook=post_test_working,
+ protocols=Servers
+).begin ()
+
+# Case #2:
+# - Requested domain is sub-domain of a domain definition in no_proxy.
+# - Domain definition in no_proxy is NOT dot-prefixed
+# Expected result: proxy settings don't apply and files are downloaded.
+pre_case_2 = {
+ "ServerFiles" : Files,
+ "Domains" : ["www.working1.localhost"]
+}
+
+err_case_2 = HTTPTest (
+ pre_hook=pre_case_2,
+ test_params=test_options,
+ post_hook=post_test_working,
+ protocols=Servers
+).begin ()
+
+# Case #3:
+# - Requested domain matches exactly the domain definition in no_proxy,
+# except for the leftmost dot (".") in no_proxy domain definition.
+# - Domain definition in no_proxy IS dot-prefixed
+# Expected result: proxy settings apply and files are downloaded. This is
+# due to the mismatch in leftmost dot.
+# NOTE: This is inconsistent with curl's behavior, but has less drawbacks.
+pre_case_3 = {
+ "ServerFiles" : Files,
+ "Domains" : ["working2.localhost"]
+}
+
+err_case_3 = HTTPTest (
+ pre_hook=pre_case_3,
+ test_params=test_options,
+ post_hook=post_test_not_working,
+ protocols=Servers
+).begin ()
+
+# Case #4:
+# - Requested domain is sub-domain of a domain definition in no_proxy.
+# - Domain definition in no_proxy IS dot-prefixed
+# Expected result: proxy settings don't apply and files are downloaded.
+pre_case_4 = {
+ "ServerFiles" : Files,
+ "Domains" : ["www.working2.localhost"]
+}
+
+err_case_4 = HTTPTest (
+ pre_hook=pre_case_4,
+ test_params=test_options,
+ post_hook=post_test_working,
+ protocols=Servers
+).begin ()
+
+# Case #5
+# - Requested domain does not match a domain definition in no_proxy.
+# - Requested domain is NOT sub-domain of a domain definition in no_proxy.
+# Expected result: proxy settings apply and files are NOT downloaded due to
+# network error when using proxy with non-existing URL.
+pre_case_5 = {
+ "ServerFiles" : Files,
+ "Domains" : ["www.example.localhost"]
+}
+
+err_case_5 = HTTPTest (
+ pre_hook=pre_case_5,
+ test_params=test_options,
+ post_hook=post_test_not_working,
+ protocols=Servers
+).begin ()
+
+# Combine error codes from all test cases
+exit (max(err_case_1, err_case_2, err_case_3, err_case_4, err_case_5))
--
2.21.0

1
ci.fmf Normal file
View File

@ -0,0 +1 @@
resultsdb-testcase: separate

11
gating.yaml Normal file
View File

@ -0,0 +1,11 @@
--- !Policy
product_versions:
- rhel-9
decision_context: osci_compose_gate
rules:
- !PassingTestCaseRule {test_case_name: osci.brew-build./plans/tier1-internal.functional}
- !PassingTestCaseRule {test_case_name: osci.brew-build./plans/public.functional}
- !PassingTestCaseRule {test_case_name: baseos-ci.brew-build.tier1.functional}
- !PassingTestCaseRule {test_case_name: baseos-ci.brew-build.tier2.functional}
- !PassingTestCaseRule {test_case_name: baseos-ci.brew-build.tier3.functional}
- !PassingTestCaseRule {test_case_name: baseos-ci.brew-build.tedude.validation}

7
plans/public.fmf Normal file
View File

@ -0,0 +1,7 @@
summary: Test plan that runs all tests from tests repo.
discover:
how: fmf
url: https://src.fedoraproject.org/tests/wget.git
execute:
how: tmt

11
plans/tier1-internal.fmf Normal file
View File

@ -0,0 +1,11 @@
summary: CI plan, picks Tier1 tests, runs in beakerlib.
discover:
- name: rhel
how: fmf
filter: 'tier: 1'
url: git://pkgs.devel.redhat.com/tests/wget
execute:
how: tmt
adjust:
enabled: false
when: distro == centos-stream-9

1
sources Normal file
View File

@ -0,0 +1 @@
SHA512 (wget-1.21.1.tar.gz) = 784efbf9fe43a1671109e32a9c36237eb2d5c19cf756bf6f6e65517fb21464d3d94b1d6f491852d23b3ddff63e38fe6b60df9125c91b139993af59875e3a0712

View File

@ -0,0 +1,43 @@
diff --git a/doc/wget.texi.old b/doc/wget.texi
index adf471d..685eb1a 100644
--- a/doc/wget.texi.old
+++ b/doc/wget.texi
@@ -513,38 +513,6 @@ treated as @samp{html} if the Content-Type matches @samp{text/html}.
Furthermore, the @var{file}'s location will be implicitly used as base
href if none was specified.
-@cindex input-metalink
-@item --input-metalink=@var{file}
-Downloads files covered in local Metalink @var{file}. Metalink version 3
-and 4 are supported.
-
-@cindex keep-badhash
-@item --keep-badhash
-Keeps downloaded Metalink's files with a bad hash. It appends .badhash
-to the name of Metalink's files which have a checksum mismatch, except
-without overwriting existing files.
-
-@cindex metalink-over-http
-@item --metalink-over-http
-Issues HTTP HEAD request instead of GET and extracts Metalink metadata
-from response headers. Then it switches to Metalink download.
-If no valid Metalink metadata is found, it falls back to ordinary HTTP download.
-Enables @samp{Content-Type: application/metalink4+xml} files download/processing.
-
-@cindex metalink-index
-@item --metalink-index=@var{number}
-Set the Metalink @samp{application/metalink4+xml} metaurl ordinal
-NUMBER. From 1 to the total number of ``application/metalink4+xml''
-available. Specify 0 or @samp{inf} to choose the first good one.
-Metaurls, such as those from a @samp{--metalink-over-http}, may have
-been sorted by priority key's value; keep this in mind to choose the
-right NUMBER.
-
-@cindex preferred-location
-@item --preferred-location
-Set preferred location for Metalink resources. This has effect if multiple
-resources with same priority are available.
-
@cindex xattr
@item --xattr
Enable use of file system's extended attributes to save the

36
wget-1.21-segfault.patch Normal file
View File

@ -0,0 +1,36 @@
diff --git a/src/ftp.c b/src/ftp.c
index ea7621e0..ab6802a6 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -992,9 +992,6 @@ Error in server response, closing control connection.\n"));
/* 2004-09-20 SMS. */
- if (target != targetbuf)
- xfree (target);
-
} /* else */
}
else /* do not CWD */
diff --git a/src/ftp.c b/src/ftp.c
index a1fcaa50..e821b0f3 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -2083,7 +2083,7 @@ ftp_loop_internal (struct url *u, struct url *original_url, struct fileinfo *f,
/* --dont-remove-listing was specified, so do count this towards the
number of bytes and files downloaded. */
{
- total_downloaded_bytes += qtyread;
+ total_downloaded_bytes += (qtyread - restval);
numurls++;
}
@@ -2098,7 +2098,7 @@ ftp_loop_internal (struct url *u, struct url *original_url, struct fileinfo *f,
downloaded if they're going to be deleted. People seeding proxies,
for instance, may want to know how many bytes and files they've
downloaded through it. */
- total_downloaded_bytes += qtyread;
+ total_downloaded_bytes += (qtyread - restval);
numurls++;
if (opt.delete_after && !input_file_url (opt.input_filename))

13
wget-1.21-strtol.patch Normal file
View File

@ -0,0 +1,13 @@
diff --git a/src/wget.h.old b/src/wget.h
index 6f20eab..f422737 100644
--- a/src/wget.h.old
+++ b/src/wget.h
@@ -144,7 +144,7 @@ typedef int64_t wgint;
#define WGINT_MAX INT64_MAX
typedef wgint SUM_SIZE_INT;
-#define str_to_wgint strtol
+#define str_to_wgint strtoll
#include "options.h"

View File

@ -1,36 +1,22 @@
Summary: A utility for retrieving files using the HTTP or FTP protocols
Name: wget
Version: 1.19.5
Release: 11%{?dist}
Version: 1.21.1
Release: 7%{?dist}
License: GPLv3+
Group: Applications/Internet
Url: http://www.gnu.org/software/wget/
Source: ftp://ftp.gnu.org/gnu/wget/wget-%{version}.tar.gz
Patch1: wget-1.17-path.patch
Patch2: wget-1.19.5-Don-t-limit-the-test-suite-HTTPS-server-to-TLSv1.patch
Patch3: wget-1.19.5-covscan-important-issues.patch
Patch4: wget-1.19.5-Add-TLS-1.3-support-for-GnuTLS.patch
Patch5: wget-1.19.5-Enable-post-handshake-auth-under-gnutls-on-TLS1.3.patch
Patch6: wget-1.19.5-Dont-use-extended-attributes---xattr-by-default.patch
Patch7: wget-1.19.5-Dont-save-userpw-with---xattr.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=692d5c5215de0db482c252492a92fc424cc6a97c
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=562eacb76a2b64d5dc80a443f0f739bc9ef76c17
Patch8: wget-1.19.5-CVE-2019-5953.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=fd85ac9cc623847e9d94d9f9241ab34e2c146cbf
Patch9: wget-1.19.5-no_proxy-dot-prefix.patch
Patch10: wget-1.19.5-no_proxy-tests.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=706e71564cadc7192ac21efbf51b661c967f35b5
Patch11: wget-1.19.5-ca-cert-too-verbose.patch
Patch12: wget-1.19.5-no-log-when-quiet.patch
Patch2: wget-1.21-strtol.patch
Patch3: wget-1.21-metalink-man.patch
Patch4: wget-1.21-segfault.patch
Provides: webclient
Provides: bundled(gnulib)
Requires(post): /sbin/install-info
Requires(preun): /sbin/install-info
# needed for test suite
BuildRequires: perl-HTTP-Daemon, python3
BuildRequires: gnutls-devel, pkgconfig, texinfo, gettext, autoconf, libidn2-devel, libuuid-devel, perl-podlators, libpsl-devel, libmetalink-devel, gpgme-devel, gcc, zlib-devel
BuildRequires: make
BuildRequires: perl(lib), perl(English), perl(HTTP::Daemon), python3
BuildRequires: gnutls-devel, pkgconfig, texinfo, gettext, autoconf, libidn2-devel, libuuid-devel, perl-podlators, libpsl-devel, gpgme-devel, gcc, zlib-devel
%description
GNU Wget is a file retrieval utility which can use either the HTTP or
@ -48,18 +34,7 @@ support for Proxy servers, and configurability.
sed -i "s|\(PACKAGE_STRING='wget .*\)'|\1 (Red Hat modified)'|" configure
grep "PACKAGE_STRING='wget .* (Red Hat modified)'" configure || exit 1
%patch1 -p1 -b .path
%patch2 -p1 -b .tlsv1_testsuite
%patch3 -p1 -b .covscan_imp_issues
%patch4 -p1 -b .tls1_3
%patch5 -p1 -b .post_auth_tls13
%patch6 -p1 -b .no_xattr_by_default
%patch7 -p1 -b .no_userpw_in_xattr
%patch8 -p1 -b .CVE-2019-5953
%patch9 -p1 -b .no_proxy-dot-prefix
%patch10 -p1 -b .no_proxy-test
%patch11 -p1 -b .too_verbose
%patch12 -p1 -b .no-log-quiet
%autopatch -p1
%build
%configure \
@ -71,34 +46,22 @@ grep "PACKAGE_STRING='wget .* (Red Hat modified)'" configure || exit 1
--enable-ntlm \
--enable-nls \
--enable-ipv6 \
--disable-rpath \
--with-metalink
--disable-rpath
make %{?_smp_mflags}
%{make_build}
%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT CFLAGS="$RPM_OPT_FLAGS"
%{make_install} CFLAGS="$RPM_OPT_FLAGS"
rm -f $RPM_BUILD_ROOT/%{_infodir}/dir
%find_lang %{name}
%find_lang %{name}-gnulib
%check
make check
%post
/sbin/install-info %{_infodir}/wget.info.gz %{_infodir}/dir || :
%preun
if [ "$1" = 0 ]; then
/sbin/install-info --delete %{_infodir}/wget.info.gz %{_infodir}/dir || :
fi
%clean
rm -rf $RPM_BUILD_ROOT
%files -f %{name}.lang
%defattr(-,root,root)
%files -f %{name}.lang -f %{name}-gnulib.lang
%doc AUTHORS MAILING-LIST NEWS README COPYING doc/sample.wgetrc
%config(noreplace) %{_sysconfdir}/wgetrc
%{_mandir}/man1/wget.*
@ -106,35 +69,80 @@ rm -rf $RPM_BUILD_ROOT
%{_infodir}/*
%changelog
* Tue Dec 13 2022 Michal Ruprich <mruprich@redhat.com> - 1.19.5-11
- Resolves: #2152731 - Running wget with -O and -q in the background yields a file wget-log
* Tue Nov 02 2021 Michal Ruprich <mruprich@redhat.com> - 1.21.1-7
- Resolves: #2017842 - Two different segfaults when downloading multiple files
- Removing metalink from manpage
* Tue Mar 31 2020 Tomas Hozza <thozza@redhat.com> - 1.19.5-10
- Fix wget being too verbose when using --no-verbose and --ca-certificate (#1807267)
* Tue Aug 10 2021 Mohan Boddu <mboddu@redhat.com> - 1.21.1-6
- Rebuilt for IMA sigs, glibc 2.34, aarch64 flags
Related: rhbz#1991688
* Thu Nov 21 2019 Tomáš Hozza <thozza@redhat.com> - 1.19.5-9
- Fix issue with dot-prefixed domain names in no_proxy ENV (#1763702)
* Tue Jun 08 2021 Michal Ruprich <mruprich@redhat.com> - 1.21.1-5
- Resolves: #1967216 - consider disabling metalink support in wget
* Sun Apr 07 2019 Tomas Hozza <thozza@redhat.com> - 1.19.5-8
- Fix CVE-2019-5953 (#1696736)
* Fri Apr 16 2021 Mohan Boddu <mboddu@redhat.com> - 1.21.1-4
- Rebuilt for RHEL 9 BETA on Apr 15th 2021. Related: rhbz#1947937
* Thu Jan 10 2019 Tomas Hozza <thozza@redhat.com> - 1.19.5-7
- Fix information exposure in set_file_metadata function in xattr.c (CVE-2018-20483)
* Wed Apr 14 2021 Michal Ruprich <michalruprich@gmail.com> - 1.21.1-3
- Resolves: #1949045 - wget in F33 i686 is unable to download files larger than 2GiB
* Fri Oct 12 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-6
- Enable post handshake auth under gnutls on TLS1.3 (#1636903)
* Wed Jan 27 2021 Fedora Release Engineering <releng@fedoraproject.org> - 1.21.1-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild
* Wed Oct 03 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-5
- Allow specification of TLSv1_3 in --secure-protocol option (#1623997)
* Fri Jan 22 2021 Michal Ruprich <mruprich@redhat.com> - 1.21.1-1
- Update to 1.21.1
* Wed Aug 29 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-4
- Add zlib-devel to BuildRequires to enable compression of WARC files (#1623004)
* Thu Nov 19 2020 Michal Ruprich <mruprich@redhat.com> - 1.20.3-9
- Use make macros
- https://fedoraproject.org/wiki/Changes/UseMakeBuildInstallMacro
* Mon Aug 27 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-3
- Fixed resource leaks found by Coverity (#1602729)
* Thu Jul 30 2020 Tomas Hozza <thozza@redhat.com> - 1.20.3-8
- Fix too verbose output even with --no-verbose
* Fri Aug 10 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-2
- Fix FTBFS due to test suite HTTPS server forcing use of TLSv1 (#1611753)
* Wed Jul 29 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.20.3-7
- Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
* Wed Jul 01 2020 Artem Egorenkov <aegorenk@redhat.com> - 1.20.3-6
- Fix Perl module build dependencies
* Wed Jun 24 2020 Artem Egorenkov <aegorenk@redhat.com> - 1.20.3-5
- Fix FTP VERIFCERTERR handling (#1475861)
* Tue Feb 25 2020 Tomas Hozza <thozza@redhat.com> - 1.20.3-4
- Fix FTBFS with new gcc (#1800250)
* Fri Jan 31 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.20.3-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_32_Mass_Rebuild
* Sat Jul 27 2019 Fedora Release Engineering <releng@fedoraproject.org> - 1.20.3-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_31_Mass_Rebuild
* Fri Apr 05 2019 Tomas Hozza <thozza@redhat.com> - 1.20.3-1
- Update to 1.20.3
- Fix CVE-2019-5953
* Sun Feb 03 2019 Fedora Release Engineering <releng@fedoraproject.org> - 1.20.1-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_30_Mass_Rebuild
* Wed Jan 02 2019 Tomas Hozza <thozza@redhat.com> - 1.20.1-1
- Update to 1.20.1
- Fix CVE-2018-20483
* Thu Dec 06 2018 Tomas Hozza <thozza@redhat.com> - 1.20-1
- Update to 1.20
- --secure-protocol=TLSv1_3 now works (#1623994)
* Thu Aug 29 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-5
- Avoid creating empty wget-log when using -O and -q in background (#1484411)
* Tue Aug 28 2018 Tomas Korbar <tkorbar@redhat.com> - 1.19.5-4
- Add zlib-devel to buildrequires (#1612891)
* Sat Jul 14 2018 Fedora Release Engineering <releng@fedoraproject.org> - 1.19.5-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
* Tue Jun 26 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-2
- Don't install info files in scriptlets
* Wed May 09 2018 Tomas Hozza <thozza@redhat.com> - 1.19.5-1
- Update to 1.19.5 fixing CVE-2018-0494