rpms/wget
7
0
Fork 0
Code Issues Pull Requests Releases Activity

Fixed use of .netrc (#1425097)

Browse Source
This commit is contained in:
Tomas Hozza 2017-06-02 12:01:50 +02:00
parent 9222fc4c05
commit 34da3b1c06
6 changed files with 527 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

139
wget-1.19.1-Add-command-line-option-to-disable-use-of-.netrc.patch Normal file
View File

@ -0,0 +1,139 @@
From 876def8ebe56d483921cf645371d277b615373e5 Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Fri, 12 May 2017 19:17:32 +0200
Subject: [PATCH 3/4] Add command line option to disable use of .netrc
Although internally code uses option for (not) reading .netrc for
credentials, it was not possible to turn this behavior off on command
line. Note that it was possible to turn it off using wgetrc.
Idea for this change came from Bruce Jerrick (bmj001@gmail.com).
Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1425097
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
doc/wget.texi | 6 ++++
src/main.c | 3 ++
testenv/Makefile.am | 1 +
testenv/Test-auth-basic-no-netrc-fail.py | 59 ++++++++++++++++++++++++++++++++
4 files changed, 69 insertions(+)
create mode 100755 testenv/Test-auth-basic-no-netrc-fail.py
diff --git a/doc/wget.texi b/doc/wget.texi
index a2bf9dc..e4e0bf6 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -703,6 +703,12 @@ Before (over)writing a file, back up an existing file by adding a
files are rotated to @samp{.2}, @samp{.3}, and so on, up to
@var{backups} (and lost beyond that).
+@cindex authentication credentials
+@item --no-netrc
+Do not try to obtain credentials from @file{.netrc} file. By default
+@file{.netrc} file is searched for credentials in case none have been
+passed on command line and authentication is required.
+
@cindex continue retrieval
@cindex incomplete downloads
@cindex resume download
diff --git a/src/main.c b/src/main.c
index 8e9d6e9..297499e 100644
--- a/src/main.c
+++ b/src/main.c
@@ -359,6 +359,7 @@ static struct cmdline_option option_data[] =
#endif
{ "method", 0, OPT_VALUE, "method", -1 },
{ "mirror", 'm', OPT_BOOLEAN, "mirror", -1 },
+ { "netrc", 0, OPT_BOOLEAN, "netrc", -1 },
{ "no", 'n', OPT__NO, NULL, required_argument },
{ "no-clobber", 0, OPT_BOOLEAN, "noclobber", -1 },
{ "no-config", 0, OPT_BOOLEAN, "noconfig", -1},
@@ -629,6 +630,8 @@ Download:\n"),
-nc, --no-clobber skip downloads that would download to\n\
existing files (overwriting them)\n"),
N_("\
+ --no-netrc don't try to obtain credentials from .netrc\n"),
+ N_("\
-c, --continue resume getting a partially-downloaded file\n"),
N_("\
--start-pos=OFFSET start downloading from zero-based position OFFSET\n"),
diff --git a/testenv/Makefile.am b/testenv/Makefile.am
index 7104314..ef4158a 100644
--- a/testenv/Makefile.am
+++ b/testenv/Makefile.am
@@ -78,6 +78,7 @@ if HAVE_PYTHON3
Test-auth-basic-netrc.py \
Test-auth-basic-netrc-user-given.py \
Test-auth-basic-netrc-pass-given.py \
+ Test-auth-basic-no-netrc-fail.py \
Test-auth-both.py \
Test-auth-digest.py \
Test-auth-no-challenge.py \
diff --git a/testenv/Test-auth-basic-no-netrc-fail.py b/testenv/Test-auth-basic-no-netrc-fail.py
new file mode 100755
index 0000000..fad15e9
--- /dev/null
+++ b/testenv/Test-auth-basic-no-netrc-fail.py
@@ -0,0 +1,59 @@
+#!/usr/bin/env python3
+from sys import exit
+from test.http_test import HTTPTest
+from misc.wget_file import WgetFile
+
+"""
+ This test ensures that Wget will not use credentials from .netrc
+ when --no-netrc option is specified and Basic authentication is required
+ and fails.
+"""
+############# File Definitions ###############################################
+File1 = "I am an invisble man."
+
+User = "Sauron"
+Password = "TheEye"
+
+File1_rules = {
+ "Authentication" : {
+ "Type" : "Basic",
+ "User" : User,
+ "Pass" : Password
+ }
+}
+
+Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, Password)
+
+A_File = WgetFile ("File1", File1, rules=File1_rules)
+Netrc_File = WgetFile (".netrc", Netrc)
+
+WGET_OPTIONS = "--no-netrc"
+WGET_URLS = [["File1"]]
+
+Files = [[A_File]]
+LocalFiles = [Netrc_File]
+
+ExpectedReturnCode = 6
+ExpectedDownloadedFiles = [Netrc_File]
+
+################ Pre and Post Test Hooks #####################################
+pre_test = {
+ "ServerFiles" : Files,
+ "LocalFiles" : LocalFiles
+}
+test_options = {
+ "WgetCommands" : WGET_OPTIONS,
+ "Urls" : WGET_URLS
+}
+post_test = {
+ "ExpectedFiles" : ExpectedDownloadedFiles,
+ "ExpectedRetcode" : ExpectedReturnCode
+}
+
+err = HTTPTest (
+ pre_hook=pre_test,
+ test_params=test_options,
+ post_hook=post_test
+).begin ()
+
+exit (err)
--
2.7.5

275
wget-1.19.1-Added-tests-for-HTTP-authentication-using-credential.patch Normal file
View File

@ -0,0 +1,275 @@
From 17960b57d51ffb19b2b20df3e53da42c555f022c Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Fri, 12 May 2017 19:17:30 +0200
Subject: [PATCH 1/4] Added tests for HTTP authentication using credentials
from .netrc
Getting credentials from .netrc has been broken from time to time, thus
adding a test coverage to prevent regressions.
Also added setting of "HOME" environment variable when executing wget,
to make sure LocalFiles like .netrc, which are created just for the
test, are actually used.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
testenv/Makefile.am | 3 ++
testenv/Test-auth-basic-netrc-pass-given.py | 68 +++++++++++++++++++++++++++++
testenv/Test-auth-basic-netrc-user-given.py | 68 +++++++++++++++++++++++++++++
testenv/Test-auth-basic-netrc.py | 66 ++++++++++++++++++++++++++++
testenv/test/base_test.py | 2 +-
5 files changed, 206 insertions(+), 1 deletion(-)
create mode 100755 testenv/Test-auth-basic-netrc-pass-given.py
create mode 100755 testenv/Test-auth-basic-netrc-user-given.py
create mode 100755 testenv/Test-auth-basic-netrc.py
diff --git a/testenv/Makefile.am b/testenv/Makefile.am
index 3febec7..7104314 100644
--- a/testenv/Makefile.am
+++ b/testenv/Makefile.am
@@ -75,6 +75,9 @@ if HAVE_PYTHON3
TESTS = Test-504.py \
Test-auth-basic-fail.py \
Test-auth-basic.py \
+ Test-auth-basic-netrc.py \
+ Test-auth-basic-netrc-user-given.py \
+ Test-auth-basic-netrc-pass-given.py \
Test-auth-both.py \
Test-auth-digest.py \
Test-auth-no-challenge.py \
diff --git a/testenv/Test-auth-basic-netrc-pass-given.py b/testenv/Test-auth-basic-netrc-pass-given.py
new file mode 100755
index 0000000..43dfe34
--- /dev/null
+++ b/testenv/Test-auth-basic-netrc-pass-given.py
@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+from sys import exit
+from test.http_test import HTTPTest
+from misc.wget_file import WgetFile
+
+"""
+ This test ensures Wget uses credentials from .netrc for Basic Authorization Negotiation.
+ In this case we test that .netrc credentials are used in case only
+ password is given on the command line.
+ Also, we ensure that Wget saves the host after a successful auth and
+ doesn't wait for a challenge the second time.
+"""
+############# File Definitions ###############################################
+File1 = "I am an invisble man."
+File2 = "I too am an invisible man."
+
+User = "Sauron"
+Password = "TheEye"
+
+File1_rules = {
+ "Authentication" : {
+ "Type" : "Basic",
+ "User" : User,
+ "Pass" : Password
+ }
+}
+File2_rules = {
+ "ExpectHeader" : {
+ "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
+ }
+}
+
+Netrc = "machine 127.0.0.1\n\tlogin {0}".format(User)
+
+A_File = WgetFile ("File1", File1, rules=File1_rules)
+B_File = WgetFile ("File2", File2, rules=File2_rules)
+Netrc_File = WgetFile (".netrc", Netrc)
+
+WGET_OPTIONS = "--password={0}".format(Password)
+WGET_URLS = [["File1", "File2"]]
+
+Files = [[A_File, B_File]]
+LocalFiles = [Netrc_File]
+
+ExpectedReturnCode = 0
+ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
+
+################ Pre and Post Test Hooks #####################################
+pre_test = {
+ "ServerFiles" : Files,
+ "LocalFiles" : LocalFiles
+}
+test_options = {
+ "WgetCommands" : WGET_OPTIONS,
+ "Urls" : WGET_URLS
+}
+post_test = {
+ "ExpectedFiles" : ExpectedDownloadedFiles,
+ "ExpectedRetcode" : ExpectedReturnCode
+}
+
+err = HTTPTest (
+ pre_hook=pre_test,
+ test_params=test_options,
+ post_hook=post_test
+).begin ()
+
+exit (err)
diff --git a/testenv/Test-auth-basic-netrc-user-given.py b/testenv/Test-auth-basic-netrc-user-given.py
new file mode 100755
index 0000000..57b6148
--- /dev/null
+++ b/testenv/Test-auth-basic-netrc-user-given.py
@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+from sys import exit
+from test.http_test import HTTPTest
+from misc.wget_file import WgetFile
+
+"""
+ This test ensures Wget uses credentials from .netrc for Basic Authorization Negotiation.
+ In this case we test that .netrc credentials are used in case only
+ user login is given on the command line.
+ Also, we ensure that Wget saves the host after a successful auth and
+ doesn't wait for a challenge the second time.
+"""
+############# File Definitions ###############################################
+File1 = "I am an invisble man."
+File2 = "I too am an invisible man."
+
+User = "Sauron"
+Password = "TheEye"
+
+File1_rules = {
+ "Authentication" : {
+ "Type" : "Basic",
+ "User" : User,
+ "Pass" : Password
+ }
+}
+File2_rules = {
+ "ExpectHeader" : {
+ "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
+ }
+}
+
+Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, Password)
+
+A_File = WgetFile ("File1", File1, rules=File1_rules)
+B_File = WgetFile ("File2", File2, rules=File2_rules)
+Netrc_File = WgetFile (".netrc", Netrc)
+
+WGET_OPTIONS = "--user={0}".format(User)
+WGET_URLS = [["File1", "File2"]]
+
+Files = [[A_File, B_File]]
+LocalFiles = [Netrc_File]
+
+ExpectedReturnCode = 0
+ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
+
+################ Pre and Post Test Hooks #####################################
+pre_test = {
+ "ServerFiles" : Files,
+ "LocalFiles" : LocalFiles
+}
+test_options = {
+ "WgetCommands" : WGET_OPTIONS,
+ "Urls" : WGET_URLS
+}
+post_test = {
+ "ExpectedFiles" : ExpectedDownloadedFiles,
+ "ExpectedRetcode" : ExpectedReturnCode
+}
+
+err = HTTPTest (
+ pre_hook=pre_test,
+ test_params=test_options,
+ post_hook=post_test
+).begin ()
+
+exit (err)
diff --git a/testenv/Test-auth-basic-netrc.py b/testenv/Test-auth-basic-netrc.py
new file mode 100755
index 0000000..5710fe7
--- /dev/null
+++ b/testenv/Test-auth-basic-netrc.py
@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+from sys import exit
+from test.http_test import HTTPTest
+from misc.wget_file import WgetFile
+
+"""
+ This test ensures Wget uses credentials from .netrc for Basic Authorization Negotiation.
+ In this case we test that .netrc credentials are used in case no user
+ login and no password is given on the command line.
+ Also, we ensure that Wget saves the host after a successful auth and
+ doesn't wait for a challenge the second time.
+"""
+############# File Definitions ###############################################
+File1 = "I am an invisble man."
+File2 = "I too am an invisible man."
+
+User = "Sauron"
+Password = "TheEye"
+
+File1_rules = {
+ "Authentication" : {
+ "Type" : "Basic",
+ "User" : User,
+ "Pass" : Password
+ }
+}
+File2_rules = {
+ "ExpectHeader" : {
+ "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
+ }
+}
+
+Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, Password)
+
+A_File = WgetFile ("File1", File1, rules=File1_rules)
+B_File = WgetFile ("File2", File2, rules=File2_rules)
+Netrc_File = WgetFile (".netrc", Netrc)
+
+WGET_URLS = [["File1", "File2"]]
+
+Files = [[A_File, B_File]]
+LocalFiles = [Netrc_File]
+
+ExpectedReturnCode = 0
+ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
+
+################ Pre and Post Test Hooks #####################################
+pre_test = {
+ "ServerFiles" : Files,
+ "LocalFiles" : LocalFiles
+}
+test_options = {
+ "Urls" : WGET_URLS
+}
+post_test = {
+ "ExpectedFiles" : ExpectedDownloadedFiles,
+ "ExpectedRetcode" : ExpectedReturnCode
+}
+
+err = HTTPTest (
+ pre_hook=pre_test,
+ test_params=test_options,
+ post_hook=post_test
+).begin ()
+
+exit (err)
diff --git a/testenv/test/base_test.py b/testenv/test/base_test.py
index b0087e9..bb706d8 100644
--- a/testenv/test/base_test.py
+++ b/testenv/test/base_test.py
@@ -102,7 +102,7 @@ class BaseTest:
time.sleep(float(os.getenv("SERVER_WAIT")))
try:
- ret_code = call(params)
+ ret_code = call(params, env={"HOME": os.getcwd()})
except FileNotFoundError:
raise TestFailed("The Wget Executable does not exist at the "
"expected path.")
--
2.7.5

30
wget-1.19.1-Fix-two-Metalink-tests-if-HOME-is-changed.patch Normal file
View File

@ -0,0 +1,30 @@
From 5d4ada1b7b0b79f8053f3d6ffddda2e2c66d9dce Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Tim=20R=C3=BChsen?= <tim.ruehsen@gmx.de>
Date: Tue, 16 May 2017 10:24:52 +0200
Subject: [PATCH 4/4] Fix two Metalink tests if $HOME is changed
* conf/expected_files.py (gen_local_fs_snapshot): Skip processing
of 'pubring.kbx'
---
testenv/conf/expected_files.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/testenv/conf/expected_files.py b/testenv/conf/expected_files.py
index 5362771..4e3ace9 100644
--- a/testenv/conf/expected_files.py
+++ b/testenv/conf/expected_files.py
@@ -24,6 +24,11 @@ class ExpectedFiles:
snapshot = {}
for parent, dirs, files in os.walk('.'):
for name in files:
+ # pubring.kbx will be created by libgpgme if $HOME doesn't contain the .gnupg directory.
+ # setting $HOME to CWD (in base_test.py) breaks two Metalink tests, so we skip this file here.
+ if name == 'pubring.kbx':
+ continue
+
f = {'content': ''}
file_path = os.path.join(parent, name)
with open(file_path) as fp:
--
2.7.5

38
wget-1.19.1-Fixed-getting-of-credentials-from-.netrc.patch Normal file
View File

@ -0,0 +1,38 @@
From f8c3df1f40f09dd61078436614d06e7ad818536e Mon Sep 17 00:00:00 2001
From: Tomas Hozza <thozza@redhat.com>
Date: Fri, 12 May 2017 19:17:31 +0200
Subject: [PATCH 2/4] Fixed getting of credentials from .netrc
There seemed to be a copy&paste error in http.c code, which decides
whether to get credentials from .netrc. In ftp.c "user" and "pass"
variables are char*, while in http.c, these are char**. For this reason
they should be dereferenced when determining if password and user login
is set to some value.
Also since both variables are dereferenced on lines above the changed
code, it does not really make sense to check if they are NULL.
This patch is based on fix from Bruce Jerrick <bmj001@gmail.com>.
Fedora bug: https://bugzilla.redhat.com/show_bug.cgi?id=1425097
Signed-off-by: Tomas Hozza <thozza@redhat.com>
---
src/http.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/http.c b/src/http.c
index 8b77a10..323f559 100644
--- a/src/http.c
+++ b/src/http.c
@@ -1900,7 +1900,7 @@ initialize_request (const struct url *u, struct http_stat *hs, int *dt, struct u
*passwd = NULL;
/* Check for ~/.netrc if none of the above match */
- if (opt.netrc && (!user || (!passwd || !*passwd)))
+ if (opt.netrc && (!*user || !*passwd))
search_netrc (u->host, (const char **) user, (const char **) passwd, 0);
/* We only do "site-wide" authentication with "global" user/password
--
2.7.5

21
wget-1.19.1-skip-failing-https-tests.patch Normal file
View File

@ -0,0 +1,21 @@
diff --git a/testenv/Makefile.am b/testenv/Makefile.am
index ef4158a..a534e77 100644
--- a/testenv/Makefile.am
+++ b/testenv/Makefile.am
@@ -96,16 +96,12 @@ if HAVE_PYTHON3
Test-cookie.py \
Test-Head.py \
Test-hsts.py \
- Test--https.py \
Test--https-crl.py \
Test-missing-scheme-retval.py \
Test-O.py \
- Test-pinnedpubkey-der-https.py \
Test-pinnedpubkey-der-no-check-https.py \
- Test-pinnedpubkey-hash-https.py \
Test-pinnedpubkey-hash-no-check-fail-https.py \
Test-pinnedpubkey-pem-fail-https.py \
- Test-pinnedpubkey-pem-https.py \
Test-Post.py \
Test-recursive-basic.py \
Test-recursive-include.py \

26
wget.spec
View File

@ -1,7 +1,7 @@
Summary: A utility for retrieving files using the HTTP or FTP protocols
Name: wget
Version: 1.19.1
Release: 2%{?dist}
Release: 3%{?dist}
License: GPLv3+
Group: Applications/Internet
Url: http://www.gnu.org/software/wget/
@ -9,7 +9,19 @@ Source: ftp://ftp.gnu.org/gnu/wget/wget-%{version}.tar.xz
Patch1: wget-1.17-path.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=4d729e322fae359a1aefaafec1144764a54e8ad4
patch2: wget-1.19.1-CVE-2017-6508.patch
Patch2: wget-1.19.1-CVE-2017-6508.patch
# https://bugzilla.redhat.com/show_bug.cgi?id=1425097
# 4 upstream commits total
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=17960b57d51ffb19b2b20df3e53da42c555f022c
Patch3: wget-1.19.1-Added-tests-for-HTTP-authentication-using-credential.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=f8c3df1f40f09dd61078436614d06e7ad818536e
Patch4: wget-1.19.1-Fixed-getting-of-credentials-from-.netrc.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=876def8ebe56d483921cf645371d277b615373e5
patch5: wget-1.19.1-Add-command-line-option-to-disable-use-of-.netrc.patch
# http://git.savannah.gnu.org/cgit/wget.git/commit/?id=5d4ada1b7b0b79f8053f3d6ffddda2e2c66d9dce
Patch6: wget-1.19.1-Fix-two-Metalink-tests-if-HOME-is-changed.patch
# http://lists.gnu.org/archive/html/bug-wget/2017-06/msg00009.html
Patch7: wget-1.19.1-skip-failing-https-tests.patch
Provides: webclient
Provides: bundled(gnulib)
@ -18,6 +30,8 @@ Requires(preun): /sbin/install-info
# needed for test suite
BuildRequires: perl-HTTP-Daemon, python3
BuildRequires: gnutls-devel, pkgconfig, texinfo, gettext, autoconf, libidn2-devel, libuuid-devel, perl-podlators, libpsl-devel, libmetalink-devel
# needed because of added test in Patch3. Need to remove this after rebase
BuildRequires: automake
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
%description
@ -38,6 +52,11 @@ grep "PACKAGE_STRING='wget .* (Red Hat modified)'" configure || exit 1
%patch1 -p1 -b .path
%patch2 -p1 -b .CVE-2017-6508
%patch3 -p1 -b .netrc_1
%patch4 -p1 -b .netrc_2
%patch5 -p1 -b .netrc_3
%patch6 -p1 -b .netrc_4
%patch7 -p1 -b .failing-tests
%build
%configure \
@ -84,6 +103,9 @@ rm -rf $RPM_BUILD_ROOT
%{_infodir}/*
%changelog
* Wed May 31 2017 Tomas Hozza <thozza@redhat.com> - 1.19.1-3
- Fixed use of .netrc (#1425097)
* Fri May 12 2017 Tomas Hozza <thozza@redhat.com> - 1.19.1-2
- Fix CVE-2017-6508 (#1429986)