From self[at]sungpae.com Mon Nov 8 16:59:48 2021 Date: Mon, 8 Nov 2021 16:59:48 -0600 From: Sung Pae To: security@docker.com Subject: Permissive forwarding rule leads to unintentional exposure of containers to external hosts Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QR1yLfEBO/zgxYVA" Content-Disposition: inline X-PGP-Key: fp="4BC7 2AA6 B1AE 2B5A C7F7 ADCF 9D1A A266 D2BC 9C2D" X-TUID: Avm8Mn+0Qq5s --QR1yLfEBO/zgxYVA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, The documentation for "docker run --publish" states: > Note that ports which are not bound to the host (i.e., -p 80:80 instead of > -p 127.0.0.1:80:80) will be accessible from the outside. This also applies > if you configured UFW to block this specific port, as Docker manages his own > iptables rules. https://docs.docker.com/engine/reference/commandline/run/#publish-or-expose-port--p---expose The statement above is accurate, but terribly misleading, since traffic to the container's published ports from external hosts will still be forwarded due to an explicit forwarding rule added to the DOCKER chain: # iptables -nvL DOCKER Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.2 tcp dpt:80 An attacker that sends traffic to 172.17.0.2:80 *through* the docker host will match the rule above and successfully connect to the container, obviating any security benefit of binding the published port on the host to 127.0.0.1. What's worse, users who bind their published ports to 127.0.0.1 operate under a false sense of security and may not bother taking further precautions against unintentional exposure. ## Proof of Concept Here is a simple proof of concept: 1. [VICTIM] Start a postgres container and publish its main port to 127.0.0.1 on the host. victim@192.168.0.100# docker run -e POSTGRES_PASSWORD=password -p 127.0.0.1:5432:5432 postgres 2. [ATTACKER] Route all packets destined for 172.16.0.0/12 through the victim's machine. attacker@192.168.0.200# ip route add 172.16.0.0/12 via 192.168.0.100 3. [ATTACKER] Discover open ports on the victim's internal docker networks. attacker@192.168.0.200# nmap -p5432 -Pn --open 172.16.0.0/12 Starting Nmap 7.92 ( https://nmap.org ) at 2021-11-05 15:00 CDT Nmap scan report for 172.17.0.2 Host is up (0.00047s latency). PORT STATE SERVICE 5432/tcp open postgresql 4. [ATTACKER] Connect to the victim's container. attacker@192.168.0.200# psql -h 172.17.0.2 -U postgres Password for user postgres: ## Scope of Exposure Port publishing in docker and docker-compose is a popular way to expose applications and databases to developers in a cross-platform development environment. Web searches for the pitfalls of "--publish", as well as discussions with other developers, suggest that Docker users who are aware of the security implications of port publishing also believe that specifying an IP address to bind on the host will effectively constrain access to the service they are attempting to share. This is a reasonable conclusion that can be drawn from the documentation, but the reality is that simply publishing a port exposes a container to external machines regardless of the IP address bound on the host. Github contains tens of thousands of projects that publish container ports to "127.0.0.1:xxx:xxx": * https://github.com/search?q=docker+run+%22-p+127.0.0.1%3A%22&type=code * https://github.com/search?q=docker+run+%22--publish+127.0.0.1%3A%22&type=code * https://github.com/search?p=5&q=%22127.0.0.1%3A5432%3A5432%22&type=Code * https://github.com/search?q=%22127.0.0.1%3A15432%3A5432%22&type=code * https://github.com/search?q=%22127.0.0.1%3A3306%3A3306%22&type=Code * https://github.com/search?p=5&q=%22127.0.0.1%3A8080%3A80%22&type=Code * And many more! Here is a sampling of commit messages that specifically mention the security rationale behind publishing to "127.0.0.1": https://github.com/rubyforgood/abalone/commit/764a619babc7ac05fe9fe6edc63e9128a2c86af3 > Forward the "db" service's port to the host's loopback interface, so > that a developer could choose to use docker-compose only for a container > to run the database while running all the Ruby processes on their host > computer. "127.0.0.1:5432:5432" was chosen over "5432:5432" so that the > PostgreSQL would not be available to all other computers on the host > computer's network (say, a coffee shop wifi). https://github.com/MayankTahil/pref/commit/f3056408867a227e9ff6b338c51ef37d605f5dad > [SECURITY] Limit port export to localhost > > It's prevents leak private developed projects vie Eth & Wi-Fi interfaces. > You now must use `localhost` host or use host mapped directly to 127.0.0.1 https://github.com/open-edge-insights/eii-core/commit/7a85ab8ed818af73a83489554eb5737394a4cf0c > Docker Security: Port mapping and default security options > > Changes: > > 1) Provide secuiry options in docker-compose file related to selinux and resticted privilages > 2) Set HOST_IP as Environment Variable in Compose startup > 2) Bind all ports to either 127.0.0.1 or Host IP ## Mitigation While the unintentional exposure of published container ports can be mitigated by constraining access to containers in the DOCKER-USER chain, my observation is that most Linux users do not know how to configure their firewalls and have not added any rules to DOCKER-USER. The few users that do know how to configure their firewalls are likely to be unpleasantly surprised that their existing FORWARD rules have been preceded by Docker's own forwarding setup. In light of this, an effective mitigation should: 1. Restrict the source addresses and/or interfaces that are allowed to communicate with the published container port. For example, "docker run -p 127.0.0.1:5432:5432" creates the following rule in the DOCKER chain: Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 172.17.0.2 tcp dpt:5432 It should, however, restrict the source ip address range to 127.0.0.1/8 and the in-interface to the loopback interface: Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- lo docker0 127.0.0.1/8 172.17.0.2 tcp dpt:5432 The values of "127.0.0.1/8" and "lo" can be retrieved from the interface on which 127.0.0.1 is defined. For instance, if a machine has an IP address of 192.168.0.100 on a /24 network on eth0 and the user runs "docker run -p 192.168.0.100:5432:5432", we would expect to see the following: Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- eth0 docker0 192.168.0.0/24 172.17.0.2 tcp dpt:5432 2. Default to "127.0.0.1" when a bind address is not supplied to "--publish". This is a breaking change, but it should have been the default from the beginning. ## Conclusion Docker port publishing is an *extremely* popular feature, and at present, virtually all users that use containers with published ports are exposed to attackers that have noticed the oversight outlined in this email. I have not noticed any discussion online of attackers using custom routes to gain access to containers, but it is an obvious attack, and perhaps unfortunately, I posted a comment about this vulnerability in a related Github issue: https://github.com/moby/moby/issues/22054#issuecomment-962202433 Thank you for your attention to this. Sung Pae https://github.com/guns --QR1yLfEBO/zgxYVA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEES8cqprGuK1rH963PnRqiZtK8nC0FAmGJq+IACgkQnRqiZtK8 nC3TvBAAka0sVXX4X2k8BIzVUoojrM1OkOBzAZl76cdI1Zmv4P6sp/zmkR7iE5eV lUQ57cLwnalbbn9e0QyVA2/jcuB96cx8bKL8jy+JnJ0IuQ4VUYEWGTkLORIojDRJ I8imGY83Bz4fyffoMUxG3DBeuJJCOHIHFbcoijI4xYPz2ujY3KR0vC0UYxcZLv92 bD1thh/bFXaPOPBHlVCUB9hFq1/JZ27XaH9GZ7X7TeuOp25JriU1h3U/A6gsGTkK OBOjRVJV30tDnsVZa8TBvL27JfLGyRvGACpnOhaozSvVgePERDBeMsH6bjDNzWEs mb9QIsxA6brZdJdH1uXJDM36nhG1eT3OM7jrZzI76+7FT2yzrQcKsk2Oes0t9ZCq wyZbVoZGExam2bPiWvu9XJVb9TPwKpxXpLPyiuFZrlrOaBfDV1ZqFMSBXZJxFOuu PEysiYTUpq+FufGJxH5JqWERLh79TV/f+654DG/UtOas+A7Rjy6hF9OsDXDWzpz/ lo7w3OKaXqvNZ2ysL8ihHp963fFLPkhMn2JAOBsoFa3s/hCCBwFJjxHnzF9gNRhZ cr9f3wlk6IVJMSARPJsZCD+g5uaU1gzDbndem3SlMLjkJ4D6rLoZ3zhmkddjKhsv CLBL6R7nEmuBcb4e97EVmCnYR8221uXmqvc2bQwPpeTLeGMG5BQ= =+qVS -----END PGP SIGNATURE----- --QR1yLfEBO/zgxYVA--