Skip to content

Instantly share code, notes, and snippets.

@thesamesam
Last active December 10, 2021 22:39
Show Gist options
  • Save thesamesam/a36ff15235f5cbe5004972f80f254123 to your computer and use it in GitHub Desktop.
Save thesamesam/a36ff15235f5cbe5004972f80f254123 to your computer and use it in GitHub Desktop.
Migrating a Gentoo install to libxcrypt (OBSOLETE)

This document is oboslete!

Please read the news item and wiki page instead.

Migrating a Gentoo install to libxcrypt

This is a set of instructions for migrating a Gentoo glibc system's libcrypt provider from glibc[crypt] to libxcrypt[system].

See wiki notes for background, etc.

Fedora did this ~3 years ago, so we're not expecting many problems.

How can I help?

Please share the results of: wgetpaste -c 'grep "libcrypt\.so" /var/db/pkg/*/*/NEEDED.ELF.2'

The more exotic packages installed on your system, the better. The results are still useful even if you have not migrated the system in question to libxcrypt yet.

Expected issues

  • Almost certainly just missing virtual/libcrypt dependencies in ebuilds
  • genkernel may fail to build an initramfs if libxcrypt is used?

If you hit any issues, please file a new bug blocking bug 699422 and CC sam@.

Caveats

  • We're not yet in a place where the migration is safe with FEATURES="-preserved-libs" because of missing subslot deps on consumers which need libcrypt. But this is an extremely rare configuration.

Migrating

Steps

  1. Fully update world and depclean (emerge --sync ; emerge -a -uvDU @world ; emerge -acv). This reduces the chances of conflicts and aids Portage in finding a clean path to do the rebuilds shortly. Don't skip this.
  2. Make /etc/portage changes (see below).
  3. Fully update world (if it's working, you should see a bunch of rebuilds caused by virtual/libcrypt) and depclean.

Portage config changes

$ cat /etc/portage/package.use
# Disable libcrypt in glibc
sys-libs/glibc -crypt
# Provide libcrypt
sys-libs/libxcrypt system
$ cat /etc/portage/package.accept_keywords
# Allow the new libcrypt virtual which includes libxcrypt
>=virtual/libcrypt-2
# Needed if you're on non-amd64/x86 for now (stabilisation ongoing)
# (* copies stable keywords anywhere)
sys-libs/libxcrypt *
$ cat /etc/portage/package.unmask
# Allow virtual which specifies libxcrypt
~virtual/libcrypt-2
$ cat /etc/portage/profile/package.use.mask
# Allow libxcrypt to be the system provider of libcrypt, not glibc
sys-libs/libxcrypt -system -split-usr
$ cat /etc/portage/profile/package.use.force
# Don't force glibc to provide libcrypt
sys-libs/glibc -crypt

Example output

If everything has gone well, we should see a bunch of rebuilds caused by virtual/libcrypt.

$ emerge -a -uvDU @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-crypt/openpgp-keys-libidn-20210517::gentoo  0 KiB
[ebuild     U  ] net-libs/nghttp2-1.41.0-r2:0/1.14::gentoo [1.41.0:0/1.14::gentoo] USE="threads -cxx -debug -hpack-tools -jemalloc -static-libs -test -utils -xml (-abi_riscv_lp64%) (-abi_riscv_lp64d%) (-libressl%)" 0 KiB
[ebuild     U ~] net-analyzer/openbsd-netcat-1.195-r1::gentoo [1.195::gentoo] 0 KiB
[ebuild     U  ] dev-util/source-highlight-3.1.9-r1::gentoo [3.1.9::gentoo] USE="-doc -static-libs" 0 KiB
[ebuild     U  ] net-libs/libslirp-4.6.1::gentoo [4.6.0::gentoo] USE="-static-libs" 133 KiB
[ebuild     U  ] net-vpn/tor-0.4.5.9-r1::gentoo [0.4.5.9::gentoo] USE="caps man seccomp (selinux) server verify-sig -doc -lzma -scrypt (-systemd) -test -tor-hardening -zstd" 0 KiB
[ebuild     U  ] net-misc/curl-7.77.0-r1::gentoo [7.77.0::gentoo] USE="ftp http2 imap ipv6 openssl pop3 progress-meter smtp ssl tftp threads -adns -alt-svc -brotli -gnutls -gopher -hsts -idn -kerberos -ldap -mbedtls -metalink (-nghttp3) -nss (-quiche) -rtmp -samba -ssh -sslv3 -static-libs -telnet -test (-winssl) -zstd" CURL_SSL="openssl -gnutls -mbedtls -nss (-winssl)" 0 KiB
[ebuild     U  ] sys-kernel/dracut-053-r1::gentoo [053::gentoo] USE="(selinux)" 0 KiB
[ebuild  NS   ~] sys-kernel/gentoo-kernel-bin-5.4.127-r1:5.4.127::gentoo [5.4.117-r1:5.4.117::gentoo, 5.4.118-r1:5.4.118::gentoo, 5.4.119-r1:5.4.119::gentoo, 5.4.120-r1:5.4.120::gentoo, 5.4.121-r1:5.4.121::gentoo, 5.4.123-r1:5.4.123::gentoo, 5.4.125-r1:5.4.125::gentoo] USE="initramfs -test" 55073 KiB
[ebuild  r  U ~] virtual/dist-kernel-5.4.127:0/5.4.127::gentoo [5.4.125:0/5.4.125::gentoo] 0 KiB
[ebuild     U  ] net-dns/libidn2-2.3.1:0/2::gentoo [2.3.0:0/2::gentoo] USE="verify-sig%* -static-libs (-abi_riscv_lp64%) (-abi_riscv_lp64d%)" 0 KiB
[ebuild  rR   ~] sys-fs/zfs-2.0.4-r1:0/4::gentoo  USE="dist-kernel pam rootfs (split-usr) -custom-cflags -debug (-kernel-builtin) -minimal -nls -python -static-libs (-test-suite)" PYTHON_TARGETS="python3_8 python3_9" 0 KiB
[ebuild   R    ] sys-libs/glibc-2.33:2.2::gentoo  USE="caps multiarch (selinux) ssp (static-libs) (-audit) (-cet) -compile-locales -crypt* -custom-cflags -doc -gd -headers-only (-multilib) -multilib-bootstrap -nscd -profile -static-pie -suid -systemtap -test (-vanilla)" 0 KiB
[ebuild  N    ~] sys-libs/libxcrypt-4.4.20:0/1::gentoo  USE="compat static-libs system (-split-usr) -test" 522 KiB
[ebuild  r  U #] virtual/libcrypt-2:0/2::gentoo [1-r1:0/1::gentoo] USE="static-libs (-abi_riscv_lp64%) (-abi_riscv_lp64d%)" 0 KiB
[ebuild  rR    ] sys-libs/pam-1.5.1::gentoo  USE="filecaps (selinux) (split-usr) (-audit) -berkdb -debug -nis" 0 KiB
[ebuild  rR    ] sys-apps/busybox-1.32.1-r1::gentoo  USE="ipv6 (selinux) static -debug -livecd -make-symlinks -math -mdev -pam -savedconfig -sep-usr -syslog (-systemd)" 0 KiB
[ebuild  rR    ] dev-lang/perl-5.32.1:0/5.32::gentoo  USE="gdbm -berkdb -debug -doc -ithreads -minimal" 0 KiB
[ebuild  rR    ] sys-apps/util-linux-2.36.2::gentoo  USE="caps cramfs logger ncurses pam readline (selinux) (split-usr) suid (unicode) (-audit) -build -cryptsetup -fdformat -hardlink -kill -magic% -nls -python -slang -static-libs -su (-systemd) -test -tty-helpers -udev" PYTHON_TARGETS="python3_8 python3_9 (-python3_7%)" 0 KiB
[ebuild  rR    ] sys-apps/shadow-4.8.1-r3::gentoo  USE="acl pam (selinux) (split-usr) su xattr (-audit) -bcrypt -cracklib -nls (-skey)" 0 KiB
[ebuild  rR   ~] sys-fs/zfs-kmod-2.0.4-r1:0/2.0.4-r1::gentoo  USE="dist-kernel rootfs -custom-cflags -debug" 0 KiB
[ebuild  rR    ] dev-lang/python-3.9.5_p2:3.9::gentoo  USE="gdbm hardened ipv6 ncurses readline sqlite ssl verify-sig (xml) -bluetooth -build -examples -test -tk -wininst" 0 KiB
[ebuild  rR    ] dev-lang/python-3.8.10_p2:3.8::gentoo  USE="gdbm hardened ipv6 ncurses readline sqlite ssl verify-sig (xml) -bluetooth -build -examples -test -tk -wininst" 0 KiB
[ebuild     U ~] dev-lang/python-3.10.0_beta3:3.10::gentoo [3.10.0_beta2:3.10::gentoo] USE="gdbm hardened ipv6 ncurses readline sqlite ssl verify-sig (xml) -bluetooth -build -examples -test -tk -wininst" 18213 KiB
[ebuild     U  ] net-misc/openssh-8.6_p1-r2::gentoo [8.6_p1-r1::gentoo] USE="pam pie scp (selinux) ssl -X -X509 (-audit) -bindist (-debug) -hpn -kerberos (-ldns) -libedit -livecd -sctp -security-key -static -test -xmss" 0 KiB

Total: 25 packages (12 upgrades, 2 new, 1 in new slot, 10 reinstalls), Size of downloads: 73939 KiB

The following packages are causing rebuilds:

  (virtual/libcrypt-2:0/2::gentoo, ebuild scheduled for merge) causes rebuilds for:
    (sys-apps/shadow-4.8.1-r3:0/0::gentoo, ebuild scheduled for merge)
    (dev-lang/python-3.8.10_p2:3.8/3.8::gentoo, ebuild scheduled for merge)
    (dev-lang/perl-5.32.1:0/5.32::gentoo, ebuild scheduled for merge)
    (dev-lang/python-3.9.5_p2:3.9/3.9::gentoo, ebuild scheduled for merge)
    (sys-apps/busybox-1.32.1-r1:0/0::gentoo, ebuild scheduled for merge)
    (sys-libs/pam-1.5.1:0/0::gentoo, ebuild scheduled for merge)
    (sys-apps/util-linux-2.36.2:0/0::gentoo, ebuild scheduled for merge)
  (virtual/dist-kernel-5.4.127:0/5.4.127::gentoo, ebuild scheduled for merge) causes rebuilds for:
    (sys-fs/zfs-2.0.4-r1:0/4::gentoo, ebuild scheduled for merge)
    (sys-fs/zfs-kmod-2.0.4-r1:0/2.0.4-r1::gentoo, ebuild scheduled for merge)

Troubleshooting

Nudge the virtual

In some cases, confusing conflicts have appeared.

If this happens, it's recommended to nudge the virtual first:

$ emerge -v1 "~virtual/libcrypt-2" --autounmask=n --usepkg=n

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U ~] sys-libs/glibc-2.33-r1:2.2::gentoo [2.33:2.2::gentoo] USE="multiarch ssp (static-libs) (-audit) -caps (-cet) -compile-locales -crypt* -custom-cflags -doc -gd -headers-only (-multilib) -mult
ilib-bootstrap -nscd -profile (-selinux) -static-pie -suid -systemtap -test (-vanilla)" 0 KiB
[ebuild  N    ~] sys-libs/libxcrypt-4.4.20:0/1::gentoo  USE="compat static-libs system (-split-usr) -test" 0 KiB
[ebuild  r  U #] virtual/libcrypt-2:0/2::gentoo [1-r1:0/1::gentoo] USE="static-libs" 0 KiB
[ebuild  rR    ] sys-libs/pam-1.5.1::gentoo  USE="berkdb filecaps (split-usr) (-audit) -debug -nis (-selinux)" 0 KiB
[ebuild  rR    ] sys-apps/busybox-1.32.1-r1::gentoo  USE="ipv6 static -debug -livecd -make-symlinks -math -mdev -pam -savedconfig (-selinux) -sep-usr -syslog -systemd" 0 KiB
[ebuild  rR    ] sys-apps/shadow-4.8.1-r3::gentoo  USE="acl nls pam (split-usr) su xattr (-audit) -bcrypt -cracklib (-selinux) (-skey)" 0 KiB
[ebuild  rR    ] dev-lang/perl-5.32.1:0/5.32::gentoo  USE="berkdb gdbm -debug -doc -ithreads -minimal" 0 KiB
[ebuild  rR    ] sys-apps/util-linux-2.36.2::gentoo  USE="cramfs logger ncurses nls pam readline (split-usr) suid udev (unicode) (-audit) -build -caps -cryptsetup -fdformat -hardlink -kill -magic -python (-selinux) -slang -static-libs -su -systemd -test -tty-helpers" PYTHON_TARGETS="python3_8 python3_9 (-python3_7%)" 0 KiB
[ebuild  rR    ] dev-lang/python-3.9.5_p2:3.9::gentoo  USE="bluetooth gdbm hardened ipv6 ncurses readline sqlite ssl xml -build -examples -test -tk -verify-sig -wininst" 0 KiB
[ebuild  rR    ] dev-lang/python-3.8.10_p2:3.8::gentoo  USE="bluetooth gdbm hardened ipv6 ncurses readline sqlite ssl xml -build -examples -test -tk -verify-sig -wininst" 0 KiB
[ebuild  r  U  ] net-misc/openssh-8.6_p1-r2::gentoo [8.6_p1-r1::gentoo] USE="X pam pie scp ssl -X509 (-audit) -bindist (-debug) -hpn -kerberos (-ldns) -libedit -livecd -sctp -security-key (-selinux) -static -test -xmss" 0 KiB

Total: 11 packages (3 upgrades, 1 new, 7 reinstalls), Size of downloads: 0 KiB

The following packages are causing rebuilds:

  (virtual/libcrypt-2:0/2::gentoo, ebuild scheduled for merge) causes rebuilds for:
    (sys-libs/pam-1.5.1:0/0::gentoo, ebuild scheduled for merge)
    (dev-lang/python-3.8.10_p2:3.8/3.8::gentoo, ebuild scheduled for merge)
    (dev-lang/python-3.9.5_p2:3.9/3.9::gentoo, ebuild scheduled for merge)
    (dev-lang/perl-5.32.1:0/5.32::gentoo, ebuild scheduled for merge)
    (sys-apps/shadow-4.8.1-r3:0/0::gentoo, ebuild scheduled for merge)
    (sys-apps/util-linux-2.36.2:0/0::gentoo, ebuild scheduled for merge)
    (net-misc/openssh-8.6_p1-r2:0/0::gentoo, ebuild scheduled for merge)
    (sys-apps/busybox-1.32.1-r1:0/0::gentoo, ebuild scheduled for merge)

Then complete a full world upgrade.

Quirk

I have the following EMERGE_DEFAULT_OPTS: EMERGE_DEFAULT_OPTS="--keep-going --deep --complete-graph --with-bdeps=y".

On one machine, I somehow ended up in a situation where I had no provider of libcrypt, but the virtual was still installed. The nudge with --ignore-default-opts worked, followed by a world update.

In extremis, you could force an upgrade of libxcrypt and the virtual, but this is really not recommended. Try everything else first.

static-libs conflicts

Even with the nudge, you may still get some conflicts. It's possible they're to do with USE=static-libs.

$ emerge -v1 "~virtual/libcrypt-2" --autounmask=n --usepkg=n --backtrack=9999

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] sys-libs/glibc-2.33-r1:2.2::gentoo [2.33:2.2::gentoo] USE="caps multiarch ssp (static-libs) -audit (-cet) -compile-locales -crypt* -custom-cflags -doc -gd -headers-only (-multilib) -multili
b-bootstrap -nscd -profile (-selinux) (-static-pie) -suid -systemtap -test (-vanilla)" 59 KiB
[ebuild  N     ] sys-libs/libxcrypt-4.4.22:0/1::gentoo  USE="compat static-libs system (-split-usr) -test" 524 KiB
[ebuild     U #] virtual/libcrypt-2:0/2::gentoo [1-r1:0/1::gentoo] USE="-static-libs* (-abi_riscv_lp64%) (-abi_riscv_lp64d%)" 0 KiB

Total: 3 packages (2 upgrades, 1 new), Size of downloads: 582 KiB

!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:

virtual/libcrypt:0

  (virtual/libcrypt-2:0/2::gentoo, ebuild scheduled for merge) USE="-static-libs" pulled in by
    >=virtual/libcrypt-2 (Argument)

  (virtual/libcrypt-1-r1-1:0/1::gentoo, installed) USE="static-libs -abi_riscv_lp64 -abi_riscv_lp64d" pulled in by
    virtual/libcrypt[static-libs] required by (sys-apps/busybox-1.33.1:0/0::gentoo, installed) USE="ipv6 static -debug -livecd -make-symlinks -math -mdev -pam -savedconfig (-selinux) -sep-usr -syslog -syste
md"
                     ^^^^^^^^^^^


    virtual/libcrypt:0/1= required by (dev-lang/python-3.9.5_p1:3.9/3.9::gentoo, installed) USE="gdbm ipv6 ncurses readline sqlite ssl xml (-bluetooth) -build -examples (-hardened) -test -tk -verify-sig -wi
ninst"
                    ^^^^^


    virtual/libcrypt:0/1= required by (dev-lang/python-3.10.0_beta1:3.10/3.10::gentoo, installed) USE="gdbm ipv6 ncurses readline sqlite ssl xml (-bluetooth) -build -examples (-hardened) -test -tk -verify-s
ig -wininst"
                    ^^^^^


    virtual/libcrypt:0/1= required by (net-misc/openssh-8.6_p1-r1:0/0::gentoo, installed) USE="X pam pie scp ssl -X509 -audit -bindist (-debug) (-hpn) -kerberos -ldns (-libedit) -libressl -livecd -sctp (-se
curity-key) (-selinux) -static -test -xmss"
                    ^^^^^



    =virtual/libcrypt-1-r1 required by (sys-apps/util-linux-2.37:0/0::gentoo, installed) USE="caps cramfs logger ncurses nls pam readline (split-usr) suid (unicode) -audit -build -cryptsetup -fdformat -hard
link -kill -magic -python (-selinux) -slang -static-libs -su -systemd -test -tty-helpers -udev" PYTHON_TARGETS="python3_8 python3_9"
    ^                 ^^^^




    virtual/libcrypt:0/1= required by (dev-lang/python-3.8.10_p1:3.8/3.8::gentoo, installed) USE="gdbm ipv6 ncurses readline sqlite ssl xml (-bluetooth) -build -examples (-hardened) -test -tk -verify-sig -wininst"
                    ^^^^^

    virtual/libcrypt:0/1= required by (dev-lang/python-3.8.10_p1:3.8/3.8::gentoo, installed) USE="gdbm ipv6 ncurses readline sqlite ssl xml (-bluetooth) -build -examples (-hardened) -test -tk -verify-sig -wininst"
                    ^^^^^
    (and 15 more with the same problems)

sys-libs/glibc:2.2

  (sys-libs/glibc-2.33-r1:2.2/2.2::gentoo, ebuild scheduled for merge) USE="caps multiarch ssp (static-libs) -audit (-cet) -compile-locales -crypt -custom-cflags -doc -gd -headers-only (-multilib) -multilib-bootstrap -nscd -profile (-selinux) (-static-pie) -suid -systemtap -test (-vanilla)" pulled in by
    sys-libs/glibc[-crypt(+)] required by (sys-libs/libxcrypt-4.4.22:0/1::gentoo, ebuild scheduled for merge) USE="compat static-libs system (-split-usr) -test"


  (sys-libs/glibc-2.33:2.2/2.2::gentoo, installed) USE="caps crypt multiarch ssp (static-libs) -audit (-cet) -compile-locales -custom-cflags -doc -gd -headers-only (-multilib) -multilib-bootstrap -nscd -profile (-selinux) (-static-pie) -suid -systemtap -test (-vanilla)" pulled in by
    sys-libs/glibc[crypt(+),static-libs(+)?] required by (virtual/libcrypt-1-r1-1:0/1::gentoo, installed) USE="static-libs -abi_riscv_lp64 -abi_riscv_lp64d"


NOTE: Use the '--verbose-conflicts' option to display parents omitted above

!!! The slot conflict(s) shown above involve package(s) which may need to
!!! be rebuilt in order to solve the conflict(s). However, the following
!!! package(s) cannot be rebuilt for the reason(s) shown:

  (dev-lang/python-3.8.10_p1:3.8/3.8::gentoo, installed): ebuild is masked or unavailable
  (dev-lang/python-3.10.0_beta1:3.10/3.10::gentoo, installed): ebuild is masked or unavailable
  (dev-lang/python-3.9.5_p1:3.9/3.9::gentoo, installed): ebuild is masked or unavailable
  (net-misc/openssh-8.6_p1-r1:0/0::gentoo, installed): ebuild is masked or unavailable


It may be possible to solve this problem by using package.mask to
prevent one of those packages from being selected. However, it is also
possible that conflicting dependencies exist such that they are
impossible to satisfy simultaneously.  If such a conflict exists in
the dependencies of two different packages, then those packages can
not be installed simultaneously.

For more information, see MASKED PACKAGES section in the emerge man
page or refer to the Gentoo Handbook.

Make sure it is consistently enabled on both libxcrypt and virtual/libcrypt if a package e.g. util-linux or busybox requires it:

$ cat /etc/portage/package.use
# Disable libcrypt in glibc
sys-libs/glibc -crypt
# Provide libcrypt
sys-libs/libxcrypt system static-libs
virtual/libcrypt static-libs

Then try the nudge again, then a full world upgrade.

Copy link

ghost commented Sep 17, 2021

Thank you for this tutorial, it saved my sanity

But I also had additionally to mask libcrypt-1 to solve cellular dependency issue

echo '<virtual/libcrypt-2' >> /etc/portage/package.mask

https://dpaste.com/A5G4NVEE7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment