Adding PPA update to libstdc++6

wdurocher · October 22, 2018, 5:27pm

We are building a custom Ubuntu 16.04 image on a BeableBone black derived project (as in our own board). We have used omap-image-builder-v2018.01 as the base for the distribution. It compiles fine and we habe been using the image since the beginning of the year for development.

Recently, we added some logging using spdlog. However, it requires an update to libstdc++6 as the version in the xenial repos is not at the same version as that required by spdlog. We can manually update the library doing the following:

###Needed for add-apt-repository
sudo apt install -y software-properties-common

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update

Recommended by source but not something we want to do.

#sudo apt-get upgrade

Instead we do this

sudo apt install libstdc++6

The procdure above works on our target. However, we want to add this to the image. I tried modifying the chroot.sh script in the scripts folder doing this:

if [ "x${enable_ppa_launchpad}" = "xenable" ] ; then
    ## this one comes for the internet, not the proxy !!!
    apt-get -y \${apt_options} install software-properties-common || true

    echo "Log: (chroot) add-apt-repository -y ppa:ubuntu-toolchain-r/test"
    add-apt-repository -y ppa:ubuntu-toolchain-r/test
    #Disable by RobertCNelson
    #patch_launchpad_sources_list

    ## Only update the PPA sources
    apt-get update -o Dir::Etc::sourcelist="sources.list.d/ubuntu-toolchain-r-ubuntu-test-xenial.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"

    if [ ! "x${ppa_launchpad_pkg_list}" = "x" ] ; then
        echo "Log: (chroot) Installing (from ppa.launchpad.net): ${ppa_launchpad_pkg_list}"
        #Missing install.... by RobertCNelson
        apt-get -y install ${ppa_launchpad_pkg_list}
    fi
fi

in the config file we have:

enable_ppa_launchpad="enable"
ppa_launchpad_pkg_list=" \
    libstdc++6 \
"

The build works up to the apt-get -y install libstdc++6 which fails to find the package !

Any help would be appreciated !

RobertCNelson · October 22, 2018, 5:44pm

@wdurocher, building now, lets see what happens…

edit, What’s the apt source.list for ppa:ubuntu-toolchain-r/test?

I don’t have an ubuntu based linux box in the lab…

edit 2:

i think it’s:

deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial main

Regards,

wdurocher · October 22, 2018, 5:50pm

The add-apt-ropository command adds the file /etc/apt/sources.list.d/ubuntu-toolchain-r-ubuntu-test-xenial.list
Which contains:
deb Index of /ubuntu-toolchain-r/test/ubuntu xenial main

RobertCNelson · October 22, 2018, 5:52pm

Correct it does, i was trying to deduce what you were doing in this bash call:

patch_launchpad_sources_list

Regards,

wdurocher · October 22, 2018, 5:53pm

That was a debugging step I forgot to remove where I was manually creating the file ubuntu-toolchain-r-ubuntu-test-xenial.list (This did not work)

RobertCNelson · October 22, 2018, 6:16pm

Okay this is where it currently bombs…

Get:1 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease [17.5 kB]
Get:2 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages [60.8 kB]
Fetched 78.3 kB in 2s (37.9 kB/s)  
Reading package lists... Done
Log: (chroot) Installing (from ppa.launchpad.net):      libstdc++6 
E: Command line option 'y' [from -y] is not understood in combination with the other options.
Log: umount: [/opt/github/omap-image-builder/ignore/tmp.PAfe7ay95V/dev/pts]
Log: umount: [/opt/github/omap-image-builder/ignore/tmp.PAfe7ay95V/proc]
Log: umount: [/opt/github/omap-image-builder/ignore/tmp.PAfe7ay95V/sys]
Log: umount: [/opt/github/omap-image-builder/ignore/tmp.PAfe7ay95V/run]

Your original post didn’t have an install, so i edited with that piece…

wdurocher · October 22, 2018, 7:05pm

Sorry for the delay. Meeting oblige !

I had an issue with apt_options for xenial being set to :
apt_options="–allow-downgrades --allow-remove-essential --allow-change-held-packages"

I removed that and used the flag -y instead. But I see that I missed something in the line it should have been:

apt-get -y install ${ppa_launchpad_pkg_list}

wdurocher · October 22, 2018, 7:09pm

I will try also on my side but it takes a while.

RobertCNelson · October 22, 2018, 7:28pm

Okay, it works for me now:

Log: (chroot) add-apt-repository -y ppa:ubuntu-toolchain-r/test
gpg: keyring `/tmp/tmpna61yach/secring.gpg' created
gpg: keyring `/tmp/tmpna61yach/pubring.gpg' created
gpg: requesting key BA9EF27F from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpna61yach/trustdb.gpg: trustdb created
gpg: key BA9EF27F: public key "Launchpad Toolchain builds" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
Get:1 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease [17.5 kB]
Get:2 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages [60.8 kB]
Fetched 78.3 kB in 2s (35.3 kB/s)  
Reading package lists... Done
Log: (chroot) Installing (from ppa.launchpad.net): 		libstdc++6	
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  gcc-8-base
The following NEW packages will be installed:
  gcc-8-base
The following packages will be upgraded:
  libstdc++6
1 upgraded, 1 newly installed, 0 to remove and 13 not upgraded.
Need to get 376 kB of archives.
After this operation, 155 kB of additional disk space will be used.
Get:1 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf gcc-8-base armhf 8.1.0-5ubuntu1~16.04 [18.5 kB]
Get:2 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf libstdc++6 armhf 8.1.0-5ubuntu1~16.04 [357 kB]
Fetched 376 kB in 1s (323 kB/s)    
Selecting previously unselected package gcc-8-base:armhf.
(Reading database ... 31451 files and directories currently installed.)
Preparing to unpack .../gcc-8-base_8.1.0-5ubuntu1~16.04_armhf.deb ...
Unpacking gcc-8-base:armhf (8.1.0-5ubuntu1~16.04) ...
Setting up gcc-8-base:armhf (8.1.0-5ubuntu1~16.04) ...
(Reading database ... 31457 files and directories currently installed.)
Preparing to unpack .../libstdc++6_8.1.0-5ubuntu1~16.04_armhf.deb ...
Unpacking libstdc++6:armhf (8.1.0-5ubuntu1~16.04) over (5.4.0-6ubuntu1~16.04.10) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...
Setting up libstdc++6:armhf (8.1.0-5ubuntu1~16.04) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...

Here is the full diff, i no longer have “xenial” support in the git repo, so there was a few things to add back apt_options…

diff --git a/scripts/chroot.sh b/scripts/chroot.sh
index d1590c2f..63916e08 100755
--- a/scripts/chroot.sh
+++ b/scripts/chroot.sh
@@ -562,6 +562,12 @@ cat > "${DIR}/chroot_script.sh" <<-__EOF__
 			dpkg-divert --local --rename --add /sbin/initctl
 			ln -s /bin/true /sbin/initctl
 		fi
+
+		apt_options=""
+		if [ "x\${deb_codename}" = "xxenial" ] ; then
+			apt_options="--allow-downgrades --allow-remove-essential --allow-change-held-packages"
+		fi
+		echo "Log: (chroot): apt using extra: [\${apt_options}]"
 	}
 
 	install_pkg_updates () {
@@ -594,8 +600,8 @@ cat > "${DIR}/chroot_script.sh" <<-__EOF__
 		fi
 
 		apt-get update
-		apt-get upgrade -y
-		apt-get dist-upgrade -y
+		apt-get upgrade -y \${apt_options}
+		apt-get dist-upgrade -y \${apt_options}
 
 		if [ ! -f /etc/resolv.conf ] ; then
 			#'/etc/resolv.conf.bak' -> '/etc/resolv.conf'
@@ -647,7 +653,7 @@ cat > "${DIR}/chroot_script.sh" <<-__EOF__
 			#Install the user choosen list.
 			echo "Log: (chroot) Installing: ${deb_additional_pkgs}"
 			apt-get update
-			apt-get -y install ${deb_additional_pkgs}
+			apt-get -y \${apt_options} install ${deb_additional_pkgs}
 		fi
 
 		if [ "x${chroot_enable_debian_backports}" = "xenable" ] ; then
@@ -659,7 +665,7 @@ cat > "${DIR}/chroot_script.sh" <<-__EOF__
 
 		if [ ! "x${repo_external_pkg_list}" = "x" ] ; then
 			echo "Log: (chroot) Installing (from external repo): ${repo_external_pkg_list}"
-			apt-get -y install ${repo_external_pkg_list}
+			apt-get -y  \${apt_options} install ${repo_external_pkg_list}
 		fi
 
 		if [ ! "x${repo_ros_pkg_list}" = "x" ] ; then
@@ -674,12 +680,28 @@ cat > "${DIR}/chroot_script.sh" <<-__EOF__
 			apt-mark hold chromium || true
 		fi
 
+		if [ "x${enable_ppa_launchpad}" = "xenable" ] ; then
+			## this one comes for the internet, not the proxy !!!
+			apt-get -y \${apt_options} install software-properties-common || true
+
+			echo "Log: (chroot) add-apt-repository -y ppa:ubuntu-toolchain-r/test"
+			add-apt-repository -y ppa:ubuntu-toolchain-r/test
+
+			## Only update the PPA sources
+			apt-get update -o Dir::Etc::sourcelist="sources.list.d/ubuntu-toolchain-r-ubuntu-test-xenial.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"
+
+			if [ ! "x${ppa_launchpad_pkg_list}" = "x" ] ; then
+				echo "Log: (chroot) Installing (from ppa.launchpad.net): ${ppa_launchpad_pkg_list}"
+				apt-get -y \${apt_options} install ${ppa_launchpad_pkg_list} || true
+			fi
+		fi
+
 		##Install last...
 		if [ ! "x${repo_rcnee_pkg_version}" = "x" ] ; then
 			echo "Log: (chroot) Installing modules for: ${repo_rcnee_pkg_version}"
-			apt-get -y install libpruio-modules-${repo_rcnee_pkg_version} || true
-			apt-get -y install rtl8723bu-modules-${repo_rcnee_pkg_version} || true
-			apt-get -y install ti-cmem-modules-${repo_rcnee_pkg_version} || true
+			apt-get -y \${apt_options} install libpruio-modules-${repo_rcnee_pkg_version} || true
+			apt-get -y \${apt_options} install rtl8723bu-modules-${repo_rcnee_pkg_version} || true
+			apt-get -y \${apt_options} install ti-cmem-modules-${repo_rcnee_pkg_version} || true
 			depmod -a ${repo_rcnee_pkg_version}
 			update-initramfs -u -k ${repo_rcnee_pkg_version}
 		fi

Regards,

wdurocher · October 22, 2018, 7:55pm

So to resume, missing install and repo_rcnee_pkg_version must be last.

Your base seems different from mine.

Did you add the changes to install_pkgs_update () ?
But mine was in install_pkgs ()

I will grab the latest version of chroot in https://rcn-ee.com/rootfs/eewiki/minfs/ubuntu-18.04.1-minimal-armhf-2018-07-30.tar.xz and apply the patch to it. So that I cab make the correct comparison and backport the changes to my version !

But since yours work, I will stick as closely as possible to yours. I will give you my results as soon as I have done it !

RobertCNelson · October 22, 2018, 8:00pm

Correct, that was from git master https://github.com/RobertCNelson/omap-image-builder. I added the call into install_pkgs_updates, vs install_pkgs, as to minimize any issues with a newer libc when installing packages built against the old library.

Regards,

wdurocher · October 23, 2018, 2:10pm

Still no go.

The repo at ppa.launchpad is not providing the file Packages for armhf. Which then makes it impossible to get the libstdc++6 package. Since the same commands work on the target and libstdc++6 is pulled it then it’s not a case of ppa.launchpad not having the file. It must be something with my build setup. Also since your version works it points to other differences in my build process. Here is the error log:

[Note I had to break the links below to publish this. Not allowed to put more than 2 links, all the htp were actually http]

gpg: keyring `/tmp/tmpu4lgddej/pubring.gpg’ created
gpg: requesting key BA9EF27F from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpu4lgddej/trustdb.gpg: trustdb created
gpg: key BA9EF27F: public key “Launchpad Toolchain builds” imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK
Ign:1 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease
Ign:2 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial Release
Ign:3 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages
Ign:4 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main all Packages
Ign:5 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en
Ign:3 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages
Ign:4 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main all Packages
Ign:5 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en
Ign:3 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages
Ign:4 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main all Packages
Ign:5 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en
Ign:3 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages
Ign:4 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main all Packages
Ign:5 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en
Ign:3 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages
Ign:4 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main all Packages
Ign:5 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en
Err:3 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main armhf Packages
404 Not Found
Ign:4 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main all Packages
Ign:5 htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en
Reading package lists… Done
W: Can’t drop privileges for downloading as file ‘/var/lib/apt/lists/partial/ppa.launchpad.net_ubuntu-toolchain-r_test_ubuntu_dists_xenial_InRelease’ couldn’t be accessed by user ‘_apt’. - pkgAcquire::Run (13: Permission denied)
W: The repository ‘htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial Release’ does not have a Release file.
N: Data from such a repository can’t be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch htp://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu/dists/xenial/main/binary-armhf/Packages 404 Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
Log: umount: [/home/wdurocher/ArmLinux/LF102/Distro/Ubuntu/omap-image-builder-v2018.01/ignore/tmp.psg89YB6Mm/dev/pts]
Log: umount: [/home/wdurocher/ArmLinux/LF102/Distro/Ubuntu/omap-image-builder-v2018.01/ignore/tmp.psg89YB6Mm/proc]
Log: umount: [/home/wdurocher/ArmLinux/LF102/Distro/Ubuntu/omap-image-builder-v2018.01/ignore/tmp.psg89YB6Mm/sys]
Log: umount: [/home/wdurocher/ArmLinux/LF102/Distro/Ubuntu/omap-image-builder-v2018.01/ignore/tmp.psg89YB6Mm/run]

Here is what I think may be an issue:

I am using a proxy (mirror of sources) for the regular build (we want to freeze the packages we use to build to a specific version). It is still possible to build from the internet though.

wdurocher · October 23, 2018, 2:43pm

Indeed, the use of the proxy is preventing ppa.launchpad from being reached. When I build from the internet sources the build proceeds to the endI will have to fix that issue.

But in the mean time the problem I initially opened has been fixed

RobertCNelson · October 23, 2018, 2:43pm

What proxy mirror are you using? I use apt-cacher-ng from sid, on my local deb server, which was used in my previous run.

Regards,

wdurocher · October 23, 2018, 2:50pm

I am using apt-mirror and created a mirror of:
ports. ubuntu. com xenial and xenial-updates in main, universe and multiverse
As well as repos. rcn-ee. com/ubuntu xenial main

I did have to do some dancing to get the script to access the repos as
http://192.168.1.228/[Actual repo URL]
The apt_proxy was not working otherwise.

RobertCNelson · October 23, 2018, 3:03pm

Try patching your sources.list with “deb [trusted=yes] http…” then it should bypass the keyring during the chroot run…

Regards,

wdurocher · October 23, 2018, 3:04pm

I will do that !

wdurocher · October 23, 2018, 3:08pm

Do I also need to do that for the file in sources.list.d ?