update and install too slow (raspi 3 b)

[acconciox]

hi, lately installation or update is too slow, not the material download, after download i check with htop there is not activity relevant on cpu, but the pacmanager spend a lot of time to ultimate update or installation and the desktop seems frozen.

$this->bbcode_second_pass_code('', ':: Processing package changes...
(1/1) installing gnome-keyring [######################] 100%
:: Running post-transaction hooks...
(1/2) Compiling GSettings XML schema files...
(2/2) Updating manpage index...
')
Is there some command i can do to make order in packages or to speed up package manager?
14 G free / 28 G
at the moment with openbox and file manager+firefox+lxterminal/htop 241M Mem used (htop) so there is some bottle neck or some clean i have to do?

ciao e grazie

[sdrider]

This is new behavior in the last month or so. I keep checking Arch forums and here for someone else to have asked and solved the issue. But I guess it's really only an issue on low-power ARM boxes and therefore not even noticed by the general Arch community.

The issue is that about a month back (maybe a little more?), pacman started running manpage indexes as the last part of the install. Or, if it WAS doing this before, I never noticed it and now it's doing something that takes an extremely long time to process. This is a cpu-bound process, there is very little sdcard I/O while this process is going on.

Typically updating packages is something I do in the background and no big deal if it takes a little while, but when just trying to install a quick package or two that I need, this man page update takes forever. Perhaps the biggest complaint I have is the speed of pacman is one of the things I have *LOVED* about Arch since Day 1. Every other distro, installing just a package or two is a 5-30 second affair. On Arch, after I hit <enter> to install a package or two, I sometimes get the prompt back in < 1 second. On low-power ARM hardware no less. Amazing! Or.. it used to be.

Now, that seems to be gone and that makes me sad. I wish there was some way to say "don't update the man page right now" and just have it deferred to some nightly cron run instead. I just tested installing the tiny 'bc' package. Even on my lowest powered pi, the install completes in < 1 second and it then spends the next 7 seconds on 'updating man page' and 'updating info directory'. I would love to know how to turn off these post-install updates and/or defer them to some regularly scheduled time.

[modul]

I noticed this too. The Pi 3 here is a little bit unusable while updating the manpage index.

[sdrider]

So I did some digging through pacman changelogs. It appears this was added in v5 as a feature of pacman hooks.

The wiki states hooks are in /usr/lib/pacman/hooks or /etc/pacman.d/hooks, the latter being highest priority so it can be used to override default hooks. However, there is no /usr/lib/pacman/hooks on any of my alarm installs.

So I ran a pacman install in debug mode and it showed this:

$this->bbcode_second_pass_code('', 'debug: parsing hook file /usr/share/libalpm/hooks/texinfo-remove.hook
debug: parsing hook file /usr/share/libalpm/hooks/texinfo-install.hook
debug: parsing hook file /usr/share/libalpm/hooks/gio-querymodules.hook
debug: parsing hook file /usr/share/libalpm/hooks/update-ca-trust.hook
debug: parsing hook file /usr/share/libalpm/hooks/udev-hwdb.hook
debug: parsing hook file /usr/share/libalpm/hooks/man-db.hook
debug: parsing hook file /usr/share/libalpm/hooks/man-db-purge.hook
debug: parsing hook file /usr/share/libalpm/hooks/glib-compile-schemes.hook
:: Running post-transaction hooks...
(1/2) Updating manpage index...')

Following the wiki's suggestion, I linked the man page and texinfo ones to /dev/null:

$this->bbcode_second_pass_code('', '> sudo mkdir /etc/pacman.d/hooks
> sudo ln -s /dev/null /etc/pacman.d/hooks/man-db.hook
> sudo ln -s /dev/null /etc/pacman.d/hooks/texinfo-install.hook')

After doing this, in debug mode you can see the slow hook transactions being skipped:

$this->bbcode_second_pass_code('', 'debug: parsing hook file /etc/pacman.d/hooks/texinfo-install.hook
debug: parsing hook file /etc/pacman.d/hooks/man-db.hook
debug: parsing hook file /usr/share/libalpm/hooks/texinfo-remove.hook
debug: skipping overridden hook /usr/share/libalpm/hooks/texinfo-install.hook
debug: parsing hook file /usr/share/libalpm/hooks/gio-querymodules.hook
debug: parsing hook file /usr/share/libalpm/hooks/update-ca-trust.hook
debug: parsing hook file /usr/share/libalpm/hooks/udev-hwdb.hook
debug: skipping overridden hook /usr/share/libalpm/hooks/man-db.hook
debug: parsing hook file /usr/share/libalpm/hooks/man-db-purge.hook
debug: parsing hook file /usr/share/libalpm/hooks/glib-compile-schemes.hook')

And now my pacman installs run full speed again! I'm sure my man page index will be out of date as I continue to install software, so at some point I'll need to setup a systemd timer to run man page updates in the wee hours. I'll work on that later.

[acconciox]

$this->bbcode_second_pass_code('', ' > sudo ln -s /dev/null /etc/pacman.d/hooks/man-db.hook
> sudo ln -s /dev/null /etc/pacman.d/hooks/texinfo-install.hook
')
i have to create

$this->bbcode_second_pass_code('', 'mkdir /etc/pacman.d/hooks')

first ? otherwise it does not fit the commands ln -s ...

[sdrider]

Yes, sorry. You do have to create the directory first. I did and just forgot to include it in the commands I listed. I've edited my previous message to include the extra command, thanks.

[sdrider]

To add to this for those who might find this thread later:

Here's how you setup systemd timers to put the manpage and info indexers on a scheduled update, so you get the immediate performance boost of not having to run them with pacman, but still keep them up to date daily. I chose to run them at 1:30 and 2:30am local time, pick whatever time you want. Or apparently you can not pick any time at all and just use "daily" and systemd will pick a time for you. I much preferred to specifically set a late night run so it doesn't kick up randomly in the middle of the day when I might be using the system.

Note you could do this with cron just as easily, I used systemd since it's already in every Arch install and doesn't require an additional cron package.

$this->bbcode_second_pass_code('', '> pwd
/etc/systemd/system
> more *-index*
::::::::::::::
manpage-index.service
::::::::::::::
[Unit]
Description=Run manpage indexer

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'mkdir -p /var/cache/man; exec mandb --quiet --no-purge'
Nice=19
IOSchedulingClass=best-effort
IOSchedulingPriority=7
::::::::::::::
manpage-index.timer
::::::::::::::
[Unit]
Description=Run manpage indexer daily

[Timer]
OnCalendar=*-*-* 01:30:00
Persistent=true
::::::::::::::
texinfo-index.service
::::::::::::::
[Unit]
Description=Run texinfo updater

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'for f in /usr/share/info/*.gz; do install-info "$f" /usr/share/info/dir 2> /dev/null; done'
Nice=19
IOSchedulingClass=best-effort
IOSchedulingPriority=7
::::::::::::::
texinfo-index.timer
::::::::::::::
[Unit]
Description=Run texinfo updater daily

[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true
')

After these files are in place, start the timers:

$this->bbcode_second_pass_code('', '# systemctl daemon-reload
# systemctl start manpage-index.timer
# systemctl start texinfo-index.timer')

and you can confirm they are scheduled by running 'list-timers'. Example output:

$this->bbcode_second_pass_code('', '> systemctl list-timers
NEXT LEFT LAST PASSED UNIT ACTIVATES
Mon 2016-06-20 23:05:43 PDT 2h 0min left Sun 2016-06-19 23:05:43 PDT 21h ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Tue 2016-06-21 00:00:00 PDT 2h 54min left Mon 2016-06-20 00:00:33 PDT 21h ago logrotate.timer logrotate.service
Tue 2016-06-21 00:00:00 PDT 2h 54min left Mon 2016-06-20 00:00:33 PDT 21h ago shadow.timer shadow.service
Tue 2016-06-21 01:30:00 PDT 4h 24min left Mon 2016-06-20 01:30:33 PDT 19h ago manpage-index.timer manpage-index.service
Tue 2016-06-21 02:30:00 PDT 5h 24min left Mon 2016-06-20 02:30:33 PDT 18h ago texinfo-index.timer texinfo-index.service

5 timers listed.')

[acconciox]

thanks you sdrider i think your post is the best possible we can do elegant and efficient.

[acconciox]

i need a confirmation about this, if i remove the internet connection after the download of packages by pacman, the man page update and generally the update is faster, can you check and tell me if this is true or just on my ppp0 connection'?

ciao & thanks

[graysky]

Great post and idea.

https://aur.archlinux.org/packages/mandb-ondemand