Randomly built binary search trees

In Introduction to Algorithms (CLRS) 3rd Edition, page 299, the section attempts to prove:

The expected height of a randomly built binary search tree on $ n$ distinct keys is $ O(\lg n)$ .

We define “randomly built binary search tree on $ n$ keys” as:

a binary search tree that arises from inserting the keys in random order into an initially empty tree, where each of the $ n!$ permutations of the input keys is equally likely.

In the proof, we defined some random variables:

Let $ X_n$ denotes the height of a randomly built binary search tree on $ n$ keys and the exponential height $ Y_n = 2^{X_n}$ . Of the $ n$ keys, we choose one key as the root of the tree, and we let $ R_n$ denotes the random variable that holds the position that this key would occupy if the set of keys were sorted (also know as ‘rank’ in the text). If we know that $ R_n=i$ , it follows that $ Y_n=2\cdot max(Y_{i-1},Y_{n-1})$ .

We also define indicator random variables $ Z_{n,1}, Z_{n,2}, …, Z_{n,n}$ , $ Z_{n,i} = I\{R_n=i\}$ .

\begin{equation} \begin{split} E[Y_n] & = E\Big[\sum_{i=1}^{n} Z_{n,i} (2\cdot max(Y_{i-1}, Y_{n-i}))\Big] \ & = \sum_{i=1}^{n} E[Z_{n,i} (2\cdot max(Y_{i-1}, Y_{n-i}))] && \text{(by linearity of expectation)}\ & = \sum_{i=1}^{n} E[Z_{n,i}] E[(2\cdot max(Y_{i-1}, Y_{n-i}))] && \text{(by independence)}\ \end{split} \end{equation}

I would like to ask why is the last equality correct? In other words, why can we assume independence of $ Z_{n,i}$ and $ max(Y_{i-1},Y_{n-i})$ ?

In the proof, there was a brief explanation:

Having chosen $ R_n = i$ , the left subtree (whose ranks are less than $ i$ . This subtree is just like any other randomly built binary search tree on $ i-1$ keys. Other than the number of keys it contains, this subtree’s structure is not affected at all by the choice of $ R_n=i$ , and hence the random variables $ Y_{i-1}$ and $ Z_{n,i}$ are independent. Likewise, the right subtree, whose exponential height is $ Y_{n-i}$ , is randomly built on the $ n-i$ keys whose ranks are greater than $ i$ .

However, since $ R_n$ does affect the number of keys $ Y_{i-1}$ and $ Y_{n-i}$ contain (as acknowledged by the explanation above), wouldn’t it mean that $ Y_{i-1}$ and $ Y_{n-i}$ are dependent on $ Z_{n,i}$ ?

A note regarding similar questions posted on CS stackexchange

I have read the answers for the following questions which are very similar to mine:

  1. Proof that a randomly built binary search tree has logarithmic height
  2. Average height of a BST with n Nodes.
  3. Randomized BST height analysis : How $ Z_{n,i}$ and $ Y_{k-1}$ are independent?

For 1 & 2, the question was a more general one which asked for an explanation for the entire proof. Also, the answer for both questions did not attempt to justify the independence and simply used the result.

For 3, the question was based on a MIT lecture, https://www.youtube.com/watch?v=vgELyZ9LXX4 and it lacked some of the details which I have included above. Also, the answer was also not clear.

Returning from lock randomly takes too long in 18.04

When unlocking the computer it sometimes takes too long (1:30 minutes) to arrive to the desktop. This is happening more than one time a day.

Setup:

  • Ubuntu 18.04 with Unity 7 + Lightdm (but issue can be reproduced with Gnome3 with Gdm3 too)
  • uname -a: Linux adamhostname 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
  • Login is through Active Directory using Kerberos and Sssd

Logs of journalctl -f when login is fast (Login took place from 12:21:46 to 12:21:49 – 3 seconds)

jul 04 12:19:52 adamhostname systemd[2263]: Started Backing Service for the Unity Panel in Lockscreen mode. jul 04 12:19:52 adamhostname systemd[2263]: Reached target A target that, when running, represents the screen being locked. jul 04 12:20:05 adamhostname gnome-session[2579]: gnome-session-binary[2579]: WARNING: Could not get session path for session. Check that logind is properly installed and pam_systemd is getting used at login. jul 04 12:20:05 adamhostname gnome-session-binary[2579]: WARNING: Could not get session path for session. Check that logind is properly installed and pam_systemd is getting used at login. jul 04 12:20:05 adamhostname unity-settings-[2556]: failed to turn the kbd backlight off: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface 'org.freedesktop.UPower.KbdBacklight' on object at path /org/freedesktop/UPower/KbdBacklight jul 04 12:21:46 adamhostname kernel: [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS. jul 04 12:21:46 adamhostname gnome-session[2579]: gnome-session-binary[2579]: WARNING: Could not get session path for session. Check that logind is properly installed and pam_systemd is getting used at login. jul 04 12:21:46 adamhostname gnome-session-binary[2579]: WARNING: Could not get session path for session. Check that logind is properly installed and pam_systemd is getting used at login. jul 04 12:21:48 adamhostname compiz[2673]: pam_unix(unity:auth): authentication failure; logname= uid=1099222340 euid=1099222340 tty= ruser= rhost=  user=adamn jul 04 12:21:48 adamhostname audit[24081]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/systemd/users/1099222340" pid=24081 comm="krb5_child" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.330:8086): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/systemd/users/1099222340" pid=24081 comm="krb5_child" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24081]: AVC apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/krb5/plugins/authdata/sssd_pac_plugin.so" pid=24081 comm="krb5_child" requested_mask="m" denied_mask="m" fsuid=1099222340 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.354:8087): apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/krb5/plugins/authdata/sssd_pac_plugin.so" pid=24081 comm="krb5_child" requested_mask="m" denied_mask="m" fsuid=1099222340 ouid=0 jul 04 12:21:48 adamhostname compiz[2673]: pam_sss(unity:auth): authentication success; logname= uid=1099222340 euid=1099222340 tty= ruser= rhost= user=adamn jul 04 12:21:48 adamhostname compiz[2673]: gkr-pam: unlocked login keyring jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=24083 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=24083 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.466:8088): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=24083 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.466:8089): apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=24083 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.470:8090): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.470:8091): apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.470:8092): apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.470:8093): apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=24083 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/samba/gensec/krb5.so" pid=24083 comm="gpo_child" requested_mask="m" denied_mask="m" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.482:8094): apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/samba/gensec/krb5.so" pid=24083 comm="gpo_child" requested_mask="m" denied_mask="m" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 12:21:48 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 12:21:48 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 12:21:48 adamhostname kernel: audit: type=1400 audit(1562235708.506:8095): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/dev/urandom" pid=24083 comm="gpo_child" requested_mask="wc" denied_mask="wc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/dev/urandom" pid=24083 comm="gpo_child" requested_mask="wc" denied_mask="wc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="mknod" profile="/usr/sbin/sssd" name="/var/lib/sss/gpo_cache/domain.local/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INIRNrUek" pid=24083 comm="gpo_child" requested_mask="c" denied_mask="c" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/var/lib/sss/gpo_cache/domain.local/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INIRNrUek" pid=24083 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="chmod" profile="/usr/sbin/sssd" name="/var/lib/sss/gpo_cache/domain.local/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INIRNrUek" pid=24083 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="rename_src" profile="/usr/sbin/sssd" name="/var/lib/sss/gpo_cache/domain.local/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INIRNrUek" pid=24083 comm="gpo_child" requested_mask="wrd" denied_mask="wrd" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="rename_dest" profile="/usr/sbin/sssd" name="/var/lib/sss/gpo_cache/domain.local/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INI" pid=24083 comm="gpo_child" requested_mask="wc" denied_mask="wc" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[24083]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/var/lib/sss/gpo_cache/domain.local/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INI" pid=24083 comm="gpo_child" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 jul 04 12:21:48 adamhostname audit[1614]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name=2F7661722F6C69622F7373732F67706F5F63616368652F7469732E6C6F63616C2F506F6C69636965732F7B33314232463334302D303136442D313144322D393435462D3030433034464239383446397D2F4D616368696E652F4D6963726F736F66742F57696E646F7773204E542F536563456469742F477074546D706C2E696E66 pid=1614 comm="sssd_be" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 jul 04 12:21:49 adamhostname unity-panel-ser[2681]: menus_destroyed: assertion 'IS_WINDOW_MENU(wm)' failed jul 04 12:21:49 adamhostname unity-panel-ser[2681]: menus_destroyed: assertion 'IS_WINDOW_MENU(wm)' failed jul 04 12:21:49 adamhostname compiz[2673]: WARN  2019-07-04 12:21:49 unity.glib.dbus.proxy GLibDBusProxy.cpp:487 Calling method "EmitEvent" on object path: "/com/ubuntu/Upstart" failed: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name com.canonical.Unity.Test.Upstart was not provided by any .service files jul 04 12:21:49 adamhostname systemd[2263]: Stopped target A target that, when running, represents the screen being locked. jul 04 12:21:49 adamhostname systemd[2263]: Stopping Backing Service for the Unity Panel in Lockscreen mode... jul 04 12:21:51 adamhostname unity-panel-ser[24048]: Source ID 4294967295 was not found when attempting to remove it jul 04 12:21:51 adamhostname systemd[2263]: Stopped Backing Service for the Unity Panel in Lockscreen mode. jul 04 12:21:51 adamhostname compiz[2673]: ERROR 2019-07-04 12:21:51 unity.decoration.shape DecorationsShape.cpp:73 Failed to get shape rectangles jul 04 12:21:51 adamhostname compiz[2673]: ERROR 2019-07-04 12:21:51 unity.decoration.shape DecorationsShape.cpp:73 Failed to get shape rectangles ... 

Logs of journalctl -f when login is slow (Login credentials entered at: 14:31:38 Actual login succeeded at: 14:33:06 – a bit more than 1 minute 30 seconds)

... jul 04 14:17:01 adamhostname CRON[26960]: pam_unix(cron:session): session opened for user root by (uid=0) jul 04 14:17:01 adamhostname CRON[26961]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly) jul 04 14:17:01 adamhostname CRON[26960]: pam_unix(cron:session): session closed for user root jul 04 14:20:42 adamhostname kernel: perf: interrupt took too long (4011 > 3997), lowering kernel.perf_event_max_sample_rate to 49750 jul 04 14:22:09 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:22:09 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:22:09 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:22:09 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:29:45 adamhostname kernel: CIFS VFS: Free previous auth_key.response = 00000000d051e350 jul 04 14:31:32 adamhostname kernel: usb 1-4: new high-speed USB device number 10 using xhci_hcd jul 04 14:31:32 adamhostname kernel: usb 1-4: New USB device found, idVendor=05ac, idProduct=12a8 jul 04 14:31:32 adamhostname kernel: usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 jul 04 14:31:32 adamhostname kernel: usb 1-4: Product: iPhone jul 04 14:31:32 adamhostname kernel: usb 1-4: Manufacturer: Apple Inc. jul 04 14:31:32 adamhostname kernel: usb 1-4: SerialNumber: a3f06c9dc81e21861c818fa61646c5f3af7638ad jul 04 14:31:32 adamhostname mtp-probe[27190]: checking bus 1, device 10: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-4" jul 04 14:31:32 adamhostname mtp-probe[27190]: bus: 1, device: 10 was not an MTP device jul 04 14:31:32 adamhostname fwupd[4335]: failed to add USB device 05ac:12a8: 05ac:12a8 is not supported: USB error on device 05ac:12a8 : Entity not found [-5] jul 04 14:31:32 adamhostname systemd[1]: Started Socket daemon for the usbmux protocol used by Apple devices. jul 04 14:31:32 adamhostname upowerd[2143]: unhandled action 'bind' on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4 jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.772][3] usbmuxd v1.1.0 starting up jul 04 14:31:32 adamhostname unity-settings-[2556]: failed to connect to device: Failed to connect to missing device /org/freedesktop/ColorManager/devices/sysfs__null_ jul 04 14:31:32 adamhostname unity-settings-[2556]: failed to connect to device: Failed to connect to missing device /org/freedesktop/ColorManager/devices/sysfs__null_ jul 04 14:31:32 adamhostname compiz[2673]: 14:31:32.780-warning default unknown@0 # UdevQt: unhandled device action "bind" jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.804][3] Successfully dropped privileges to 'usbmux' jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.807][3] Could not get old configuration descriptor for device 1-10: -5 jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.810][3] Connecting to new device on location 0x1000a as ID 1 jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.810][3] Initialization complete jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.810][3] Enabled exit on SIGUSR1 if no devices are attached. Start a new instance with "--exit" to trigger. jul 04 14:31:32 adamhostname usbmuxd[27196]: [14:31:32.810][3] Connected to v2.0 device 1 on location 0x1000a with serial number a3f06c9dc81e21861c818fa61646c5f3af7638ad jul 04 14:31:32 adamhostname kernel: ipheth 1-4:4.2: Apple iPhone USB Ethernet device attached jul 04 14:31:32 adamhostname NetworkManager[1314]: <info>  [1562243492.8116] manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/10) jul 04 14:31:32 adamhostname colord[1748]: CdMain: failed to emit DeviceAdded: failed to register object: An object is already exported for the interface org.freedesktop.ColorManager.Device at /org/freedesktop/ColorManager/devices/sysfs__null_ jul 04 14:31:32 adamhostname compiz[2673]: 14:31:32.814-warning default unknown@0 # UdevQt: unhandled device action "bind" jul 04 14:31:32 adamhostname colord[1748]: CdMain: failed to emit DeviceAdded: failed to register object: An object is already exported for the interface org.freedesktop.ColorManager.Device at /org/freedesktop/ColorManager/devices/sysfs__null_ jul 04 14:31:32 adamhostname nm-applet[2802]: gtk_widget_destroy: assertion 'GTK_IS_WIDGET (widget)' failed jul 04 14:31:32 adamhostname nm-applet[2802]: gtk_widget_destroy: assertion 'GTK_IS_WIDGET (widget)' failed jul 04 14:31:32 adamhostname systemd-udevd[27219]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable. jul 04 14:31:32 adamhostname kernel: ipheth 1-4:4.2 enp0s20f0u4c4i2: renamed from eth0 jul 04 14:31:32 adamhostname nm-applet[2802]: Can't set a parent on widget which has a parent jul 04 14:31:32 adamhostname upowerd[2143]: unhandled action 'bind' on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:4.1 jul 04 14:31:32 adamhostname NetworkManager[1314]: <info>  [1562243492.8387] devices added (path: /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:4.2/net/enp0s20f0u4c4i2, iface: enp0s20f0u4c4i2) jul 04 14:31:32 adamhostname NetworkManager[1314]: <info>  [1562243492.8387] device added (path: /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:4.2/net/enp0s20f0u4c4i2, iface: enp0s20f0u4c4i2): no ifupdown configuration found. jul 04 14:31:32 adamhostname NetworkManager[1314]: <info>  [1562243492.8392] device (enp0s20f0u4c4i2): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external') jul 04 14:31:32 adamhostname upowerd[2143]: unhandled action 'bind' on /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:4.2 jul 04 14:31:32 adamhostname NetworkManager[1314]: <info>  [1562243492.8411] keyfile: add connection in-memory (2c58623b-fb52-3aad-b209-8cd1f0746a30,"Wired connection 1") jul 04 14:31:32 adamhostname kernel: IPv6: ADDRCONF(NETDEV_UP): enp0s20f0u4c4i2: link is not ready jul 04 14:31:32 adamhostname kernel: IPv6: ADDRCONF(NETDEV_UP): enp0s20f0u4c4i2: link is not ready jul 04 14:31:32 adamhostname compiz[2673]: 14:31:32.841-warning default unknown@0 # UdevQt: unhandled device action "bind" jul 04 14:31:32 adamhostname NetworkManager[1314]: <info>  [1562243492.8416] settings: (enp0s20f0u4c4i2): created default wired connection 'Wired connection 1' jul 04 14:31:32 adamhostname nm-applet[2802]: gtk_widget_destroy: assertion 'GTK_IS_WIDGET (widget)' failed jul 04 14:31:32 adamhostname nm-applet[2802]: gtk_widget_destroy: assertion 'GTK_IS_WIDGET (widget)' failed jul 04 14:31:32 adamhostname compiz[2673]: 14:31:32.843-warning default unknown@0 # UdevQt: unhandled device action "move" jul 04 14:31:32 adamhostname nm-applet[2802]: Can't set a parent on widget which has a parent jul 04 14:31:32 adamhostname nm-applet[2802]: gtk_widget_destroy: assertion 'GTK_IS_WIDGET (widget)' failed jul 04 14:31:32 adamhostname nm-applet[2802]: gtk_widget_destroy: assertion 'GTK_IS_WIDGET (widget)' failed jul 04 14:31:32 adamhostname nm-applet[2802]: Can't set a parent on widget which has a parent jul 04 14:31:33 adamhostname gvfs-afc-volume-monitor[3669]: creating volume for device uuid 'a3f06c9dc81e21861c818fa61646c5f3af7638ad' jul 04 14:31:35 adamhostname ModemManager[1329]: <info>  Couldn't check support for device '/sys/devices/pci0000:00/0000:00:14.0/usb1/1-4': not supported by any plugin jul 04 14:31:35 adamhostname kernel: [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS. jul 04 14:31:36 adamhostname gnome-session[2579]: gnome-session-binary[2579]: WARNING: Could not get session path for session. Check that logind is properly installed and pam_systemd is getting used at login. jul 04 14:31:36 adamhostname gnome-session-binary[2579]: WARNING: Could not get session path for session. Check that logind is properly installed and pam_systemd is getting used at login. jul 04 14:31:38 adamhostname compiz[2673]: pam_unix(unity:auth): authentication failure; logname= uid=1099222340 euid=1099222340 tty= ruser= rhost=  user=adamn jul 04 14:31:38 adamhostname audit[27245]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/systemd/users/1099222340" pid=27245 comm="krb5_child" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: kauditd_printk_skb: 7 callbacks suppressed jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.125:8120): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/systemd/users/1099222340" pid=27245 comm="krb5_child" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27245]: AVC apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/krb5/plugins/authdata/sssd_pac_plugin.so" pid=27245 comm="krb5_child" requested_mask="m" denied_mask="m" fsuid=1099222340 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.153:8121): apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/krb5/plugins/authdata/sssd_pac_plugin.so" pid=27245 comm="krb5_child" requested_mask="m" denied_mask="m" fsuid=1099222340 ouid=0 jul 04 14:31:38 adamhostname compiz[2673]: pam_sss(unity:auth): authentication success; logname= uid=1099222340 euid=1099222340 tty= ruser= rhost= user=adamn jul 04 14:31:38 adamhostname compiz[2673]: gkr-pam: unlocked login keyring jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=27248 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=27248 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.273:8122): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=27248 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.273:8123): apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/var/cache/samba/gencache.tdb" pid=27248 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.277:8124): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="wrc" denied_mask="wrc" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.277:8125): apparmor="ALLOWED" operation="file_lock" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="wk" denied_mask="wk" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.277:8126): apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.277:8127): apparmor="ALLOWED" operation="truncate" profile="/usr/sbin/sssd" name="/run/samba/gencache_notrans.tdb" pid=27248 comm="gpo_child" requested_mask="w" denied_mask="w" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/samba/gensec/krb5.so" pid=27248 comm="gpo_child" requested_mask="m" denied_mask="m" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.293:8128): apparmor="ALLOWED" operation="file_mmap" profile="/usr/sbin/sssd" name="/usr/lib/x86_64-linux-gnu/samba/gensec/krb5.so" pid=27248 comm="gpo_child" requested_mask="m" denied_mask="m" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:31:38 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:31:38 adamhostname systemd-resolved[1041]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP. jul 04 14:31:38 adamhostname audit[27248]: AVC apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/dev/urandom" pid=27248 comm="gpo_child" requested_mask="wc" denied_mask="wc" fsuid=0 ouid=0 jul 04 14:31:38 adamhostname kernel: audit: type=1400 audit(1562243498.313:8129): apparmor="ALLOWED" operation="open" profile="/usr/sbin/sssd" name="/dev/urandom" pid=27248 comm="gpo_child" requested_mask="wc" denied_mask="wc" fsuid=0 ouid=0 jul 04 14:33:08 adamhostname unity-panel-ser[2681]: menus_destroyed: assertion 'IS_WINDOW_MENU(wm)' failed jul 04 14:33:08 adamhostname unity-panel-ser[2681]: menus_destroyed: assertion 'IS_WINDOW_MENU(wm)' failed jul 04 14:33:08 adamhostname compiz[2673]: WARN  2019-07-04 14:33:08 unity.glib.dbus.proxy GLibDBusProxy.cpp:487 Calling method "EmitEvent" on object path: "/com/ubuntu/Upstart" failed: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name com.canonical.Unity.Test.Upstart was not provided by any .service files jul 04 14:33:08 adamhostname systemd[2263]: Stopped target A target that, when running, represents the screen being locked. jul 04 14:33:08 adamhostname systemd[2263]: Stopping Backing Service for the Unity Panel in Lockscreen mode... jul 04 14:33:08 adamhostname unity-fallback-[2824]: GtkDialog mapped without a transient parent. This is discouraged. jul 04 14:33:08 adamhostname hud-service[3779]: void DBusMenuImporter::slotGetLayoutFinished(QDBusPendingCallWatcher*): "No such interface 'com.canonical.dbusmenu' on object at path /org/ayatana/bamf/window/31458319" jul 04 14:33:10 adamhostname unity-panel-ser[25760]: Source ID 4294967295 was not found when attempting to remove it jul 04 14:33:10 adamhostname systemd[2263]: Stopped Backing Service for the Unity Panel in Lockscreen mode. 

Based on this: New alert keeps showing up: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001 the NXDOMAIN issue seems irrelevant.

Apache 2.4 randomly return wrong certificate

I’ve a server with 50 different Virtual Hosts, ~10 with SSL enabled. Everything worked perfectly until last week, when I added another host with SSL, ,t is configured exactly like the other hosts, with the same certificate, and Apache is randomly returning wrong certificate only for this host.

The strangest thing is that 50% of the time it returns the correct certificate, and the other 50% of the time it returns a wrong certificate, from another of the sites.

This is the .conf file of the new host:

<VirtualHost *:80>         ServerName      new_site.com         Redirect    /   https://new_site.com/ </VirtualHost>  SSLStaplingCache shmcb:/tmp/stapling_cache(128000)  <VirtualHost *:443>         ServerName      new_site.com          SSLEngine on          SSLCertificateFile /var/ca/new_site/ServerCertificate.cer         SSLCertificateKeyFile /var/ca/new_site/private.key         SSLCertificateChainFile /var/ca/new_site/CACertificate.cer          DocumentRoot /var/www/new_site          SSLUseStapling on </VirtualHost> 

My server is running Ubuntu Server 16.04.6 LTS with updated Apache from repositories:

Server version: Apache/2.4.18 (Ubuntu) Server built:   2019-04-03T13:34:47 

apachectl -t returns Syntax OK

Stock OnePlus 3 randomly rebooting multiple times per day

It might have been since the update to Oreo or just coincidence but my phone started rebooting by itself. Also noticed when it reboots after doing the usual booting animation which takes much longer than usual it will ask for unlock pattern and do the boot animation again. I notice sometimes after reboot the phone feels weird, a bit sluggish and some apps don’t work properly. For example anything that calls the camera will say it is not available. Other things like for example VoIP features in messaging app won’t work. Another reboot would fix the issue or it would stay the same and require another reboot. This issue got worse and worse where it would require multiple reboots or take longer and longer for the reboot sometimes just switching off during the reboot.

Of course I tried wiling cache. Didn’t help. I wiped the phone and gave it to my mom and it seemed to fix it at first. It started appearing again eventually. So I see multiple possibilities.

  1. Rogue app. But my mom has only a few apps, different from what I used.
  2. OS update messed up? Phone is currently on latest OTA on Pie. Would a custom rom help?
  3. Battery issue? This would be simple to fix but the phone rebooted regardless of charge state, once I saw it even doing so while hooked up. I didn’t observe a drop in charge after reboot either.
  4. Mainboard / Nand issue.

Ubuntu 19.04 logs out randomly after first sleep

I just installed Ubuntu 19.04 on my computer, and ever since I installed it, I’ve been getting this weird bug where, after the first sleep, if I try to log in again, Ubuntu will keep logging out. The only way I’ve been able to bypass this is by powering off my computer (by holding down the power button), and logging in again. Any thoughts on how to fix this?

Design Review: Randomly generated primary keys in database


Problem:

In my web application entities have unique IDs, and for some type of entities(e.g. user, order etc.), the IDs are visible to users via URLs. MySQL is used for storing the entities.

Using auto increment integer as primary key is simple and convenient, but if it’s exposed in URL, people are able to estimate the entity amount or increasing rate by observing the IDs(German Tank Problem), and we might want to avoid that.

Having read a lot of posts on the topic, I’m aware of several approaches to mitigate the information disclose, e.g. generate random IDs by the app, using UUIDs, encode/hash last insert ID etc.

But finally I’ve come up with my approach.

Design

Firstly, generate a list of unique integers, and preload them into the database.

Since I’m not going to have billions of records, I could make a large sequential list and shuffle it in memory.

table to store the integers would be:

CREATE TABLE IF NOT EXISTS entity1_id (     id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,     entity_id INT UNSIGNED NOT NULL UNIQUE ); 

The entity_id column is assigned the integer value.

Secondly, prepare the entity table and a counter table:

CREATE TABLE IF NOT EXISTS entity1 (     id INT UNSIGNED PRIMARY KEY,     create_time DATETIME DEFAULT CURRENT_TIMESTAMP ) ENGINE=InnoDB;  CREATE TABLE IF NOT EXISTS id_counter (     entity_name CHAR(30)  PRIMARY KEY,     counter INT NOT NULL DEFAULT 1 );  INSERT INTO id_counter (entity_name) VALUES ('entity1'); 

Finally, use a “BEFORE INSERT” trigger to fetch a ID from the entity1_id table, and increment the ID counter.

The counter is used to keep track of used IDs in the entity1_id table. Think of entity1_id as a map, entity1_id[counter] gives next entity ID.

Here is the trigger:

DELIMITER |  CREATE TRIGGER entity1_idgen BEFORE INSERT ON entity1 FOR EACH ROW BEGIN     DECLARE cnt, eid INT UNSIGNED;      IF (NEW.id IS NULL) THEN         SET cnt = (SELECT counter FROM id_counter WHERE entity_name='entity1' FOR UPDATE);          SET eid = (SELECT entity_id FROM entity1_id WHERE id=cnt);         IF (eid IS NULL) THEN             SET @errMsg = CONCAT("table `entity1_id` doesn't have row where id=", cnt);             SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT=@errMsg;         END IF;          SET NEW.id = eid;         UPDATE id_counter SET counter=counter+1 WHERE entity_name='entity1';     END IF; END|  DELIMITER ; 

I’ve briefly tested this approach, when inserting into the entity1 table, if id is not specified, an ID is taken from the entity1_id table; if a ID is specified, it would be used, and entity1_id and id_counter is not visited; if id is not specified and there’re no enough IDs in entity1_id, the error is reported.

Analysis

I think this approach is good for me:

  1. I get randomly generated IDs
  2. I can decide what IDs to use
  3. It’s not slow

From my observation, this approach has 3 downsides:

  1. it’s not a straightforward solution
  2. the risk of running out of IDs
  3. extra space for the pre-generated IDs

Question

  • Am I doing something wrong?
  • Are there any other risks I’m not aware of?

Drives randomly disconnect on USB Hub

When I try to read from multiple external desktop drives connected to a USB 3.0 hub it will work fine for a couple minutes before:

[  462.908496] usb 9-2.2: reset SuperSpeed USB device number 8 using xhci_hcd [  469.467983] xhci_hcd 0000:02:00.0: xHCI host not responding to stop endpoint command. [  469.468235] xhci_hcd 0000:02:00.0: xHCI host controller not responding, assume dead [  469.468279] xhci_hcd 0000:02:00.0: HC died; cleaning up [  469.468366] usb 8-2: USB disconnect, device number 2 [  469.468401] usb 9-2.1: USB disconnect, device number 7 [  469.469441] usb 9-2: USB disconnect, device number 6 [  469.488016] sd 7:0:0:0: [sdg] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK 

shows up in dmesg and the drives disconnect. I’ve tried two different hubs and they both do this. When I connect the drives directly into the USB ports they don’t have this problem.

Here’s my kernel:

4.15.0-51-generic #55-Ubuntu SMP Wed May 15 14:27:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux 

I’ve tried disabling legacy USB mode in the BIOS and it still fails.

“Failed to Create cuSolverDN Instance” Appears Seemingly Randomly

I am aware that this issue has been highlighted multiple times by others(Failed to create cuSolverDN instance and Tensorflow GPU: Failed to create cuSolverDN instance); however, these were asked some time ago and were recreated under different circumstances.

I have been using Tensorflow for creating an RNN that classifies handwriting(here is my code), and it has produced entirely different results seemingly randomly. At times it will work completely fine, whereas sometimes it will create the following error:

2019-06-21 12:52:48.561331: I T:\src\github\tensorflow\tensorflow\core\kernels\cuda_solvers.cc:159] Creating CudaSolver handles for stream 000001C0904E7E70 2019-06-21 12:52:49.111234: F T:\src\github\tensorflow\tensorflow\core\kernels\cuda_solvers.cc:94] Check failed: cusolverDnCreate(&cusolver_dn_handle) == CUSOLVER_STATUS_SUCCESS Failed to create cuSolverDN instance. 


The problem arises when using CuDNNLSTM in order to make the RNN much faster. Just an hour ago, the program ran without any difficulties; now it ends in the same error regarding the cuSolverDN instance. I am running the code with a GeForce MX250 graphics card.