Is there any scenario whereby randomly shufflying a sequence improves it’s compressibility?

I’m performing some correlation assessment à la NIST Recommendation for the Entropy Sources Used for Random Bit Generation, § 5.1.

You take a test sequence and compress it with a standard compression algorithm. You then shuffle that sequence randomly using a PRNG, and re-compress. We expect that the randomly shuffled sequence to be harder to compress as any and all redundancy and correlations will have been destroyed. It’s entropy will have increased.

So if there is any auto correlation, $ \frac{\text{size compressed shuffled}} {\text{size compressed original}} > 1$ .

This works using NIST’s recommended bz2 algorithm, and on my data samples, the ratio is ~1.03. This indicates a slight correlation within the data. When I switch to LZMA, the ratio is ~0.99 which is < 1. And this holds over hundreds of runs so it’s not just a stochastic fluke.

What would cause the LZMA algorithm to repetitively compress a randomly shuffled sequence (slightly) better than a non shuffled one?

“An error occurred when navigating to:” error occurs randomly approximately every third time certain users try to access particular documents

At my company, certain users, when trying to access documents via links outside of our SharePoint environment, are getting an error saying “An error occurred when navigating to:…”. This error is happening for the users irregularly. If they get the error and then click the link again, the error goes away and they get access to the document. I am not an expert in SharePoint, but would this imply that one of our content servers in our load balance is not working properly while the others ARE working? Can anyone tell me how I could go about solving this issue? Any thoughts or comments would be appreciated!

Ubuntu 18.04 Gnome shell crashing randomly

I am running a clean install of 18.04 on a Dell Inspiration desktop with a Nvidia Quadro FX1800 graphics card. Every so often – most days – my GNOME session just ends, logging me out. There is no obvious pattern to this in terms of actions or open programs. The last time I was just moving the mouse. The trace from syslog is long, but having looked at a few of them the following seem to be common features:

Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) glamor0: GL error: GL_OUT_OF_MEMORY in glTexSubImage Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (WW) glamor: Failed to allocate 7x12 FBO due to GL_OUT_OF_MEMORY. Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (WW) glamor: Expect reduced performance. Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Backtrace: Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 0: /usr/lib/xorg/Xorg (OsLookupColor+0x139) [0x55ef0cc59829] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 1: /lib/x86_64-linux-gnu/libpthread.so.0 (funlockfile+0x50) [0x7f86e5d158df] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 2: /usr/lib/xorg/modules/libglamoregl.so (glamor_get_pixmap_texture+0x7d) [0x7f86e27b257d] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 3: /usr/lib/xorg/modules/libglamoregl.so (glamor_create_gc+0x78f4) [0x7f86e27c4664] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 4: /usr/lib/xorg/modules/libglamoregl.so (glamor_create_gc+0x7caf) [0x7f86e27c4d8f] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 5: /usr/lib/xorg/modules/libglamoregl.so (glamor_create_gc+0x8668) [0x7f86e27c58a8] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 6: /usr/lib/xorg/modules/libglamoregl.so (glamor_create_gc+0xa26f) [0x7f86e27c909f] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 7: /usr/lib/xorg/Xorg (DamageRegionAppend+0x6c6) [0x55ef0cbdcb46] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 8: /usr/lib/xorg/modules/libglamoregl.so (glamor_create_gc+0x1111f) [0x7f86e27d749f] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 9: /usr/lib/xorg/Xorg (AddTraps+0x423d) [0x55ef0cbd576d] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 10: /usr/lib/xorg/Xorg (SendErrorToClient+0x368) [0x55ef0cafc548] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 11: /usr/lib/xorg/Xorg (InitFonts+0x3c8) [0x55ef0cb00528] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 12: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0xe7) [0x7f86e5933b97] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) 13: /usr/lib/xorg/Xorg (_start+0x2a) [0x55ef0caea18a] Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Segmentation fault at address 0x0 Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: Fatal server error: Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Caught signal 11 (Segmentation fault). Server aborting Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: Please consult the The X.Org Foundation support Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: #011 at http://wiki.x.org Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]:  for help. Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Please also check the log file at "/home/nigel/.local/share/xorg/Xorg.0.log" for additional information. Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (II) AIGLX: Suspending AIGLX clients for VT switch Aug 28 13:36:55 mill /usr/lib/gdm3/gdm-x-session[2616]: (EE) Server terminated with error (1). Closing log file. Aug 28 13:36:56 mill systemd[1]: Starting Process error reports when automatic reporting is enabled... Aug 28 13:37:01 mill nautilus[3334]: nautilus: Fatal IO error 11 (Resource temporarily unavailable) on X server :0. Aug 28 13:37:01 mill keepassxc[8945]: keepassxc: Fatal IO error 11 (Resource temporarily unavailable) on X server :0. Aug 28 13:37:01 mill org.gnome.Shell.desktop[2750]: Gdk-Message: 13:37:01.648: /usr/lib/firefox/firefox: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.  ... a lot of other apps croaking similarly ...  Aug 28 13:37:01 mill gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.Wacom.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session[2628]: gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.Wacom.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.Color.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session[2628]: gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.Color.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session[2628]: gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.MediaKeys.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session[2628]: gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.Keyboard.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.MediaKeys.desktop' exited with code 1 Aug 28 13:37:01 mill gnome-session-binary[2628]: WARNING: App 'org.gnome.SettingsDaemon.Keyboard.desktop' exited with code 1 Aug 28 13:37:01 mill rtkit-daemon[1488]: Supervising 1 threads of 1 processes of 1 users. Aug 28 13:37:01 mill rtkit-daemon[1488]: Successfully made thread 10328 of process 10312 (n/a) owned by '1000' RT at priority 5. Aug 28 13:37:01 mill rtkit-daemon[1488]: Supervising 2 threads of 1 processes of 1 users. Aug 28 13:37:01 mill rtkit-daemon[1488]: Supervising 2 threads of 1 processes of 1 users. Aug 28 13:37:01 mill rtkit-daemon[1488]: Successfully made thread 10329 of process 10312 (n/a) owned by '1000' RT at priority 5. Aug 28 13:37:01 mill rtkit-daemon[1488]: Supervising 3 threads of 1 processes of 1 users. Aug 28 13:37:01 mill dbus-daemon[971]: [system] Activating via systemd: service name='org.bluez' unit='dbus-org.bluez.service' requested by ':1.138' (uid=1000 pid=10312 comm="/usr/bin/pulseaudio --start --log-target=syslog " label="unconfined") Aug 28 13:37:01 mill at-spi-bus-launcher[2725]: XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0" Aug 28 13:37:01 mill at-spi-bus-launcher[2725]:       after 2981 requests (2981 known processed) with 0 events remaining. Aug 28 13:37:01 mill gnome-shell[2750]: gnome-shell: Fatal IO error 4 (Interrupted system call) on X server :0. Aug 28 13:37:01 mill gsd-print-notif[2873]: Source ID 2 was not found when attempting to remove it Aug 28 13:37:01 mill kernel: [22074.902509] rfkill: input handler enabled Aug 28 13:37:01 mill systemd[2596]: Stopping D-Bus User Message Bus... Aug 28 13:37:01 mill gvfsd[2757]: A connection to the bus can't be made Aug 28 13:37:01 mill systemd[2596]: xdg-permission-store.service: Main process exited, code=exited, status=1/FAILURE Aug 28 13:37:01 mill systemd[2596]: xdg-permission-store.service: Failed with result 'exit-code'. Aug 28 13:37:01 mill deja-dup-monito[3966]: Owner of volume monitor org.gtk.vfs.UDisks2VolumeMonitor disconnected from the bus; removing drives/volumes/mounts Aug 28 13:37:01 mill deja-dup-monito[3966]: Owner of volume monitor org.gtk.vfs.MTPVolumeMonitor disconnected from the bus; removing drives/volumes/mounts Aug 28 13:37:01 mill deja-dup-monito[3966]: Owner of volume monitor org.gtk.vfs.GPhoto2VolumeMonitor disconnected from the bus; removing drives/volumes/mounts Aug 28 13:37:01 mill deja-dup-monito[3966]: Owner of volume monitor org.gtk.vfs.GoaVolumeMonitor disconnected from the bus; removing drives/volumes/mounts Aug 28 13:37:01 mill deja-dup-monito[3966]: Owner of volume monitor org.gtk.vfs.AfcVolumeMonitor disconnected from the bus; removing drives/volumes/mounts Aug 28 13:37:01 mill systemd[2596]: Stopped D-Bus User Message Bus. Aug 28 13:37:01 mill systemd[2596]: Started D-Bus User Message Bus. Aug 28 13:37:01 mill dbus-daemon[10362]: [session uid=1000 pid=10362] AppArmor D-Bus mediation is enabled Aug 28 13:37:03 mill systemd[1]: Started Session c5 of user gdm. 

There are no obvious customisations to look at and system memory is fine. Where to start with a solution?

Touchpad randomly disabled upon reboot on lenovo x1c7

Just switched from apple to lenovo (carbon 7th gen), installed Ubuntu (19.04, with 5.0.0-25-generic). However, the touchpad randomly is disabled upon reboot (it is activated in the system settings). After rebooting the touchpanel sometimes starts to work, sometimes doesn’t. This happens only with the ubuntu partition, not with Windows. Can somebody help?

Secret PHP File executed randomly

In a wordpress installation i have created a file named test_random_name.php.

Last night something strange happened. Somebody executed the file but i have no clue how they did it. No directory listing is allowed. Also, there are no viruses. The file îs newly created and it is not public on the web.

It can be accessed only of you know the filename. So? How can a bot or person guess so fast a random filename and execute it?

Previously it was set a cron job on the file but i removed that comand right before it was executed.

Ubuntu 18.04 freezes randomly after RAM and SSD upgrade

I previously had an 8GB of RAM and 1TB of hard disk on my ASUS VivoBook R542UQ-DM153. I’ve added a 250GB WD Blue SSD and an 8GB of RAM. Total RAM now is 16GB. I’ve also made a fresh installation of Ubuntu on SSD. Now every time there is somewhat like heavy use the system freezes and I am left with no other option but to hard shutdown. Please help how can I fix that.

$   sudo dmidecode --type memory # dmidecode 3.1 Getting SMBIOS data from sysfs. SMBIOS 3.0.0 present.  Handle 0x0008, DMI type 16, 23 bytes Physical Memory Array     Location: System Board Or Motherboard     Use: System Memory     Error Correction Type: None     Maximum Capacity: 64 GB     Error Information Handle: Not Provided     Number Of Devices: 4  Handle 0x0009, DMI type 17, 40 bytes Memory Device     Array Handle: 0x0008     Error Information Handle: Not Provided     Total Width: 64 bits     Data Width: 64 bits     Size: 8192 MB     Form Factor: SODIMM     Set: None     Locator: ChannelA-DIMM0     Bank Locator: BANK 0     Type: DDR4     Type Detail: Synchronous Unbuffered (Unregistered)     Speed: 2400 MT/s     Manufacturer: 859B     Serial Number: E0F2D27D     Asset Tag: 9876543210     Part Number: CB8GS2400.C8D            Rank: 1     Configured Clock Speed: 2133 MT/s     Minimum Voltage: 1.2 V     Maximum Voltage: 1.2 V     Configured Voltage: 1.2 V  Handle 0x000A, DMI type 17, 40 bytes Memory Device     Array Handle: 0x0008     Error Information Handle: Not Provided     Total Width: Unknown     Data Width: Unknown     Size: No Module Installed     Form Factor: Unknown     Set: None     Locator: ChannelA-DIMM1     Bank Locator: BANK 1     Type: Unknown     Type Detail: None     Speed: Unknown     Manufacturer: Not Specified     Serial Number: Not Specified     Asset Tag: Not Specified     Part Number: Not Specified     Rank: Unknown     Configured Clock Speed: Unknown     Minimum Voltage: Unknown     Maximum Voltage: Unknown     Configured Voltage: Unknown  Handle 0x000B, DMI type 17, 40 bytes Memory Device     Array Handle: 0x0008     Error Information Handle: Not Provided     Total Width: 64 bits     Data Width: 64 bits     Size: 8192 MB     Form Factor: SODIMM     Set: None     Locator: ChannelB-DIMM0     Bank Locator: BANK 2     Type: DDR4     Type Detail: Synchronous Unbuffered (Unregistered)     Speed: 2400 MT/s     Manufacturer: 04CB     Serial Number: 01210200     Asset Tag: 9876543210     Part Number: AO1P24HC8T1-BSFS         Rank: 1     Configured Clock Speed: 2133 MT/s     Minimum Voltage: 1.2 V     Maximum Voltage: 1.2 V     Configured Voltage: 1.2 V  Handle 0x000C, DMI type 17, 40 bytes Memory Device     Array Handle: 0x0008     Error Information Handle: Not Provided     Total Width: Unknown     Data Width: Unknown     Size: No Module Installed     Form Factor: Unknown     Set: None     Locator: ChannelB-DIMM1     Bank Locator: BANK 3     Type: Unknown     Type Detail: None     Speed: Unknown     Manufacturer: Not Specified     Serial Number: Not Specified     Asset Tag: Not Specified     Part Number: Not Specified     Rank: Unknown     Configured Clock Speed: Unknown     Minimum Voltage: Unknown     Maximum Voltage: Unknown     Configured Voltage: Unknown 

Hd randomly becomes System read only even after beeing fixed

I’m using an six years old notebook. The old hd died after start to make some noises. The new one, after moths of good use, start with noises, but the noise would stop for a whole day if i just restart it twice before using it. So yerstady the screen freezes while using a lot of the processor, and after the forced reboot, ubuntu booted initramfs terminal. I ran fsck /dev/sda1 -y and ubuntu boot normaly, but the hd was read-only (and freezing for some seconds, wich isn’t common). After trying some solutions i figured out that the Hd dosen’t boot read-only, it becames read-only after using it for a while. Usually during downloads, such as updates, but i’m not quite sure what causes it to become read-only.

I’m not good with hardware, but i think this notebook might be a hd killer.

Wi-fi randomly stops. Supplicant network-manager issues. Failed to set WFD IES and re-acquiring supplicant in wlo1

I’ve been having this issue for weeks. I looked it up in the web but doesn’t seem to be working solutions.

The thing is that after a couple of hours of use my wifi randomly disconnects.

While the network is working, if I do

sudo service network-manager status 

There’s no issue at all. The device (wlo1) activation is “successful, device activated”. and the supplicant is okay, saying “supplicant: wpa_supplicant die count reset”.

Meanwhile, if I check the network manager status when is down. I’ve two warns.

The status explain in one hand

supplicant: failed to set WFD IEs on wpa_supplicant: GDBus.Error:org.freedesktop.DBus.Error.NoReply: Message recipient disconneected from message bus without replying 

And, in the other hand

device (wlo1): re-acquiring supplicant interface (#1) 

Any help really would help. Thanks.

Details of my wireless card:

*-network                         description: Ethernet interface        product: RTL810xE PCI Express Fast Ethernet controller        vendor: Realtek Semiconductor Co., Ltd.        physical id: 0        bus info: pci@0000:01:00.0        logical name: eno1        version: 07        serial: b0:5a:da:d0:bb:31        size: 10Mbit/s        capacity: 100Mbit/s        width: 64 bits        clock: 33MHz        capabilities: pm msi pciexpress msix vpd bus_master cap_list rom ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd autonegotiation        configuration: autonegotiation=on broadcast=yes driver=r8169 duplex=half firmware=rtl8106e-1_0.0.1 06/29/12 latency=0 link=no multicast=yes port=MII speed=10Mbit/s        resources: irq:16 ioport:2000(size=256) memory:91004000-91004fff memory:91000000-91003fff memory:91500000-9150ffff   *-network        description: Wireless interface        product: RTL8188EE Wireless Network Adapter        vendor: Realtek Semiconductor Co., Ltd.        physical id: 0        bus info: pci@0000:02:00.0        logical name: wlo1        version: 01        serial: 18:4f:32:b6:95:3a        width: 64 bits        clock: 33MHz        capabilities: pm msi pciexpress bus_master cap_list ethernet physical wireless        configuration: broadcast=yes driver=rtl8188ee driverversion=5.0.0-20-generic firmware=N/A ip=192.168.0.110 latency=0 link=yes multicast=yes wireless=IEEE 802.11        resources: irq:121 ioport:1000(size=256) memory:91100000-91103fff 

Randomly built binary search trees

In Introduction to Algorithms (CLRS) 3rd Edition, page 299, the section attempts to prove:

The expected height of a randomly built binary search tree on $ n$ distinct keys is $ O(\lg n)$ .

We define “randomly built binary search tree on $ n$ keys” as:

a binary search tree that arises from inserting the keys in random order into an initially empty tree, where each of the $ n!$ permutations of the input keys is equally likely.

In the proof, we defined some random variables:

Let $ X_n$ denotes the height of a randomly built binary search tree on $ n$ keys and the exponential height $ Y_n = 2^{X_n}$ . Of the $ n$ keys, we choose one key as the root of the tree, and we let $ R_n$ denotes the random variable that holds the position that this key would occupy if the set of keys were sorted (also know as ‘rank’ in the text). If we know that $ R_n=i$ , it follows that $ Y_n=2\cdot max(Y_{i-1},Y_{n-1})$ .

We also define indicator random variables $ Z_{n,1}, Z_{n,2}, …, Z_{n,n}$ , $ Z_{n,i} = I\{R_n=i\}$ .

\begin{equation} \begin{split} E[Y_n] & = E\Big[\sum_{i=1}^{n} Z_{n,i} (2\cdot max(Y_{i-1}, Y_{n-i}))\Big] \ & = \sum_{i=1}^{n} E[Z_{n,i} (2\cdot max(Y_{i-1}, Y_{n-i}))] && \text{(by linearity of expectation)}\ & = \sum_{i=1}^{n} E[Z_{n,i}] E[(2\cdot max(Y_{i-1}, Y_{n-i}))] && \text{(by independence)}\ \end{split} \end{equation}

I would like to ask why is the last equality correct? In other words, why can we assume independence of $ Z_{n,i}$ and $ max(Y_{i-1},Y_{n-i})$ ?

In the proof, there was a brief explanation:

Having chosen $ R_n = i$ , the left subtree (whose ranks are less than $ i$ . This subtree is just like any other randomly built binary search tree on $ i-1$ keys. Other than the number of keys it contains, this subtree’s structure is not affected at all by the choice of $ R_n=i$ , and hence the random variables $ Y_{i-1}$ and $ Z_{n,i}$ are independent. Likewise, the right subtree, whose exponential height is $ Y_{n-i}$ , is randomly built on the $ n-i$ keys whose ranks are greater than $ i$ .

However, since $ R_n$ does affect the number of keys $ Y_{i-1}$ and $ Y_{n-i}$ contain (as acknowledged by the explanation above), wouldn’t it mean that $ Y_{i-1}$ and $ Y_{n-i}$ are dependent on $ Z_{n,i}$ ?

A note regarding similar questions posted on CS stackexchange

I have read the answers for the following questions which are very similar to mine:

  1. Proof that a randomly built binary search tree has logarithmic height
  2. Average height of a BST with n Nodes.
  3. Randomized BST height analysis : How $ Z_{n,i}$ and $ Y_{k-1}$ are independent?

For 1 & 2, the question was a more general one which asked for an explanation for the entire proof. Also, the answer for both questions did not attempt to justify the independence and simply used the result.

For 3, the question was based on a MIT lecture, https://www.youtube.com/watch?v=vgELyZ9LXX4 and it lacked some of the details which I have included above. Also, the answer was also not clear.