Tag: slower

Generalization of code is slower than particular case

I wrote the following Mathematica module:

QNlalternative[NN_, l_, f_] := Module[{s, wz, w, z, j, lvec},    s = 0;    Do[       wz = Table[weightsNodesQ1l@lvec@i, {i, NN}];       w = Table[wz[[i]][[1, All]], {i, NN}];       z = Table[wz[[i]][[2, All]], {i, NN}];       s = s + Function[Sum[(f @@ (Table[z[[i]][[j[i]]], {i, NN}]))*(Times @@ (Table[                 w[[i]][[j[i]]], {i, NN}])), ##]] @@                  Table[{j[k], 2^lvec[k] + 1}, {k, NN}],       ##       ] & @@ Table[{lvec[i], l + NN - 1 - Total@Table[lvec[k], {k, i - 1}]}, {i, NN}];    Return[s]    ];

This module calls another module:

sumPrime[v_List] := First[v]/2 + Total[Delete[v, 1]]  weightsNodes[NN_] := Module[{w, z},    w = Table[4/NN*sumPrime[Table[1/(1 - n^2)*Cos[n*k*Pi/NN], {n, 0., NN, 2}]], {k, 0., NN}];    z = Table[Cos[k*Pi/NN], {k, 0., NN}];    Return[{w, z}]    ];  weightsNodesQ1l[l_] := weightsNodes[2^l]

This code is related to a mathematical problem I am solving (it is a modification). When I first was thinking about how to write the module QNlalternative, I wrote the particular case of NN=5 in a sloppy manner, using repeated statements, as follows:

Q5l[l_, f_] :=    Module[{s, wzl1, wzl2, wzl3, wzl4, wzl5, wl1, zl1, wl2, zl2, wl3,      zl3, wl4, zl4, wl5, zl5},    s = 0;    Do[     wzl1 = weightsNodesQ1l[l1];     wzl2 = weightsNodesQ1l[l2];     wzl3 = weightsNodesQ1l[l3];     wzl4 = weightsNodesQ1l[l4];     wzl5 = weightsNodesQ1l[l5];     wl1 = wzl1[[1, All]]; zl1 = wzl1[[2, All]];     wl2 = wzl2[[1, All]]; zl2 = wzl2[[2, All]];     wl3 = wzl3[[1, All]]; zl3 = wzl3[[2, All]];     wl4 = wzl4[[1, All]]; zl4 = wzl4[[2, All]];     wl5 = wzl5[[1, All]]; zl5 = wzl5[[2, All]];     s = s +  Sum[f[zl1[[i1]], zl2[[i2]], zl3[[i3]], zl4[[i4]], zl5[[i5]]]*         wl1[[i1]]*wl2[[i2]]*wl3[[i3]]*wl4[[i4]]*wl5[[i5]], {i1, 1,          2^l1 + 1}, {i2, 1, 2^l2 + 1}, {i3, 1, 2^l3 + 1}, {i4, 1,          2^l4 + 1}, {i5, 1, 2^l5 + 1}],     {l1, 1, l + 5 - 1}, {l2, 1, l + 5 - 1 - l1}, {l3, 1,       l + 5 - 1 - l1 - l2}, {l4, 1, l + 5 - 1 - l1 - l2 - l3}, {l5, 1,       l + 5 - 1 - l1 - l2 - l3 - l4}     ];    Return[s]    ];

The module Q5l is much faster than QNlalternative:

AbsoluteTiming[QNlalternative3[5, 6, Sin[Plus[##]]^2 &]] (* {19.4634, 6213.02} *)  AbsoluteTiming[Q5l[6, Sin[Plus[##]]^2 &]] (* {6.64357, 6213.02} *)

Why is QNlalternative slower? Which step of the generalization of Q5l to an arbitrary NN is too slow?

MariaDB Inserts are getting slower and slower (7x tables, ~ 2.8M and 200MB)

I have auto increment on each of a table. One unique ID that is consistent of 10 numbers, and each table have ~ 6 big int columns (values are small from 1-60k), and from 0 to 4 var chars (~ up to 500 characters, on average from 5 to 50 characters).

I am fighting with this for months and can’t make it production stage :(, basically it drops from ~ 170 inserts (from app perspective) to ~ 40 just after ~ 200-500k inserts.

This is nothing as I’ve worked with DB that was holding trillions of columns and auto increment and huge varchars. (however paid solution :().

I already tweaked the config so many times but still getting to the point where server is using ~ 950% & .net core 25% (of all cores).

Machine has i9 9900k 8c/16t, 64GB RAM, 2x NVME 2TB

I can’t even run @ 5 minute API test as it won’t be able to process all data from a queue 🙁 (API can accept ~ 20k/s).

Buffers, read io, inno_db other tweaks for a commit etc. were applied, nothing seems to be working.

Looks like it cannot for some reason handle just so little data and I cannot figure out why (I never had any real experience with free databases, so I only assume that it should be able to insert 300k records within 60 seconds and sustain this for ~ 10TB).

Is there some theoretical verification or explanation of why KDTree gets slower when the dimensionality gets higher?

This post and this post indicate that when the dimensionality gets higher, KDTree gets slower.

Per scikit-learn doc, the KDTree becomes inefficient as D(dimensions) grows very large (say, $ D>20$ ): this is one manifestation of the so-called “curse of dimensionality”.

Is there some theoretical verification or explanation of why KDTree gets slower when the dimensionality gets higher?

TP-LINK WN821N MADE MY INTERNET SLOWER?? Whyyy…What am i doing wrong?

I downloaded the driver for windows 10 on my laptop and my internet went from 20-10 basically cut it in half instead of improving it. I saw some posts about different drivers, kernal, bios info, and other possible solutions but i am not familiar with most of it. Any help would be greatly appreciated.

How to deliberately slow internet connection to better understand UX on slower connections?

I would like to better understand UX on slower connections.

Is there a set of tools or instructions on how to achieve this on mac/windows?

FYI my software isn’t a website or app, it’s desktop software that runs on mac and windows.

Perhaps there are tools available to help? (I don’t mind a DIY solution though)

Why is RAID 0 slower then single NVME drive

I created a MD RAID 0 array using two NVME drives. I realize using two different drives, rather than identical drives is not optimal, but when I just bought the new one they are all faster than my 1 year old drive. Using Patriot Scorch 512GB Nvme and Mushkin Helix 256GB. I made a 32GB partition on each, and created the MD RAID 0 array using 16k chunks, then formated to EXT4 to test. 

kde-swiebel@T5600-kde:~$   cat /proc/mdstat Personalities : [raid0]  unused devices: <none> kde-swiebel@T5600-kde:~$   sudo mdadm -Cv -l0 -c16 -n2 /dev/md0   /dev/nvme0n1p3 /dev/nvme1n1p2  mdadm: /dev/nvme0n1p3 appears to contain an ext2fs file system        size=34277376K  mtime=Wed Dec 31 17:00:00 1969 mdadm: /dev/nvme1n1p2 appears to contain an ext2fs file system        size=34277376K  mtime=Wed Dec 31 17:00:00 1969 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. kde-swiebel@T5600-kde:~$   cat /proc/mdstat Personalities : [raid0]  md0 : active raid0 nvme1n1p2[1] nvme0n1p3[0]       68487168 blocks super 1.2 16k chunks  unused devices: <none> kde-swiebel@T5600-kde:~$   sudo mkfs.ext4 -F /dev/md0 mke2fs 1.44.6 (5-Mar-2019) Discarding device blocks: done                             Creating filesystem with 17121792 4k blocks and 4284416 inodes Filesystem UUID: f9826f0f-7ba4-40b2-9c9f-861379e82703 Superblock backups stored on blocks:          32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,          4096000, 7962624, 11239424  Allocating group tables: done                             Writing inode tables: done                             Creating journal (131072 blocks): done Writing superblocks and filesystem accounting information: done

Benchmarks from Gnome-disk-utility 3.32.1 1000 samples, 1MB size

Patriot Scorch average read: 1362.2 MB/s Average write: 228.7 MB/s Average access time 0.07 msec

Mushkin Helix Average read: 1529.5 MB/s Average write: 1003.1 MB/s Average access time 0.06 msec

/dev/md0 Average read: 1751.3 MB/s Average write: 26.9 MB/s Average access time 0.05 msec

Change to 100 samples, 100MB size, read-only /dev/md0 Average read: 2927.6 MB/s Average write: NA Average access time 0.04 msec

The read speed, while it is faster than either drive alone, is not too impressive but when I changed the test to read-only, it greatly improved. The write speed, however, was horrible. Write speed was 10% of the slowest drive.

I tried removing the RAID array and creating it with different chunk sizes (4K, 32K, 128K) but little changed with each. Write speed was between 11MB/s to 32MB/s

According to Phoronix and other sites I’m doing this right and should see much more gain.

Using mdadm version 4.1-1ubuntu1, kernel 5.0.0-25-generic Running on a Dell T5600 with dual Xeon E5-2609(Quad core 2.4Ghz)

Any ideas what I’ve done wrong?

Do database actions inside a transaction become slower as the transaction grows?

I have a PostgreSQL database running a transaction. Inside that transaction I process around 160 records from an Excel file. As it runs, I see the processing becomes slower with each record (0.05 secs for the first to 0.20 for the last).

Could this be because of the transaction, or should I like elsewhere?

Wifi is 10x slower on laptop vs phone

I was having some lag spikes while playing so I decided to run a speed tests, turns out my laptop is getting 30Mbs while my phone is getting 280Mbs. Both devices connected to the same wifi and test ran on the same location.

I had run this tests before and it was the same on both devices (280Mbs). I recently tried to configure Bluetooth connection to my headset I think it might be related with the issue but I am not sure.

I already tried the top solutions I found on google: Wifi Slow when connected to bluetooth on 18.04 Ubuntu and https://itsfoss.com/speed-up-slow-wifi-connection-ubuntu/

I also tried reverting all configuration I made related to Bluetooth but the problem remained.

I already tried live booting from a usb drive using ubuntu 18.04 and I have the same issue there.

My system: https://pastebin.com/CarSiWnC

Would appreciate any tips or suggestions on fixes for this. Much appreciated!

Edit: Speed test differences between wifi and wired connection: https://imgur.com/a/Xf0e963

Docker Container is much more slower than Server

  1. Server: Im using Ubuntu 18.10 and run magento2.3.1 with redis, varnish end elasticsearch.

    • when clear the cache, ubuntu take only 10-20seconds to reload the page.
    • when running setup:static-content:deploy ubuntu take 50seconds to finish
    • with varnish cache, it takes 1second to reload the page

  2. Docker: I create a docker-compose on my ubuntu server, no volume added

    • when clear the cache, docker take 1-2min to reload the page
    • when running setup:static-content:deploy docker take 300seconds (5min) to finish
    • with varnish cache, it takes 1second the same as Ubuntu Server.

My question is what happen to Docker that take the deploy time so long? Im thinking of read/write files speed b/w Docker and Server are different, Docker read/write speed is much slower than ubuntu.

Below are my docker-compose and Dockerfile

docker-compose.yml version: '3.7' services:   web:     build: ./projectx     networks:       - magento_bacth_net       - 94now_elasticsearch_net     container_name: magento_bacth_web     links:       - db       - redis     ports:       - "8080:80"     env_file:       - .env       - database.env   redis:     build: ./redis     networks:       - magento_bacth_net     container_name: magento_bacth_redis   varnish:     build: ./varnish     networks:       - magento_bacth_net     container_name: magento_bacth_varnish     depends_on:       - web     ports:       - "8082:82"   db:     image: mysql:5.7.26     networks:       - magento_bacth_net     container_name: magento_bacth_db     environment:       - MYSQL_ROOT_PASSWORD=ecomsys       - MYSQL_USER=ecomsys       - MYSQL_PASSWORD=ecomsys       - MYSQL_DATABASE=magento_ecomsys networks:   magento_bacth_net:     name: magento_bacth_net     ipam:       driver: default       config:         - subnet: "170.10.10.16/29"   94now_elasticsearch_net:     external:       name: 94now_elasticsearch_net  project: Dockerfile FROM centos:latest  ENV INSTALL_DIR /home/ecommage/public_html ENV SETUP_FOLDER /home/ecommage/bamboo_setup ENV COMPOSER_HOME /home/ecommage/.composer/ ENV SSH_HOME /home/ecommage/.ssh/  RUN rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \  && rpm -Uvh https://rpms.remirepo.net/enterprise/remi-release-7.rpm  # normal updates RUN yum -y update  # php && httpd RUN yum -y install php72 php72-php php72-php-opcache php72-php-bcmath php72-php-cli php72-php-common php72-php-gd php72-php-intl php72-php-json php72-php-mbstring php72-php-pdo php72-php-pdo-dblib php72-php-pear php72-php-pecl-mcrypt php72-php-xmlrpc php72-php-xml php72-php-mysql php72-php-soap php72-php-pecl-zip php72-php-pecl-mongodb php72-php-pecl-xdebug php72-php-pecl-yaml httpd  # tools RUN yum -y install epel-release iproute at curl crontabs git redis mysql htop  RUN curl -O https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_x86_64.rpm \  && rpm -U mod-pagespeed-*.rpm \  && yum clean all \  && php72 -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" \  && php72 composer-setup.php --install-dir=bin --filename=composer \  && php72 -r "unlink('composer-setup.php');" \  && rm -rf /etc/localtime \  && ln -s /usr/share/zoneinfo/Europe/Berlin /etc/localtime \  && ln -s /bin/php72 /bin/php  #create ecommage user RUN useradd ecommage RUN echo "ecommage:ecommage" | chpasswd  # we want some config changes COPY config/50-php_settings.ini /etc/opt/remi/php72/php.d/ COPY config/v-host.conf /etc/httpd/conf.d/  # create webserver-default directory RUN mkdir -p $  {INSTALL_DIR} COPY ./Setup $  {SETUP_FOLDER}  RUN chown -R ecommage:ecommage /home/ecommage/.  RUN su ecommage -c "mkdir -p $  {SSH_HOME}"  COPY ./install-magento /usr/local/bin/install-magento RUN chmod +x /usr/local/bin/install-magento  COPY ./install-magento-cache /usr/local/bin/install-magento-cache RUN chmod +x /usr/local/bin/install-magento-cache  COPY ./install-magento-theme /usr/local/bin/install-magento-theme RUN chmod +x /usr/local/bin/install-magento-theme  COPY ./switch-git-repos /usr/local/bin/switch-git-repos RUN chmod +x /usr/local/bin/switch-git-repos  #change apache default user to ecommage RUN sed -i 's/User apache/User ecommage/g' /etc/httpd/conf/httpd.conf RUN sed -i 's/Group apache/Group ecommage/g' /etc/httpd/conf/httpd.conf  #install modman RUN curl -SL https://raw.githubusercontent.com/colinmollenhour/modman/master/modman -o modman RUN mv ./modman /usr/local/bin/modman RUN chmod +x /usr/local/bin/modman  #give /var/opt/remi/php72/lib/php/session 777 permission RUN chmod -R 777 /var/opt/remi/php72/lib/php/session  EXPOSE 80  RUN systemctl enable httpd \  && systemctl enable crond  CMD ["/usr/sbin/init"] CMD ["/usr/sbin/httpd","-D","FOREGROUND"]