For every imperative function, is there a functional counterpart with identical performance or even instructions?

Currently, I haven’t learned about a functional language that can achieve the same performance as C/C++. And I have learned that some languages that favor functional programming to imperative programming, such as Scala and Rust, use imperative ways to implement their library functions for better efficiency.

So here comes my question, on today’s comptuters that execute imperative instructions, is this a limitation of the compiler or functional programming itself? For every imperative function with no side effects, either in a language without GC such as C/C++/Rust/assembly or one with GC such as Java, is there a pure functional counterpart in Haskell, Scala, etc. that can be compiled to run with identical performance in time and space (not just asymptotic but exactly the same) or even to the same instructions, with an optimal functional compiler that utilizes all modern and even undiscovered optimization techniques such as tail recursion, laziness, static analysis, formal verification, and so on which I don’t know about?

I am aware of the equivalence between λ-computable and Turing computable, but but I couldn’t find an answer to this question online. If there is, please share a compiler example or a proof. If not, please explain why and show a counter-example. Or is this a non-trivial open question?

Near identical MySQL deployments behaving very different – High CPU Usage problem

So I have five identical websites, running on five machines provisioned in the same way. The only thing that differs between these installations are the language files and the languages of the text stored in MySQL tables.

Four of them have no problems what so ever. One is struggling a LOT under the same or somewhat less load than the other four.

I cannot understand why this is.

Things I’ve done so far:

  1. Checked slow queries. All queries uses indexes and are in the realm of 0.0008 Sec execution time i.e. very fast
  2. I’ve noticed that the thing that causes most trouble for this MySQL instance is UPDATE and INSERT, so much so, I’ve turned off the UPDATE’s that were there, for this instance. Bear in mind that these UPDATE’s doesn’t cause a blip on the other servers.
  3. Tried to eliminate external factors i.e. noisy neighbours (moved host) etc.

Worth noticing is that the machines are deployed the same i.e. a vanilla Debian 10 installation with a LEMP stack, nothing out of the ordinary at all.

Still, the problem persists. I can see the load of the machine struggling to keep under 1.00. The other machines are in the 0.10 – 0.20 range all the time.

Looking at CPU for the MySQL process on this machine (with 2 CPU cores as the other machines have as well) it is quite often above 100%. The other machines are never – EVER – over 60% for the MySQL process.

So, any help is much appreciated.

Please do let me know if you need me to run a command that you need to see the output from in order to help.

Thanks.

EDIT Spelling and clarifications

If an NPC and a PC roll identical initiative who goes first?

Normally, when 2 players roll identical initiative in combat, they decide between themselves the order they do things in and then stick to that on subsequent rounds.

How should the DM handle it if the NPC rolls the same initiative, especially if they are in combat directly with the player who rolled the identical initiative?

If rolling bulk initiative for a group of NPCs, how should identical rolls with players be handled here? It feels like a lot can swing on whether the GM chooses for the group of NPCs to attack the player before he can decide his action.

Order of resolution of several identical initiative Attacks

It sometimes happens that several identical monsters, all using the same initiative, attack a single character.

In my games, this is often a volley of missile fire at a caster.

For pure practicality, I typically roll all the attack rolls together, count up the hits, and then roll all the damage, and assign it to the player as a single total. If the potential damage total is less than their current hit points, there is little difference between this approach and RAW.

However, when there is a potential for damage to render a character unconscious, this approach does differ substantially from RAW, at least for my understanding of them. RAW, I should roll each attack and record each instance of damage separately. Upon the first hit that renders the character unconscious, they immediately drop prone. The subsequent attacks, although occurring on the same initiative, in some sense come ‘after’ the character has fallen prone. Thus they are at disadvantage to hit (assuming missile attacks) but each one that does hit indicates a failed death save, so that three such hits would result in the character’s death.

Questions:

  1. Is my understanding of the situation with RAW correct?

  2. Is there any difference in this situation between multiple attackers on the same initiative and an attacker with multiattack?

  3. Suppose I choose to roll all attacks and damage at once, even in situations where the potential damage was more than a PC’s current hp. [In this case, the chance of the attackers hitting would increase, the chance of the PC going unconscious would increase, but the chance of the PC dying from failed death saves would decrease.] Can this decision be reconciled with (Initiative; PHB, p. 189):

    If a tie occurs, the DM decides the order among tied DM-controlled creatures,

    that is, I have decided to resolve these ties simultaneously? Or would such a choice violate RAW and require me to invoke Rule 0?

Related: How do creatures moving on the same initiative handle the effects of Sleep and Hypnotic Pattern?

Scheduling jobs online on 3 identical machines – a lower bound of 5/3

Consider the Online Scheduling Problem with $ 3$ identical machines. Jobs, with arbitrary size, arrive online one after another and need to be scheduled on one of the $ 3$ machines without ever moving them again.

How can I show, that there can’t be any deterministic Online-Algorithm which achieves a competitive ratio of $ c<\frac{5}{3}$ .

This should be solved by just giving some instance $ \sigma$ and arguing that no det. algorithm can do better. Same can easily be done for $ 2$ machines and $ c<\frac{3}{2}$ . Sadly I can’t find any solution to this (more or less) textbook answer.

What happens when identical overlapping effects have their end-condition met?

The Dungeon Master’s Guide errata (direct download) added the “Combining Game Effects” section which states (emphasis mine):

Different game features can affect a target at the same time. But when two or more game features have the same name, only the effects of one of them—the most potent one—apply while the durations of the effects overlap. […]

So with overlapping things, one instance will be “active” (will have effects) and the other will be “inactive” (won’t have effects). That said, I think my question will make more sense with examples:

  1. The Fire Elemental’s Fire Form trait:

    […] The first time it enters a creature’s space on a turn, that creature takes 5 (1d10) fire damage and catches fire; until someone takes an action to douse the fire, the creature takes 5 (1d10) fire damage at the start of each of its turns.

    If a creature is under the effects of multiple instances of Fire Form, and somebody uses their action to douse the fire, are both instances removed or only one?

  2. The booming blade spell:

    […] If the target willingly moves before then, it immediately takes 1d8 thunder damage, and the spell ends. […]

    If a creature is under the effects of multiple instances of booming blade, and they move, are both instances removed or only one?

  3. The hold monster spell:

    […] At the end of each of its turns, the target can make another Wisdom saving throw. On a success, the spell ends on the target. […]

    If a creature is under the effects of multiple instances of hold monster, and they succeed on a save, are both instances removed or only one?

Can an “inactive” effect end when its end-condition is met (letting them end simultaneously), or do these sorts of things always end one at a time? Or perhaps the answer is something in-between?


There is also the following related question:

  • Can multiple creatures grapple a single target?

Both answers there supports that if you are grappled by multiple creatures and you make a check to remove a grapple, you only remove one of the grapples. It’s not perfectly analogous situation but it is somewhat similar.

Identical records in IIS logs

I am not very knowledgeable about IIS 7, so I thought this was the right place to ask.

While inspecting the web server logs, I came across several instances of separate records that look just the same. For example:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken 2020-04-21 00:00:10 ABC.128.138.15 GET /MY/API/HERE departmentId=&prodLineId=&prodUnitId=&puGroupId=&FLId=null&startTime=2019-09-08T21:00:00.000Z&endTime=2019-09-09T21:00:00.000Z&status=null&itemType=&itemComponent=&howFound=&priority=&foundBy=&fixedBy=&flList= 80 - ABC.128.138.15 Apache-HttpClient/4.5.6+(Java/1.8.0_92) - 200 0 0 1453 2020-04-21 00:00:10 ABC.128.138.15 GET /MY/API/HERE departmentId=&prodLineId=&prodUnitId=&puGroupId=&FLId=null&startTime=2019-09-08T21:00:00.000Z&endTime=2019-09-09T21:00:00.000Z&status=null&itemType=&itemComponent=&howFound=&priority=&foundBy=&fixedBy=&flList= 80 - ABC.128.138.15 Apache-HttpClient/4.5.6+(Java/1.8.0_92) - 200 0 0 1453 

The question is – do those records correspond to two actual separate requests, or perhaps it was just one that for some reason got duplicated? This is not an isolated occurrence (there are hundreds more). Just for the record, these are all GET requests coming from the same source (an Apache Tomcat-based application that resides in the same web server and is invoking APIs in different application pools).

Thanks in advance,

Gabriel

Is multiplying hashes a valid way to ensure two sets of data are identical (but in arbitrary order)

Let’s say “User A” has a set of data like below. Each entry has been hashed (sha256) to ensure integrity within a single entry. You can’t modify data of a single entry without also modifying the corresponding hash:

[ { data: "000000", hash: "91b4d142823f7d20c5f08df69122de43f35f057a988d9619f6d3138485c9a203" },  { data: "111111", hash: "bcb15f821479b4d5772bd0ca866c00ad5f926e3580720659cc80d39c9d09802a" },  { data: "345345", hash: "dbd3b3fcc3286d927ec214c5648fbb226353a239789750f51430b1e6e9d91f4f" },  ] 

And “User B” has the same data but in a slightly different order. Hashes are the same of course:

[ { data: "345345", hash: "dbd3b3fcc3286d927ec214c5648fbb226353a239789750f51430b1e6e9d91f4f" },  { data: "111111", hash: "bcb15f821479b4d5772bd0ca866c00ad5f926e3580720659cc80d39c9d09802a" },  { data: "000000", hash: "91b4d142823f7d20c5f08df69122de43f35f057a988d9619f6d3138485c9a203" },  ] 

I want to allow both users to verify they have the exactly same set of data, ignoring sort order. If, as an extreme example, a hacker is able to replace User B’s files with otherwise valid-looking data, the users should be able to compare a hash of their entire datasets and detect a mismatch.

I was thinking to calculate a “total hash” which the users could compare to verify. It should be next to impossible to fabricate a valid looking dataset that results in the same “total hash”. But since the order can change, it’s a bit tricky.

I might have a possible solution, but I’m not sure if it’s secure enough. Is it, actually, secure at all?

My idea is to convert each sha256 hash to integer (javascript BigInt) and multiply them with modulo to get a total hash of similar length:

 var entries = [ { data: "345345", hash: "dbd3b3fcc3286d927ec214c5648fbb226353a239789750f51430b1e6e9d91f4f" },  { data: "111111", hash: "bcb15f821479b4d5772bd0ca866c00ad5f926e3580720659cc80d39c9d09802a" },  { data: "000000", hash: "91b4d142823f7d20c5f08df69122de43f35f057a988d9619f6d3138485c9a203" },  ];  var hashsize = BigInt("0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"); var totalhash = BigInt(1); // arbitrary starting point  for (var i = 0; i < entries.length; i++) {   var entryhash = BigInt("0x" + entries[i].hash);   totalhash = (totalhash * entryhash) % hashsize;  } totalhash = totalhash.toString(16); // convert from bigint back to hex string 

This should result in the same hash for both User A and User B, unless other has tampered data, right? How hard would it be to create a slightly different, but valid-looking dataset that results in the same total checksum? Or is there a better way to accomplish this (without sorting!).

Aurora PostgreSQL database using a slower query plan than a normal PostgreSQL for an identical query?

Following the migration of an application and its database from a classical PostgreSQL database to an Amazon Aurora RDS PostgreSQL database (both using 9.6 version), we have found that a specific query is running much slower — around 10 times slower — on Aurora than on PostgreSQL.

Both databases have the same configuration, be it for the hardware or the pg_conf.

The query itself is fairly simple. It is generated from our backend written in Java and using jOOQ for writing the queries:

with "all_acp_ids"("acp_id") as (     select acp_id from temp_table_de3398bacb6c4e8ca8b37be227eac089 )  select distinct "public"."f1_folio_milestones"."acp_id",      coalesce("public"."sa_milestone_overrides"."team",      "public"."f1_folio_milestones"."team_responsible")  from "public"."f1_folio_milestones"  left outer join      "public"."sa_milestone_overrides" on (         "public"."f1_folio_milestones"."milestone" = "public"."sa_milestone_overrides"."milestone"          and "public"."f1_folio_milestones"."view" = "public"."sa_milestone_overrides"."view"          and "public"."f1_folio_milestones"."acp_id" = "public"."sa_milestone_overrides"."acp_id" ) where "public"."f1_folio_milestones"."acp_id" in (     select "all_acp_ids"."acp_id" from "all_acp_ids" ) 

With temp_table_de3398bacb6c4e8ca8b37be227eac089 being a single-column table, f1_folio_milestones (17 million entries) and sa_milestone_overrides (Around 1 million entries) being similarly designed tables having indexes on all the columns used for the LEFT OUTER JOIN.

When we run it on the normal PostgreSQL database, it generates the following query plan:

Unique  (cost=4802622.20..4868822.51 rows=8826708 width=43) (actual time=483.928..483.930 rows=1 loops=1)   CTE all_acp_ids     ->  Seq Scan on temp_table_de3398bacb6c4e8ca8b37be227eac089  (cost=0.00..23.60 rows=1360 width=32) (actual time=0.004..0.005 rows=1 loops=1)   ->  Sort  (cost=4802598.60..4824665.37 rows=8826708 width=43) (actual time=483.927..483.927 rows=4 loops=1)         Sort Key: f1_folio_milestones.acp_id, (COALESCE(sa_milestone_overrides.team, f1_folio_milestones.team_responsible))         Sort Method: quicksort  Memory: 25kB         ->  Hash Left Join  (cost=46051.06..3590338.34 rows=8826708 width=43) (actual time=483.905..483.917 rows=4 loops=1)               Hash Cond: ((f1_folio_milestones.milestone = sa_milestone_overrides.milestone) AND (f1_folio_milestones.view = (sa_milestone_overrides.view)::text) AND (f1_folio_milestones.acp_id = (sa_milestone_overrides.acp_id)::text))               ->  Nested Loop  (cost=31.16..2572.60 rows=8826708 width=37) (actual time=0.029..0.038 rows=4 loops=1)                     ->  HashAggregate  (cost=30.60..32.60 rows=200 width=32) (actual time=0.009..0.010 rows=1 loops=1)                           Group Key: all_acp_ids.acp_id                           ->  CTE Scan on all_acp_ids  (cost=0.00..27.20 rows=1360 width=32) (actual time=0.006..0.007 rows=1 loops=1)                     ->  Index Scan using f1_folio_milestones_acp_id_idx on f1_folio_milestones  (cost=0.56..12.65 rows=5 width=37) (actual time=0.018..0.025 rows=4 loops=1)                           Index Cond: (acp_id = all_acp_ids.acp_id)               ->  Hash  (cost=28726.78..28726.78 rows=988178 width=34) (actual time=480.423..480.423 rows=987355 loops=1)                     Buckets: 1048576  Batches: 1  Memory Usage: 72580kB                     ->  Seq Scan on sa_milestone_overrides  (cost=0.00..28726.78 rows=988178 width=34) (actual time=0.004..189.641 rows=987355 loops=1) Planning time: 3.561 ms Execution time: 489.223 ms 

And it goes pretty smoothly as one can see — less than a second for the query. But on the Aurora instance, this happens:

Unique  (cost=2632927.29..2699194.83 rows=8835672 width=43) (actual time=4577.348..4577.350 rows=1 loops=1)   CTE all_acp_ids     ->  Seq Scan on temp_table_de3398bacb6c4e8ca8b37be227eac089  (cost=0.00..23.60 rows=1360 width=32) (actual time=0.001..0.001 rows=1 loops=1)   ->  Sort  (cost=2632903.69..2654992.87 rows=8835672 width=43) (actual time=4577.348..4577.348 rows=4 loops=1)         Sort Key: f1_folio_milestones.acp_id, (COALESCE(sa_milestone_overrides.team, f1_folio_milestones.team_responsible))         Sort Method: quicksort  Memory: 25kB         ->  Merge Left Join  (cost=1321097.58..1419347.08 rows=8835672 width=43) (actual time=4488.369..4577.330 rows=4 loops=1)               Merge Cond: ((f1_folio_milestones.view = (sa_milestone_overrides.view)::text) AND (f1_folio_milestones.milestone = sa_milestone_overrides.milestone) AND (f1_folio_milestones.acp_id = (sa_milestone_overrides.acp_id)::text))               ->  Sort  (cost=1194151.06..1216240.24 rows=8835672 width=37) (actual time=0.039..0.040 rows=4 loops=1)                     Sort Key: f1_folio_milestones.view, f1_folio_milestones.milestone, f1_folio_milestones.acp_id                     Sort Method: quicksort  Memory: 25kB                     ->  Nested Loop  (cost=31.16..2166.95 rows=8835672 width=37) (actual time=0.022..0.028 rows=4 loops=1)                           ->  HashAggregate  (cost=30.60..32.60 rows=200 width=32) (actual time=0.006..0.006 rows=1 loops=1)                                 Group Key: all_acp_ids.acp_id                                 ->  CTE Scan on all_acp_ids  (cost=0.00..27.20 rows=1360 width=32) (actual time=0.003..0.004 rows=1 loops=1)                           ->  Index Scan using f1_folio_milestones_acp_id_idx on f1_folio_milestones  (cost=0.56..10.63 rows=4 width=37) (actual time=0.011..0.015 rows=4 loops=1)                                 Index Cond: (acp_id = all_acp_ids.acp_id)               ->  Sort  (cost=126946.52..129413.75 rows=986892 width=34) (actual time=4462.727..4526.822 rows=448136 loops=1)                     Sort Key: sa_milestone_overrides.view, sa_milestone_overrides.milestone, sa_milestone_overrides.acp_id                     Sort Method: quicksort  Memory: 106092kB                     ->  Seq Scan on sa_milestone_overrides  (cost=0.00..28688.92 rows=986892 width=34) (actual time=0.003..164.348 rows=986867 loops=1) Planning time: 1.394 ms Execution time: 4583.295 ms 

It effectively has a lower global cost, but takes almost 10 times as much time than before!

Disabling merge joins makes Aurora revert to a hash join, which gives the expected execution time — but permanently disabling it is not an option. Curiously though, disabling nested loops gives an even better result while still using a merge join…

Unique  (cost=3610230.74..3676431.05 rows=8826708 width=43) (actual time=2.465..2.466 rows=1 loops=1)   CTE all_acp_ids     ->  Seq Scan on temp_table_de3398bacb6c4e8ca8b37be227eac089  (cost=0.00..23.60 rows=1360 width=32) (actual time=0.004..0.004 rows=1 loops=1)   ->  Sort  (cost=3610207.14..3632273.91 rows=8826708 width=43) (actual time=2.464..2.464 rows=4 loops=1)         Sort Key: f1_folio_milestones.acp_id, (COALESCE(sa_milestone_overrides.team, f1_folio_milestones.team_responsible))         Sort Method: quicksort  Memory: 25kB         ->  Merge Left Join  (cost=59.48..2397946.87 rows=8826708 width=43) (actual time=2.450..2.455 rows=4 loops=1)               Merge Cond: (f1_folio_milestones.acp_id = (sa_milestone_overrides.acp_id)::text)               Join Filter: ((f1_folio_milestones.milestone = sa_milestone_overrides.milestone) AND (f1_folio_milestones.view = (sa_milestone_overrides.view)::text))               ->  Merge Join  (cost=40.81..2267461.88 rows=8826708 width=37) (actual time=2.312..2.317 rows=4 loops=1)                     Merge Cond: (f1_folio_milestones.acp_id = all_acp_ids.acp_id)                     ->  Index Scan using f1_folio_milestones_acp_id_idx on f1_folio_milestones  (cost=0.56..2223273.29 rows=17653416 width=37) (actual time=0.020..2.020 rows=1952 loops=1)                     ->  Sort  (cost=40.24..40.74 rows=200 width=32) (actual time=0.011..0.012 rows=1 loops=1)                           Sort Key: all_acp_ids.acp_id                           Sort Method: quicksort  Memory: 25kB                           ->  HashAggregate  (cost=30.60..32.60 rows=200 width=32) (actual time=0.008..0.008 rows=1 loops=1)                                 Group Key: all_acp_ids.acp_id                                 ->  CTE Scan on all_acp_ids  (cost=0.00..27.20 rows=1360 width=32) (actual time=0.005..0.005 rows=1 loops=1)               ->  Materialize  (cost=0.42..62167.38 rows=987968 width=34) (actual time=0.021..0.101 rows=199 loops=1)                     ->  Index Scan using sa_milestone_overrides_acp_id_index on sa_milestone_overrides  (cost=0.42..59697.46 rows=987968 width=34) (actual time=0.019..0.078 rows=199 loops=1) Planning time: 5.500 ms Execution time: 2.516 ms 

We have asked the AWS support team, they are still looking at the issue, but we are wondering what could cause that issue to happen. What could explain such a behaviour difference?

While looking at some of the documentation for the database, I read that Aurora favors cost over time — and hence it uses the query plan that has the lowest cost.

But as we can see, it’s far from being optimal given its response time… Is there a threshold or a setting that could make the database use a more expensive — but faster — query plan?