How to execute an HTTP call in addition to RewriteRule (mod_rewrite) in apache?

I have an apache server which redirects URLs to a different one based on some conditions.

I would like to “intercept” this, make a call to an API server, and then continue with the redirect.

So it would somewhat look like this:

RewriteRule ^(.)$ {make API call with $ 1 as a param} RewriteRule ^(.)$ newhost.com/$ 1 [R,L]

I know the above doesn’t work because perhaps RewriteRule isn’t meant for this.

How would I go about accomplish this? I looked in to cgi scripts, but it looks like they need to be called explicitly with a certain URL pattern.

My requirement is no matter what, make an intermediate API call with whatever URL was originally requested, then do a redirect to another URL.

Apache ssl.conf RewriteRule

Trying to modify an ssl.conf configuration to do the following…

RewriteCond %{REQUEST_URI} ^/$ #### misc.org #### RewriteRule ^/(.*) https://misc.org/scores [L,R=301] ####takes misc.org and makes it misc.org/scores#####

1

RewriteRule ^/scores/liverpool https://misc2.org/liverpool2 [L,R=301] ####SO REDIRECT FOR ‘/scores/liverpool to misc2.org/liverpool2’ WORKS#####

2

RewriteRule ^/scores/liverpool/child https://misc2.org/liverpool2/child [L,R=301] ####only redirects to /misc2.org/liverpool2 and never makes it to /misc2.org/liverpool2/child####

If I comment out #1…#2 works but I need both to work.

What am I missing?

Thanks in advance

Apache web service inaccessible from external IP/DNS host name (Windows Server 2012, Apache 2.4, xampp 3.2.3)

For the record:

  • I am running a VM of Windows Server 2012, (with apache, php, and XAMPP installed) using VMware Workstation 15.
  • My VM NAT is in bridged mode and other services on the server are accessible.
  • My ISP blocks forwarding port 80.
  • I have a Domain registered through no-ip to work around both the Dynamic External IP, and my ISP’s restrictions on port 80.
  • I have checked sites like downforeveryoneorjustme.com and I am not “down”. All ports are registering as accessible.

My issue:

When I try to access my website externally, (from using the DNS Lookup on my computer, or on my phone with Wi-Fi turned off, I get an error message stating “Ex.ter.nal.IP sent an invalid response”

When I check on my Lan (using the Internal IP for the VM) my access log reads as follows:

In.ter.nal.IP - - [20/May/2019:11:53:59 -0500] "GET / HTTP/1.1" 200 3178 In.ter.nal.IP - - [20/May/2019:11:53:59 -0500] "GET /style.css HTTP/1.1" 200 3145 In.ter.nal.IP - - [20/May/2019:11:53:59 -0500] "GET /logo.png HTTP/1.1" 200 60058 In.ter.nal.IP - - [20/May/2019:11:53:59 -0500] "GET /backgournd.jpg HTTP/1.1" 200 185050 In.ter.nal.IP - - [20/May/2019:11:53:59 -0500] "GET /favicon.ico HTTP/1.1" 200 1406 

When I try to access with my phone, I get:

Ex.ter.nal.IP1 - - [20/May/2019:11:54:44 -0500] "\x16\x03\x01\x02" 400 1010 "-" "-" Ex.ter.nal.IP1 - - [20/May/2019:11:54:44 -0500] "\x16\x03\x01" 400 1010 "-" "-" Ex.ter.nal.IP2 - - [20/May/2019:11:54:45 -0500] "GET /favicon.ico HTTP/1.1" 200 1406 "http://my.ddns.lookup/" "Mozilla/5.0 (Linux; Android 6.0.1; My_Phone) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Mobile Safari/537.36" 

There is no Error Corresponding with the timestamps for the attempt at access

I have tried to follow instructions on the web for hours and I am getting to the ‘slam my head against a brick wall’ level of frustration.

What do I need to do?

Domain keeps going to the default apache config

I have an apache2 setup (Ubuntu 16, Apache 2.4.18) with a number of virtual hosts defined. Each virtualhost has it’s own .conf file in /etc/apache2/sites-available with a softlink to /etc/apache2/sites-enabled.

All but one of the virtualhosts work correctly. I have triple checked that the ServerName matches the domain (it does), but when I go to that domain, I end up at DocumentRoot for the default virtualhost and I have no idea why. I also deleted and re-wrote the whole ServerName line in case there was some hidden control character.

I have done a diff between one of the working virtualhosts and the one that doesn’t work, and the only differences are the DocumentRoot and the ServerName.

I have checked that there are no other virtual hosts with the same domain (the only other one is the SSL virtual host)

Is there anything else I can look for to help figure out the issue?

Apache Spark asigning different partitions to same executor with custom partitioner, despite idle executors

I have an issue where, using a custom partitioner, Spark will allocate two different partitions to the same executor, despite there being executors available. The partition structure with/without the custom partitioner seems to be the same, however the partition assignment seems to differ for some reason. Here is a simplified version of the problem at hand:

DEFAULT PARTITIONER:

------------------ Partitions Structure: [[(0,null)], [(1,null)], [(2,null)], [(3,null)], [(4,null)], [(5,null)]]  DAGScheduler:54 - Submitting 6 missing tasks from ResultStage 1  (MapPartitionsRDD[4] at map at xxx.java:77)  TaskSchedulerImpl:54 - Adding task set 1.0 with 6 tasks  Starting task 0.0 in stage 1.0 (TID 6, xxx.xx.xx.4,  executor 3, partition 0, PROCESS_LOCAL, 7882 bytes) Starting task 1.0 in stage 1.0 (TID 7, xxx.xx.xx.14, executor 4, partition 1, PROCESS_LOCAL, 7877 bytes) Starting task 2.0 in stage 1.0 (TID 8, xxx.xx.xx.27, executor 5, partition 2, PROCESS_LOCAL, 7882 bytes) Starting task 3.0 in stage 1.0 (TID 9, xxx.xx.xx.3,  executor 1, partition 3, PROCESS_LOCAL, 7882 bytes) Starting task 4.0 in stage 1.0 (TID 10, xxx.xx.xx.26,executor 2, partition 4, PROCESS_LOCAL, 7882 bytes) Starting task 5.0 in stage 1.0 (TID 11, xxx.xx.xx.9, executor 0, partition 5, PROCESS_LOCAL, 7882 bytes) 

CUSTOM PARTITIONER:

Partitions Structure:  [[(0,null)], [(1,null)], [(2,null)], [(3,null)], [(4,null)], [(5,null)]]  DAGScheduler:54 - Submitting 6 missing tasks from ResultStage 3  (MapPartitionsRDD[5] at map at xxx.java:77)  TaskSchedulerImpl:54 - Adding task set 3.0 with 6 tasks  Starting task 0.0 in stage 3.0 (TID 12, xxx.xx.xx.27, executor 5, partition 0, NODE_LOCAL, 7666 bytes) Starting task 4.0 in stage 3.0 (TID 13, xxx.xx.xx.26, executor 2, partition 4, NODE_LOCAL, 7666 bytes) Starting task 1.0 in stage 3.0 (TID 14, xxx.xx.xx.27, executor 5, partition 1, NODE_LOCAL, 7666 bytes) Starting task 5.0 in stage 3.0 (TID 15, xxx.xx.xx.26, executor 2, partition 5, NODE_LOCAL, 7666 bytes) Starting task 2.0 in stage 3.0 (TID 16, xxx.xx.xx.27, executor 5, partition 2, NODE_LOCAL, 7666 bytes) Starting task 3.0 in stage 3.0 (TID 17, xxx.xx.xx.27, executor 5, partition 3, NODE_LOCAL, 7666 bytes) 

Here are some key observations:

  • The Partitions Structure is the exact same
  • Default Partitioner has 1 to 1 mapping of partition to executor, Custom Partitioner has a one to many
  • Default Partitioner refers to PROCESS_LOCAL while Custom refers to NODE_LOCAL
  • Custom Partitioner always refers to MapPartitionsRDD[5] while without refers to MapPartitionsRDD[4]
  • Default Partitioner has sequential partition ordering assignment [0-5], Custom Partitioner is shuffled

Here is the code: The goal is to have foo execute once on each node on the cluster. This is achieved with the default partitioning, but unsuccessfully using the custom partitioner.

Main.java

    public static void testPartitions(){         Integer numPartitions = 6;         List<Integer> data = Arrays.asList(0, 1, 2, 3, 4, 5);         JavaRDD<Integer> dataRDD = SparkExecutor.sc.parallelize(data,numPartitions);         JavaPairRDD<Integer, Object> dataPairRDD = dataRDD.mapToPair( currData -> new Tuple2<Integer, Object>(currData, null));         TestPartitioner partitioner = new TestPartitioner(numPartitions);                 // dataPairRDD = dataPairRDD.partitionBy(partitioner); //toggle custom partitioner          logger.info("Num Partitions: {}", dataPairRDD.getNumPartitions());         logger.info("Partitions Structure: {}", dataPairRDD.glom().collect());          JavaRDD<Integer> mapRDD = dataPairRDD.map( currData -> foo(currData));                  mapRDD.collect();     }      public static Integer foo(Tuple2<Integer, Object> data)throws Exception{         Integer num = data._1();         TimeUnit.SECONDS.sleep(5); //Simulate some work being done         return num;     } 
public class TestPartitioner extends Partitioner{     private int numPartitions;      //Constructor     public TestPartitioner(int numPartitions){         this.numPartitions = numPartitions;     }      @Override     public int getPartition(Object key) {         int bucket = (int) key;         return bucket;     }      @Override     public int numPartitions() {         return this.numPartitions;     }      @Override     public boolean equals(Object obj) {         if (obj instanceof TestPartitioner) {              TestPartitioner partitionerObject = (TestPartitioner) obj;             if (partitionerObject.numPartitions == this.numPartitions)                 return true;         }         return false;     } } 

Как подключить SSL Apache + Node.JS?

Столкнулся с проблемой подключения SSL Apache + Node.JS, может у кого есть пример конфига, прилагаю ниже www.js и conf для apache

<VirtualHost *:80> ServerName moy-sait.com ServerAlias www.moy-sait.com  DocumentRoot /var/www/html/moy-sait/public  DirectoryIndex index.hbs  <Directory "/var/www/html/moy-sait"> AllowOverride All Require all granted </Directory>  ProxyRequests off <Location /> #ProxyPreserveHost On ProxyPass http://localhost:3000/     ProxyPassReverse http://localhost:3000/ </Location>  ErrorLog logs/moy-sait.error.log CustomLog logs/moy-sait.requests.log combined </VirtualHost>  <VirtualHost *:443>     ServerName moy-sait.com ServerAlias www.moy-sait.com  DocumentRoot /var/www/html/moy-sait/public  DirectoryIndex index.hbs  SSLEngine on SSLCertificateFile "/var/www/httpd-cert/voishunter/moy-sait.crt" SSLCertificateKeyFile "/var/www/httpd-cert/voishunter/moy-sait.key" SSLCertificateChainFile "/var/www/httpd-cert/voishunter/moy-sait.ca" SSLHonorCipherOrder on SSLProtocol +TLSv1 +TLSv1.1 +TLSv1.2 SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH  <Directory "/var/www/html/moy-sait"> AllowOverride All Require all granted </Directory>  ProxyRequests off <Location /> #ProxyPreserveHost On ProxyPass http://localhost:4000/     ProxyPassReverse http://localhost:4000/ </Location>  ErrorLog logs/moy-sait.error.log CustomLog logs/moy-sait.requests.log combined </VirtualHost>`  #!/usr/bin/env node "use strict"; Object.defineProperty(exports, "__esModule", { value: true }); var app_1 = require("./app"); var app_2 = require("./app"); var http = require("http");   /*For SSL Connection*/  var express = require("express"); var app_2 = express(); var fs = require("fs");  var key = fs.readFileSync("/var/www/httpd-cert/voishunter/moy-sait.key"); var cert = fs.readFileSync( "/var/www/httpd-cert/voishunter/moy-sait.crt" ); var ca = fs.readFileSync( "/var/www/httpd-cert/voishunter/moy-sait.ca" );  var options = {  key: key,  cert: cert,  ca: ca };  var https = require("https"); https.createServer(options, app_2.app).listen(4000);  /**/  var Server = /** @class */ (function () { function Server() {     this.port = this.normalizePort(process.env.PORT || '3000');     app_1.app.set('port', this.port);     this.server = http.createServer(app_1.app);     this.server.listen(this.port);     this.server.on('error', this.onError); } Server.prototype.normalizePort = function (val) {     var port = parseInt(val, 10);     if (isNaN(port)) {         // named pipe         return val;     }     if (port >= 0) {         // port number         return port;     }     return false; }; Server.prototype.onError = function (error) {     if (error.syscall !== 'listen') {         throw error;     }     var bind = typeof this.port === 'string'         ? 'Pipe ' + this.port         : 'Port ' + this.port;     // handle specific listen errors with friendly messages     switch (error.code) {         case 'EACCES':             console.error(bind + ' requires elevated privileges');             process.exit(1);             break;         case 'EADDRINUSE':             console.error(bind + ' is already in use');             process.exit(1);             break;         default:             throw error;     } }; return Server; }()); var kek = new Server(); 

PHP7.3-FPM with Apache: Unknown script request execution timed out

I configured Apache 2.4 MPM Event with PHP7.3-FPM on a pretty busy web server as follows:

Timeout 90  <Proxy "unix:/run/php/php7.3-fpm.sock|fcgi://php-fpm">     ProxySet disablereuse=on timeout=90 </Proxy>  <FilesMatch ".+\.php$  ">         SetHandler proxy:fcgi://php-fpm </FilesMatch> 

The PHP config in /etc/php/7.3/fpm/php.ini is set to

max_execution_time=60 

The PHP-FPM is configured in /etc/php/7.3/fpm/pool.d/www.conf to

request_terminate_timeout=90 

The read timeouts are configured in /etc/apache2/mods-enabled/reqtimeout.conf like this:

RequestReadTimeout header=20-120,minrate=50 RequestReadTimeout body=60-120,minrate=50 

In FPM log now I can see every minute 1-2 workers for unknown requests which are killed after 90+ seconds.

[16-May-2019 09:25:32] WARNING: [pool www] child 105567, script '' (request: " ") execution timed out (113.002653 sec), terminating [16-May-2019 09:25:32] WARNING: [pool www] child 105567 exited on signal 15 (SIGTERM) after 4050.136381 seconds from start [16-May-2019 09:25:32] NOTICE: [pool www] child 110414 started 

If I don’t set request_terminate_timeout to kill these workers they stay longer (~5 minutes) in state “Reading headers” and block the PHP-FPM pool.

The threads in Apache MPM seem not to get blocked anyway. Nothing is hitting the max workers values.

How can I see which script/request these PHP workers belong to? Why these scripts are running even after max_execution_time of 60 seconds has expired? How can I avoid blocking the PHP pool by such requests?

I suspect there could be sometimes incomplete HTTPS requests which are starting the PHP worker somehow. Is there a way to avoid starting a PHP worker for these?

Unusual Get/Connect requests to Apache

Not sure if that has been asked before but over the last week I am getting hundreds of CONNECT and GET requests to my Apache web server for domains that are not on my server. Some are for well known sites like instagram.com. Here is a few that happened this morning:

120.78.240.35 - - [15/May/2019:09:15:10 +0200] "CONNECT book.spicejet.com:443 HTTP/1.1" 405 519 "-" "Apache-HttpClient/4.5.3 (Java/1.8.0_111)" 120.78.240.35 - - [15/May/2019:09:15:09 +0200] "CONNECT book.spicejet.com:443 HTTP/1.1" 405 519 "-" "Apache-HttpClient/4.5.3 (Java/1.8.0_111)" 120.78.240.35 - - [15/May/2019:09:15:09 +0200] "CONNECT book.spicejet.com:443 HTTP/1.1" 405 519 "-" "Apache-HttpClient/4.5.3 (Java/1.8.0_111)" 120.78.240.35 - - [15/May/2019:09:15:09 +0200] "CONNECT book.spicejet.com:443 HTTP/1.1" 405 519 "-" "Apache-HttpClient/4.5.3 (Java/1.8.0_111)" 51.75.12.139 - - [15/May/2019:09:15:07 +0200] "CONNECT api.goldenfrog.com:443 HTTP/1.1" 400 0 "-" "-" 120.76.246.242 - - [15/May/2019:09:15:05 +0200] "CONNECT api.nokair.com:443 HTTP/1.1" 405 516 "-" "-" 54.37.76.200 - - [15/May/2019:09:15:03 +0200] "CONNECT hq.uis.kaspersky.com:443 HTTP/1.1" 400 0 "-" "-" 72.130.166.116 - - [15/May/2019:09:15:03 +0200] "CONNECT stockx.com:443 HTTP/1.1" 400 0 "-" "-" 51.77.245.206 - - [15/May/2019:09:15:00 +0200] "CONNECT 160.153.202.75:443 HTTP/1.0" 405 535 "-" "-" 

Any idea what could be causing those requests to be redirected to my server? Initially I thought that it might be a scan to see if the server could be used as a proxy but this has been going on for the better part of a week now.