Synchronize multiples REST api, centralized into 1 web application

I am developing a web application for a client, connected to 4 differents sources :

  • E-commerce website
  • CRM
  • Emailing platform
  • Surveys platform

All the sources have REST Api’s.

The web application has to get the data from the sources and provide a list of customers. Also the client can create/edit customers, so the web application have to send the new / updated customer to the sources.

On the sources, the client can also create/edit customers.

The 4 sources have the same columns (email, firstname, lastname, etc.)

The web application has to provide a unique list of customers (unique by email address).

On the web application and the source i have access to an “updatedAt” value.

This is the workflow i want to setup :

  • A crontab each 5 minutes > get last customers from source 1
  • Check on the web application if a more recent version exists
  • If not insert / update customer
  • Then send to the other sources the new / updated customer (the updatedAt value will change on each source)

But on the next cron execution (for source 2) the previous updated customers will be returned because of the updatedAt column, so i’m afraid, the last customers list will increase and increase..

I can’t control the change of the updatedAt column on the sources.

I feel stuck!

My questions are :

  • What is the best way to handle this kind of system ?
  • Is my workflow idea in a good direction or totally wrong ?

Thanks in advance 🙂

How, as a group, should we move towards creating a simple chat GUI application with Python?

I am sorry if this isn’t the correct community to ask this in. I don’t know and I didn’t really find any suitable place. Stackoverflow itself seemed wrong because it focuses on technical and specific problems.

So, we are a group in our 4th semester studying SWE. We are taking Computer Networks-1 and we have a project. We have a project for 4 courses (Computer Networks-1, Databases-1, Data structures-1, SWE-1) and we thought we’d make one mega project that integrates concepts learnt at all 4 courses. Professors gave us the greenlight.

So, first idea we had (and only, so far) was a chat application. You sign up, log in and send messages. (basic functionality which we intend to add to after we reach that goalpost)

Thing is, we don’t know whether this idea if feasible for people of our experience. (Which I wish the more experienced would help us see through whether or not we are fit for it because we don’t want to commit too much time for an idea that we won’t be able to see through)

Secondly, we don’t know where to begin with the project if we decide to move on with this idea. We have no idea how the architecture of such an application should be and that’s my primary question.

To make this application, what should we know? How well should we know it? We just need anyone of experience to point us in the right direction and we’ll take it from there. The building blocks of such an application.

I am sorry if this is in the wrong community. If it is, just tell me what’s the right place to post this in and I’ll do that. Other than that, guidance will be GREATLY appreciated. Especially the very elaborate kind of guidance.

Also worth noting that we are still in the beginning of the semester in my country (going into week 3) so it isn’t too late to brush that idea under the rug if it’s “too big” for us. (and if you do suggest we do that, we’d appreciate if you could suggest any more suitable ideas)

The basic functionality we have sketched up for it is:

  • Sign up
  • Log in
  • Send messages either to one person or a group of people (which the user chooses)
  • View messages in real-time with names of the senders

We’ll be using SQL for the database since that’s what the databases course is using so I doubt we can change that.

Good Way to Implement a Web-based Chat Application

I have developed some web applications in the past, but all of them only worked with databases.

This means that they work using a fire-and-forget approach where the following sequence takes place:

  1. A Client application (the UI for example) makes a request to my app.

  2. The app connects to the database.

  3. The app performs an operation.

  4. The app disconnects from the database.

  5. The app returns a result to the UI.

NOTE: My app creates a DB Connection for each request to handle unexpected Server outages and other cases where the DB might be unavailable.

Now, I am trying to develop a web-based Chat application similar to the one in LinkedIn or Zoom Meetings.

I plan to use XMPP with Openfire as the Server and the Smack API as the XMPP Client Library. This is because XMPP is a generally good choice for Instant Messaging apps.

My main issue is that the above fire-and-forget approach will not be applicable because User connections need to be kept alive for as long as the User is logged in to the UI (otherwise they will appear to be Offline to other Users).

Another factor to consider is that instead of the usual 1-is-to-1 relationship of standalone clients like WhatsApp and Skype (where the app installed on a User’s machine only needs to worry about the specific User logged in to it), this application will need to take a 1-is-to-many approach.

My questions are as follows:

  1. What is a good way to implement a web-based Chat application to handle these requirements?

  2. One possible approach to the 1-is-to-many relationship is to create a new Thread for each User that logs in to the UI. However, I am having some trouble visualizing how the Chat application will keep track of which Thread belongs to which UI.

  3. I assumed that I need to keep Connections connected for each User that logs in through my Chat app. Is this necessary?

  4. Am I missing some concepts here that will make this easier to implement?

Any advice would be greatly appreciated.

Application clone for various domain extensions hurts SEO?

On the server I have 3 different directories which are named by domains I own in hosting/domains provider.

  • example.com
  • example.co.uk
  • example.de

In every of this directory there is clone of exact same single page application (SPA).

There isn’t any changes in code of this app clones. The only change is default language of page content but application itself determine the language basing on extension or browser language (when .com).

Because the hosting provider automatically created directories basing on domains I bought it seemed to be the easiest way to just put clone in each folder.

It worked and I have my SPA working correctly at .com, .co.uk, and .de extensions but now I wonder if such architecture is good for SEO. Shouldn’t I have one SPA in some separate directory and just do some redirections in .htaccess for various domain extensions?

[Download] Packt Hands-on Web Application Development with jQuery

[Download] Packt Hands-on Web Application Development with jQuery
Want to build outstanding visual and interactive web applications by adding amazing javascript functionality with ease? jQuery, a fast, extensible, and feature-rich javascript library, will come to your rescue. It is easy to get started with and helps you build complex scripting functionality in just a few lines of code

https://drive.google.com/drive/u/1/folders/1Blp_P4pJTCjHNfeAaw74JK5A04DzEPkr

Can I query from two databases if my web application is connected to one database?

I have a local database, DB1, that I created and is connected to my web application. My web application takes in stored procedures from DB1 and queries them. My application connects to SQL Server using my Windows Authentication account.

I have access to another database, DB2, on a different server that I have read only access to using the same windows account. Can I still query from DB2 if my application is only connected to DB1?

When I execute stored procedures in DB1 that also query stuff from DB2 it works on SQL Server Management Studio but I get an error when I try to do it on my application.

Name an application to stream audio (from files and/or mic) from pc to multiple android devices

I was working on a project to make mini radio for kids. We will be in a place with no service so streaming audio over internet is not an option. So I brought 3 routers and connected them together and I want to stream audio from my pc over the wifi network. in short I need the receivers to type an IP in the application to receive my stream.

p.s. I found a great app called soundwire, it does stream audio from an app on pc over the LAN to android devices; however, it can stream to only one device. I need to stream to more than 35 devices.

How do I get my Django + React Application running in production mode on apache2 with Pipenv?

I am doing a group project with Django and React and each of us has been given its own VM to run it on. At the moment we are only able to start a development server for Django with the “runserver” command and a server to compile our react code with “npm start”. The backend is then accessible on port 8000 and the frontend on port 3000. We are now trying to get this configuration running so that it is continually accessible and we can configure HTTPS. We have tried many different internet tutorials (and probably broke some things underway) but we can’t seem to get it running. Last one we tried was https://docs.djangoproject.com/en/2.1/howto/deployment/wsgi/modwsgi/ The server is still not accessible at its IP. Our VM is a Debian GNU/Linux 8.11 (jessie), we are using Pipenv version 2018.10.13 as out virtual environment with Python 3.4.2 and Django 2.0.10. Our apache is ver. 0.91-ubuntu1.

Here is my apache2.conf:

# This is the main Apache server configuration file.  It contains the # configuration directives that give the server its instructions. # See http://httpd.apache.org/docs/2.4/ for detailed information about # the directives and /usr/share/doc/apache2/README.Debian about Debian specific # hints. # # # Summary of how the Apache 2 configuration works in Debian: # The Apache 2 web server configuration in Debian is quite different to # upstream's suggested way to configure the web server. This is because Debian's # default Apache2 installation attempts to make adding and removing modules, # virtual hosts, and extra configuration directives as flexible as possible, in # order to make automating the changes and administering the server as easy as # possible.  # It is split into several files forming the configuration hierarchy outlined # below, all located in the /etc/apache2/ directory: # #       /etc/apache2/ #       |-- apache2.conf #       |       `--  ports.conf #       |-- mods-enabled #       |       |-- *.load #       |       `-- *.conf #       |-- conf-enabled #       |       `-- *.conf #       `-- sites-enabled #               `-- *.conf # # # * apache2.conf is the main configuration file (this file). It puts the pieces #   together by including all remaining configuration files when starting up the #   web server. # # * ports.conf is always included from the main configuration file. It is #   supposed to determine listening ports for incoming connections which can be #   customized anytime. # # * Configuration files in the mods-enabled/, conf-enabled/ and sites-enabled/ #   directories contain particular configuration snippets which manage modules, #   global configuration fragments, or virtual host configurations, #   respectively. # #   They are activated by symlinking available configuration files from their #   respective *-available/ counterparts. These should be managed by using our #   helpers a2enmod/a2dismod, a2ensite/a2dissite and a2enconf/a2disconf. See #   their respective man pages for detailed information. # # * The binary is called apache2. Due to the use of environment variables, in #   the default configuration, apache2 needs to be started/stopped with #   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not #   work with the default configuration.   # Global configuration #  # # ServerRoot: The top of the directory tree under which the server's # configuration, error, and log files are kept. # # NOTE!  If you intend to place this on an NFS (or otherwise network) # mounted filesystem then please read the Mutex documentation (available # at <URL:http://httpd.apache.org/docs/2.4/mod/core.html#mutex>); # you will save yourself a lot of trouble. # # Do NOT add a slash at the end of the directory path. # #ServerRoot "/etc/apache2"  # # The accept serialization lock file MUST BE STORED ON A LOCAL DISK. # Mutex file:$  {APACHE_LOCK_DIR} default  # # PidFile: The file in which the server should record its process # identification number when it starts. # This needs to be set in /etc/apache2/envvars # PidFile $  {APACHE_PID_FILE}  # # Timeout: The number of seconds before receives and sends time out. # Timeout 300 # # KeepAlive: Whether or not to allow persistent connections (more than # one request per connection). Set to "Off" to deactivate. # KeepAlive On  # # MaxKeepAliveRequests: The maximum number of requests to allow # during a persistent connection. Set to 0 to allow an unlimited amount. # We recommend you leave this number high, for maximum performance. # MaxKeepAliveRequests 100  # # KeepAliveTimeout: Number of seconds to wait for the next request from the # same client on the same connection. # KeepAliveTimeout 5   # These need to be set in /etc/apache2/envvars User $  {APACHE_RUN_USER} Group $  {APACHE_RUN_GROUP}  # # HostnameLookups: Log the names of clients or just their IP addresses # e.g., www.apache.org (on) or 204.62.129.132 (off). # The default is off because it'd be overall better for the net if people # had to knowingly turn this feature on, since enabling it means that # each client request will result in AT LEAST one lookup request to the # nameserver. # HostnameLookups Off  # ErrorLog: The location of the error log file. # If you do not specify an ErrorLog directive within a <VirtualHost> # container, error messages relating to that virtual host will be # logged here.  If you *do* define an error logfile for a <VirtualHost> # container, that host's errors will be logged there and not here. # ErrorLog $  {APACHE_LOG_DIR}/error.log  # # LogLevel: Control the severity of messages logged to the error_log. # Available values: trace8, ..., trace1, debug, info, notice, warn, # error, crit, alert, emerg. # It is also possible to configure the log level for particular modules, e.g. # "LogLevel info ssl:warn" # LogLevel warn  # Include module configuration: IncludeOptional mods-enabled/*.load IncludeOptional mods-enabled/*.conf  # Include list of ports to listen on Include ports.conf # Sets the default security model of the Apache2 HTTPD server. It does # not allow access to the root filesystem outside of /usr/share and /var/www. # The former is used by web applications packaged in Debian, # the latter may be used for local directories served by the web server. If # your system is serving content from a sub-directory in /srv you must allow # access here, or in any related virtual host. <Directory />         Options FollowSymLinks         AllowOverride None         Require all denied </Directory>  <Directory /usr/share>         AllowOverride None         Require all granted </Directory>  <Directory /var/www/>         Options Indexes FollowSymLinks         AllowOverride All         Require all granted </Directory>  #<Directory /srv/> #       Options Indexes FollowSymLinks #       AllowOverride None #       Require all granted #</Directory>     # AccessFileName: The name of the file to look for in each directory # for additional configuration directives.  See also the AllowOverride # directive. # AccessFileName .htaccess  # # The following lines prevent .htaccess and .htpasswd files from being # viewed by Web clients. # <FilesMatch "^\.ht">         Require all denied </FilesMatch>   # # The following directives define some format nicknames for use with # a CustomLog directive. # # These deviate from the Common Log Format definitions in that they use %O # (the actual bytes sent including headers) instead of %b (the size of the # requested file), because the latter makes it impossible to detect partial # requests. # # Note that the use of %{X-Forwarded-For}i instead of %h is not recommended. # Use mod_remoteip instead. # LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %O" common LogFormat "%{Referer}i -> %U" referer LogFormat "%{User-agent}i" agent  # Include of directories ignores editors' and dpkg's backup files, # see README.Debian for details.  # Include generic snippets of statements IncludeOptional conf-enabled/*.conf  # Include the virtual host configurations: IncludeOptional sites-enabled/*.conf  # vim: syntax=apache ts=4 sw=4 sts=4 sr noet  # TODO LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so # This is just the root of your virtualenv directory, i.e. two levels above activate WSGIPythonHome /root/.local/share/virtualenvs/backend-nwAt4pcf/bin/ WSGIPythonPath /root/.local/share/virtualenvs/backend-nwAt4pcf/bin/python3.4   <Directory /var/www/>   Options Indexes FollowSymLinks   AllowOverride All   Require all granted </Directory>  

And my backend.conf:

Listen 8000 <VirtualHost *:8000>   ServerName webspec68.mi   ServerAdmin xxx@xxx-xxx.xx   DocumentRoot /var/www/html/puppy-webshop/backend    # Point this to the wsgi.py in the same directory as your settings.py file   WSGIScriptAlias / /var/www/html/puppy-webshop/backend/backend/wsgi.py    <Directory /var/www/html/puppy-webshop/backend>     <Files wsgi.py>       Require all granted       Allow from all     </Files>   </Directory> </VirtualHost>  

And finally the frontend.conf:

<VirtualHost *:80>   ServerName webspec68.mi   ServerAdmin xxx@xxx-xxx.xx   DocumentRoot /var/www/frontend    # Serve static files like the minified javascript from npm run-script build   Alias /static /var/www/todo/frontend/build/static   <Directory /var/www/todo/frontend/build/static>     Require all granted   </Directory> </VirtualHost>  

The error.log is constantly logging these errors:

"ImportError: No module named 'encodings' [Sun Jan 20 17:59:59.887686 2019] [core:notice] [pid 20413:tid 140450917902208] AH00052: child pid 20430 exit signal Aborted (6) [Sun Jan 20 18:00:00.889711 2019] [wsgi:warn] [pid 20431:tid 140450917902208] (13)Permission denied: mod_wsgi (pid=20431): Unable to stat Python home /root/.local/share/virtualenvs/backend-nwAt4pcf/bin/. Python interpreter may not be able to be initialized correctly. Verify the supplied path and access permissions for whole of the path. Fatal Python error: Py_Initialize: Unable to get the locale encoding 

The project is currently under /var/www/html/my-project/. Should our backend part be in /var/www/backend and our frontend in /var/www/frontend? Also the WSGIPythonHome and WSGIPythonPath are just a guess as I can’t find a tutorial for doing this with pipenv. These seem like the main problems for now. There is probably a lot wrong here but I am pretty new to all of this. Appreciate any help!