The right way of doing images manipulation in a serverless app

This question is not specific to images, but this is the current issue im having so im using this as an example.

So basically, I have some kind of a pipeline that some data(in this case, an image), has to go through before It can be used in my app(in this case, a web app).

If we split the pipeline into functions it would look like this: 1. The image is uploaded to a storage(cloud storage). 2. The image is converted into multiple formats and sizes. 3. All of the different sizes and formats are registered in the database in relation to the original file.

My question is, what would be the right way of managing this process, and keeping track of errors, retries, updating the front-end etc.

I think im trying to solve this task with the wrong mindset, and thus im getting a bad solution. But maybe functions is not the right way. Maybe I need to opt out of cloud functions, and use some other serverless solution, like for example “Google’s AppEngine”.

Currently, I have a function with an HTTP endpoint that: 1. Receive the file and stream it into the Cloud storage(I do this because I try to avoid a heavy front end). 2. Save the name of the file in the DB with the temporary name, and status(status is currently waiting for optimization/cropping). 3. The upload into Cloud Storage triggers another function that starts the optimization and cropping of the image. 4. Update the DB with the status and all of the subfile names. 5. delete the original file.

The problem is that between 2 and 3, there is a broken link. I can’t really know what is going on on the client side without multiple requests to the DB to see the status of the image.

So is this the right way? is there any better way? I found some similar problems all over my app development, like for example with data validation, and that made me think that maybe im getting it wrong.

♞♞♞ SEOSpartans Reliable Catch-all Emails ♞♞♞ Stop Worrying About Emails RIGHT NOW ♞♞♞ FREE Trial

         NO MORE         
 DISPOSABLE EMAILS!!! 
It’s Time To Up Your Game!
———————————————————————
   You need something that’s     
     100% reliable     
 
image
     You need a simple      
     set & forget solution     
  
image
  
     You need something that     
     doesn’t break your bank!     
image
  
———————————————————————
And most importantly… 
You need to  stop worrying  about emails 
  once and for all!   
———————————————————————
 
Presenting…
image
And our…
     GSA SER Catch-all Emails     
also known as…
The Most Reliable
Email Solution
On The Planet
———————————————————————
   This is a NextGen Blackhat Email Solution   
  • 100% Private Emails – No sharing emails between users or similar nonsense. Only you will be able to access the inbox of your email(s). Period. 
  • Unlimited Use – You can use our emails on 10 projects on a single server OR on 1000 projects across 10 servers, we don’t really care! Your only limitation is the 300mb inbox / email. And let me tell you…that’s A LOT of emails.
  • Instant Account / Email Setup – Our backend is completely automated! As soon as you purchase a package and create an account, you will have a unique download link waiting for you in members area so you can get started RIGHT AWAY.
  • 1-click Import To GSA SER – Your email(s) will be in a GSA SER format, with spintax and all the settings pre-configured. You just need to download the .txt file, import it to as many projects as you wish and click start!
  • 300mb Inbox / Email – You would have to be spamming really hard with 200+ LPM 24/7 to fill up just one 300mb inbox. But if you want to do it, that’s perfectly fine!
  • Automatic Inbox Cleanup – Our system automatically deletes all emails older than 5 days. This will make ensure a fast connection and email download and at the same time prevent your inbox getting maxed out.
  • Cancel Anytime – If for some weird reason you don’t want to use our service anymore, just cancel your account directly from your dashboard and be done with it!
   We Laugh In The Face Of Blacklists   
  • UNLIMITED Alias Generation – All you need is a big-ass spintax before the @ (which is already included in the SER template) and you have a never ending supply of unique emails. Specific email blacklisting is a thing of the past.
  • Branded Looking Domains – Each email you get is a unique subdomain on a “branded” domain. So these aren’t the typical “…@a3k5.kfjte84.com” looking emails that scream SPAM at first glance. No no, our emails look as legit as possible and this alone eliminates a lot of potential blacklistings.
  • Swap Emails On Request – Even if some webmasters might blacklist your entire email subdomain (the part after @), it’s absolutely no problem! You can request a new catch-all email (on a new unique subdomain) once per month and never look back! But most of the time this isn’t necessary and you can use the same email for a long time.

   We Have Serious Buletproof Hardware   
  • Dedicated Email Servers – We’re dead serious when it comes to hardware. No baby league stuff like VPS or (lol) shared hosting. We’re running top of the line hardware in the form of super fast dedicated servers.
  • Multiple SSDs Per Server – Each server comes with multiple redundant SSDs to ensure maximum speeds & reliability when processing emails. That means instant almost email retrieval via pop3.
  • Lightning Fast POP3 Access – Dedicated servers, SSDs, 1gbps lines? Nuff said.
  • Advanced Server Monitoring – Our servers are CONSTANTLY monitored and there are a number of people at any given time watching over them. Their job is to make sure that servers run as smooth as possible. And in the rare case that something does happen, they’re instantly on top of the problem to ensure an almost perfect guaranteed uptime.
  • 99.9% Uptime Guarantee – If for some cosmic-reason-that’s-yet-to-happen we don’t uphold our 99.9% uptime guarantee in any given month, you will get a full refund!
————————————————————————–
 
 Stop Worrying About Emails 
          Once And For All!          
 
 
Start Your FREE Trial NOW! <
 
 
*For more info on the recommended LPM, pricing and other stuff, please visit our site.
————————————————————————–

                   FAQ                   

   Q: What is a catch-all (master) email anyway?   
Catch-all is a mailbox that will “catch” all of the emails sent to non-existent email addresses on that specific domain. 
For example: If your catch-all email is “catch@domain.com”, you can register on the same site with “john@domain.com” and “lucy@domain.com” (and as many other aliases as you want, even tho these email addresses technically don’t exist) and receive all confirmation links in 1 mailbox.
That means that you can make as many accounts as you want on a specific site using “imaginary” emails and confirm all of them by using 1 “master” email in your projects. Pretty cool, huh?
   Q: Why is this better than using thousands of cheap disposable emails?   
First of all, are you sure that those 1000 disposable emails will work in a tomorrow / next week / next month? 
With our catch-all emails, you can be sure they will work until the end of time. Also, they’re ridiculously “hands-off” and simple to use. You just import the txt file (which you can download IMMEDIATELY after paying and registering on our site) in your template projects and you’re done thinking about emails for a few months. 
So every few month you just swap your emails for free, select all of your projects and import your new ones (click yes on the “remove old email accounts?” prompt) and you’re done!
   Q: How does SER handle catch-alls?   
SER does a very good job at handling catch-all emails out of the box:
– It connects to the master email via POP3
– It correctly tracks which emails belong to which project
– It uses a spin macro to create a new unique alias on each new submission
Simply put, once you give SER a catch-all email, it will automatically and correctly handle everything no matter how many projects use that same email.
   Q: Why wouldn’t I just setup a catch-all email myself?   
Sure, go ahead! 
But it’s not just a matter of buying a cPanel shared hosting and a domain, creating a catch-all and blasting away. 
First of all, shared hosting just doesn’t cut it. 
When you’re on shared hosting (or better said, when you don’t have root access), there’s a firewall standing between you and the server and there’s nothing you can do about it. 
That means that you’re going to have to go real easy on POP3 if you don’t want to get your IP banned. Sure proxies and longer time between logins helps, but it’s hardly ideal. Not to mention the fact that many hosting providers won’t be happy about what you’re doing on their shared server, especially since catch-all emails properly abuse the server’s disk.
The next step would seem to be a VPS. That way, you can tweak the firewall as needed and providers are generally much more easy going about what you’re doing. However, you’re still abusing the disk that other people are using. And this means trouble. 
But even if you manage to get a VPS for < $ 10, you still need a cPanel license which is around $ 15 / month. Not to mention the fact that it needs to be a min of 1gb ram server to operate “efficiently” (more like barely running). 
So you’re already looking at $ 25+ / month. 
On top of that, there’s setting everything up, taking care of the server, etc. And at that price point the VPS is semi-managed at best. You better know your way around WHM as well as cPanel!
   Q: Will the email I get be private?   
Yes, 100% private, there’s ABSOLUTLY NO SHARING involved.
   Q: How does your pricing work?   
Simple, you’re paying the monthly fee as long as you’re using our emails.
   Q: What is your refund policy ?   
In the practically impossible case that you experience less than 99.9% uptime in any month, just contact our support and we will refund your full monthly fee, no questions asked!
   Q: How soon can I get started?   
Within 1 minute in 3 steps! 
1. Go to our site and choose your plan. 
2. Pay the bill and you will be redirected to a page where you will create your account 
(Make sure that you let paypal redirect you naturally otherwise you won’t be able to instantly create an account.)
3. Go to your members area, login with your credentials and there will be a .txt file with email(s) that you purchased waiting for you, ready to be imported to SER in 1 click.

Right menu links

My company website has main page (header images, our services, testimonials, contacts) and few store pages: all products page, single product page (only catalog, no buying function).

My headeache is what items should I use in menu.

I have two ideas:

  1. Home // Catalog
  2. Only Catalog (instead of Home item use clickable logo)

What can you recommend to do? Is only one item menu good for UX?

What is the right term/theory for prediction of Binary Variables based upon their continuous value?

I am working with a linear programming problem in which we have around 3500 binary variables. Usually IBM’s Cplex takes around 72 hours to get an objective with a gap of around 15-20% with best bound.In the solution, we get around 85-90 binaries which have value of 1 and others are zero. The objective value is around 20 to 30 million. I have created an algorithm in which I am predicting (fixing their values) 35 binaries (with the value of 1) and letting the remaining ones solved through the Cplex. This has reduced the time to get the same objective to around 24 hours (the best bound is slightly compromised). I have tested this approach with the other (same type of problems) and it worked with them also. I call this approach as “Probabilistic Prediction”, but I don’t know what is the standard term for it in mathematics?

Below is the algorithm:

Let y=ContinousObjective(AllBinariesSet); WriteValuesOfTheContinousSolution(); Let count=0;  Let processedbinaries= EmptySet; while (count < 35 ) { Let maxBinary =AllBinariesSet.ExceptWith(processedJourneys).Max();//Having Maximum Value between 0 & 1 (usually lesser than 0.6)             processedJourneys.Add(maxBinary); maxBinary=1; Let z = y; y = ContinousObjective(AllBinariesSet); if (z > y + 50000)                  { //Reset maxBinary maxBinary.LowerBound = 0; maxBinary.UpperBound = 1; y = z; } else { WriteValuesOfTheContinousSolution(); count=count+1; }              } 

According to me, it’s working because the solution matrix is very sparse and there are too many good solutions.