Interface for users that need to fill in data for thousands of items

The problem I encounter is that I have thousands of items a day (e.g. transactions of toys that have been purchased in a store). My goal is to provide the users with an interface to fill in manually 10+ data fields on this transaction (e.g. toy category, is it the main toy or something you purchase as extra, what age is it for, etc.). Then, I would categorize the data by the information the users provided me. How would you recommend doing it?

Thanks!

Ask google to remove thousands of links in its index

I’ve had my website hacked. I have cleaned it and searched google site:site.com and made a list of all the results. Removed the actual links that are ok and made a list of the ones in the format of site.com/ad0-b1fermarte54eb17chb-1244425. I’ve submited these links to Google to remove and Google removed these links.

The problem here is that after my initial search, that resulted in 200 URLs which we’ve removed, other new links appeared out of nowhere. These links don’t actually work anymore and return a 404 page, but Google still has these in its results.

We’ve tried deleting our sitemap and re-submitting it. Google has crawled it, but again, it has not removed the dummy search results.

Looking in our google webmaster console, we saw that in the coverage tab the number of indexed links spiked from 230 (normal usage) to 10.900 links – I’m guessing these are all dummy links that were infected and now cleaned.

Any magic way of having Google remove dead links automatically? Or a way to force it to recrawl the entire website?

Thank you!

How do attackers hit a website with thousands of similar but distinct IP addresses?

I have a website that is being hit with invalid URL requests by thousands of distinct IP addresses, never the same one used twice. Most of them are in a few ranges of IP addresses and often just go up sequentially.

Could this be a zombie botnet of compromised devices, or is it possible the attacker is spoofing these addresses?

The clustering of IP addresses into a handful of ranges seems inconsistent with what I would expect from random devices across the world being compromised and part of a botnet.

User agents are all legitimate and quite varied, but I know that is simple to spoof.

It doesn’t feel like a DDOS attack as it is “just” a few thousand per hour. If they really wanted to DDOS it seems like they would crank the volume up more. Once I adjusted some exception handling I was able to get my server to resume being responsive to legitimate use.

I suspect it is a malicious (poorly constructed) crawler/spider.

Is IP address spoofing easily done and common now in these scenarios?

Combobox with a lot of (thousands) of options

in my project there is a form. One of the field is problematic for me. User should choose an option from a very long list (thousands of options). I wanted to do it by combobox, but I don’t know how searching should works.

We can’t display options after typing by user first letter, because list will be still very long .

I thought that we could display options after 3 letters.

Example:

User have to choose one name from the list (10 000 of names).

enter image description here

Is it clear? Do you have any ideas do it better?

The most important is that user must choose an option from the list (can’t type new name) and list is veeeery long.

transcribe thousands of words from PDF or image to words for $5

I will do your data entries 100% accurately. I have experience in working in large batches of data entries. I can extract data from pdf, online doc or images to be extracted and entered into any application like Word or Excel. Services provided: 1.PDF to Word 2.Data scrape from website to Word 3.Image to Word 4.Image table to Excel 5.PDF table to Excel 6.Copy paste to Word / Excel Regard: USMAN

by: 94usman
Created: —
Category: Data Entry
Viewed: 170


How to modify and compile ONLY few java files among many thousands and deploy?

This is basically a code development question that deals with frequent changes to one or more java files among many thousands.

A few years ago, as a software intern, I was given a large java / jsp code base and my mentor set me up using intellij IDE to build a WAR file for a web application (a student portal). I would then copy the War file to the tomcat container and launch it. I don’t recall, but I recall to the effect that all I had to do was to press that one button (build?) to generate the WAR file. It took a long while to generate this War file. Intellij used an Ant file to build the war file. The team later migrated to Maven, but that’s when my term was up.

Software development was very slow because I wanted to write code, compile and test code often in the web browser. There wasn’t unit code testing.

So here’s my question:

What does the industry do (for example FANG) where if I wanted to just modify ONE java file, how would I go about in a much faster fashion to compile that one file instead of recompiling every file to create a WAR file and then put it into the tomcat container and run it?

I just want to minimize the amount of time from inserting code to checking out how it behaves.

I tried to write my own batch script but taking into account all the thousands of java dependency was a nightmare. this this post. This is my first post since many years ago. I appreciate for someone to shed light on how I can approach this.

I wasn’t happy with using this IDE solution because there was a very senior software engineer who didn’t use an ide, isntead, used emacs on his mac. He got things done fast, but I didn’t understand how he would go modify a few files and got it to run on tomcat. I can’t imagine that he’d recompile everything. So please share with me some work flow using IDE and also preferably not to use an IDE.

I did my research and google but couldn’t find an examnple tutorial of how such would be done. One reason I ask this question is because if I ever get a software engineering job, I would not want to modify one file and take forever to recompile everything.. it just doesn’t sound like the way to go. This wasn’t taught in school.

If there is a git project that deals with the above.. I’d be grateful if someone can send me an exampile git project that deals with the above webapp scenario.

Thank you. Gordon

Communication Architecture for hundreds or thousands of devices deployed in field [on hold]

I’m building an application (both client and server side code) where probably thousands of devices will be deployed in field and these devices will be connected to internet via cellular connection. Each device will have a SIM in it. Now, these devices are embedded devices with very limited capability in terms of software options. The only comm protocol support by these are TCP, UDP and HTTP/s(server and client). You can’t even add any external libraries to support more protocols like MQTT etc. The development environment is very closed and only allows very specific and limited type of external libraries to be added.

So I’ve decided not to use UDP and TCP because they’re very very low-level and the development time and hence cost will be much high. So, I’m left with HTTP. The application requires to poll some data from the devices at defined intervals and sometimes on-demand as well(when the user requests). This suggests to put the server on devices, and just use GET/POST etc. to acquire the data periodically or whenever the user demands. But since these devices will be put in very remote and geographically different locations, we’ll have to assign a fixed IP(or a domain name) to each device so that the server can send requests. Now this architecture doesn’t seem very good because managing the IPs will be a huge headache.

The last option I have is to put a HTTP server on the actual physical server, and have each device ping periodically(like every 5 or 10 seconds ?) via POST request, for example, and as a response to this post request the server can ask for whatever it wants in json format, be it new data or any configuration changes etc. This ping will also act as a heartbeat. On the server side, due to some limitations I’m bound to use python. So I’m thinking of using Flask with some webserver like Nginx etc.

I need some suggestions if this is the only way I can implement this or are there any better solutions/architecture ? Also, will the flask+nginx be able to handle thousands of requests ? Or are there better options to implement the webserver in python ?

Magento 2 generates thousands of temp table queries on product update

I have a Magento 2.2.5 store with ~20k products and ~250 categories arranged in a tree of maximum 2 sub categories.

One of the top categories has 5k products and when I try to remove a product from the category, or remove the category for a specific product the server takes more than 2 minutes and times out.

After reading the general_log, I saw thousands of queries like this:

SELECT `e`.* FROM `tmp_select_e5e424ef98e4a8a4026e18df9d8088e0` AS `e` WHERE (hash_key = '1_44083') 

peaking the database server to 100%: enter image description here

I think this has something to do with url_rewrites, but I can’t seem to understand how is this a problem with only 5k products.

Are there any configurations I can set maybe loosing functionalities, to be able to do something as simple as removing a product from a category?

Thanks a lot.

Server Hunter – Easily browse thousands of virtual and dedicated servers from hundreds of providers

Server Hunter is an online platform for searching and comparing virtual and dedicated server plans from hundreds of different hosting providers. Offers and pricing are automatically retrieved and updated every 24 hours. Visit us at serverhunter.com.

Now you can easily search and compare among thousands of virtual, hybrid, and dedicated servers from hundreds of providers. Learn more about Server Hunter: https://www.serverhunter.com/about/.

Speeding up thousands of string parses

I have a mapping application which takes string arguments in the form of string arrays. I parse these and then perform an action on the map. One of these is MoveVertex which lets the user pass in the name of an existing shape/feature on the map, the index of the vertex to move, and a new x,y location to move that vertex to.

I’m currently getting thousands of these in a row, all for the same shape. It takes anywhere from 3-10ish seconds to parse them all and complete the action. I’m looking for any improvements I could make.

One thing I did so far was stop the map from refreshing every time a vertex is moved. This helped a bit, but it’s still slow.

Example of the string[] commands being sent:

command[0] = "move" //command type command[1] = "vertex" //what to move command[2] = "testLine[10]" //move the vertex at index 10 in testLine's list of vertices command[3] = "x" //new x value for the vertex command[4] = "y" //new y value for the vertex 

Here’s the function that does the work:

public static void MoveVertex(string[] command, Layer inMemoryFeatureLayer, Map map) {     int index; //this will be the index of the vertex in the collection of vertices of the shape     string featureName; //name of the shape/feature that has the vertex to move      if (command.Length < 5) //can't be more than 5 parameters     {         return;     }      try     {         if (!command[2].Contains('[')) //if it doesn't contain brackets which contain an index of the vertices         {             return;         }          const string pattern = @"\[(.*?)\]";         var query = command[2];         var matches = Regex.Matches(query, pattern); //Gets anything inside the brackets         index = Convert.ToInt32(matches[0].Groups[1].Value); //should be an int          featureName = command[2].Substring(0, command[2].IndexOf('[')).ToUpper(); //everything before the bracket is the name of the object     }     catch (Exception ex)     {         return;     }      if (!double.TryParse(command[3], out double longitude) || !double.TryParse(command[4], out double latitude))     {         return;     }      try     {         BaseShape shape;          inMemoryFeatureLayer.Open(); //make sure the layer is open         if (inMemoryFeatureLayer.Name.Contains(GlobalVars.LineType)) if it's the layer holding line shapes         {             shape = (LineShape)inMemoryFeatureLayer.FeatureSource.GetFeatureById(featureName, ReturningColumnsType.NoColumns).GetShape();             ((LineShape)shape).Vertices[index] = new Vertex(longitude, latitude); //set the vertex to a new location         }         else //it's the layer holdilng polygone shapes         {             shape = (PolygonShape)inMemoryFeatureLayer.FeatureSource.GetFeatureById(featureName, ReturningColumnsType.NoColumns).GetShape();             ((PolygonShape)shape).OuterRing.Vertices[index] = new Vertex(longitude, latitude); //set the vertex to a new location         }          inMemoryFeatureLayer.EditTools.BeginTransaction();         inMemoryFeatureLayer.EditTools.Update(shape);         inMemoryFeatureLayer.EditTools.CommitTransaction();         map.Refresh(); //this won't happen because I suspend refreshes beforehand     }     catch (Exception ex)     {         //log it     } }