I crawled my WordPress site using the Screaming Frog program and noticed a few issues which might have SEO implications and was hoping someone could help identify the problem (if there is one).
N.B: I have obscured the domain name in the image as I do not want it made public.
The canonical URL for my domain is prefixed by
https://www but as you can see from the tree graph image below, there are a number of URLs accessible under the
http protocol and the non-
www version of the domain. Those URLs are non-indexable but I’m curious as to why they’re accessible at all. I was thinking that shouldn’t happen if the proper redirects were in place.
In the WordPress admin settings I have entered the correct version of my domain for the WordPress and site addresses (i.e.,
It’s possible my
.htaccess file might require editing so I have pasted it below:
My company is renting a CentOS server, with one main domain and ~30 subdomains (the number varies, but it’s usually around 30) hosted on it. The subdomains themselves are used for various purposes (mostly development), and are usually deleted once they outlive their use.
Recently it has come to our attention that some of the search engines have indexed some of our defunct and some of our active subdomains.
What would be the best way of preventing search engines and crawlers from crawling and indexing the main site and its subdomains?
I am aware that it’s possible to create a
robots.txt file with
noindex, nofollow, etc directives, but I’d like to avoid it, if possible (going through ~30 subdomains, setting up a new
robots.txt when a new subdomain is created, etc).
I’m also aware that it’s possible to use:
<IfModule mod_headers.c> Header set X-Robots-Tag "noindex, noarchive, nosnippet, nofollow, noodp, noydir" </IfModule>
.htaccess. What I’m unsure of is whether I could place this directive inside of main domain’s
.htaccess, in hopes of preventing crawling for both the main domain and its subdomains.
Is there a better way of doing this? I’ve come across a StackOverflow answer that
Alias can be used in
robots.txt. Can this be used for
.htaccess as well?
We are developing an app, aprox 50k RPM read, 1k RPM write, it ask via key and get a JSON. The search is always made by key. I’m inclined to use one MySQL 8 table with a Id field and JSON field with innodb. Seems simple, cheap and fast accessing by index. Each index can have n rows (30 max), total size of table less than 100gb. Response time is important, I think 2-10ms are achievable on MySQL. The other, more expensive options that I have are DynamoDB and ElasticSearh (can’t use another tool). Can’t find a comparison for this use case to help me know if I’m in the correct path. Do you see any cons of using MySql or I’m missing something? Thanks!!
Can you name other search engines than Google?
Does it hinder processing a lot, making it slower?
Would you mind elaborating a bit more on the new engines added to SER? I’d like to know their category and names.
I understand that game engines have their own level design methods and different implementations of levels but why don’t they allow full control of the level design to the game developer? Without actually stating that, "The game developer should do/ use this to get this." rather than "Use these tools to get this done. Have full control over your product." ideology?
I guess its because not all game developers are interested in dealing with low level aspects of the game development phase but wouldn’t it be more efficient if the user can optimize certain aspects of the game to better suit for their product? For example, a mesh object, resource submissions, or other areas which game engines define a certain method?
since the recent update which i just updated to 14.75
i cannot select the engines on the left (when the engines are showing) & cannot right click on engines on left (i.e. to select just dofollow engines)
the engines on the left only show when you restart gsa ser, other wise there is just a blank space on the left where the engines should be, it says “where to submit 343” but below that just a blank space, this is just since the lest update
I am not sure which stack exchange site to post a question like this, but:
I am particularly wondering if job search engines like Glassdoor, when they repost a job, delete the old job post or if there are duplicates on the website.
If you think this question should be posted somewhere else or should be taken down, let me know and I will do so!
my computer isn’t really great and I was wondering if there are game engines that do support development on my laptop’s specs, while there not trash, most game engines require Gpu which I unfortunately don’t have.
I’ve seen libGDX and Monogame which are java and C# game engines respectively that does meet with my specs.
My development experience isn’t low as I have made a couple of simple games and an AI in python. And have low experience java and C# (sorting algos)
If any of you know any Game engines with the specs below I’ll be grateful, thanks!
My specs are : 1.8 GHz processor 8 GB ram 256 GB SSD No GPU
PS : it’ll be great if it’s runs C# or java as I have experience in them but any other languages will work, thanks
Thanks in advance!