Lots of products in wrong category indexed by Google

Hi,

The part gezichtsverzorging-algemeen/lipverzorging of the URL is not the category but the menu item.
This means that somewhere, HikaShop had to generate a link to the product without knowing which menu item to use and selected a random one as it didn't know which one would be the correct one.
For example, if you use the Google products plugin to generate a XML of the products data, HikaShop doesn't know which menu item you want to use to access a specific product when generating the data of the product for the XML. So it will select one menu item to use for the URLs.
The solution is to fill the "canonical URL" setting of each product with the URL you want to be used for each product. That way, when search engine crawlers access the page of a product, the canonical URL will be provided to it and it will know you want this URL to be indexed and not others.
This also means that just adding the URL in the canonical URL setting won't immediately fix the URL indexed on search engines as they need to recrawl the pages and this can take weeks. So then you can resubmit a XML or a sitemap or ask to recrawl these URLs.

Hi,

Well, the problem is that what's important here is to understand where these faulty URLs are generated and when they were generated.

The "product" part of the URL corresponds to what you have entered in the "Product SEF name" setting of the HikaShop configuration.
So if you empty that setting the URL without the "product" part will be valid (and not the URLs with "product" in them).

So I could imagine that at some point you had URLs without the "product" generated because you had emptied the "product SEF name" setting, then search engines crawled these URLs and then you changed the "Product SEF name" setting to revert to "product" and thus the crawled URLs became invalid.
But it could also be that a third party extension (like a product slider module for example) is generating the URLs for the products and doesn't take into account the canonical URL and generates a faulty URL without the "product" part for some reason.
Or it could be that the canonical URL entered at some point was faulty.

Basically, that's hard for me to say what to do exactly from where I am because I don't know where these faulty URLs where found and when and thus I can't check the situation there and say "it comes from this part and you need to change that".
So the big question is do you have any information on where these faulty URLs (at least one) can be found on your website ?

Hi,

Let's breakdown the different things into different points.

1.

I did empty one field in de SEF configuration but that was a long time ago and it was the category SEF name and not the Product SEF name [att.1].
I read on this forum that you can only empty one SEF name.

I must admit I made a mess by not using canonical urls in the first place and I changed some menu links without redirecting them.
So now I'm trying my best to 'repair' all of this and all of my Dutch products have canonical urls now.
Not all my products are translated yet but I started last month and every product I translate is given a canonical url immediately.

If you look for instance on the Dutch product 'Kapper Schaar' than this product is last edited by me on may 13th. It already had a canonical url on the Dutch version and on may 13th I translated it and gave it an English canonical url.[att.2]

This is great to set the canonical URLs properly. The index might not take into account the changes immediately, but you're definitely on the good track in having a clean referencing of your products.

2.

One goes to the product but with the wrong url: /component/hikashop/product/kapper-schaar instead of the right category Styling & Accessoires.

But this url was crawled by Google on april 22nd. As you can see this product already had a canonical url but yet the search result goes to the wrong url /component/hikashop/product/kapper-schaar [att.5]

I do indeed see that first result on your link. I can also confirm that the canonical URL is correctly added to the header of the page.
However, everything is in Dutch on your screenshot 5 so I'm not sure what is what.
Also, I can see that you have "alternate" links on your headers which seem to be wrong and not take into account the canonical URLs of HikaShop that you setup in the product :
i.imgur.com/cm5o7MD.png
So it might be why you get that /component/hikashop/product/kapper-schaar URL indexed ?
HikaShop doesn't add these "alternate" tags to the page. So I'm not sure what is doing that.
Do you have Falang installed ? I think it might be linked to that:
forum.joomla.org/viewtopic.php?t=884219
Do you have a recent version of Falang ? Updating might help ?

3.

Now if you look at the second search result this page points to the English page without the translation (Hair Dresser Scissors) and in the wrong category so it gives a 404.
In the search results you see a partial translated text like Common Price and Our Price. The rest is stil in Dutch.
Might be logical because the product wasn't translated yet at the time.
But when I inspect this url in Search console you see that this url is also crawled on april 22nd including the Dutch canonical url.[att.6]
So why is it still pointing to the wrong category including a 404 page?

That's not what I see on your link. For me, the second link points to the URL /gezichtsverzorging/gezichtsverzorging-algemeen/probleemhuid/product/kapper-schaar
That page displays in Dutch completely. It has the same canonical URL in the headers so this should be fine. However, I still see a problem with the "alternate" links:
i.imgur.com/YsAufqb.png

4.

But the Dutch page has a different redirection page, the same strange "pathway" url as I spoke about in one of my previous posts.[att.7]
And suddenly there's no canonical url anymore or at least n.v.t. (means N/A) allthough it's listed in the OSMap sitemap.

That's strange. If I check that same sitemap URL, I do get the correct URL from the canonical URL of the product and no other URL for that product:
i.imgur.com/PI7l15K.png
I can see it was generated on the 13th of May so it should have been good already at the time.
The URL from your screenshot seems to indicate that you had turned off the SEF of your website when the sitemap was crawled on the 17th since that URL from screenshot 7 is the non-SEFed URL of the product when generated dynamically.
One thing though is that I would configure OSMap to reduce the change frequency to daily for the short term. Because right now, you're telling crawlers to not reindex the pages less than monthly. So that doesn't help getting consistent results with your changes.

5.

So I hope you understand that I don't know what to do anymore to get my pages/products properly indexed by Google.
I already have 382 Redirects in my .htaccess file [att.8]. And most of these are from product pages that don't give a 404 but from search results that DO point to the product but in the wrong category. Most 404 pages I handle in redirect management of Joomla itself.
So when will this stop and will I be finally ready with all these redirects?

From what I can see with this example, you're definitely on the right track. I would recommend continuing to fix the canonical URLs, modify the sitemap to reduce the change frequency, and check what is going on with the "alternate" links. Besides that, I think it's just a matter of time.

Hi,

1. On your page lichaam/dames/douche-bad you have two robots meta tags:
i.imgur.com/mkg6MW6.png
i.imgur.com/HjAv7eb.png
If you set the robots setting of the menu item to "index, follow" then HikaShop will add a "index, follow" robots meta tag to the page (so I suppose the first one is added by HikaShop).
If you set the "robots" setting of the menu item to "user global" and the global setting is set to "index, follow" then no robots meta tag will be added to the page as it considers it is not necessary.
So I suppose that before you changed the menu item's robots setting, you had only the second robots meta tag added by something else on that page. And now that you change, you have two tags.
So the question is where that second robots meta tag comes from on that page.
I suppose it comes from a third party plugin ( a SEO plugin probably ?).

2. Regarding the alternate meta data tags, I would recommend you to first check with Falang's support if you're using Falang.

3.

The URL from your screenshot seems to indicate that you had turned off the SEF of

I don't know exactly what you mean by that but I did not turn anything on or of hat has to do with SEF

I'm talking about the URL pointed by your arrow on this image: www.hikashop.com/media/kunena/attachment...5/007_2022-05-20.JPG
This URL is the URL of the page when SEF is turned off in the Joomla configuration. So when the SEF is activated, this URL should never appear on your website, unless you add the non SEF URL as "canonical URL" of the product with the "force canonical URLs on listings" setitng activated.

4.

One positive thing though. The pages with error in Search Console have reduced from 240 last week to 199 this morning.
Too bad you are only allowed to reindex 10 pages a day.

Yes, still a few things to iron out and reindex and you'l get a nice SEO of your pages.