Feature Request: Google Products Generation

  • Posts: 31
  • Thank you received: 0
10 years 9 months ago #144278

Could we get a way to "Trigger" the Google Products XML file outtside of using the Adminisration area? Once you have thousands of products, the script times out.

Even with "A Lot" set, our Trigger fails and gives us the empty XML file. We have to modify google_products.php to be "0" for timeout so PHP won't time out. And then we have to modify the Apache config to also have a "0" timeout. This is to keep Apache alive so we can actually download the XML file once PHP finishes building the file.

We could skip all the unlimited timeouts (which could lead to DOS) if we could kick off the Trigger from the PHP command line. PHP command line timeout is already set to be "0" so the script could run and generate the XML without any HTTP involved.

Thanks.

Please Log in or Create an account to join the conversation.

  • Posts: 82863
  • Thank you received: 13372
  • MODERATOR
10 years 9 months ago #144418

Hi,

In the latest version of HikaShop that we released last week, you can now configure the plugin to generate the xml file on the server automatically with the cron task.
That way, you would just have to change the google_products.php to set the timeout to 0 and you would retrieve the XML where the plugin stores it without having to change the apache configuration.
We'll also add an option to set to timeout to 0 in the options of the plugin in next version.

Please Log in or Create an account to join the conversation.

  • Posts: 31
  • Thank you received: 0
10 years 9 months ago #144421

Awesome! Joomla did not notify us that an extension was out of date!!

Thanks, we'll give it a try.

Please Log in or Create an account to join the conversation.

  • Posts: 31
  • Thank you received: 0
10 years 9 months ago #145335

Joomla Error Reporting set to Maximum
PHP max memory set to 4GB
PHP timeout set to never timeout
Apache timeout set to never timeout
After 2 days of generating our product XML file, we still get a 500 error in the browser. No errors are reported anywhere, and the file does not show up on disk.

The command line option will work better for shops as they keep growing. Will Hikashop never really work with 50k products and/or 3k categories??

Please Log in or Create an account to join the conversation.

  • Posts: 82863
  • Thank you received: 13372
  • MODERATOR
10 years 9 months ago #145359

Of course that you still get the error when you do it via the browser. It's not meant to do that for such an amount of products and that's why I recommended you to use the new options to generate the XML files automatically so that you can just have a download link for the xml file.
It's not really a matter of HikaShop. No matter what you do, having a process loading 50 000 elements with all their information (prices, discounts, variants, images etc) on one single PHP process requires a lot of resources.
On a normal products listing on your website, you display 20 products at once with a pagination to see the rest so 50K products is not a problem. But here, the system has to load everything on the page, and that's why you get that resources issue. Normally, with so much products, you wouldn't generate the XML dynamically each time someone request it. Instead, you would have it generated automatically once every day and then provide the link to the file (the same method I recommended in my previous message).

Please Log in or Create an account to join the conversation.

  • Posts: 31
  • Thank you received: 0
10 years 8 months ago #146402

The file is never generated with the new method. No worries, we use the Hikashop database + Java to generate the XML file and it works great. Google received all 85000 products successfully. It appears the queries take too long with the number of products we have. The long queries mixed with PHP just don't seem to be a good mix. We create the 100MB XML file for google using Java in 32 seconds, queries included. Not bad!!

I think that once more of the hikashop users start getting bigger shops, then more attention can be paid to the slow performance. For now, it's ok with most users having smaller shops.

Thx.

Please Log in or Create an account to join the conversation.

  • Posts: 82863
  • Thank you received: 13372
  • MODERATOR
10 years 8 months ago #146450

Yes, HikaShop is first designed for small shops even though it works better than a lot of mainstream solutions when you have a lot of products. Now, if you already have 30 secs of processing in Java (which is a hell faster than PHP) with your own queries (which are made for your specific case), it's no wonder that our script has a hard time doing the same thing. In such cases, not using PHP at all is better for the task :)
Don't hesitate to share your java script if you want. I'm sure it could help someone else in the future with the same need.

Please Log in or Create an account to join the conversation.

  • Posts: 223
  • Thank you received: 1
10 years 6 months ago #156559

We are having the same issue...

Would you be willing to share your script?

Thank you

Please Log in or Create an account to join the conversation.

  • Posts: 344
  • Thank you received: 3
8 years 1 day ago #255840

Joomla 3.6.4.
Hikashop 2.6.4

I have 10 000 products in my store, and until now I've just clicked the "trigger" function in to create the google xml file, and manually upload it to Google.

But on my demo page I've added another 10 000 products (now total count 20 000). So now when I click "trigger" I get 500 error timeout.

My host have incresed the creation time for the server, but I still get the 500 error timeout.

Is there any other way to create the feed ?
Without automatically upload it into google.

Please Log in or Create an account to join the conversation.

  • Posts: 82863
  • Thank you received: 13372
  • MODERATOR
8 years 21 hours ago #255845

Hi,

Maybe the problem is not with the max execution time, but with the memory limit.
If you double the amount of products, the system will require twice the amount of memory in order to store all the product data coming from the database and twice the processing time.
So both limits should be increased. Then the trigger should work fine.
Even if you automatically upload it into google, you'll have the same problem as it will still require loading all the product data into memory. So really, increasing the limits of time and memory is the solution here.

Please Log in or Create an account to join the conversation.

Time to create page: 0.299 seconds
Powered by Kunena Forum