Hi, I hope ECT Users here are all well.

For many years I have been using A1 Sitemap Generator. Works fine, but the software is installed on my Desktop PC and must be run manually.

What would be much better is a Server Side sitemap.xml Generator that can be installed on a cPanel server and set to run automatically every 7 days. Yes, I can Google search. But maybe you have one already that works really well ?

If so, could you kindly post the details here ? All replies will be much appreciated.

Gary

Hi Gary,

Mike has one here that might work for you:
http://dlss.ca/proddetail.asp?prod=GoogleXMLSitemap

Once configured you have the option of submitting the PHP file to Google rather than the xml file.
This way it's always up to date.

You can also add non store pages in the config.

Regards,
David

Thank you David,
I have looked at Mike Beebe’s XML Sitemap Generator.
However, the LIVE DEMO file does not follow the Sitemaps.org XML File Protocol.
Also his script needs to be run manually in order to update, which is basically what I have now.

What I am looking for is a script that runs automatically, using a server side Cron Job scheduled task I imagine

Gary

Hi Gary,

Maybe Mike can come back on the Protocol but we submitted the PHP url in our Google Search Console rather than submitting an xml file so each time it gets fetched it's up to date with no need to run anything or set up cron jobs in cPanel to handle it.

Regards,
David

Gary,

I may need to look at the demo version I placed on the site as it has been there since inception.

With my script you use the schedule in Google to come and fetch your sitemap. Instead of using the XML file as the file to fetch, you use the PHP file (as David indicated) and when ever Google returns it will have a fresh set of results to that instant.

You will need to add all your static pages to the list manually.



Mike Beebe
President,
DataLinks Software Solutions
www.dlss.ca
Rate Me Here

ASP and PHP mods - www.dlss.ca/products.asp
A Tremendous Home - www.ATremendousHome.com
Buy Cigars Online www.CigarSmokeShop.net

Hi Gary,

It is very easy to install a cron job to run everyday:

0 0 * * * curl YOURWEBSITE.COM/GMerchLister.php >/var/www/html/gbase.xml 2>/dev/null
1 0 * * * curl YOURWEBSITE.COM/GoogleSitemap.php >/var/www/html/sitemap.xml 2>/dev/null

Just change to your website and the path and file name you want to save.

Has worked a treat for many years with no issue.

John

No need for the cron job with the sitemap generator.

As mentioned, in the search console, you just place GoogleSitemap.php instead of sitemap.xml.
When google hits the GoogleSitemap.php it will parse that and get the fresh sitemap.

I hope that clears things up.

David

Hi Gary,

We’ve been using xml-sitemaps.com (php install vers) for years on 3 sites.
Not completely straightforward to setup (you have to set some folder permissions)
Once uploaded to your server is is easily scheduled with a cron job.
Lots of options to choose from to produce exactly the sitemap you want but once setup you can pretty much forget it and it works very well.

Steve

I had no idea that you could enter GoogleSitemap.php in the search console instead of sitemap.xml. I wish I'd known as I've been manually updating the XML file generated by GoogleSitemap.php and uploading to my server periodically. This is so much better..

Richard.

You should not just rely on Google. There are many many sites that read the sitemap.xml file therefore it is better practice to have it updated on the web server everyday with a cron job and point the various search consoles to that.

John

That's a valid point John.

David

I am working on a way to do both currently. As it is good to have an XML file on the server for other robots.

Mike Beebe
President,
DataLinks Software Solutions
www.dlss.ca
Rate Me Here

ASP and PHP mods - www.dlss.ca/products.asp
A Tremendous Home - www.ATremendousHome.com
Buy Cigars Online www.CigarSmokeShop.net

Thank you all for the replies and discussion; very helpful. Looking further into this, perhaps I can share some more information.

Just for interest, here is a list of Top 50 Sitemap Generator Tools >>
I wouldn't take much notice of the rank order. Some of those are deprecated. Others lack features like:
> Obey robots.txt Disallow.
> Obey meta tag Noindex / Nofollow.
> Obey link tag rel Canonical
> Calculate page Importance rank 0.1 to 1.0
> Sort URL's in sitemap by Importance, rather than structure.
> Ping Google and Bing when sitemap.xml updated.

The server installed PHP script found that ticked most boxes was Unlimited Sitemap Generator >>
Very reasonable price. However, look in their User Forum to find a pinned post "We have a problem where the pages added to the sitemap maxes out at 32". For websites with many pages, it appears the host server runs out of PHP memory and exceeds max execution time.
Mike Beebe, perhaps that is an issue you know more about?

Reading more about "A1 Sitemap Generator" (the Windows software I own) I found a page explaining why they chose NOT to offer a server installed sitemap tool that had "a limited capacity and low amount of functionality". The page says "Most online sitemap tools only allow you to scan a low amount of pages". There is also a page explaining how to Schedule and Automate Sitemap Generator Tool with Command Line Interface (CLI) in the Windows Task Scheduler.

So maybe I was on the wrong track thinking a "Server Side sitemap.xml Generator" would be a good way to go. Now I will try using "A1 Sitemap Generator" with the Windows Task Scheduler, creating a separate task for each website project.

Gary

Hi Gary,

FYI The Unlimited Sitemap Generator / xml-sitemaps has never caused me any problems.
I have had it run daily on 3 sites on the same server, one hour apart in the early hours of each morning, for probably about the last 6-7 years.

Current server spec is VPS 3Gb ram / 4cpu / SSD

Steve