Documentation

5.1.14 Overview

Using Google and IndexNow APIs, you can easily and quickly push indexing requests to all of the major search engines.  Google has its own indexing feature (Web Search Indexing API, inspiration for the name of this plugin), while Bing and several others participate in a collaboration known as IndexNow - https://indexnow.org.

At the time of this writing - this is the ONLY free extension to offer Google Web Search Indexing API OR Bing/IndexNow, and it offers BOTH.

Google Web Search Indexing API

This really needs no introduction.  The world uses Google search, and this is the same interface you have in the Google Search Console where you can submit a URL to be indexed - except this is automatic, occurring when you save an article.

IndexNow

IndexNow is a collaboration and cooperative effort, whereas all participants share indexing requests with each other.  A submission to one is a submission to all.  At the time of this writing, the following search engines participate:  IndexNow, Microsoft Bing, Naver, Seznam.cz, Yandex, and Yep.

This plugin acts as an agent to submit your articles as they are saved, saving time and hopefully speeding up the process vs sitemap scanning.

How to Use This Document

While the operation and configuration of this plugin are relatively simple, the preparation is not.  This document is divided into several sections, allowing easy navigation to the topics of interest.  It is written in almost reverse order of setup.  While Google and Bing API key setup is listed last, it is almost the first thing you need to do.  Because operational instruction is more likely to be needed after installation, it is listed first.  I suggest first-time users follow the following steps.

1. Install the plugin
   • Because of the size of the installation file, this presents its own challenges
2. Obtain a Google Key
   • Most difficult, biggest reward
   • If you choose, the plugin requires at least one indexing target.
3. Obtain a Bing Key
   • Easiest to obtain, lesser reward, submits to multiple search engines
   • If you choose, the plugin requires at least one indexing target.
4. Configure the plugin
   • It's dead simple, there are only 6 settings.

This entire page is irrelevant!  When creating the extension, I missed a line in the Google documentation allowing me to clean up unused code.  This source clean up reduced the size of the install zip file from 30MB to less than 2MB.  No PHP/HTTP server configuration changes are necessary for this extension post version 5.1.6

Carry on!

Installing the plugin is simple, and yet difficult.  Because of the infrastructure required (PHP libraries), the size of the plugin is nearly 30MB.  This requires special PHP and HTTP server configurations.

Why is it so big?  Genetics - I mean dependencies.  Interactions with IndexNow take only a few lines of code.  It's super easy, barely an inconvenience.  Google, however, requires a ton of code.  99.99997% of this plugin is dependencies to satisfy requirements of the connection to Google.  Without these dependencies, this plugin would be less than 40KB uncompressed!

PHP Considerations

Upload Size

There are 2 settings within your php.ini file which affect the maximum size of your uploads.  Both of these values need to be above the 30MB size of this plugin.  A nice round number would be 32MB, and there isn't any reason to leave these settings in place once the installation is complete, although you may need to reinstate them to perform an update.

upload_max_filesize = 32M
post_max_size = 32M

You could go bigger if you wanted, but 32 will get you by for this plugin.  Joomla provides an easy way to find your current settings: Under the System menu in /administrator, open the System Information link and have a look at PHP Information.  You'll find the current values for both of these settings there, which will tell you if you need to modify them.

Execution Time

Because of the size of the plugin, extraction takes some time.  This will depend on many factors, but you may run into a situation where your server cannot extract the zip file fast enough - generating a strange blank error message.  In these cases, try adjusting your max execution time in the same place you made the upload size changes.

max_execution_time = 60

How those modifications are made in your environment is beyond the scope of this document.  Consult the maintainer of your server operating system for that information.

Apache2 Considerations

There are several ways to alter the maximum upload file size in Apache.  One easy method is using .htaccess

Joomla provides an example .htaccess file, and if you're using that file then you only need to add a single line of code.  The location where you insert this code is important.  The example file provided by Joomla contains a lot of comments at the very beginning.  There is a section titled "READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE"

I suggest inserting the following line immediately following that comment block (the block of lines beginning with #), there should be a blank line where you can safely insert this configuration option:

LimitRequestBody 33544432

Another option is to make this exact same configuration in your httpd.conf - but this is a more wide-reaching configuration change.  If you don't know how to do this, definitely ask for help or you can take your server down.

nginx Considerations

Similar to Apache2, nginx requires a setting to increase maximum upload file sizes.

In my environment, this configuration is easily made by creating a single file at this location /etc/nginx/conf.d/upload.conf

The contents are as follows:

client_max_body_size 32M;

Restarting NGINX applies this change.  Be certain that it isn't already present in other files by testing the configuration before restarting the service.  As suggested with the Apache2 configuration, if you don't know how to do this - ask for help.

Configuring this plugin is relatively simple once you have obtained your keys.  There are only 7 options:

1. Force HTTPS
   • Useful for sites behind a load balancer or using reverse proxy. If your site runs on https, turning this on cannot hurt.
2. All Categories
   • Enabling this feature will activate the index submission for all articles across all categories.  Disabling this feature enables the next configuration option (Select a Category)
3. Select a Category (there are several of these, I'm counting them as 1)
   • You may select one or more categories to activate automatic index submission.
4. Google (Enable/Disable)
   • When enabled, you will be able to enter a Google API Key.  Indexing with Google requires this to be enabled, and an API key to be present.
5. Google API Key
   • The process to obtain this key will provide you with a JSON file.  Copy the contents of this file into this field.  Make no alterations.
6. Bing (Enable/Disable)
   • When enabled, you will be able to enter a Bing/IndexNow API Key.  Indexing with IndexNow requires this to be enabled, and an API key to be present.
7. Bing API Key
   • This is a normal key, a relatively short alphanumeric string.
   • Paste the key you obtained in the instructions below.  

Index submission will always come with status responses.  These come in the form of numeric HTTP response codes.  200, 202, 400, 403 (and others) - each index has a predefined list of codes that this plugin translates into the index defined meaning.  Generally, 200s are forms of success, 400s are forms of failure.

This example shows complete success.  Both index reported 200.

1. The URL submitted
2. Google response
3. Bing/IndexNow response
4. The article save status
5. The actual response code returned by the index.

If there is an error, the message will include it and the color will reflect that there was an error (red).  Ambiguous responses responses (responses not defined by the index) are still displayed, but in yellow.

Google API Key creation is a multi-step process.  This is the most difficult part of the process.  Do not skip any of these steps - they are ALL required.  Some sections allow you to use previously created accounts, but be certain that you know what you're doing when using these existing accounts.

Google has their own version of these API enrollment instructions published here:
https://developers.google.com/search/apis/indexing-api/v3/prereqs

Theirs are not as detailed as this document.

Google Cloud Platform

1. Create a Google Cloud Platform account.
   • https://console.developers.google.com/flows/enableapi?apiid=indexing.googleapis.com&credential=client_key
2. Select or create a project
   • This is normally named after your website, which makes keeping your keys organized.  It doesn't have to be a domain name, it could easily be the name of your company. The name is entirely for your benefit, to make it easier to recognize later.

Service Account

1. Log into Google Cloud Platform
   • https://console.developers.google.com/flows/enableapi?apiid=indexing.googleapis.com&credential=client_key
2. In the hamburger menu to the left (3 horizontal bars), locate and open IAM & Admin
3. In the left side menu, locate and open Service Accounts
4. In the Service Accounts page, locate and open + CREATE SERVICE ACCOUNT
   • If you already have a service account that you want to use, you can skip this part.
5. Copy the service account ID that looks like an email address
6. Find the Actions menu button next to that account ID (3 dots in a vertical line), and select Manage Keys
7. Locate the ADD KEY button, and choose Create new key - when prompted, choose JSON as the Key type
8. This action initiates a JSON file download - save it somewhere that you can keep it safe.  You will need this for the plugin configuration.

Web Search Indexing API

1. Log into Google Cloud Platform
   • https://console.developers.google.com/flows/enableapi?apiid=indexing.googleapis.com&credential=client_key
2. In the hamburger menu to the left (3 horizontal bars), locate and open APIs & Services
3. In the APIs & Services page, locate and open the + ENABLE APIS AND SERVICES link
4. In the search bar near the top of the page, search for "Indexing"
5. Select Web Search Indexing API
6. On the Web Search Indexing API page, click the ENABLE button.
   • Now that this API is enabled, you can return to its configuration page by returning to the APIs & Services page, and finding it in the Enabled APIs & Services menu.

Google Search Console

We're going to assume that you already have a working Google Search Console for your site.

1. Open your Google Search Console
2. Open the Settings menu
3. Choose Users and permissions
4. Choose + ADD USER
5. Paste the Service Account email address you copied earlier and give it Owner permissions

The Bing/IndexNow API Key is far easier to obtain than the Google key.  

Click here: https://www.bing.com/indexnow/getstarted

1. Visit the link above
2. Scroll down the page until you see "Generate API Key"
3. Press the "Generate" button until you're happy with the generated key
4. Copy that key value into a text file, store it in the same place as you stored your Google key JSON file (or, more specifially, somewhere you can find it later and where you can keep it safe).

That's it, the plugin needs the key and will handle the rest.