Creating a Top Blog Posts List in Gatsby
In this post you will learn how to create a list of your top blog posts in your Gatsby site. You will use data from Google Search Console API.
Prerequisites
You must have already set up a property for your blog on Google Search Console.
Furthermore, you need to have a project on Google Cloud Platform with the following done:
- Have the Google Search Console API enabled.
- Set up a service account and create credentials for it.
- Have the service account added as a user to your Search Console.
You can read my post How to Use Google Search Console API in Node.js, if you need help setting it all up.
Setting Up Project Environment
To use Google Search Console API, you must install the client library and authenticate it.
Start by running the installation command.
npm i @googleapis/searchconsole
Next, create a .env file in the root directory of your project. Add the values of client_email and private_key fields from your service account’s key file. It’s a .json file you download after creating a key pair for a service account.
CLIENT_EMAIL=service-account-name@project-id.iam.gserviceaccount.com
PRIVATE_KEY=-----BEGIN PRIVATE KEY-----\nLONG_STRING\n-----END PRIVATE KEY-----\n
Make sure to add the .env file to .gitignore. Never commit your secrets to version control.
It is safe to use environment variables in Gatsby, because they are only used during build time. They are not included in the resulting code that you serve on your website.
Don’t forget to add these environment variables for your deployed website on your hosting platform of choice.
To use environment variables in your Gatsby application, set up dotenv on the first line of gatsby-config.js file.
// gatsby-config.js
require('dotenv').config();
Authenticating Google API Client
To access your Search Console property data, you need to authenticate the client using your service account credentials.
Decide where you want to place all the Google API code. You create a src/services/top-posts.js file for now.
Import the Search Console client library and create a function that authenticates it. The function should return the authenticated client instance.
// src/services/top-posts.js
const searchConsole = require('@googleapis/searchconsole');
const getAuthenticatedClient = () => {
const auth = new searchConsole.auth.GoogleAuth({
credentials: {
private_key: process.env.PRIVATE_KEY.replaceAll('\\n', '\n'),
client_email: process.env.CLIENT_EMAIL,
},
scopes: [
'https://www.googleapis.com/auth/webmasters.readonly',
],
});
return searchConsole.searchconsole({
version: 'v1',
auth,
});
};
Since the private key in your .env file contains \n characters, you need to replace them with real line breaks by using the replaceAll method.
Getting URLs of Your Top Posts
After authenticating the Search Console API client, you can perform a Search Analytics query to get your most clicked pages.
Use the authenticated client from previous section and call query method on client.searchanalytics. You must specify your siteUrl, and a range from startDate to endDate in YYYY-MM-DD format.
// src/services/top-posts.js
const getTopPageURLs = async () => {
const client = getAuthenticatedClient();
const res = await client.searchanalytics.query({
siteUrl: 'https://your-site.com',
requestBody: {
startDate: '2022-01-01',
endDate: new Date().toLocaleDateString('en-CA'),
dimensions: ['page'],
dimensionFilterGroups: [
{
filters: [
{
dimension: 'PAGE',
operator: 'INCLUDING_REGEX',
expression: '(/posts/).S*',
},
{
dimension: 'PAGE',
operator: 'notContains',
expression: '/page/',
},
],
},
],
rowLimit: 10,
},
});
// If no pages matched the criteria
if (!res.data.rows) {
return [];
}
return res.data.rows.map(row => {
return row.keys[0];
});
};
A couple of interesting things are happening here.
Since you must specify dates in YYYY-MM-DD format. To do so, use the toLocaleDateString method and give it en-CA locale.
To query your top page URLs, you need to set dimensions: ['page']. You can also apply a filter with dimensionFilterGroups to control which pages you would like to retrieve.
For example, let’s assume your blog posts are under /posts/ subdirectory. You can use a regular expression with the INCLUDING_REGEX operator to only pick pages that contain /posts/ in their URL. Here’s a reference to RE2 syntax that Google expects you to use in your regular expression.
You can prevent certain pages from being fetched with the notContains operator. For example, to exclude a page with https://your-site.com/posts/pages/2/ URL, use expression: '/page/'.
Adjust the rowLimit property to specify the number of posts you want to fetch.
Passing Post Slugs Via Context
In the previous section you created an array with your top most clicked blog post URLs. Now you need to retrieve the slugs out of these URLs.
// src/services/top-posts.js
const getTopPostSlugs = async () => {
const urls = await getTopPageURLs();
return urls.map(url => {
return url.split('/posts')[1];
});
};
module.exports = {
getTopPostSlugs,
};
You can use the split method to extract the slug from a URL. After splitting https://your-site.com/posts/post-name/, you can grab the /post-name/ part.
Pass these slugs via context to a page component where you want to query your top posts.
In this example, you recreate the index page to include topPostSlugs in its context.
// gatsby-node.js
exports.onCreatePage = async ({ page, actions }) => {
const { createPage, deletePage } = actions;
if (page.path === '/') {
deletePage(page);
const topPostSlugs = await getTopPostSlugs();
createPage({
...page,
context: {
...page.context,
topPostSlugs,
}
});
}
};
You will later use the topPostSlugs as a variable in your GraphQL query to filter the top blog posts.
Creating a Top Posts List Component
You will now create a component that displays a list of your top posts.
You need to pass two props to this component. The first one, posts, is an array of MarkdownRemark nodes that represent the top posts. The second one, sortedSlugs is an array that contains slugs in their sorted order. You need it to reorder your posts.
// src/components/TopPostsList.js
import React from 'react';
import { Link } from 'gatsby';
const TopPostsList = ({ posts, sortedSlugs }) => {
const sortedPosts = [];
posts.forEach(post => {
const index = sortedSlugs.findIndex(slug => slug === post.fields.slug);
sortedPosts[index] = post;
});
return (
<ul>
{sortedPosts.map((post, index) => {
const { id, title } = post.frontmatter;
return (
<li key={id}>
<Link to={`/posts${post.fields.slug}`}>
{title}
</Link>
</li>
);
})}
</ul>
);
};
export default TopPostsList;
You need to sort your posts in the order of sortedSlugs, before you can display them in a list. When you will query your posts with GraphQL, they won’t be sorted by their click amount. You reorder them to match the order of sortedSlugs array.
The last step is to query the posts in a page component and pass them to the component you just created.
Filtering Top Posts With a GraphQL Query
In your page component, query the top posts and pass them to the TopPostsList component.
// src/pages/index.js
import React from 'react';
import { graphql } from 'gatsby';
import TopPostsList from '../components/TopPostsList';
const IndexPage = ({ data, pageContext }) => {
const topPosts = data.allMarkdownRemark.nodes;
return (
<>
<h1>Page Title</h1>
<section>
<h2>Top posts</h2>
<TopPostsList posts={topPosts} sortedSlugs={pageContext.topPostSlugs} />
</section>
</>
);
};
export default IndexPage;
export const query = graphql`
query TopPosts($topPostSlugs: [String]) {
allMarkdownRemark(filter: { fields: { slug: { in: $topPostSlugs } } }) {
nodes {
id
frontmatter {
title
}
fields {
slug
}
}
}
}
`;
The $topPostSlugs variable in your query is replaced with pageContext.topPostSlugs, which you set in gatsby-node.js.
Summary
In this tutorial you learned a couple of things to create a top blog post list. These things include:
- Using Search Console API to fetch your top pages by click amount.
- Recreating a page to set its context.
- Querying and filtering data using variables.
- Passing the top posts to a component that displays them.