Caching with Fathom, Redis, and SvelteKit
I had a bit of a surprise land in my inbox at the start of the month from my analytics provider Fathom, the subject line “You’ve been quite popular”.
Not the good type of surprise though! 😅
On opening the email I discovered that I’d exceeded the limit on my current pricing plan, by a factor of 5. With the base plan you get with Fathom (for $140 a year) you get 100,000 pageviews per month. This also includes API calls I’d hit over 500,000.
So I go into full panic mode and remove all of the calls to the the Fathom API which I documented in the Adding real-time analytics to my SvelteKit site with Fathom post at the end of Feb.
I reached out to the Fathom support team and they were awesome and I didn’t need to move up to the $540 annual plan. I did offer to pay the difference but they said that wasn’t necessary. Thanks Ash!
Preamble over, let’s get into the caching.
Scene setting
This example is a enhancement to the already existing sveltekit-and-fathom project over on GitHub you can check out the code over there.
The project gets live visitors and page analytics from the Fathom
Analytics API, they’re in two endpoints on the project
/current-visitors.json
and /analytics.json
.
All the changes for this post are in the PR and I’ll be going over the changes in this post.
For more information on the project for using Fathom and SvelteKit check out the following:
So, this is taking into account the previous posts listed above.
I have an existing SvelteKit project with Fathom Analytics configured and live analytics already. Now the purpose of this post is to implement caching for the analytics and live visitors data.
This is the third instalment and an addition to the previous two posts. If you just want to implement caching into a SvelteKit project then this will be useful for you as well.
Thanks!
First up, before we get into the details I want to thank Geoff Rich for his post on the Upstash blog for Building SvelteKit Applications with Serverless Redis Geoff is a great blogger who’s site a get a lot of value out from.
If you haven’t checked out his site then I highly recommend you do so!
Get started
Before I get started I should mention the prerequisites needed:
- Fathom Analytics account with API access
- SvelteKit project with Fathom Analytics configured
- Redis instance, I’m using upstash
The only thing I’ll be covering in this post will be configuring Redis, the rest of the setup is covered in the previous posts.
From the Upstash console I’ll create a database, give it a name and keep the rest of the default options, for the region it’s advised that you chose the region closest to where your application is. I’m using Vercel so to find where the build was I went into the deployment build logs and found the region from there.
Once the database is created I’ll need to get the connection string, at the time of writing this I can get the connection string from the details tab. It looks a bit like this:
redis://default:...
Copy that, add it to my .env
file with a variable name of
REDIS_CONNECTION
and I’m good to go.
The data I’ll be caching will be coming from the two endpoints
/current-visitors.json
and /analytics.json
I mentioned earlier.
I’ll start with current visitors.
Current visitors
In the src/routes/current-visitors.json/+server.ts
file as it was
before this refactor did a straight up call to the Fathom API to get
the current visitors data. This is how it looked:
export const GET: RequestHandler = async () => {
try {
const headers_auth = new Headers()
headers_auth.append(`Authorization`, `Bearer ${FATHOM_API_KEY}`)
const res = await fetch(
`https://api.usefathom.com/v1/current_visitors?site_id=${PUBLIC_FATHOM_ID}&detailed=true`,
{
headers: headers_auth,
}
)
let data = await res.json()
return json({
visitors: data,
})
} catch (error) {
return json({
error: `Error: ${error}`,
status: 500,
})
}
}
To start caching the visitors
data I’ll need to add a Redis client
to the project. I’m using ioredis as this is the one used in Geoff’s
guide! 😅
pnpm i -D ioredis
The path for the Fathom Analytics API response for visitors isn’t
going to change so for the key value pair on the visitors data I’ll
add a VISITORS_KEY
environment variable.
This is just a unique identifier for the visitors data, it can be
whatever you like. I used a GUID generator to create a random string
and added that to my .env
file.
I’ll create a new file in the src/lib
folder called redis.ts
and
add the following code:
import { REDIS_CONNECTION, VISITORS_KEY } from '$env/static/private'
import Redis from 'ioredis'
export function get_current_visitors(): string {
return `visitors:${VISITORS_KEY}`
}
// I'll be needing this later
export function get_page_analytics(slug: string): string {
return `slug:${slug}`
}
export default REDIS_CONNECTION
? new Redis(REDIS_CONNECTION)
: new Redis()
I’m defining a couple of functions to help access the data from the Redis instance.
The last part creates a new instance of the Redis client using either
the REDIS_CONNECTION
(if available) or the default configuration.
A quick rundown of what’s happening in this file:
- I’m importing the environment variables I need
- I’m importing the Redis client
- I’m creating a function to get the visitors key
- I’m creating a function to get the page analytics key (I’ll be using this later)
- I’m creating a new Redis instance
I’ll need to update the src/routes/current-visitors.json/+server.ts
file now to use the Redis client.
I’ll add in some additional functions, first I’ll need to check if the
data is in the cache with get_visitors_from_cache
, if it is then
I’ll return that, if not then I’ll make the call to the Fathom API
with get_visitors_from_api
, cache the response (in Redis) then
return the data.
import { FATHOM_API_KEY, VISITORS_KEY } from '$env/static/private'
import { PUBLIC_FATHOM_ID } from '$env/static/public'
import redis, { get_current_visitors } from '$lib/redis'
import { json } from '@sveltejs/kit'
import type { RequestHandler } from './$types'
export const GET: RequestHandler = async () => {
const cached_visitors = await get_visitors_from_cache()
if (cached_visitors) {
return json({ visitors: cached_visitors })
}
const visitors = await get_visitors_from_api()
return json({ visitors })
}
const get_visitors_from_api = async () => {
try {
const headers_auth = new Headers()
headers_auth.append('Authorization', `Bearer ${FATHOM_API_KEY}`)
const res = await fetch(
`https://api.usefathom.com/v1/current_visitors?site_id=${PUBLIC_FATHOM_ID}&detailed=true`,
{
headers: headers_auth,
}
)
if (res.ok) {
const data = await res.json()
await cache_fathom_response(VISITORS_KEY, { visitors: data })
return data
}
} catch (error) {
console.error(`Error fetching visitors from API: ${error}`)
}
return null
}
const get_visitors_from_cache = async () => {
try {
const cached = await redis.get(get_current_visitors())
if (cached) {
return JSON.parse(cached)
}
} catch (e) {
console.error(`Error fetching visitors from cache: ${e}`)
}
return null
}
const cache_fathom_response = async (
id: string = VISITORS_KEY,
visitors: any
) => {
try {
const cache = { visitors }
await redis.set(id, JSON.stringify(cache), 'EX', 15 * 60)
} catch (e) {
console.error(`Error caching Fathom response: ${e}`)
}
}
I’ll summarise what the functions are doing:
get_visitors_from_cache
checks if the data is in the cache and returns it if it is.cache_fathom_response
caches the data in Redis.get_visitors_from_api
makes the call to the Fathom API and caches the data in Redis before returning the data for use in theGET
handler.
Page analytics
Before I added in caching the src/lib/utils/index.ts
file looked
like this, it’s behind the index.ts
button here:
One dirty great function to get the daily, monthly and yearly visits.
I took a similar approach with this file as with the GET
handler.
Added in a function to first check the cache
get_analytics_from_cache
, if there’s no cached data, go off and
request the data from the Fathom API then cache the data with
cache_analytics_response
.
I’ll detail them individually here, then put the whole file behind the
index.ts
button again.
Get analytics from cache:
const get_analytics_from_cache = async (cache_key: string) => {
try {
const cached = await redis.get(cache_key)
if (cached) {
return JSON.parse(cached)
}
} catch (e) {
console.error(`Error fetching analytics from cache: ${e}`)
}
return null
}
Cache analytics response:
const cache_analytics_response = async (
cache_key: string,
analytics_data: any
) => {
try {
await redis.set(
cache_key,
JSON.stringify(analytics_data),
'EX',
15 * 60
)
} catch (e) {
console.error(`Error caching analytics response: ${e}`)
}
}
Check out the whole file here:
You’ll notice that this time around I’ve wrapped all the call to the
Fathom API in a Promise.all
to speed things up a bit.
That’s it! Now I can go to the data browser tab over on Upstash and see the data being cached.
The one drawback here is that I’m caching the data for 15 minutes, so if I want to see the latest data, I’ll have to wait 15 minutes. 😅
Conclusion
As I discovered (the hard way), caching is an essential technique for minimizing the impact of API calls in your projects.
I’m hoping this guide will give you an idea on how to implement caching in your own SvelteKit projects.
There's a reactions leaderboard you can check out too.
Analytics Information