18.4 Fetching Remote Data: resources.GetRemote
Right, so you want to fetch some data from the internet and jam it into your Hugo site. Maybe it’s a JSON API from a third-party service, maybe it’s a CSV file you’re keeping on GitHub, maybe it’s the latest manifesto from your favorite obscure band. You’ve heard about resources.GetRemote, and you’re thinking, “Great! A simple GET request. How hard can it be?”
Famous last words. Let’s pull up a chair. This is one of those Hugo features that is incredibly powerful but has more sharp edges than a bag of broken glass if you don’t know how to handle it properly. I’m here to make sure you don’t bleed all over your build script.
The Absolute Basics
At its core, resources.GetRemote is a function you can use in your templates to, well, get a remote resource. You call it with a URL, and it returns a resource object. Simple. But here’s the first “gotcha”: you can’t just slap this anywhere. It has to be used within a .Permalink or .RelPermalink property to actually trigger the fetch. Hugo’s lazy evaluation is both a blessing and a curse.
{{/* This does NOTHING. It just creates a promise of a resource */}}
{{ $dataPromise := resources.GetRemote "https://api.example.com/data.json" }}
{{/* This actually triggers the HTTP GET request */}}
{{ with $dataPromise }}
{{ $data := . | transform.Unmarshal }}
<pre>{{ $data | jsonify (dict "indent" " ") }}</pre>
{{ else }}
{{ errorf "Oh no, the fetch failed!" }}
{{ end }}
See that? We have to use with and then reference . (the resource) to force Hugo to actually go get it. This is Hugo’s way of being efficient—it won’t make the network call until it absolutely has to.
Caching: Your Best Friend and Worst Enemy
By default, Hugo is smart. It will cache the result of this remote fetch for the duration of the build. This is fantastic for performance and being a good netizen; it prevents you from hammering an API with a thousand requests while you’re live-reloading your site.
But this is also the number one thing that will make you tear your hair out during development. You’ll change the data at the remote URL, rebuild your site, and see… the exact same old data. “Is Hugo broken? Am I broken?” Nope. It’s the cache.
You have two weapons against this:
- The Nuclear Option: Run
hugo --ignoreCache. This nukes the entire cache, including your remote resources. - The Scalpel: Use the
keyparameter to create a unique cache key for this specific resource. Change the key, and you bust the cache for just this resource.
{{/* This fetches and caches with a key derived from the URL */}}
{{ $data := resources.GetRemote "https://api.example.com/data.json" }}
{{/* This lets you control the cache key. A timestamp is a classic trick. */}}
{{ $cacheKey := print "https://api.example.com/data.json" (now.Unix) }}
{{ $data := resources.GetRemote "https://api.example.com/data.json" (dict "key" $cacheKey) }}
For production, you want a stable key (like just the URL) so you get the performance benefit. For development, that timestamp trick can be a lifesaver.
Headers, Timeouts, and Errors
Want to access an API that requires an Authorization header? Of course you do. resources.GetRemote has you covered, but the syntax is a little… particular.
{{ $headers := dict "Authorization" (printf "Bearer %s" site.Params.myApiKey) "User-Agent" "Hugo Static Site Generator" }}
{{ $options := dict "method" "get" "headers" $headers "timeout" 5 }}
{{ with resources.GetRemote "https://api.secure.com/data" $options }}
{{/* Success! */}}
{{ else }}
{{/* The fetch failed. Likely a timeout or 404. */}}
{{ end }}
Let’s talk about that timeout. It defaults to a slightly generous 10 seconds. For a build process, that’s an eternity. I always set it lower, like 3-5 seconds. If an API is that slow, I don’t want it holding my entire site build hostage. A timeout will cause the function to return nil, which is why we check with with/else.
And for the love of all that is holy, please set a custom User-Agent. It’s just polite. It helps the API owners know who’s asking, and it might prevent you from getting accidentally blocked.
Transforming the Data Immediately
You fetched some JSON. Now you need to parse it. You could do it in the template with transform.Unmarshal, but there’s a better way: chain the transformation right onto the fetch. This keeps your logic tidy.
{{ $data := resources.GetRemote "https://api.example.com/data.json" | transform.Unmarshal }}
<h2>{{ $data.title }}</h2>
But what if the fetch fails? The whole thing fails. This is where a more robust pattern comes in handy:
{{ $remoteData := resources.GetRemote "https://api.example.com/data.json" }}
{{ if $remoteData }}
{{ $data := $remoteData.Content | transform.Unmarshal }}
{{/* Work with your $data object */}}
{{ else }}
{{ errorf "Failed to retrieve remote data from example.com" }}
{{ end }}
This way, you can handle the error gracefully, perhaps loading a local fallback file instead of crashing the entire build.
The bottom line? resources.GetRemote is a Swiss Army knife. It’s incredibly useful for pulling dynamic data into a static site, but it requires a thoughtful approach. Always set timeouts, always consider caching, and always, always plan for failure. The internet is a wild place, and your build process shouldn’t depend on it being calm and reliable.