Improving Twitter Embeds using the Twitter API

For a long time now, I've been frustrated by how slow the official twitter embed is. Any page where I've embedded a tweet, becomes noticeably slower. The warnings on lighthouse are always the same - "Reduce the impact of third-party code". When broken down further, you can see that you spend on average 730ms exclusively on twitter javascript files. This is frankly unusable. However, there is a solution, albeit one that requires some effort.

Twitter Embed Javascript is extreemly slow
Twitter Embed Javascript is extreemly slow

We can remove all of this javascript if we instead fetch our tweets on the backend and create a custom element for displaying them. As with any other components I've made for the blog, I'll be making them in sveltekit but you can use whatever you are comfortable with.

Fetching the tweets

In order to fetch tweets, we need a twitter API key (this is one downside over the basic twitter embeds) which will allow twitter to ensure we're within usage limits. That being said, by default you can fetch 4,000,000 tweets per month at no cost. To get a twitter API key, we go to the Twitter Development Portal, where we can create a new project and get our key. Creating a project requires us to answer a few questions but honestly nothing too detailed that you couldn't complete within a couple of minutes.

Setting up our twitter API Key
Setting up our twitter API Key

Once we have our key, we need to decide how to fetch the tweets. Ideally we want to do this before we send the post to the user and we'd want to cache the response so that we can respond quickly to rapid user requests. Thankfully, there is a great - community supported - twitter api library called twitter-api-v2 that will do much of the heavy lifting for us. By combining that with p-memoize, we can cache the tweets server side for a short period to speed up subsequent page loads.

We're going to create our new function within the blog post fetching endpoint. The twitter API allows us to fetch a series of tweets in a single request so this should be performant no matter how many tweets we choose to embed. By doing this here rather than in a secondary endpoint, we can ensure we don't just swap third party code for first party tweet rendering code. It also ensures there isn't a tweet endpoint that we need to secure in order to stop spammers using it to perform their own twitter api requests on our api key.

Before we create our function though, we need to set up our api client library and cache instance. The code for that is quite straight forward but you'll need to remember to pass in your own bearer token (Best practice is to store this as some form of environment secret).

Now we can write our actual fetch function. Our function will take an array of tweet IDs and return an array of tweet objects. Notice that we aren't doing any error handling and are returning only response.data. This is because we can handle deleted/invalid tweets at render time and it makes our explanations that much cleaner.

You might notice that we aren't exporting our function and that it's called fetchTweetsInternal right now. That's because we're going to export it wrapped in a cache handler that makes use of the cache we created before.

With that written and exported, we can read the tweet Ids from our blog post and pass them through to the render pipeline. I add my tweets to markdown using the following shorthand [tweet #tweetId] which means extracting them is fairly simple as I can do it in regex.

After implementing this in your endpoint and checking the result, you may notice that it only contains the tweetId and the text content. This is obviously less than ideal but can be expanded by adding an options object to your tweets(tweetIds) call so that it becomes tweets(tweetIds, { ...yourOptions }). You can get the author of a tweet using response.includes.users to get all mentioned users and then filter based on the authorId (which you will have to request via the options). More info can be found on how to get specific pieces of information over on their github repo.

Rendering Tweets

When it came to rendering the tweets, it was as simple as parsing the data from the above function and rendering the it correctly. While I wanted to keep it looking like Twitter, I did also try to adapt the styles to match my site. To begin I started with a basic tweet. It has no image attachments, no polls, no videos and no links, making it the perfect jumping off point. I made sure to put the profile info in the correct place and add a like and retweet button via twitter intents. You can create a like intent using the format https://twitter.com/intent/like?tweet_id= [tweetId] and retweet as https://twitter.com/intent/retweet?tweet_id= [tweetId]. I think the basic tweet came out really well, with the added benefit of matching the style of my site a lot closer than the official one ever did.

Being an early adopter of a project always means that design choices that you liked will eventually be changed because the wider audience prefers something else... #webdev #dev
0 1 link to tweet

From there, I next created a style for polls. This was a little trickier but still didn't take too long. The biggest difficulty was ensuring that I had the correct colors for the progress bar vote representations as twitter highlights the winning option in another color and these weren't listed anywhere that I could find. In case you've never made a progress bar before, it's essentially a div inside another that has had the vote percentage set as the elements width style ({(100*(option.votes/total))}%).

Working on a blog post at the moment that might end up being quite long. What's the maximum length of a blog post you'd read? #webdev #asktwitter
2 minutes 40%
5 minutes 20%
10 minutes 40%
30 minutes or more 0%
0 0 link to tweet

Because a tweet can contain multiple images. Recreating the various image layouts required some clever usage of css-grid. thankfully, I think I managed to pull it off. I've included an example of a single image, a triple image and a grid of images for you to judge my styling on. While alt values have been pulled through. I haven't added the alt overlay that is visible in the official twitter application. I might think of it as a nice feature for version 2.

Matthew Anderson
Things native English speakers know, but don't know we know: https://t.co/Ex0Ui9oBSL
The embeded tweet image
77886 49378 link to tweet
I needed a tweet with 3 images on it for testing. Here's a #cat dump. https://t.co/1act0kdjyW
The embeded tweet imageThe embeded tweet imageThe embeded tweet image
0 1 link to tweet
Joel Burgess
I said goodbye today to River, who most of you know as Fallout 4’s Dogmeat. Heartbroken doesn't cover it, but I won’t eulogize her here. For twitter, I thought it'd be appropriate to look back at her impact on that game. (plus, writing about game dev hurts less than grieving) https://t.co/ayN1Vd6oqQ
The embeded tweet imageThe embeded tweet imageThe embeded tweet imageThe embeded tweet image
169911 32435 link to tweet

While I haven't embeded any video tweets on the blog up until this point, I believe it's still an important feature. It unfortunately requires falling back on the v1 twitter API to work as video sources have not yet been added to the twitter v2 api. They've been saying that it's coming soon for a few years now but I feel like waiting would be a bad move. There is also an api endpoint that is used within the official embed that has all the needed data for video displays but it isn't officially documented. You can send a request to it via https://cdn.syndication.twimg.com/tweet?id=${tweetId} but do so at your own risk as this is not a supported endpoint. The video embed itself shares most of it's styles with the image embed.

Stefan Judis
The new VS Code version comes with an experimental "sticky" mode and I quite like it! 👏 `"editor.experimental.stickyScroll.enabled": true` Video alt: Scrolling in VS Code showing that function signatures are sticky. https://t.co/uEWLtmaykx
8524 1258 link to tweet

Quote tweets were a quite simple one, as I just render the quoted tweet inside of the parent. I've added an option to hide the metrics on quoted tweets for clarity but other than that. I'm happy to leave it as is.

Jake Archibald
This was an issue for Wordle too. You might have a significant number of users that leave the tab open, so they don't pick up updates to your app. The window focus event is a good time to check for updates https://t.co/ksTq6wYd2x But what should you do if you find an update?… https://t.co/OH4nyjOwBA
Framed
A movie list reshuffle went out recently, apologies to anyone seeing a movie they have already seen in the past couple of days - it should hopefully normalise in the next few days. Please refresh the game to make sure you are on the latest version! Thanks for playing!
79 14 link to tweet

A Note on Site Embed Cards

I've previously done an entire blog post on ensuring that posts from this site show up nicely on social media via OG Images. Unfortunately, twitter's v2 api does not send through card information as far as I can see meaning that the blog posts where I was showing these off have had a warning appended to the top of the article. There does appear to be an option on the ads API to get what's needed to show these correctly but it's something that will need more research.

Hashtags and Mentions

In order to highlight hashtags and mentions, twitter supplies a library that you can install with npm i twitter-text. This library handles all the twitter text linking that is needed to make tweets look right. All you have to do to get the escaped and then marked up text is call the following on your tweet text.

Conclusion

Overall I hope this was an interesting read and I hope it inspires you to build your own twitter embeds to speed up your pages. If you've been reading my posts for a while then you've likely noticed the speed that this page loaded even with the sheer number of embeded tweets. Feel free to leave comment or @ me on twitter @mikevdev. Make sure to share this article with anyone you know who still uses the official twitter embeds.

github issue

Webmentions

What's this?

This site uses Webmentions to handle likes, comments and other interactions. To leave a comment, you can reply on Twitter, GitHub, or Reddit. While the site will count likes from any source, only twitter likes are currently displayed in the facepile above.

Nice writeup. Are you thinking of publishing this to npm? If not, would you mind/care if I did as a test project...?

It's a tough one to publish to npm as it doesn't fit nicely in its own component. Given it has both a backend function and frontend component, and my particular implementation would only work if you were using sveltekit. I don't think there's much point releasing it if it's not flexible.

I'd happily help out anyone who was looking to make it a set of npm components though so go for it if you want to try.

Posted via github.com

Hmm, so is publishing a SvelteKit backend function (e.g. a frontend component + utility endpoint) not a thing? (My background is as a Java dev, used to Maven & Spring Boot, recently a SvelteKit convert).

I haven't published a Svelte/SvelteKit component (or anything else) to npm, was thinking this would be a nice tiny thing to try out. Excluding the upcoming changes to SvelteKit, isn't a backend endpoint fundamentally a .js/.ts in the right path with the right method[s] on it? Or is there more to it for publishing?

Posted via github.com

Hmm, so is publishing a SvelteKit backend function (e.g. a frontend component + utility endpoint) not a thing? (My background is as a Java dev, used to Maven & Spring Boot, recently a SvelteKit convert).

I haven't published a Svelte/SvelteKit component (or anything else) to npm, was thinking this would be a nice tiny thing to try out. Excluding the upcoming changes to SvelteKit, isn't a backend endpoint fundamentally a .js/.ts in the right path with the right method[s] on it? Or is there more to it for publishing?

For routes yes. But this is a component that you include on a page. I think the new backend changes might make this a tad more straight forward but we'll have to see.

Posted via github.com