Under the hood of github-readme-stats project
Intro
Recently I came across this "stats picture" in someone's Readme profile on Github, displaying profile information, and it caught my attention. Yes, I am talking about this github-readme-stats
widget, here is an example for my profile.
Example of the stats picture for my profile
What is interesting, it is dynamic, meaning, every time you see this image it is going to be up to date with the current stats. But, how does it render dynamic information inside of the static markdown page?
Looks cool though, so I started digging - inspected source and quickly found out there is app running at github-readme-stats.vercel.app so can be used by anyone, and it is an open source project github-readme-stats so we can look at the implementation as well.
Awesome, let's look under the hood then.
How it works
Well, all you need to do is simply use the img
with API endpoint, specifying Github username.
There is plenty of other parameters to customise the widget, like what information to display, what colour theme to use, etc.
So, let's have a look what exactly this end-point returns.
It returns SVG!
It returns SVG! Yes, you can use SVG
as a source for an image HTML
element. SVG
loses its interactivity because of that, but it's not critical in this use-case. And if you think about it, it kind of make sense, I mean, it's way easier obviously to manipulate with vector graphics and generate SVG
markup than work with raster binary
data (as we know it for PNG
/JPEG
/etc. images formats).
Under the hood
Where do we start? It is JavaScript
project, so looking inside package.json
is always good starting point (as a standard).
1{2 "name": "github-readme-stats",3 "version": "1.0.0",4 "description": "Dynamically generate stats for your github readmes",5 "main": "index.js",6 "scripts": {7 "test": "jest --coverage",8 "test:watch": "jest --watch",9 "theme-readme-gen": "node scripts/generate-theme-doc",
We can see main
points 1 to index.js
but there is not such file. What we do have though are some other hints...
Interesting... Well, we saw this already (by vercel.app
piece in the url) but these 2 are hints that this package is not "standard", it is "Powered by Vercel" with Serverless Functions - we can tell that by presence of /api
directory (standard directory used by Vercel
) for functions.
That means that api/index.js
is in fact the entry point of the application and the place where we should start reading code.
11const blacklist = require("../src/common/blacklist");12const { isLocaleAvailable } = require("../src/translations");1314module.exports = async (req, res) => {15 const {16 username,17 hide,18 hide_title,+36 } = req.query;37 let stats;3839 res.setHeader("Content-Type", "image/svg+xml");+49 try {50 stats = await fetchStats(51 username,52 parseBoolean(count_private),53 parseBoolean(include_all_commits),54 );+64 return res.send(65 renderStatsCard(stats, {66 hide: parseArray(hide),67 show_icons: parseBoolean(show_icons),68 hide_title: parseBoolean(hide_title),69 hide_border: parseBoolean(hide_border),70 hide_rank: parseBoolean(hide_rank),71 include_all_commits: parseBoolean(include_all_commits),
84 }85};8687async function fetchStats(88 username,89 count_private = false,90 include_all_commits = false,91) {92 if (!username) throw Error("Invalid username");9394 const stats = {95 name: "",96 totalPRs: 0,97 totalCommits: 0,98 totalIssues: 0,99 totalStars: 0,100 contributedTo: 0,101 rank: { level: "C", score: 0 },102 };103104 let res = await retryer(fetcher, { login: username });+114 const user = res.data.data.user;+137 stats.totalStars = user.repositories.nodes.reduce((prev, curr) => {138 return prev + curr.stargazers.totalCount;139 }, 0);140+151 return stats;152}153154module.exports = fetchStats;155
45 `;46};4748const renderStatsCard = (stats = {}, options = { hide: [] }) => {49 const {50 name,51 totalStars,52 totalCommits,53 totalIssues,54 totalPRs,55 contributedTo,56 rank,57 } = stats;+214 const card = new Card({+227 });228+235 return card.render(`+238 <svg x="0" y="0">239 ${flexLayout({240 items: statItems,241 gap: lheight,242 direction: "column",243 }).join("")}244 </svg>245 `);246};247248module.exports = renderStatsCard;249
114 : "";115 }116117 render(body) {118 return `119 <svg120 width="${this.width}"121 height="${this.height}"122 viewBox="0 0 ${this.width} ${this.height}"123 fill="none"124 xmlns="http://www.w3.org/2000/svg"125 >+162 <g163 data-testid="main-card-body"164 transform="translate(0, ${165 this.hideTitle ? this.paddingX : this.paddingY + 20166 })"167 >168 ${body}169 </g>170 </svg>171 `;172 }173}174175module.exports = Card;
Let's see what we have here:
0 | All starts at api/index.js default function, it's called by Vercel , providing req and res parameters; essentially giving us a way to read request parameters for GET call and a res object to send back response. As you can see first thing we do, we set proper headers for the payload: res.setHeader("Content-Type", "image/svg+xml"); |
1 | Next we need to fetch some stats data, count stars, etc. The graphql end-point for Github API is used here (more details about that later). |
2 | Once stats are received, passing them to cards/stats-card.js , as well as extra parameters how to style the widget. Card class is used to generate complete SVG markup and fill up with stats data. |
3 | And, finally we send SVG back to the client. |
Once again, how Github stats were fetched? Using graphql
!
6const { request, logger, CustomError } = require("../common/utils");78require("dotenv").config();910const fetcher = (variables, token) => {11 return request(12 {13 query: `14 query userInfo($login: String!) {15 user(login: $login) {16 name17 login18 contributionsCollection {19 totalCommitContributions20 restrictedContributionsCount21 }+44 `,45 variables,46 },47 {48 Authorization: `bearer ${token}`,49 },50 );51};+87async function fetchStats(88 username,89 count_private = false,90 include_all_commits = false,91) {+104 let res = await retryer(fetcher, { login: username });105106 if (res.data.errors) {107 logger.error(res.data.errors);
77 fallbackColor78 );79}8081function request(data, headers) {82 return axios({83 url: "https://api.github.com/graphql",84 method: "post",85 headers,86 data,87 });88}8990/**91 *
Alternatively one can use @octokit/rest client to simplify interactions with Github API (instead of create graphql queries you can call particular helper for the same operation).
Interesting finds
There are few things I've also noticed that worth to mention.
First thing is blacklisting hook. To prevent spamming (and other bad behaviour) we filter out requests by username and just return an error instead.
6 clampValue,7 CONSTANTS,8} = require("../src/common/utils");9const fetchStats = require("../src/fetchers/stats-fetcher");10const renderStatsCard = require("../src/cards/stats-card");11const blacklist = require("../src/common/blacklist");12const { isLocaleAvailable } = require("../src/translations");1314module.exports = async (req, res) => {15 const {16 username,+41 if (blacklist.includes(username)) {42 return res.send(renderError("Something went wrong"));43 }4445 if (locale && !isLocaleAvailable(locale)) {46 return res.send(renderError("Something went wrong", "Language not found"));47 }
1const blacklist = ["renovate-bot", "technote-space", "sw-yx"];23module.exports = blacklist;4
Another one is retryer for API calls. It does exactly what you think - re-trying API calls.
84 }85};8687async function fetchStats(88 username,89 count_private = false,90 include_all_commits = false,91) {+104 let res = await retryer(fetcher, { login: username });105106 if (res.data.errors) {107 logger.error(res.data.errors);
1const { logger, CustomError } = require("../common/utils");23const retryer = async (fetcher, variables, retries = 0) => {4 if (retries > 7) {5 throw new CustomError("Maximum retries exceeded", CustomError.MAX_RETRY);6 }7 try {8 // try to fetch with the first token since RETRIES is 0 index i'm adding +19 let response = await fetcher(10 variables,11 process.env[`PAT_${retries + 1}`],12 retries,13 );1415 // prettier-ignore16 const isRateExceeded = response.data.errors && response.data.errors[0].type === "RATE_LIMITED";1718 // if rate limit is hit increase the RETRIES and recursively call the retryer19 // with username, and current RETRIES20 if (isRateExceeded) {21 logger.log(`PAT_${retries + 1} Failed`);22 retries++;23 // directly return from the function24 return retryer(fetcher, variables, retries);25 }2627 // finally return the response28 return response;
For stability of our App (API endpoint) we want to make calls, we have internally, more predictable and stable. Often network calls can fail (for different reasons) but it is not necessary means something wrong, we might just do extra attempt.
Here it is!