pysn blogs

How to make a URL shortener

Mind the typos, trying to put stuff out. Will get better with time, I assure you. Thanks for taking the time to read. Enjoy!

I am currently learning backend( in Nodejs lol ) and I suck at frontend designs ( i have to practice a lot to get to great level of designs ) and so i decided i need a project which showcased my backend skills all the while not compromising my design skills. So i did what every good engineer does, ask on twitter and I was suggested by this lad @saleh to make a url shortener.

So I did. Project at : url-shortener

I will not bore you with the research bit, long story short : read of alot of everything you can find about the topic at hand and scope out the project.

So what does a url shortener do ? It essentially transforms a long messy url into a small abstract url which you can easily share and bonus, run analytics on. like this url https://www.sitepoint.com/community/t/are-these-urls-too-long/30053 to something like this https://shorturl.at/xroBq

where the xroBq is the converted url.

so how does this essentially work ? there are many ways but one way is that :

you would need to first hash the url so that it stays secure and cant be reverse engineered easily, so we use a hashing algorithm like sha256 ( there are many more algos you can find ) and this allows us to make consistent urls for the same domain and reduces the chances for collision as it is rare in algos like sha256/

const hash = crypto.createHash("sha256").update(orgUrl).digest("hex");

here we use the inbuilt library of node to create a hash of our original Url ( the long one ) into a hexadecimal string and then we encode it.

the question arises what type of encoding should we do ? we are making a short url, so we would need a higher base and something that is alphanumeric in nature because we want it work seamlessly across query strings and paths without an issue and most importantly it should be easier to read, copy and share.

Alphanumeric characters consists of 0-9, a-z and A-Z, so we decide to use base62 encoding as all of these character adds up to 62 and if we were to use base64 , we would also get special characters.

Making a encoding function is simple, we would need to convert the hashed hex string into a short readable alphanumeric strings and preferably 7 characters long as then we can get around 3.5 trillion unique urls ( 62^7).

the code for this is

const encodingFunc = (hexStr) =>{

    const referenceTable = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";

    let num = BigInt("0x" + hexStr) 
    //helps us get hex to number to encode this

    let encoded ="";

    while(num){

        encoded += referenceTable[parseInt(num % 62n)]; 
        // get the corresponding number value from the ref table

        num = num / 62n;

    }

    return encoded;

}

and then we can use the first 7 digits of the encoded strings to use and attach it to our urls and return them.

Now we can simply create a database model in which we can store our longurls to shorturls, and also some additional parameters like expires in time and clicks to do basic analytics. We can always add more to our model to advanced analytics and even setup user dashboards to show them everything they need, like all the urls created, their clicks, ability to set their active indicator, change expiry date and so much more.

But we would be keep our project simple and be build 3 routes, i.e

app.post("/create"); // route to create a shorturl
app.get('/:url'); // route on which shorturl redirects
app.get('/analytics/:url'); // route to get the no of clicks 

I will be sharing with you the simple code to get an idea how to implement and then you can always add on to it . Like validation and sanitization of requests , user accounts, dashboards and much more.

You can also check the entire code of my project at url-shortener Go through the backend files to get an idea how i implemented everything.

Basic skeleton for routes

**POST Route to create shortURL

app.post('/create',(req,res) => {
	const longUrl = req.body;
	const hash = crypto.createHash("sha256").update(longUrl).digest("hex"); 
	const encoded = encodingFunc(hash); \\ funtion is defined above
	const shortUrlHash = encoded.slice(0, 7); // get 7 digits
	const shortUrl = URL + shortUrlHash;
	\\ URL is the variable which hold the base url such as tinyUrl, bit.ly etc
	\\ ideally store the shortUrl in your db
	res.status(201).json({message:"Short Url Created"},shortUrl)
})

GET ROUTE to redirect from shortURL

app.get('/:url',(req,res) => {
	const shortUrlHash = req.params.url;
	const shortUrl = URL + shortUrlHash
	\\ Check if shortURL exists in db,  
	\\ if yes return corrsesponding longURL and redirect
	\\ here you should increment clicks 
	const longUrl = result.longUrl; \\ result is the db response 
	res.redirect(longUrl);
})