Side Project #1 of 2024 as a Software Developer: From Ideation to Deployment
Hey! Let me quickly share with you some of my thoughts first…
I feel like as a software developer, you wake up everyday thinking in 10 new ideas you could develop that would be cool to have and that will succeed, but then I never take action and do any of them. It is a feeling of knowing that if it was a do or die situation you would be capable of doing it but for one reason or another you never do anything and another day goes by. This is my feeling right now and that’s why I decides to stop it (let’s see how long it lasts), and in my new year goals I set myself to do at least 2 full projects this year as well as step up my number of articles in Medium.
💡 Ideation
So as soon as the year started, I decided to check the long list of notes that I have on my phone with business ideas to code. One of the recent ones that I came across with, was a newsletter that would basically grab the daily Spanish Government announcements of the day (such as announcing new laws, change of existing ones, …) that are usually published with a very technical and incomprehensible language (at least for the younger population that is not into Law or Politics), and make it more comprehensible and in a simpler format.
This document that contains all these communications is called Boletín Oficial del Estado (BOE), more info here. I am not saying that I am going to be able to make them fun, but at least not as insufferable as it is right now.
I extracted this idea from a book that has helped me a lot finding new product ideas which is called “Generating Product Ideas” (I recommend it specially for developers, even though it is for everyone). One of its chapters mentions that you can get product ideas from public data and APIs, in this case, I used BOE’s OpenData in this project.
Once I had the idea, I needed to sketch all the components needed for it to work: a backend in charge of running a cronjob for generating the daily email, a landing page where users are going to be able to subscribe to the newsletter, and some other ones that I created along the way.
💻 It’s Coding Time
I decided to start with the backend instead of the landing page. The main reason along with the fact that without the cronjob there is no product and therefore I couldn’t know if it was feasible, was that I suck at frontend and I also don’t enjoy it because of the reason mentioned before.
⛓️ Backend: Node.js, Mailjet, ChatGPT, …
The cronjob was going to be done using Node.js, a framework that Ia m very familiarized with. I saw that the BOE has an API that given params like the date, it returns to you in XML format the bullet points of the day along with some other additional data. At least it was something, even if the data fetched was in XML and I needed to parte it into JSON. I sketched in my head the format that I wanted to give to the newsletter and then I applied by using types (Typescript 🤘) for better managing the data.
This was not a hard task but rather a boring and a long one where I tested it many times until I got the right format. Then the integration with 2 external services: developer’s beloved ChatGPT’s API and Mailjet.
I chose Mailjet because I have worked before with it, it gives you a very reasonable free plan and the most important thing, it accepted dynamic templates. For the other choice, I guess it is the cheaper and the most trustable option for an LLM.
The ChatGPT integration was a piece of cake as I had done some other projects with it before, as well as the Mailjet one. But now there was the first “challenge” of the project: design the newsletter template.
There were 2 requirements I set in order to create the template: it should be easily-readable (no fancy design), and it should be “scalable” in terms of the content inside, as some days there might be tons of announcements and some others might have just a few.
To be honest I thought it was going to be worse but in the end it is just an static HTML template with few styling. Once I was done with this part, I was able to trigger the cronjob and get an email in my inbox.
👨🚀 Frontend: Astro
Next up was the landing page. It was going to be a static page with just one more thing: a form where users would be able to subscribe to the newsletter. I decided that I would do it using Astro. Why? Because I am not very skilled with any framework and since this was going to be an easy project and not an SPA, why not try a new framework.
I was completely new with Astro, and to be honest, I feel like I have only taken advantage of 5% of Astro’s full potential. But I started creating my components, sketching the layout and doing some pencil and paper drawings for some parts to think how I would do them. The end result was:
The process of doing this frontend project was extremely easy and intuitive until I needed to create the form for subscribing users. Oh well, maybe I read the wrong documentation but I ended up installing React on top of Astro for this feature, so I would create a “silly” API in the backend part of my frontend that will be connected with the server that has the email subscription endpoint.
And you would be wondering, why didn’t you create a cloud function that connected the client with Mailjet and call it to subscribe a new user? Or why won’t you call Mailjet directly in the backend of the client? Well, all those options are probably easier and make more sense but I just wanted to create an Express server with an endpoint and apply some clean code to it (there is no explanation that would make sense to you).
Back to the backend component. I had the cronjob and the frontend ready, I just needed to create an API with a basic endpoint that would call Mailjet and subscribe a user. A piece of cake.
🚀 It’s Deploy Time
It should feel like a rocket, right? Just dump everything to the outside world and wait for everyone to visit your site and subscribe to the newsletter. Well, I deployed my client using Astro’s integration with Firebase Hosting. I chose Firebase because its free plan is very generous and because it has a very friendly UI. I set it with my DNS provider (GoDaddy) and try to move it to Cloudflare but for now I have failed (I will keep trying).
But basically this is the part where I struggled the most and the part where I want to improve. Once the client was deployed, I moved onto the server. I dockerized my Node.js project as it was a simple server and decided to use Google Artifacts Registry, just to use Google Cloud Platform for both backend and frontend. I followed this really straightforward article How to deploy an existing Docker container project to Google Cloud Run with the minimum amount of Google Cloud crap, which by just judging at the name of it, it was exactly what I needed.
And voilà! Both parts running in production and time to start adding stuff and improving things: monitoring, custom analytics,… but for now this is it.
Check the site out (Sorry in advanced because it is targeted to Spanish citizens so it is in Spanish). It is live! ⏰⏰⏰
💸 Costs
Whenever I read this kind of articles, I am always curious about the real costs of launching a simple project like this one. To be honest, there many third-party services to take into account and many tools needed that I wasn’t even considering, but lucky me the services I chose had really good free plans and pay as you go (obv some service choices where done by this reason).
If you have read this far, thank you for your time and I hope you found this article useful! 🚀
