Last Autumn, I deployed my blog website for recording my learning of engineer knowledge in my work. As 2020 is coming to the end, I think it’s a good time to look back the whole year to see what I have learned in my career path and what I need to improve in the following year.
I am happy to share that there are lots of tech tools I have learned and become more proficient this year. The one I grows most is Django. I started knowing that some fundamental knowledge of data engineers by developing applications with Django. The first learning point is data migration, which I think every data engineers must know as about. Data migration guarantees the data warehouse will be developed in order and easily get restore the data warehouse combining with the data dump. Data migration only moves one-way forward. Therefore if we make some mistakes while developing new features in the data warehouse, we have to make a new migration file to correct the error. The second learning point is web application development with Django. It’s great to know how to interact among the views, serializers and forms etc., which are critical components for web application development. Finally, the learning point is knowing that async programming in web application. It’s important to know async programming when concurrent requests happen. Taking requests concurrently, async handling can prevents users hanging on a web page until the tasks get finished. Instead, requests will be handled by a message queue and the workers can process them sequentially. The server will send back to users a temporary response with a code 201 or 202 saying that the task is pending and we have already received the requests and sent them to the workers. Then the server will ping worker periodically to check if the tasks get done. If the task is completed, the server will send back results back to the clients then.
Apart from Django, I took an system design course at the beginning of this year, learning some pull/push models, location based services, chat systems etc. Without covering details here, I would rather mention some parts we need to consider when we try to design a system. The first thing we need to have in mind is Scenario, i.e. what features or QPS or DAU for the system. Then we should consider Service, i.e. how many services(component) this system will consist. The next step I think is the most important step Storage, i.e. what kind of storage or combination of storage this system should use, SQL or NoSQL and how we should interact with file systems. This step requires us to know the advantages and disadvantages of different storage. Finally, we need to consider Scale, i.e. how to optimize the performance of the service and how to enable the server to serve as many users as possible. After I know these, I always has a direction to think about how a feature get designed when I surf online and it’s interesting every time I ask myself what might be behind a feature in an application.
Another achievement for me is getting the Certified Kubernetes Administrator Certificate. I mean the certificate may not mean anything but the procedure of getting the certificate is valuable. I becomes much more familiar with general Kubernetes. After I learned the knowledge required by the exam, I feel more comfortable with what my senior colleagues talks about daily. It offers me a hand to know more architecture-related topic and tools around devops like helm, terraform and vault during work, which I think is valuable for me to understand deeper about devops, definitely additional arms in my future career. Although I don’t really have too many chance working on devops(since it is vital for a company to ensure all envs healthy, I dare not destroy them), but I absolutely can look into these parts in my side projects and deploy my own with what I know in my mind, one thing on my list 2021.
Throughout the year, I also made mistakes and had lesson learned during work. The worst is the time I breaks the deployment with key change migrations, which cause some down time. I really appreciate the extra help from my colleagues and my manager. There are two impressed lessons I learned this year from work. The first lesson is the importance of the full test coverage. It helps programmers to easily do big refactor without worrying about breaking features we don’t pay attention to. I think I am benefit from the full test coverage at least and will insist on the importance of test in my future work. Besides the tests suite, I will keep pushing myself to do sanity check if applicable. For a professional engineer, it’s worthy spending extra caution during developing to prevent production environment from being broken. The second lesson is a good mindset delivered from one of my colleagues. When a problem comes up, we should always face it and figure out how others encounter the similar issue solve the problem instead of trying to seeking the excuse for dropping the part related to the problem. As an engineer, I appreciate I learn this spirit at the beginning of my career. It’s true that we are engineers and if there is something coming up, we should just go and hack it.
Last year, I added grasping Golang in my TODO list and I didn’t make it this year. I instead learned Kubernetes which is unexpected in my TODO list. So without coming up with an TODO list next year, I would like to only come up with two parts I can improve in 2021 , even my future career. The first one is improving the speed of my side projects, at least something can be deployed and live in public; keeping tracking on streaming tools and open source projects. The other one is trying to figure out a way to know more about products and being able to come up some suggestions bringing more impacts on products. Instead of only solving problems, I should cultivate my ability to figure out problems and even to prevent problems before they occur.
By looking back the whole year, I should be proud of myself in this tough year. Hope everything could become better 2021, no Covid-19 especially. Happy new year!