How to learn Python effectively
...as a data engineer
Reminder: I’m offering a limited-time 50% discount on the annual plan:
Once you claim it, the discount will be applied forever.
Now, with only $5/month, you will have access to:
200+ deep-dive data engineering articles
practice-spark: 65 LeetCode-style problems to practice Spark SQL/DataFrame
learn-spark/dbt/airflow: CLI tools to master Spark/dbt/Airflow
If you’re a Vietnamese user, please DM me for an upgrade due to payment issues
Intro
I bet some of you might think like:
“AI can code better than us; why would I need to learn to code anymore?”
I thought that too. I hesitated a bit: “Is this article necessary?“
After reflecting on my journey with Python, from 6 years ago to when I delegated nearly all code writing to AI 6 months ago, I came to this conclusion: learning to code (Python) still matters, especially in this era when nearly everyone on the Internet tells you not to.
Note: All my sharing here, including my attitude toward AI coding tools, is purely from my observation and experience.
What you're reading is my approach to learning Python effectively as a data engineer. It starts with understanding Python's core concepts and syntax.
Then gradually moves on to the production-ready skills you'll actually need: debugging, testing, dependency management, code organization, object-oriented programming, working with APIs, and DataFrame libraries, using familiar data engineering tools such as Airflow and Polars.
The motivation behind knowing how to do things right
The reason I write about Python here might not be questioned much. It’s obviously the most dominant programming language in data engineering. We speak Python all the time (only less than SQL).
Python earns that position thanks to its user-friendliness and large ecosystem of libraries, which let you do anything in data engineering.
It’s easy for humans to read and understand.
The proliferation of AI coding tools seems to make writing Python code easier.
But more Python code does not mean more quality Python code.
I don’t feel comfortable handing everything to AI, although I know it could do very well if I give it enough context, requirements, and expectations. The feeling of not knowing how the code that runs a critical function works keeps me from sleeping well.
I believe we still need to engage with understanding what AI produces, not only to guarantee correctness but also to ensure scalability, readability, and maintainability.
To achieve this with Python code, we must know how to do all of that ourselves before validating the AI’s work. You still need to have an understanding of the programming language’s nature and your own experience of writing maintainable code for the AI tool.
You can say we can achieve all of that by installing some skills for the AI; one for planning, one for reviewing, one for linting,…
That’s true; that works, even well.
But again, that’s only effective if you have the mental model for how to do things right. When you rely on an AI coding tool to do things right without knowing how to do that by yourself, two things happen:
You lose total control of what AI produces; when issues occur and features need to be added, you rely on AI. When issues occur and features need to be added again, you rely on AI another round. It just happens like that over and over again. There is no way to get back control.
You will soon lose joy and motivation. For all day long, the things you need to do are enter some prompt, install some skills, and press enter 90% of the time- no thinking, no reasoning. Just tell me the difference between doing that and scrolling short videos; you’re on autopilot and rely on external factors to give you the joy.
What I’m trying to say is that, to achieve long-term feasibility of using AI for coding, you must know how to do the right things.
This is why I think using an AI coding tool too early in your career is not a good idea.
You need time to practice and observe on your own to gain enough experience doing the right things when drafting code. When you hand everything over to AI, you lose the opportunity. Gradually, you will rely more and more on AI; without it, you don’t know how to do things right, and as we discussed above, that’s not feasible in the long term.
—
If you pick up any truly successful case study of vibe coding, you will reach the same conclusion yourself: the people behind those cases all have experience coding themselves, and they know how to do the right things before letting AI do that.
Don’t let some stories about a CEO pitching a product that is vibe coded completely demotivate you; that CEO will certainly lose total control over the app and hire an engineer, the one who actually knows how to code, to help him.
Now, after being on the same page on whether we need to learn code or not, let’s dive into how to learn Python effectively.
Syntax and the way you write code
Reminder: I’m offering a limited-time 50% discount on the annual plan:
Once you claim it, the discount will be applied forever.
Now, with only $5/month, you will have access to:
200+ deep-dive data engineering articles
practice-spark: 65 LeetCode-style problems to practice Spark SQL/DataFrame
learn-spark/dbt/airflow: CLI tools to master Spark/dbt/Airflow
If you’re a Vietnamese user, please DM me for an upgrade due to payment issues