Curious about data engineering? Frank from Bluebik breaks down the role, 4 excellence checklists, and what real career growth looks like in this field. Plus, you’ll find practical tips and actionable steps to help you get started or advance your career as a Data Engineer.
In today’s business world, data is one of the most valuable resources a company can have. High-quality data can be transformed into deep consumer insights that help organizations make sharper decisions, forecast the future more effectively, build a competitive edge, and drive operational efficiency.
Many organizations have huge amounts of data, often called Big Data, but it is usually scattered and unorganized. This is where a Data Engineer steps in. Their job is to turn raw, messy data into high-quality, usable information so other teams can use it to create real business value.
So what does a Data Engineer do day to day? How do you develop yourself to reach a level of excellence, and what does growth in this field look like? We sat down with Frank, one of Bluebik’s most accomplished Data Engineers, to hear it straight from him.
Data Engineer: The Person Who Makes Data Work at Its Best
“Our main responsibility is to design and build systems that collect data from various sources, process it, and store it in a format that other teams can easily work with,” Frank explains. He adds that the core goal of a Data Engineer is to deliver the highest-quality data possible, so that every team that receives it can put it to use efficiently and generate maximum business impact.
“If you compare it to building a house, a Data Engineer is like the person who designs the plumbing system,” Frank says. “We think of data as water that comes from many different sources. Our job is to figure out how to lay the pipes so we can draw water from all those sources in the most practical and cost-effective way. We also have to calculate how much water we actually need. And once it flows in, we run it through a filtration process to make sure it is clean and high-quality before delivering it to the people inside the house. If the water is clean enough, people can use it to cook or shower without any worry. The quality of what we deliver is everything, and that is the heart of what I do.”
As Frank describes, Data Engineers work with many types of data. Structured Data has a clear, organized format and can be pulled directly from large-scale Big Data systems. There are also Manual Files, such as CSV, Excel, or plain-text files. Then there is Semi-Structured Data, such as JSON or key-value formats. Frank is straightforward about what is most challenging to handle.
“The hardest data type for me to work with is Free Text, any open-input field where users can type whatever they want,” he says. “That data takes significant time to clean. You have to write complex programs to catch all the different patterns people use. Compared to Structured Data pulled directly from a system, which comes in a clean table format and is easier to handle. So when a new use case comes in, and I have to choose a data source, my first choice is always to connect directly to the source system. Using manual files, like someone filling out an Excel sheet and dropping it into a shared folder or Google Drive for the system to pick up, is always my last resort. I want data sources that are reliable and clearly structured whenever possible.”
To give a clearer picture of the day-to-day workflow, Frank breaks down the Data Engineer process into six key stages:
- Requirement Gathering — Start by working closely with the Business team to define the goal: what data is needed, what format it should be in, how often it needs to be updated, and what it will ultimately be used for.
- System Design — Design the overall system architecture, covering the Data Pipeline, ETL (Extract, Transform, Load) processes, and Data Model structure to meet those requirements.
- Development and SIT — Build the actual system and run a System Integration Test to evaluate performance and optimize it until everything runs at its best.
- UAT (User Acceptance Test) — Hand the system over to the end-user team for real-world testing, to confirm that the data output meets business requirements before going live.
- Production Deployment — Once everything is validated and approved, deploy the system into the production environment, where it will operate in the real world.
- Data Delivery — Deliver high-quality data to the next group of stakeholders, which typically falls into three main categories: Data Analysts, Data Scientists, and Business Teams who may have the skills to query the data themselves, so they can use it to support business decisions right away.
Data Engineer and Growth Environment at Bluebik
Frank’s journey into data engineering began during an internship on a Data and Report team at a company that did not clearly separate the roles of Data Engineer, Analyst, and Scientist. As a result, he tried a bit of everything, from building backend automation systems to creating dashboards and generating reports for executives. Through that experience, he discovered he enjoyed and excelled at backend systems work and realized this was what a Data Engineer does. From then on, he committed fully to this career path. After working in the field for a while, a senior colleague invited him to join Bluebik as a Data Engineer, opening up growth he had not expected.
“In my view, the most distinct thing about Bluebik is the project-based nature of the work and how varied it is,” Frank says. “That variety gives you a real chance to dive deep into how different organizations work and manage their data. For example, I have worked on projects for two different Retail clients. Even though they were in the same industry, inside their actual systems, the way each managed data and structured workflows was completely different. I think that makes this place unique and genuinely interesting. You are always learning something new.”
Frank also shares what he values most about working across multiple industries on a project basis.
“Switching between projects is the most exciting part of this job for me,” he says. “Every time you move to a new project, you face problems that are almost never the same, even if they seem similar on the surface. Because the tools and constraints at each client differ, you are always forced to come up with a different solution. That challenge of designing something new every time keeps this work fun and engaging.”
When asked about his most challenging project to date, Frank points to his work with a banking client. “That project had an incredibly steep learning curve, on top of extremely strict security requirements at every single stage, from development all the way through to deployment approval. The sensitivity of customer data and personally identifiable information (PII) meant there was zero room for error. But that pressure is also what pushed me to become a better engineer.”
“That project made me feel like I grew a lot,” Frank adds. “I was given the opportunity to manage the client relationship directly. I sat across from the client team, planned the work, managed timelines, and oversaw everything end-to-end on my own. The feedback I received was positive and made me realize I had grown to take full ownership of a project from start to finish.”
Frank also highlights another quality that sets Bluebik apart: the work environment. The culture is open, with no rigid seniority hierarchy. The team genuinely supports each other, and people have real space to perform at their best.
“Beyond having a great work atmosphere, what really helps you show up at your best at Bluebik is the culture of openness,” Frank says. “When you get a new assignment, your manager does not tell you to do steps one, two, three, and four. Instead, they ask you to think it through and come back with your own approach to get the best result. It is not about executing instructions. It is about pitching your idea, then working through the pros and cons with your senior, who helps you sharpen it. Combined with the autonomy to plan your own timeline and manage your work independently, I feel that environment pushes you to bring out the best in yourself.”
4 Checklists for Achieving Excellence as a Data Engineer
“As a Data Engineer, I see excellence as the ability to build a system that fulfills business requirements efficiently, using an approach that is simple and not overly complicated, while still delivering outstanding performance,” Frank says.
“What matters just as much is that the system should not be something only its creator understands. Even if I am no longer on that project, the person who takes over should understand the system I built without much difficulty and pick up where I left off. That, to me, is what excellence means.”
He adds that this perspective came from real experience. “I have seen systems so complex that only the person who built them could make sense of them. That is a pain point for everyone involved.”
Beyond that, Frank has four specific checklists he uses to measure whether a Data Engineer’s work has truly reached the level of excellence.
1. Scalability
“Besides being easy to understand and hand off, excellence means building a system that can scale up without friction. For example, if a new business use case comes in and we need to connect to a new data source, the framework we built should support a new pipeline immediately, without major rework or extra effort.”
2. Data Quality
“Another key factor is strong data quality. You can prove it by running quality checks and seeing high accuracy scores. The data needs to be correct, precise, and most importantly, trustworthy, so that whoever uses it downstream can do so with full confidence.”
3. Service Level Agreement
“Excellence also means managing delivery consistently within the Service Level Agreement (SLA) committed to with stakeholders. Maintaining timing and system stability standards is fundamental to building trust with teams that depend on your data.”
4. Business Impact
“Another core element of excellence is the business impact created through the data you deliver. The data must be something other teams can genuinely use to add value in a concrete, measurable way. I measure this by maintaining a feedback loop with end users to verify whether the data meets their needs and creates real results.”
5 Areas of Growth as a Data Engineer at Bluebik
When the topic shifts to growth, Frank is clear that since joining Bluebik, his development has been visible and real, driven by the challenges of the work, the range of industries he has experienced, and the growth environment Bluebik provides. Here are the five areas where he has grown most.
1. Technical Skill
“The first is definitely technical skill. Working across many projects means I constantly run into new problems and experiment with new tools. That has broadened my thinking about how to approach and solve problems. I take lessons from each project, both what worked and what did not, and apply them to make better decisions on the next one.”
2. People Management
“Working at Bluebik has expanded my ability to navigate people-related challenges. Dealing with diverse client teams has taught me to adapt my working style to fit different personalities and communication styles. My ability to manage people-related situations has improved dramatically compared to before.”
3. Communication and Presentation
“The third area is communication and presentation. Since moving into consulting, I have had to communicate more frequently and clearly than before. That means getting straight to the point and structuring my message so the audience can follow without getting lost. The real skill is translating complex technical concepts into language non-technical people can understand, without making them feel like you are speaking a foreign language.”
4. Teamwork
“I have become a much stronger team player. We support each other constantly, whether it’s about work or personal conversations, and we share perspectives. I have noticed that building genuine, close relationships with teammates noticeably improves collaboration quality.”
5. Self-Awareness
“One of the most meaningful areas of growth for me is self-awareness. Being honest about not knowing everything and being comfortable asking a senior colleague or manager to explain or teach you helps you learn from experts faster. I have also become more willing to ask for help when genuinely stuck. Before, I thought I needed to figure everything out on my own. Now I understand that asking for help when needed is smarter because someone else may have a perspective or approach that helps you move past the problem faster and keeps the project running smoothly.”
Frank adds one more reflection worth noting.
“Since moving into consulting, I have become more patient. When unexpected problems arise mid-project and the atmosphere gets tense, you have to think fast while staying composed and professional, so the work can keep moving forward. After the meeting, you regroup with your team and figure out how to handle similar issues if they arise again. That ability to stay calm under pressure has improved my overall work output.”
So You Want to Be a Data Engineer: Here Is How to Prepare
For anyone considering a career as a Data Engineer, Frank has some practical advice.
“If you studied computer science or a related field, you have a head start,” he says. “You come in with foundational knowledge that matters most, like database management, basic programming, and system design, all part of a standard CS curriculum. That said, you will still learn plenty on the job, regardless of your background. For people switching careers without that foundation, it is doable, but it will take more time to get up to speed. If you have that technical base, you can build on it quickly.”
When asked about the most important skills for a Data Engineer, Frank keeps it direct.
“Technical skill comes first. That means Python for programming, SQL for database work, and Spark for processing large-scale data. Those are non-negotiables. Alongside that, soft skills matter just as much, especially the ability to communicate and collaborate effectively to work smoothly with other teams.”
And on the topic of English, Frank closes with this:
“English opens doors that would otherwise stay closed, whether that is working on international projects or simply being able to access learning resources. In the tech space, the vast majority of courses, documentation, and communities operate in English. So if you invest in your language skills alongside your technical ones, you will go a lot further than you might expect.”
If you are interested in exploring Data Engineer opportunities at Bluebik, you can view the full job details and apply at https://bluebik.com/th/job/data-engineer/. We are looking for people who are serious about their craft, and we would love to have you on the team.
