I was hired 6 months ago as a freelance data engineer, and after proving myself through my work quality, I am now essentially functioning as a tech lead, with full responsibility and ownership of designing, implementing, and hiring for the projects I'm assigned.
Our company is not a tech company, so I only have a couple of tech-oriented colleagues, and I barely interact with them. Now I directly report to the director of the company, who in all senses is awesome, with 40+ years of combined experience in some of the biggest oil and drilling companies globally.
However, I have some strong FOMO about not being able to learn much technical stuff from my peers or seniors. I am trying my best to learn and pick things up on my own, learning design principles, getting code reviews from chatGPT, etc. But even then, I'm afraid I am not producing the software to the highest standards of the industry since we don't have any rigorous cross-checking, and might be missing out on a lot of learning.
Can someone who has been in positions similar to these please guide me?
1. Learn how to learn well, continuously, and sustainably. Tech changes rapidly. And you will want to hop from one domain to another, just for keeping things interesting and to move with markets. This is both a blessing and a curse. It is a blessing because you can start late and still be in the top percentile if you have the brains and work hard for it. It is a curse because you will be doing this no matter how many years of experience you have.
2. Hone your non-technical skills– caution: these are compounding over time (both good and bad habits) – being disciplined, thinking clearly, articulating clearly, being professional, being trustworthy, managing your physical and mental health, being dependable/reliable, having a growth mindset, thriving in ambiguity and uncertainty etc. then, honing your communication skills – effectively collaborating with people, give/receive effective feedback, do/get mentoring/coaching, working with cross-functional people, working with very seniors, very juniors, peers etc. read a lot, develop mental models, deeply craft your personal approach to first principles problem solving, to making tradeoffs/bets etc.
You can do the above all by yourself, through reading, and observing people from afar, and engaging with people (even strangers on forum like this one) in dialog.
1) Start contributing heavily to a popular open source project you're familiar with and use. Try to make your PRs as high quality as possible. You'll get free code reviews from some of the best engineers in the world doing this. That'll be a million times better than ChatGPT code reviews and you'll be able to learn a ton from it while also getting your code into production at thousands to millions of companies in the world that depend on the open source project/s you choose to contribute to.
2) Fill holes in your knowledge base. If you feel weak in a particular area like networking or DSA then study and practice with it until you no longer feel weak in that area. If you were on my team I would try to assign tasks to you to help you naturally build out knowledge and confidence in your weak areas but without someone to do that for you, you'll need to try to figure out your own weaknesses.
3) Always try to do your best when working professionally. This is all any of us can ever do and if you actively practice it then it'll become a habit that will guide you toward success in any environment.
Read about your technologies. A lot.
When I started working (about 3 to 4 years) , I spent 20-30 minutes a day reading DZone articles about Java, software design, architecture, OOP. Persist and do it daily. Repetition and habit are key.
And be focused. Understand everything in every article, do not accept not understanding everything. For each article, try to find out whether it makes sense, try to understand what the author wants to tell you. Think about how you would have done it. When possible, apply what seems to be useful and overcome the limitations of what is written /proposed in the article.
Occasionally, do the same with IT books instead of articles.
Then explore further, for example, find out how certain popular OOP patterns may be replaced by other, equivalent, patterns in FP. Think about how OOP classes are equivalent or not to closures in FP.
Also, become an expert in fundamental practices, like transaction management.
Application of theory is key.
That's how you do it.
You can learn from everyone around you, regardless of their status. There is no "universal developer experience curve", everyone has more or less knowledge on a field or with a specific tool/framework.
You can learn almost everything alone - I mean learning from the web. There are great forums, groups, discord chats, ask LLMs carefully and check on the answers. It may sound reassuring that someone watches your back and won't allow mistakes or would help clean up a mess, but you should not keep relying on this anyway. Learning by doing and taking responsibility will make you much more self assured, which is actually most of what makes someone senior.
Realistically, you’re not, but does that matter at all to the company? I’ve had your role in a small company, the job wasn’t really about the tech. Unless there is a dedicated product manager, that is your role now too, and it’s one to take seriously.
You have enviable freedom, do not waste it. Be straight with your boss and explain the tradeoffs regarding availability / tech debt / accuracy you may be making and why.
Above all, understand the business, how your boss sees it, and anticipate their needs.
- I'm quite well paid here (4x more than my peers)
- I have been hired full-time, and get all the company benefits as well.
- The work life is great, and I don't work more than 8-9 hours a day.
- The manager/director is great and super supportive.
- I'm not being exploited by any means :)
Kindly upvote so the disclaimer stays on top of the post!
> fresh graduate data engineer
to the company, they see this as "cheap labor"
> hired 6 months ago as a freelance
basically, they put you through the ringer without having to pay out benefits
> essentially functioning as a tech lead
a fresh out of college graduate is far from being qualified as "tech lead", you are out of depth here
> with full responsibility and ownership of designing, implementing, and hiring for the projects I'm assigned
basically you are a 1-bus factor team. hope you have negotiated pay accordingly.
> Our company is not a tech company, so I only have a couple of tech-oriented colleagues
you are a second class citizen at this company, your budget will be constrained, and be on the first to cut when it gets tough. In the O&G industry, the boom and bust cycles are quite frequent.
> Now I directly report to the director of the company
now there's internal pressure to perform at a much higher level than your experience.
If you want real advice, get the hell out of this company while you can. O&G industry itself is notorious for back stabbing/conniving.
Keep fighting the good fight.
I would like to stress that your employer should, and will likely be happy to pay (or give you the time) to do many of the following. Don't feel like you need to do all of these by any means, I just wanted to give you everything that came to mind that has helped me in my journey:
- Conferences both in your specific domain (in your case oil and gas), or your specialty, like a more general data science conference. Meet people there and keep in touch with them, both fellow attendees and also speakers and people who have booths there. Your employer should be cool with covering one or two of these a year. If they're reluctant or on the fence it has helped me in the past to writeup a little summary of the conference, what I hope to learn and how it apply to my work, and give them a written summary when you get back including concrete things that you learned that will help your company pragmatically.
- Meet mentors on platforms like this, foster and build those relationships. It seems like you are already doing this well. Other places to meet them are at local and virtual meetups for the software or packages you use, tools you use, industry societies, etc. Linkedin is also an underrated and underutilized tool for this IMO.
- Free or low cost remote courses. I'm just wrapping up a remote class at Stanford, it's been awesome. Many universities offer courses, or even course materials and videos you can get your hands on. Good universities, with good courses.
Do these things during the work day, because they are work. Don't think like you need to do all this stuff above and beyond. Any good company should be investing in you.
-
1. Turn to online communities to help you learn. Find the appropriate Reddits and Discords, hang out on Stackoverflow, join Bluesky/Mastodon and follow a bunch of people in the same space as you.
2. Start teaching others. The best way to learn is to teach. Answer questions on Stackoverflow that you know the answer to, help people out in Discord.
3. Start a blog! Even if nobody ever reads it, writing about what you're learning is an incredibly accelerant for understanding things more deeply and retaining more knowledge.
4. Conferences. Ask your boss if you can get a budget to attend conferences relevant to your work. See if they'll shell out extra for workshops.
5. Side projects. If you want to learn something new and you don't get the opportunity at work, see if there's a small side project you can spin up to start experimenting with it.
6. Find people in similar roles to you at other local companies and see if you can get into a pattern of meeting up every month or so to talk about what you're learning and share notes. Obviously don't go sharing trade secrets with the competition, but in my experience there is always a huge amount you can learn from talking to peers in other companies, in an environment where people feel safe to let their guard down a little and talk openly about problems they are facing.
Most of which can be distilled into: Don't follow fads, the latest shiny tech; they will only break your heart and maybe ruin your career. You want to deliver a quality product that just works, never breaks down, and if it does it is easily diagnosed.
What that means is (a) use software from vendors you can trust and have actual live support (this means they are usually expensive) (b) KISS. In a small shop you will be responsible for everything. You want as few moving parts as possible. You especially want to avoid all custom integrations; they suck. If software B doesn't offer an out of the box connector to software A which your company already is using. They are immediately ruled out. (c) never trust a salesman/saleswoman. Before you sign any contract, you want to try it out as proof of concept. Just because they say they have an out of the box connector, for example, does it actually sync the fields you want? Verify it works before signing any contract. If the salesman doesn't like it, tell them good bye.
Something that turned out to be extremely valuable was having the chance to make lots of technical decisions (many big, many teeny tiny) and then stick around long enough to evaluate them afterwards.
A lot of things worked out great, and a lot of things didn’t! And now I have a decent amount of insight and opinions on those sorts of things. And those insights getting me a lot of respect at my new job! Which is only my second job, after being in the hot seat at my first, which was a startup.
So my advice is try to at least stay around to evaluate your own decision making. You will end up challenging a lot of your own beliefs. I think it changed me more than anything else at that job.
Sticking with tech sounds like a safe option, but if this opportunity was given to you, maybe you could think about rising in the rankings within the oil industry.
I’m not saying this is the only choice, but this is a choice that you should consider as well.
Also, learning from more senior colleagues is really a hit or miss. I’ve been in environments where I couldn’t learn anything from senior colleagues and where I learned a ton working with relatively junior people.
The fastest way to learn is to move to the Bay Area and work with people who have been doing this for decades. That is not a sufficient criteria (anyone can be a bad teacher), but the experience is extremely useful.
If technical proficiency was all that important to your employer, you probably wouldn't be in position you're in.
I wouldn't worry about this too much. As long as what you're doing seems to vaguely match whatever information about best practices you can find online, you're probably good. As long as you're not inventing your own security tech, using floating point values for financial data or reusing feature flags in a high frequency trading scenario, any bad decisions made along the way probably won't be catastrophic and are more likely to be learning experiences.
More eyes on code is a good thing, but can go too far where a person is cargo culting their beliefs and subjective experience as objective things, leading you to learn the wrong stuff, or at least to a degree where you have an irrational attachment to some practices (e.g. some people always lean towards EAV in DB design, even though that practice has a time and place, same for microservices vs monoliths vs modular monoliths etc.).
> ...and might be missing out on a lot of learning.
This is probably what you should focus on. See if you can attend industry events, watch some talks done by people who've been in the industry for a long time, there's actually lots of nice stuff on YouTube that's not just advertisement for the latest technology or practice. Here's some people whose content I enjoy:
Dylan Beattie
Venkat Subramaniam
Kevlin Henney
Maybe look at some of the better received books. Here's a starting point: A Philosophy of Software Design
Algorithms
Building Secure & Reliable Systems
Clean Architecture A Craftsman Guide to Software Structure and Design
Clean Code
Code Complete
Design Patterns Elements of Reusable Object-Oriented Software
Next-Level Database Techniques For Developers
Patterns of Enterprise Application Architecture
Peopleware
SQL Antipatterns
The Clean Coder
The Mythical Man Month
The Pragmatic Programmer
Working Effectively With Legacy Code
Again, take none of that as gospel, but maybe give them a passing glance.Also, props to whoever mentioned learning by doing in a low stakes environment (side projects) and online communities!
Keep a tech journal.
In it, describe what you're working on, what problems you're trying to solve, and questions you have.
Make conjectures based on what you know and what you guess, and write those in the journal too. Come back later and flesh out or correct those conjectures as you learn more.
As your writing reveals holes in your knowledge, add those topics to a list of things to research.
This kind of journal is much easier to keep in a wiki or note-taking app like Obsidian, since you can then build a personal knowledge base around it. However, you can also do it the old-fashioned way, with pen and paper. Do whatever comes easiest, since that's what will stick.
Do exercises. Take 30-40 mins a day to do practice problems. Use a site like codewars, leetcode, even past advent of code challenges. Do the exercises in the text books you read.
Write. Write about what you learn. Use your writing to explain it to someone who is encountering the subject area for the first time. Doesn't matter what you write it on: a paper notebook is just as serviceable as a blog. The act of reinforcing what you discover by writing it down helps to reinforce it and challenges your knowledge by explaining it to someone else. Whatever you do make sure that the act of writing works for you: it must be seamless and flow easily.
Don't over-engineer the perfect writing system that use a ton of frameworks and require maintaining deployment scripts. You will avoid writing if it is hard to do. Writing itself is hard enough and that's what you need to focus on. If you want to go the blog route make sure it's no harder than opening up a text file and running a simple script when you're done.
Lastly, find a mentor. It can be someone outside of work -- and in your case sounds like it will have to be. Contribute to an open source project you use regularly. Go to meetups in your local area or online. Find someone you can talk shop with. And when the opportunity arises ask if they'd be interested in meeting up once in a while to review your work or talk about what you've discovered.
BTDT (Companies too cheap to hire experience, and I'm pretty smart). I am also a high school dropout with a GED, so...cheap.
In my case, I remained very humble, voraciously ate up all the literature I could, and got my company to send me to many seminars, conferences, and classes. I did a lot of traveling, and learned to be frugal (lots of cheap motels).
I made a lot of mistakes, too. My company was fairly tolerant of that, because ...cheap, and I also had a lot of successes.
"Good judgement comes from experience. Experience comes from bad judgment."
You have a marvelous opportunity, here. Stay humble, learn like crazy, and don't be afraid to make sure that your managers know where you're at.
Good luck!
You have been given an opportunity to build and lead team, hire the person you would want as a technical mentor. Focus on the big picture results, what will you be able to put on your resume? Built and Led a team that developed a technique/software/product that yielded $x result for company, or “worked on x product in y language”
It’s a different class
If someone who has a successful company wants to interact directly with you, learn everything you can from them, that is an opportunity greater than any tech bullshit
Having an insatiable drive to learn is the only requirement! Keep reading, keep doing.
Would I change it if I could? No. I think I’ve built up a strong will and understanding of the craft. The opinions I have are self formed from doing. I like that.
I at least tried to reverse the situation by doing whatever I could to help mentor my juniors over the years. But it must be said: just because someone is senior it doesn’t mean they’re good. You can easily end up learning dogmatic bullshit that takes years to shake off.
So I always say to juniors “take my words with a pinch of salt. I might be wrong. You need to find out what you believe on your own. Don’t just rely on my words alone”.
In your situation you should have an honest and frank discussion with your boss. Tell them you're feeling a bit out of your depth. Explain that you think you need more senior heads to help guide the project. He/she'll either take it seriously, or not. If not, leave and find somewhere else to work. If you don't have this conversation then any failure will be yours alone. If you do have the conversation, then you've moved the ownership of future issues to your boss.
Also, working in a company where the product is what you write is the place where you'll learn the most.
So in short, see if you can contribute to a well-run open source project with a good community.
Pairing and doing code reviews together with someone, even if they're not more skilled than you, is a nice way to feel better about technical work. Catches bugs well too.
Seeing if you can learn from seniors in or outside the company is worth it. Sounds like you won't get detailed 'here's the right algorithm for X' help but that's not the worthwhile bit of any mentorship IMO - the useful parts are more often 'here's a way to push back on unrealistic requirements / work with colleague X better / handle conflict.'
Finally, remember that there aren't some mythical grownups who Do Things Right(TM) - everyone is making it up as they go along. You're already valuable to the co, and you'll keep getting better with experience - blink and you'll be the senior producing to 'highest standards' and giving all the new hires impostor syndrome ;)
> However, I have some strong FOMO about not being able to learn much technical stuff from my peers or seniors. I am trying my best to learn and pick things up on my own, learning design principles, getting code reviews from ChatGPT, etc. But even then, I'm afraid I am not producing the software to the highest standards of the industry since we don't have any rigorous cross-checking, and might be missing out on a lot of learning.
On having the right mentors:
This sucks, and I think the rest of the comments have good advice (look to open source, read more, etc.). I'll also say, think of the upsides. You have a lot of autonomy. You have a lot of latitude to implement your ideas. You have the ability to ship something and see it fail, learn from that, and improve it.
At some of the big companies, there are so many talented people hungry for good jobs or empire building that it’s really hard to have your ideas implemented.
On high-tech quality:
I spent many years working at Airbnb and Amazon. I have friends at Facebook and Google. The honest truth is that all of these places have lots of bad code. The entire industry is held together by duct tape and glue. If Google (clear tech leader of the 2000s-2010s) and Airbnb (Tech-darling built in a post-AWS environment) both have "bad" code, where is there "good" code?
1) Companies that succeed solve business problems and don't focus on fixing tech debt that doesn’t fundamentally matter to customers.
2) Real businesses are complex and don't fit nicely into good design patterns. Every big refactor and rearchitecture I saw at Airbnb started with good design but got complicated once they hit reality.
3) Good engineers deliver good results despite bad code. Take the time to deeply understand your tech stack. Figure out which parts of the code are brittle and figure out ways to improve them. Make the case to your business partners that fixing things will improve developer time, reduce mistakes, and save them money.
And you’re in your first job, not your last job. Take the time to learn the ropes, and when you feel like you've outgrown your role, find a new one. There’s always something to learn.
If you aren't experienced already it is a very hard position to be in, with a huge responsibility and potential to screw up due to inexperience - and be promptly thrown under the bus when the proverbial shit hits the fan. Code reviews by ChatGPT can't compensate for lack of more experienced colleagues.
The best way to grow as a fresh grad is to join a medium sized, established business. There you are going to be a part of a team where you will get to learn the ropes of your field (that's something you won't find in ChatGPT or university) and at the same time the pressure won't be so hard. And while you aren't going to get the perks of working at huge companies like Google or Facebook, you likely won't have to deal with assorted corporate BS that comes with it either.
Only once you get a few years under your belt think about startups, small companies, etc.
Remember the point of industry software isn’t the software, it’s solving business problems. “Right” technically and “right” for the business are sometimes not the same thing. The only way you can learn to tell the difference is by keeping a good pulse on the rest of the company, and looking at your past decisions in the light of whether or not they enabled its success.
If you can't, find some courses, and participate in open-source development (where they need help, and will often provide feedback). If you need rigour, try test driven development. Attend meetups and conferences and try to meet people there.
I've spent my commute time to learn about these topics from youtube. For example watching EEVBlog or Uncle Bob or whatever. Then stop sometimes and just think about it regarding my past experience. Replay the stuff that I lived through while thinking about what I heard. Then find my best colleagues and sincerely ask them what they think about what I was concluding just then.
Also I like Twitter/X after just muting/notinterested-buttoning the people who just post idiot stuff. Then it is quite a good source of ideas about the present mainstream - that's also very important.
But the most important: never forget while swimming in ideas and memes: take some minutes sometime to asses what actually worked - and what is that somehow never works for ... reasons.
Great question on HN!
It sounds like you have the initiative to push your own boundaries, and that's what matters most. No matter where you are, learning starts with your willingness to learn.
I'll admit I've never been in your position precisely. I've always worked with technical people, even if not for "tech companies." But, before I had even graduated I was the only person in my entire company doing mobile dev, so I know what it feels like to be very green, and very far out on your own island. It was a great springboard. I never did mobile dev full-time again, but I experimented and explored stuff I found interesting, and the skills I learned have been invaluable to my career.
I don't know what kind of languages/environments you use, but I recommend investigating and trying out some popular technologies and architectures that are allowed in the scope of your work. Try to understand the pros and cons of each approach.
When you want to know the "right way" to do something, try reading the code of the most experienced programmers and successful projects written in the same languages/environments as yours. Very little in software is "new" so chances are someone has tried it before. Maybe LLMs are good for this, but I usually just read the code.
Focus on iterative improvement not destinations, because you'll never be "done" you just move onto the next project and the next technology. You'll get better as long as you're trying to learn new things.
Mentoring - As you build your network, you'll naturally find opportunities for informal mentorship. This could be someone with a solid career who has already navigated the tough paths and can help you tackle challenges while also offering guidance for your journey.
There have been many times in my career when I felt stuck. By having regular conversations, listening to others, and sharing work experiences, I gained a broader perspective and have always been able to found ways to move forward.
I am self taught and had no mentors for the first decade of my career. During that time I did things nobody else could do, things I was told were impossible.
Eventually I read some books and they added useful tools.
Read all the books from SICP to Superintelligence (ie concrete to speculative fiction).
But nothing is as educational as putting what you read into production, or putting it into production and fucking it up. I’d rather work with someone who has deleted a production database (ideally a preproduction database to be fair) than someone who hasn’t.
1. Learn algorithms, buy some book and start reading.
2. Learn theory of neural networks, use the Chollet book to develop some knowledge and intuition.
3. Write many small "pet" programs implementing basic stuff: implement a small database, a small programming language interpreter, a small editor, a small neural network, ... Each time try to apply good design.
4. Embrace simplicity in everything you write, don't make things more complex than needed.
5. Read good quality code, especially read open source code which is at a degree of complexity that makes reading valuable. Dont' read Kubernets... or PostgreSQL perhaps. Learn more self-contained code bases.
6. Participate to some OSS poject.
7. Start some side-project and put it on GitHub, were you are the main designer. Develop something that you need. A library, or a small utility, something that you really enojoy doing. Do it at a quality level that makes you happy. Never regret pushing code on GitHub if you feel that for the level you are right now it is appropriate.
8. Don't give a fuck about what other programmers think of your work, if you did it at the max level of what you can do. Anyway most programs on the Internet suck, including the ones of people feeling very competent.
There's lots of good advice in this thread, but I think the most important thing to keep in mind is to know that you will make mistakes and do stupid stuff. Always be evaluating yourself and use that to adjust how you do the next thing.
Anecdote: in my first real job after school, I was essentially give a one person R&D project that reported directly to the CTO to design a framework for "the future of our data analytics". I ended up designing this crazy over-engineered highly abstracted OO framework that was a nightmare to document and use. But since it was fast and met the requirements, the CTO pushed it through as the framework for the rest of the engineering team.
I spent the next couple of years justifying it and supporting it. Once I recognized that I made many dumb (well maybe not dumb, but novice) decisions. I used that as a learning experience for my next project.
I've had many of those kinds of learning experiences over the years, and as long as I keep self-examining and being self-critical, I learn from them and don't make the same mistake twice. Now 25-ish years later, I have a long list of internalized knowledge of what not to do.
I guess that's a long way of saying, don't be afraid to make mistakes, you will. Just pay attention to them and learn from them.
Failing that, I’m not sure what your relationship with your manager is like, but in most tech companies, part of your manager’s job is to help you grow. Maybe they could help you reach out to the few other tech people at the company to organize some kind of collaboration to help you all grow and improve.
I am interested in the other answers here and will add my two cents.
Recent hiring: KEEP IT STUPID-SIMPLE. Being new at the company, less experienced, and empowered, you will probably be tempted by lots of "solutions," whether they're products, frameworks, management buzzwords, etc. Most of them are not designed for a one-man-team. Make sure your solutions actually help YOU.
Non-technical boss: AGREE ON METRICS. Your boss/es will measure you eventually, whether either of you are aware of it or not. It may be better to discuss some metrics while you have a chance to do something about it. At the end of the day, it's almost always about money (and sometimes ego), especially with cantankerous O&G types. Just figure out how you can show you enable, save, or make money within the system.
Feedback and standards: YOUR BEST FEEDBACK AND STANDARDS WILL COME FROM YOU. You are in a position to act on your best judgement, and then own your mistakes. Yes, you can feel like a fish-out-of-water a lot of the time. I have tried to find a balance between pushing myself to learn personal skills on projects, innovating for the company, just getting things "done," and keeping myself sane. Your peers, "clients", and supervisors are probably not used to (or aware of) "best practices", which is a double-edged sword. They might expect you to deliver things you know could be better, but you also have a ton of power to build a solid environment for yourself. Think of progress like driving a big boat, not a racecar.
Join Discords related to the stuff you’re working on, you’ll find people much smarter than you and any of your colleagues hanging out, talking about good approaches to designs, structures, infrastructure, etc.
You’ll also find idiots not worth listening to.
Remember you’re 6 months in / just doing the job is enough of a challenge. You’re calling yourself a lead for having full responsibility of the projects you’re working on. This is typically common everywhere except large tech companies.
Read and study and learn from your mistakes and the mistakes of others – you will believe things and realize later that your beliefs were wrong.
If there was one book I had to recommend that I have read it would be Sandi Metz's 99 Bottles series.
Oh, and learn multiple programming languages.
There's no way not to learn from this, and there's things you can learn here that you wouldn't learn from years of being in a regular team. So just as you're missing out on valuable experiences here, you'll also be missing out on valuable experience if you change jobs.
The interesting part of a career is also all the non-technical people we meet, and if you're learning from them, that's also good. Maybe this road will take you somewhere more interesting than spending a lot of time on code reviews.
(PS: If this is actually in oil, make sure you're getting paid well.)
Self-directed can absolutely help. (Though, a recommendation: do a lot more reading of human-written articles and a lot less chatting with AIs.)
But you also need to find some other people to talk to, both peers and people more senior than you along one axis or another. You'll learn from them and they'll learn from you. Open Source projects, communities of practice, professional organizations, friends in other companies, online communities, etc.
As the most senior technical person in the company, you can treat this as an opportunity to create paths for technical and process development for your technical (and non-technical) colleagues. If you find opportunities, make those opportunities available to your colleagues as well, or pass on what you learn in one form or another.
My tip is to try and best practice everything. Locate the correct method online, especially if theres applicable vendor best practice. Every time the director asked me to do something new, I would go away and find out what I should be doing, and then made that the basis of our negotiations. I didn't win every one of these fights, but I managed to spin the job so far that I did IP telephony, Security Cameras, built the company's ad forest and hundreds of other small relevant to my career wins. I ended up with 2 direct reports. And in the process I took a very immature business and injected a large amount of maturity that they kept with them for more than a decade. And I did them correctly, or as correctly as I could without support. By the time I looked to move on I was immediately made a senior in my next role.
Here's what I recommend:
1. Build a corpus of materials: Conferences (in person, and you can youtube them), books, courses. Cross product these concepts with tools/techs - DBs, coding tools like IDE and source control, programming languages, programming paradigms, software craftsmanship, network stack / protocols, debuggers, Observability and Ops, Peopleware / wetware. 2. Think of attending conferences as seed materials for future research, write down every term, subject, and technology you're curious about, never heard of, are scared of, think is out of your league/level. 3. Spend an hour a day on weekdays, and 2-4 on a sixth day, sharpening the saw[2] (learning) 4. Close loops on subjects by taking action. Eg: If you finish a chapter on a certain DB, install it, insert some data, learn to bulk insert, try some complex queries. If it's a language try to write a breakable toy[3]. If it's a paradigm, try a code kata and write the same one in multiple paradigms. 5. Remember: you read a text book a month in Uni, try to finish a text book (or equivalent effort) per quarter/half for the remainder of your career.
[1] - These are definitionally top .1 or even 0.01% engineers (amongst the engineering population, not the general population)
[2] - Sharpen the saw - https://www.franklincovey.com/the-7-habits/habit-7/
[3] - https://ilango.hashnode.dev/learning-by-building-breakable-t...
I have a few years of experience under my belt, in a small team of ~10 devs. No one is a software engineer, and it's everyone else's first job essentially. There's no one senior from whom to learn, and I feel stagnant.
I improve by interacting with the open-source community. I have been getting more and more involved in several projects over the years, even making it as a maintainer for a few of them.
I recommend you look at smaller communities -- my favourite is Haskell -- as compared to very established ones like Python's. Smaller communities mean that you have more impact as a contributor, which requires you to keep pushing and learning new things. Smaller communities make it also easier to interact with senior people, for example on Reddit or a Discourse instance (e.g. https://discourse.haskell.org)
I suggest this high level idea, which I have done in isolation in the past and caused tremendous growth. Think of something you'd like to optimize, run an experiment over a year and take careful notes. For example, once I decided I wanted to minimize defects that I released into production. For a year, I tracked every single defect I released into production, and made brief notes about how I could have avoided it. This led to me completely changing my attitudes and approaches about what is important. I discovered, for example, that very few of my production failures were caused by me writing code with bugs, and that by far what I needed to do is improve my testing strategies, what kind of data I test with, and how faithfully I represent the production situation during unit testing and end to end testing, and how I approach testing in general. There were a lot of observations and approach changes I made, but the thing that made these really stick is that they were my own lessons. Everything in my process was rooted in a hard lesson I learned myself. Whenever someone asks me why I do X I have a war story to back it up.
The point is, you can figure out for yourself the big areas to improve if you set a goal for a thing you want to improve, and carefully record your results, and review those periodically and look for commonalities. You can look at the advice from others endlessly, but the best teacher is experience, usually your own. Make the most of your own experience by figuring out what's important to you and think about process improvement for yourself and those that you work with, as a first class thing.
- Learn other frameworks and/or programming languages. It gives you a wider and unique perspective on other "correct" ways of doing software. Through finding commonalities one can arrive at things that are good practices regardless of the tool being used. Through finding differences, one can better understand what might be dogmatic to a specific language/library/etc.
- Dig deep. If something isn't working, as much as possible, make it your goal not to fix it, but to fix it and understand why it was broken in the first place. Trying things until something works is a tried and tested way of fixing things, but digging through the abstraction layers and confirming why that suggestion one found online is very valuable. I think a lot of knowledge from senior developers boils down to having experience at lower levels of abstraction.
As someone who was a developer and ended up in a support role by mistake of company
Now i left that company but here’s how I relearned everything
- Read lots of Code, look how things are implemented. You can use LLMs just don’t blindly use them to copy paste ask the LLM to thoroughly explain the approach.
- Treat LLMs as someone who has lots of Knowledge whom you can ask lots of questions.
- looking at different open source projects is really helpful and having the hungry mindset of learning
- How will the company / your boss decide that you are doing a good job? - Focus on that first. Include the parts they don't directly value but where you think you need them in order to reach that goals.
- How do YOU know you are doing a good job? - to develop that taste you need peers to look up to. Find them in meetups, in un-conferences, conferences or on the internet.
- Build your skills in three main areas:
- "Behavioural" skills (a.k.a "soft" skills) - written communication, verbal communication, be good at giving and
receiving feedback, building professional relationships, organize your work, learn to "manage up" - domain skills (learn about the company & the field)
- tech skills - focus more on concepts & principles and less on the "framework of the day".
Not impossible but much harder than working closely with humble and experienced engineers. That's why I recommend that a junior engineer finds a great consulting company as soon as they can and spend a few years there. This was a kinda breakthrough period in my career for both technical and soft skills. The nature of consulting also means that you might see a greater variety of environments, technologies, and problem/solution combinations than the equivalent xp in a non-consulting role. But make no mistake, finding a "good" consulting company is key. You need to have humble and experienced engineers who want to help you and grow you. If it's a "bums in seats" kinda "body shop" or some kind of celebrity house for "hero" or "10x devs" then maybe this won't be as useful.
2. Do that by reading, here and other useful places, not necessarily about your own area.
3. Start to understand your industry, its details and quirks, watch your directory, ask them questions, learn the business in detail. Even if you move to delivering IT in another industry, the knowledge will hold you in good stead.
4. Understand how to model business workflows without worrying about technology. Learn how to communicate that with your users and customers (two different groups sometimes).
5. Nouns are more important than Verbs. Concentrate on the "things" and how they react to events and generate their own.
6. Technology comes and goes (MVC, DAO, microservices, monoliths, k8s, etc etc), processes as well (6 sigma, CASE, Agile, UML, etc etc). Learn their important details and differences and don't think any of them are silver bullets.
1 & 2 are the most important.
What I did:
* Read. All the things. Read HN comments, read tech blogs, read books. Don't put too much weight on any single book/comment/blog post—they'll almost all disagree with each other, and you'll learn the most by comparing and contrasting all of the strong opinions rather than uncritically absorbing any given opinion.
* Be okay with using work time for professional development. Your employer hired someone they knew wasn't experienced into a role that is typically filled by someone more senior. They need to be okay with you learning on the job, and if they're not you need to go somewhere else.
What I wish I'd done but failed to do:
* Don't get overly invested and burn out. This is your first job, but it's not your life, and your career doesn't depend on you completely blowing it out of the water as a brand new developer thrown into the deep end. Don't overinvest.
* Keep your eye out for red flags. Tiny companies that can't hire seniors might be small and niche but sustainable, but they might also stay small because the business side of things just doesn't work for various reasons. In my own case, it turned out the founders of the tiny company were both narcissists (which seems to be a common problem with tiny companies), which led to a ridiculous political implosion that I was lucky to escape.
My thought process was I get to learn stuff, so even if I didn't make any money off it (I mostly didn't), I learned how to ship. This included (yuck) documentation, QA, etc. Doing things like that you really learn not only how to code and how to structure your programs, but about the 80/20 rule which to me is it takes 20% to get POC and 80% to get it ready to ship.
The senior I finally got to work with was mainly to validate that I was on the right path, because I could write just as well as he could. I met this guy probably 4 years into my career and we were pretty equal at that time. He had about 5+ more years experience than me, and he was VERY sharp. I probably haven't worked with someone better since and this was 20+ years ago.
One more thing, I was the "computer guy," at my first company out of college. I did the workstation building and support, the networking, the server setup and user administration (Windows NT days), and wrote applications in MS Access for the company to use. I had to interview the users and managers on what the apps should do and whatnot. I had to learn how to scale MS Access for 250+ user base (ugh). I had to do all the reports and everything computer related. That company was a toxic shithole but it was actually a blessing. I never would have learned as much had I got a big corporate job. You tend to get pigeonholed in those places doing the same thing over and over. And yes, I was terrified of not being good enough the whole time; that helped too.
Good luck!
Practical tips on learn good techniques, do research, find the best tech companies that do similar development to yours. Check out their technical blogs, their githubs, find opensource projects which have been developed to the highest standard. Dig into them and potentially even rewrite your own simple versions to learn, maybe twin it so you could make the new implementation a part of an internal research project... so main possibilities there.
You need some example, some role models to look up to.
And it's usually not about some tech details (you can google it), but about some important style of work, ways of thinking, approaches to engineering, attitudes etc.
Something that you can't find in books, yet it is very real and important for hi quality engineering work.
At the same time it is not necessary for a relatively successfull career. Most of the companies are fine with mediocre software products (you can see this around you lol) and would not pay more for higher quality work.
So the ultimate answer depends on your definition of "grow as an engineer" :)
I work in a team with some very senior colleagues (and even famous in computer circles for a couple of them) in a reputable company. I actually don't learn much from them. They review my code, we discuss some design docs, it's certainly interesting to see how they manage their time and how they deal with various problems, but I don't learn much from them in the technical sense. Also to my surprise, we also don't produce software to the highest standards...
If you want to improve technically, you can read books on things you'd like to improve, or find technical projects outside of work.
The disadvantage is that it’s hard to grow beyond what you can visualize, so past a certain point you’ll likely be growing slower than you would working under someone with 10-20 more years of experience.
I would advise one of two paths: either stick around a year or two u til you can confidently own “tech lead” on your resume and move on, or commit to a longer stint and really reap the learning benefits of the technical leadership position.
So much for the unsolicited career advice :). To answer the direct question, a few things I found useful as a startup CTO:
- Follow the news for your tech stack. Stay up to date with the various technologies. Go to meetups. - Think about the long-term development of your system. Spend 1% of your time thinking about where you might be in a year or two, and do some experiments to explore those paths. Your codebase is the best place for you to experiment with new tools as it’s a real-world system with the warts that reality brings. You don’t have to merge these experiments! As CTO I had probably 20% of my PRs as RFC / experimental work. (I do not advise contributing to FOSS for learning in your scenario. Feel free to do so if you enjoy it though.) - keep an Architecture Decision Log. Review the big decisions you made in hindsight, and learn the lessons that you can only learn if you own a system for multiple years. Eg this DB choice seemed good and bought us a year of scale reprieve, but then required a 2 man-year rewrite down the road. Knowing the true value of “I” will make you much better at judging tradeoffs in ROI. - Beware burnout. If you are studying and leisure coding in your spare time you will eventually run the tank dry. It might take 5 years or it might take less. Just be mindful. - Don’t be terrified of burnout. The tech lead position is a unique opportunity to feel ownership of a system and have the autonomy to try things out. It’s a role where pushing harder can yield more leverage than a normal position. So putting in some extra hours can pay off in the long run for your career.
The way I excelled from beginner to a higher level engineer was self learning, curiosity, building things so I could understand them. Example: when learning dependency injection, I found a tutorial on how to build your own DI container. So i built one and saw how it worked internally. That was SUPER useful.
I recommend Podcasts, YouTube Videos, Courses, Conferences, Participating in Open Source (even writing docs that are missing is helpful). Ultimately its about using what you learn and always being curious, learning patterns, and what is useful vs what is not.
The last thing you want to do is learn from a senior. Any worthwhile mentor that actually is worth listening to will usually tell you to go try stuff and figure it out.
If you have a laptop, you have all the information and environment to learn. The only thing that you need to have is a framework of learning, which is very simply this:
Chose a feature to work on -> Write some code -> something doesn't work -> research the error, figure out why it doesn't work -> try various solutions until it works.
Rinse and repeat.
2. Get a textbook for a subject which you have taught yourself. Read the table of contents. Assess which topics you know cold, and which ones you have blind spots in. You can assess your mastery of a topic this way.
3. I learned the most by making bad choices and then having to live with the consequences. I learned the second most by having to justify the corners I had cut to other adults. Neither one is fun, but both teach you to be better.
However, if you learn well by doing, or by reading, there are loads of other great ways to improve technically. I’ve made big leaps forward in my skills by building (relatively large) side projects, where I can safely experiment with different design decisions and see the consequences over time. I’ve also got a huge amount out of just sitting down and reading the docs for tech I’m interested in - some frameworks (like React) have fantastic resources that can take you from good to great.
Good luck!
Because you might feel unsure of yourself, you will be tempted to hire folks who are less junior than you - resist that. Hire senior people (budget allowing) and start your interviews with them by asking if they are comfortable reporting and working with someone with less experience than them. As long as you are humble and want to grow, you will.
If you’re the smartest person in the room, yea, that might be a red flag.
But, here’s some different advice though: get the company to hire someone more experienced. Be willing to give up some oversight responsibilities to that person. (Or hire someone who is experienced and explicitly doesn’t want to play the politics game.)
I felt like I was in a similar situation fresh out of school. It wasn’t until my company hired an incredibly talented person that I felt I was finally learning from someone (beyond reading books… back in that day, they were still pretty relevant and up to date).
Why?
I once encountered a tech lead who was at the same job, since graduating, for about a decade. All of the novice mistakes showed. Because there was no one there to correct basic security mistakes, the lead grew very accustomed to getting their way and getting constant flattery in the process.
When the time is right, seek a position where you're generally the most junior employee, and seek to learn from everyone around you. It's important that you can learn from others before you get into a leadership role.
Standards can slow you down, which is fine in a big tech company, but probably not good for building value for your company.
It's better to think more about what may break in your code base that could be emberassing for your company and build integration / othere tests for those important cases.
Otherwise generally simple code is good code, keep building and have fun in life. Lot of those ,,standards'' just overcomplicate things and may hurt your company.
I find that it is refreshing to hear the perspectives of working and aspiring programmers, and I stand to learn a lot about the field outside my little bubble within it: what various startups are doing, the problems they're trying to solve, etc.
In short, if you cannot find humans to talk with and bounce ideas off of at work, do so outside of work (but be careful about your company's trade secrets, etc.). Do not rely on ChatGPT. ChatGPT is to discussion what ultraprocessed food is to food.
You can use a lot of modern Internet resources.
I got a Learning O'Reilly Account. There you can find courses, videos and live events where you can ask questions. It's helpful.
Read a lot of good books/e-books, ask in forums and maybe in your town is a user group for that.
Just write the software and constantly look to improve. Your code from 2 years ago is in an ideal situation, much worse than your code now.
Find some software that helps your company, and don’t be afraid to dive into the codebase. There may be times you can even contribute as part of your job.
Prepare for the day when workload increases to the point where you are hiring in your first resource. You are now the senior in the equation.
This can be an exciting time. What’s the first thing you’ll hand over? Does it make sense the way it works now? What would you do differently?
Ask yourself these questions and make the changes now, while you have time. Prepare for the team you don’t have. Yet.
It's probably not polite to ask, but I would try and form and answer the question of why that's happening - while it can be a learning experience you dont want to be the smartest person in the room (ever) when you are just starting your career.
I was in a situation where I was able to spend on avg. 14 hours a day for ~3 years programming without a single day off, trying to up-skill myself as much as I possibly could. Fortunately I already had much of the context and foreknowledge of what I needed to be learning, so all that time was spent building; instead of trying to learn what to build, or how.
After some time, I was given an opportunity to join a tech startup, and excelled. I wrote the vast majority of their code bases, I manage virtually all of their ops/infrastructure/deployments. I mentor our junior developers, I conduct all our technical interviews, and I make our implementation/design decisions, even tho we do employ other senior developers with many more years of experience than I have.
Many would be skeptical of someone with so little experience on paper being in such a position, and it honestly I believe it will be a very uphill battle you will have to fight when you are given such a high position, when it's time to move companies. To be quite honest I didn't even want the title I was given officially, because I didn't feel it was appropriate from an industry standard. However there is no doubt that it applies if you read the relevant job description.
As to your question specifically, seek out and spend time with more experienced engineers whenever possible. I am very lucky to have friends with many, many years of industry experience working for large tech companies, who I have been lucky enough to be able to work on side projects with, as well as having been able to contribute to open source, as well as building my own projects and taking on contract jobs on the side where I have solo developed a fairly substantial project as well. I am very grateful for these friends and I have learned very much from being able to build with them.
Build as much as you possibly can, work on projects that are far outside the scope of what you do for work, and take on challenges you otherwise wouldn't feel ready for. Try to befriend experienced devs and seek to be a part of communities where experienced engineers are found. There is no substitute for the hours of writing code, but surrounding yourself with others who's observations and code you will be able to learn and benefit from is a good start.
My first thought is... why? Have you tried reaching out to them to discuss what you're building? Maybe a shared chat channel (slack, teams, whatever) where you can discuss what you're doing; or even rubber duck and people can read as they want and offer their insights. Just because you're not on the same projects doesn't mean you can't talk about your work with each other.
You'll get to work with others who are more senior than you and see how they do things. It's definitely helped me a lot.
“Software standards” come and go, changing relatively quickly, so don’t sweat it too much. You will learn from mistakes, and that’s ok (for me, dunno for your employer :P)
This kind of prompting changes the conversation entirely.
Do you want to write code, and get better at writing code? Then probably find a new job with better mentorship.
Do you want to grow quickly in this company and are okay with being a manager? Then stay on your current path, hire some solid software developers to your team, and embrace the responsibility you’ve been trusted with.
It wasn't until I started going to conferences, Lisp/Python/Perl etc, that I actually met people with the same level of brain damage or worse than me.
Both paths lead to the same place, having access to mentors gets you there faster I guess.
Perhaps they feel equally isolated.
- Conferences, not my thing, but if you are new, you may learn a trick or two there. No just go there, try talking to the people. Approach a few senior looking guys and ask them for advice.
- Confs can be quite expensive, a cheaper alternative is local user groups. You can try to find the closest ones via Meetup.
- ChatGPT (yup, hallucinates but still has a reasonable answer for 90% of the questions).
As for your situation:
- You are responsible for hiring. Try to hire people with more experience than you.
- "to the highest standards of the industry" isn't that perfectionism? Most production code has much much lower quality than what we see in open source.
- I think for the time being (the job market is really bad right now), just concentrate on delivering stuff. Learn if you have some time left, but delivering solutions should be your prio #1 in my opinion.
But aren't needed to grow (but make growing so much easier and as such allow you to grow more over the same time frame).
The most important part are:
- self reflect on you work, importantly do take a emotional step back before doing so to (hopefully) have a objectifish self reflection
- if you do mistakes try to understand _why exactly_ things went wrong, instead of just finding a solution or a target of to put blame on. E.g. if some system lead to some issues don't just put if of because "that system is bad designed" but try to understand which aspects of the design are the problem and dig deeper to find out e.g. if there is a reason it's designed that way.
- in general try to learn the underlying principle independent of the fancy branding and terminology it's backed in. E.g. people all the time reinvent and rediscover long term known concepts and them give them new names and say it's something completely different even if it's not. (But also new impl. of old concepts can sometimes make the difference between unusual and a grate solution). E.g. rail track programming (e.g. in swift) is just a reinvention using Monades for error handling.
- there a many seniors sharing their knowledge freely on the internet, the tricky part her is that not everyone pretending to be a expert is one and not everyone who is a expert in one area is one in another. So never blindly follow their advice.
- when you get advice, especially from less trustable sources like the web, understand _why_/_how_ this advice is going to help instead of just following it. Also question the advice, try to find out when the advice might be a bad idea even if it seems to always be a good idea.
- reading documentation instead of just googling (or LLM generating) solutions can help with understanding (through not always, a lot of doc is terrible bad). Now speed wise you might not have time for that but double checking found solutions by checking them against the documentation is something you often should do anyways.
- be aware that even if you feel you are a senior there is a lot of thing you don't know, more importantly a lot of thing you don't know you don't know.
- don't just reflect on work you recently finished go back to older work
- if it's your thing, keep notes in a well organized way and go back to them to reflect how you understanding might have changed
there is more but I have to go
Where in the world are you? That determines your options.
It is so valuable to take the ideas and questions in your head and get them out. You can then have Claude interrogate and problem solve with you.
By default, I think Claude is very conservative and sometimes comes up with ideas that aren't quite right. But you can actually just tell it to be less concerned about X or Y, to start over from scratch, etc.
Another trick I use is to ask it to ask me one question at a time to help clarify my thinking. I find that the one by one questioning really helps me find the boundaries of my knowledge and crystallize my opinions.
Build things. Solve problems on your own, or if you are already in management instruct others to do so. You learn by failing until you don’t.
0. Writing a lot of bad code.
1. Reading a lot of good code.
Fresh grad, 6 months into their first job and working as a "tech lead". Are you leading anyone else but yourself?
I know I'll get down voted but if you were really looking to grow you'd
a) quit and get that junior job more suited to your skill level
b) hire someone to replace yourself and demote yourself to junior role in your own team. (This would not even necessarily affect your comp.)
But seriously though if you're just looking for maximal comp, stay where you are. I'm sure you can become senior director in the next 6 mo.
1) actively seek honest feedback from the director of the company, especially in the non-technical areas. It certainly feels good to receive compliments, but it takes courage to ask for weak spots and areas of improvement. The latter will actually help you improve.
2) identify your technical gaps and actively seek mentoring from qualified individuals. You will be amazed to learn that many high performers are more than willing to voluntary mentor ambitious young professionals. Note that usually its very important in those cases to demonstrate the willingness to learn and to go the extra mile in improving yourself.
3) think about what career progression would look like within your current employer. Given your current situation, its important to think out-of-the-box, as a defined career progression does not seem to apply there.
[edit: line breaks added for readability]
Here's my guidance from 30 years in the industry. Stop doing this.
You probably do want to learn some of the basics and force yourself to adhere to them until you are experienced enough to know when to break the rules. Definitely use version control, look up typical styling/formatting/etc conventions for the languages and tools you're using, probably write unit or at least integration tests, prefer to use libraries when possible for non-trivial problems that aren't core to what you're working on. If you want exposure to this I recommend joining a quality open source project's developer discord/slack/whatever just to see how they do things. You could also maybe convince your boss to pay for consultants or for you to attend to conferences or whatever if you want individual feedback (if I were your boss, I'd see this as a good compromise - you get some level of senior supervision for $2k/yr instead of $200k/yr).
My number one advice is to NOT rely on LLMs for things that they're not smart enough for, and to always make sure you actually understand what it's doing any time it creates code for you. This is the number one issue I see inexperienced engineers making these days. ChatGPT is great for converting simple to moderately complex requirements into simple to moderately complex code under a certain size, but it's NOT GOOD at giving reliable high level advice that is "non-obvious" and unlikely to be in its training data. So please do not rely on it too much to explain complicated things to you (unless that thing is very well documented on the internet, which you would have to verify by searching for it anyway) and don't ask to eg tell you if your unit test is good. It can certainly save you a lot of time and is often right, but it's also often wrong in ways that are very difficult to spot unless you know a lot about the subject, and eventually (if not already) you'll need to advance to the point where it can't help you with your problems.
Find other people working on similar/related topics, connect with them, offline or online. Do not fear being THE Tech Guy in the company.. just do not be over-confident / arrogant. Doubt quick-wins, check, prove things. 30 years ago such one-programmer-per-company was a normal thing.
My reading recommendations are below, but that's just what i have found good.. not exhaustive list. i found some of those only after 11 years doing stuff by myself, and saw i have been inventing hot-water in many "wrong"-but-somehow-working ways, so what.. Better than only applying miracle recipes you don't understand.
https://www.svilendobrev.com/rabota/
https://www.svilendobrev.com/1/
And, most important, have fun!
That being said, the one thing that you can't really replicate is getting honest feedback about what you're doing at work.
For example, you can learn all the design patterns, write the best code etc, but are you even working on the right problem? What are the assumptions that you've made that are incorrect? Is there a much simpler approach that'll get you there in one tenth of the time?
These are thing things that experience may provide which no course or single book is going to give you.
You basically need senior people to talk to.
The most straightforward way to do this is to get a job which has this structure set up for you already. Otherwise make friends, go to meetups, do some networking and find some experienced people you can get some feedback from.
Let go of FOMO, youth is the only reason you think it is possible to not miss out. You have and will miss most things because they were not hit your way. The only things you can actually miss are the opportunities that are yours.
But the only way to miss them is because your attention was somewhere else. You have an awesome opportunity for a mentor in the director. You caught a lucky bus, stay on it.
Right now people can look at you and see potential. In just a few years, they will look at you and see squandered potential. Sure technically you are an adult, but you have almost zero adult experience (your mentor has forty years of adult experience). Good luck.
That’s it.
Read books to understand fundamentals. Create projects to practice and expand on the fundamentals. Watch videos to learn how others solved problems. Document your learning. Repeat.
You don't need to FOMO that, really. It works like that in theory, in practice... In about 2 years of work in a midsize company, I learn about one or two little details from the tech leads in the company. Two third of them were spending more time asking why one would think there are worth asking them a question than replying the question itself. The level of rockstar insufferableness was off the charts.
I have a feeling that there is no-one to learn from eventually as everyone is in continuous learning and the industry is fast evolving. In general, there are circumstances where this is definitely not true, but living in it and obsserving it to my current age of considered senior I am still a complete beginner in most of the tech scene tells me this way. My encounters and direct experiences generate a strong bias yet keeping an eye out here and else draws me to this conclusion and tells me that this must be mostly true.
From an other point of view there are lots of seniors. Anyone with 3+ years experience can be considered senior in this industry, given the pace techniques and approaches change (calling improvement would be part lie to a good portion). Which again, tells the same story: not much to rely upon but yourself and keeping a sceptic curiosity about the practices, especially the famed trends. These come and go. Those famed 20 years ago (hehe, more like 10, sometimes 5) are reformed and evolved a lot, mostly past recognition, or most of the time just died. Past techniques considered baaaad practice now, new ones dragging the industry to burning discussion topics and hiring sprees are infants.
The human and social aspects are downplayed in the industry with overwhelming technical dominance.This is where seniority may be a good role model, in mentality and attitudes, but it has no focus. Or sometimes work against it by promoting procedures of replacability: expected work methods and procedures promote the non-dependence on key persons, for the investment heavy industry it is better if the workforce is easy to pick up and lay off. Replacebale parts basically. That is not really an encouragemment for investing in continuity but to show off in trendy areas.
Probably you are already doing your best for doing your best and not making stupid mistakes. Perhaps you could harden your soul like everyone else and not taking seriously after making a stpid mitake, because the future you and all will look back at it and say, this was stupid. Like for so many past things we say now.
> to the highest standard of the industry
I believe I leave this uncommeted despite my deep intrinsic urge, not to attract even more downvotes when I share my mind in this topic, about the 'highest standard of the industry', both as participant and user. ; )
Read books. Watch conference talks. Attend conferences. Find local meetups.
One benefit of this for you is that you have to take full responsibility for your own advancement. As you find things about your work that interest your or that are complicated problems, get excited about the opportunity to solve them because that’s where seniors really come from; deep experience solving and understanding problems.
Good luck!
I was similar to you out of school—ended up the lone techie in a small oilfield services company.
How do you grow as an engineer? Go get a job at a company that does that. It won't be hard. Based on the problem-solving experience you're getting now, you'll impress hiring managers much more than similar-aged / similar-comped candidates, because all they have done is focused on code, and your mindframe is probably somewhat permanently bent toward larger-picture thinking. This is an asset.
It might be the wrong question, though. The question is: are you sure you want to be an engineer? You have a seat at the table in a small business and a mentor who is a strong and experienced operator in the space. Don't under-value that setup for making a career. If you measure yourself against the yardstick of being a good technologist, then yeah, this is not the best setup to grow. But if tech is a skill, not an identity, then you have other goals: building a strong career, producing value in the market and capturing some for yourself as wealth. Lean into playing that game. Ponder how irrelevant it is that some 26-year-old vegan in Silicon Valley would sneer at your code quality. Make it your goal to have an ownership stake in your company or a similar one in the space—maybe one you start or acquire with leverage from your mentor—by the time you're 35. If you get this right, you will earn dramatically more money in your career than most engineers. You will face much better odds of business success than most startup founders. And frankly, the small business operator space is starved for high-end young talent. (It's not comparatively lucrative for the first 5-10 years, and people find the frequent nepotism annoying.) There are a ton of boomers with profitable businesses and no good succession plans. The infrastructure they built—or at least their customer bases—has to go somewhere.
My story: I left the small business I started in and went to a tech company, but I did it because my first firm was terribly unhealthy. (It folded soon after.) Never lost the vision for building and owning something, though.
Learn to write tests and apply a code linter. You will have joined the élite top 10% for doing so. Good luck!
In particular: data engineers' main task is to produce clean data. Few people care about the cleanliness of the code used to produce it. They care about whether or not the data makes sense, whether or not it's accurate, how well it's documented, and whether or not others can understand the ETL pipelines' dependencies and assumptions so that people can predict how changes to business processes and IT systems will affect their business intelligence and data science efforts.
Cleanliness of code is, at best, a secondary concern. Especially if you're mostly working solo. The code for managing data pipelines is often relatively small and tightly scoped, to the point where even a messy ETL implementation can still be easier to understand than very clean application source code. Or at least, that's been my experience as someone who's worked as both a software engineer and a data engineer.
The biggest thing is, again, just to make sure the documentation is good. In one data engineering role I found a bunch of things in the data engineering pipeline that I was pretty sure were junk that we could get rid of to simplify the whole thing. Doing so would have reduced the code for the pipeline in question by ~25%, with comparable reductions in run time and server load. But I couldn't be sure because the original author left no documentation to indicate why those bits were there, so we ultimately decided to leave it there for the sake of safety. The original author was a software engineer, not a data engineer, so the code in question was incredibly clean and readable. I had no trouble figuring out how it was doing what it was doing. But what I really needed to know was what it was doing and why.
I'd also encourage caution about adopting fancy tech stack. It's amazing what you can accomplish with just a SQL database (or Parquet files) and some Python scripts. Data engineering is a space with a lot lot lot of vendors and open source projects selling solutions to problems you might not even have. And every additional solution you bring on is another piece of technology to understand, which can quickly get overwhelming if you're on a small team. Especially be careful about committing to the open source versions of technologies like Apache Spark. I can tell you from experience that, if you can't name the specific person who will personally be responsible for supporting and operating the technology, that person will be you, you will end up spending a lot of time doing it, and it will probably not be a use of time that the person who does your performance reviews will consider valuable. Especially if "I had to spend a bunch of time diagnosing Spark executor failures" becomes the reason why you couldn't get other things done.