Yogesh Golwalkar

System Design is one of my favorite. Here is the list of system design building blocks that are important irrespective of whether you are designing a system or preparing for a system design interview ✔️ Distributed message queue: A producer puts a message Or work in a line, which the consumers pick up for further processing or work defined. This is an important concept to explain how you can scale your system, perform asynchronous processing, ✔️ DNS: Translates website names to IP address. DNS service works much like a phone directory. Services like Route53 help configure the domain and routing, and one needs to understand common types of resource records like A, CNAME, NS, MX, etc. ✔️ CDN: The content delivery network is a server between the origin and the client. It stores and delivers content to the end users, reducing latency and improving the system's overall performance. ✔️ PUB SUB: Publish-subscribe messaging works asynchronously, involving publishers, topics, subscribers, and messages. It helps to decouple the system, scalability, Durability, etc. ✔️ Load Balancer: Load balancer helps divide and divert traffic across the system. It helps drive the overall system Availability and Performance. ✔️ Distributed Caching: Caching servers store the data in memory and return it to the end users, improving the overall performance and end-user experience. A fleet of servers works together to keep and serve the data in a distributed caching system, enhancing availability. ✔️ Database: Understanding data storage, storage, types, and choosing a Database, RDMBS Vs Nosql, is vital when designing a system. It is essential to deeply understand for what requirements RDBMS is the right fit, and nosql is the right fit. ✔️ Observability: It's important to know what is happening in the system. How do we make the system easier to debug and fix? Observability helps the system to drive maintainability, resiliency, and reliability. ✔️ Task Scheduler: This is an essential component that helps one decide how the tasks must be prioritized, delegated and how much resources need to be allocated ✔️ Unstructured Data Storage: Storing photos, videos, audio, and other unstructured data is essential when designing a system since these cannot be stored in traditional databases. One needs to create a highly available, consistent, reliable solution. and efficient ✔️ Scaling Services: Scalability is an essential building block of any robust system. Understanding various scaling options and what options should be used is critical. From Serverless to ID generators, one must understand the multiple options and how to design them. ✔️ Distributed Search: Search is the core of many system designs. Understating how to build a search system and the core building blocks of a search system, including crawling and indexing, is essential. #engineering #systemdesign #interview #productdevelopment

405

37 Comments

Amazon-to-Google-to-Amazon #a2g2a thought of the day: Swimming lanes (aka Ownership).  An Amazon SDE is a jack-of-all-trades. You write code, of course. But also, you wear your QA hat and test your stuff. And you wear your SRE hat and carry a pager and respond to tickets. And you wear your TPM and PMT hats too, talk to customers, decide the shape of your product, and track your progress. There’s no such thing as “staying in your lane” when you’re an Amazon SDE. You do what you need to do to make your business successful. After all, one of our Leadership Principles, Ownership, is defined as: “Leaders are owners […] beyond just their own team. They never say ‘that’s not my job.’”   I’ve always felt comfortable operating in that environment, focusing a lot more on helping my business be successful and impactful than on what is or isn’t part of my role doc.   When I joined Google in 2020, I noticed there were very clear swimming lanes for each role. Google had a massive army of thousands of Engineering Productivity engineers dedicated to creating test tools and infrastructure, although in reality they ended up doing a lot of the testing that SDEs really should have done. Google also had an army of thousands of SREs who handled operations and carried pagers. One day, one of my SDEs said he didn’t know what the customer painpoints were for our product because he didn’t have a TPM. “Why don’t you talk to customers yourself???” I asked both puzzled and frustrated. Our customers were other Google engineers. It wasn’t rocket-science to find them and ask them. But it wasn’t an SDE task.   Amazon does have all these other roles. They just don’t have the massive armies of them that Google has. They are more leveraged (eg 100:1 dev:test ratio).   If you just want to write code and not be bothered by “all the other crap” associated with software engineering, Google is paradise. Other people are hired to deal with all that crap you don’t want to deal with it. I would be willing to bet googlers spent a lot more of their day writing code.   There’s also something to be said about the benefits of specializing in a role. EngProdders were masters of test infra, SREs were masters of operations, etc. When you have a narrow job, you go deep; when you have a broad job, you don’t. For example, as an Amazon engineer, I’ve worn TPM and PMT hats a thousand times, but I’m willing to bet I’m a pretty mediocre one.   And so here lies the tension. Specialization is a Good Thing because it brings deep expertise and craftsmanship, but it creates swimming lanes and things fall between the cracks. There’s an overall lack of end-to-end Ownership. I saw that, over, and over, and over again, in my four years at Google. So much "not my problem."   That did not jive with me. I WANT to do "all the other crap." I enjoy it. It empowers me and my team to fully own our destiny and what we deliver to customers. My product sucks? It’s mine to fix. I don’t get to pass the buck. Simple as that.

622

45 Comments

Amazon-to-Google-to-Amazon #a2g2a thought of the day: The Intensity, oh, the Intensity!!! When I went from Amazon to Google, it felt like I had just gotten off the freeway and was driving on a side street. At the time, I couldn’t quite disentangle whether the slowdown was because this was June of 2020 and covid had turned the entire world upside down or because it was a deeper part of the DNA of the company. Now that I’m back at Amazon, I feel like I was indeed driving on a side street for a while and now I’m accelerating onto the freeway onramp once again. There’s most definitely a difference in intensity between the cultures. Intensity by itself is neither good nor bad, but it depends on the context. As covid shut down the world, I appreciated Google’s emphasis on their employees well-being. In 2020, googlers received a generous 14 weeks of additional time off, fully paid, to be used any way we needed given the extraordinary world circumstances. I had two boys that we were now suddenly homeschooling, so I very much appreciated the benefit. The other side of that is that at times I wanted to step on the gas and it was quite hard. You can’t single-handedly do that, you need people around you to be as excited as you are and to follow you on the journey. Working backwards from firm dates is a bit of a foreign concept to Google and it’s table stakes at Amazon. "Intensity" is NOT necessarily the same thing as "working long hours" (although it is often confused). I have a family and my time with them is very important to me. It's about what you do with the hours that you do spend at work. The last few weeks, the intensity has been healthy. I’ve felt more alive. It’s energizing. I see the freeway up ahead and the adrenaline is beginning to kick in again!

306

28 Comments

Interesting thoughts & prospective about how software engineering job may end up soon from Russell Kaplan (link in comment below) In a nutshell : - Less code / more assembling & product management (more english for prompting, less low level programming language) - Driving « Coding agents » (like giving every engineer an army of non-human interns) - Spending more time on the higher level parts of coding (which is specifying behaviours): understanding the requirements, architecting systems, and deciding what to build - Embracing the advent of One-off apps / Single use software I have already experienced the MDA period; hence this is not the first time we will announce the end of the dev job 😄. Nonetheless, AI & models are so disruptive that it should be taken seriously IMO. If suchparadigms rules our world, I think there is a business opportunity for AI companies to propose solutions for reverse engineering agents.

15

4 Comments

Amazon-to-Google-to-Amazon #a2g2a thought of the day: In 2024, onboarding at Amazon as a new employee feels like a well-oiled machine. My embarkation (pun here) has been smooth! First time I onboarded at Amazon was in 2009. We were a smallish company of 3k engineers starting hypergrowth, so the process was pretty informal. Somebody spent 30 minutes going over benefits, then she played a 15 minute video of Jeff Bezos talking about Amazon, we were handed our laptops and badges, and an hour later I was writing my first piece of code (fun fact: that piece of code ended up in TPSGenerator many years later!). That was kinda it. You're now an Amazonian, now go do some work! I onboarded at Google in 2020 just as the pandemic shut down the world. I was hoping it’d be like The Internship (that dumb movie with Vince Vaughn and Owen Wilson) 🙂. Instead, well-intentioned Googlers were trying really hard to make us feel welcome via hours of GVC, but we were all trying to figure out how this "remote" thing even worked. I got my noogler hat 2 or 3 months after my first day. All in all it felt pretty underwhelming. I imagine it's probably better now. Where today's Amazon new employee orientation shines is in crisply explaining our culture and DNA to new amazonians. I am a returning amazon-veteran, so I knew all these things by heart, but I still watched with delight every single video, out of curiosity. The videos on Leadership Principles are pure gold. When I joined Google, I tried very hard to grasp the DNA of the company. I asked five people: What does it mean to be ‘googley’? and I got five different answers. A few scattered axioms like "Don’t be Evil" and "Respect the { User | Opportunity | each other }" help, but fall short of articulating culture. You can argue whether having a strongly defined culture is a good thing or a bad thing. Personally I think it’s a good thing, but others might feel it as cult-ish / indoctrination / fill-in-the-blanks. It is absolutely true that, as far as culture is concerned, Amazon's is so strongly defined that some personality types thrive and some do not; whereas Google's is so loosely defined that it's a more inclusive place overall. What I learned at Google and I'd like to bring back a little bit is that awareness of how to bring other people to the table. Again, personal opinion only here, but to me, Culture Eats Strategy For Breakfast

238

11 Comments

Thought I would share a personal milestone today. I reached 25 blogs on medium https://lnkd.in/dJc3YsKE This is a short blurb on my journey and process. I had started blogging in 2020 and actively from 2021 partly inspired by people I met along the way and also as a way to express my thoughts better. I changed my focus and approach multiple times, from org design, promotion processes and tech frameworks to in depth content on specific areas like cryptography. Sometimes all it took for me to go past a writers block was to just pen down my thoughts even if it doesn't form a cohesive whole and refine it as I continued writing. I decided I would always put in links to the articles or blogs of the articles I had been inspired by. Most of them have better content than mine. I realised that if I had to explain something complex I had to build concepts around it brick by brick so when I started writing about OPAQUE PAKE, I made sure I had covered cryptography in ample depth even if it meant repeating commonly known concepts. I also wanted to focus on the why rather than the what so that my detailed blog doesn't end up like an abstract white paper with references. I wanted the blog to focus on my own interests instead of clickbait. So I kept it free of prophetic albeit popular articles like AI doomerism. So what's your excuse for not blogging?

61

4 Comments

Offtopic Since my feed is filled with layoffs news here is my personal take on what can be done to reduce cost and layoffs. 1. Give permanent work from home for software engineers. We don't need lavish offices. What we need is challenging work, handsome pay, peace of mind, good equipment, clear career progression path, supportive management, safe working environment, access to best learning resources. 2. If you really have to cut cost, then cut x% across the board for all employees. 3. Don't over hire, ever. 4. If a unit in your firm is not making profit and you don't see it to make profit in next few years then shut it down and move the people from that unit to other core units of the firm. 5. Don't fire people. Fire poor projects, poor products, #layoffs2024 #softwareengineering

2,584

114 Comments

🚘 Behind Uber’s recovery and growth in India Many had written off Uber India's operation but since then it has had surprising turnaround. Today, Uber India is earning higher revenue per user, primarily because of an influx of new riders drawn by budget commuting options like bike taxis and autos. As these riders gradually transition to utilising services such as car rentals and parcel deliveries, their spending on the platform increases. Recently, the company received permission to run buses in Delhi NCR. This takes it closer to becoming a full-fledged provider of mobility solutions. Read how Uber turnaround its India biz https://lnkd.in/gJDnQXJh

14

Amazon-to-Google-to-Amazon #a2g2a thought of the day: Reasons Not to Promote. When you endorse somebody for a promotion at Amazon, you MUST provide Reasons Not to Promote. This is in recognition that we all have gaps and places where we could use a little growth. Nobody is perfect. In fact, if an endorsement lacked Reasons Not to Promote, it was removed from the promotion package. Or if it had too many weasel words and platitudes, it too was often challenged (“X is a very nice person and a good developer” – did you spot the weasel words?).   Occasionally this was weaponized. It also led to some awkward situations, like when I was up for promotion to Senior Engineer and I got a frantic text from my manager saying “Director X is citing ABC as reasons not to promote you, do you have any data to show improvement? I need it in the next 3 minutes if you wanna get promoted!!!” But most of the times I think it worked as intended. The desire was to trigger the right objective, data-driven, and honest discussions during the promotion decision meeting.   A couple of months after I joined Google in 2020, I was asked to endorse an engineer I worked with. I wrote my endorsement in Amazon-style, ending with Reasons Not to Promote. About a day later I got a worried text from the manager, “Hummm… do you NOT want X to be promoted???”  I was confused, “I’m a huge fan of X!” I clarified. The manager explained in no uncertain terms that I was to remove the Reasons Not to Promote.   “Also, why did you say ‘Endorse’ and not ‘Strong Endorse’?”, the manager probed. At Amazon, a Strong Endorse means I’m willing to debate anybody regardless of seniority, tooth-and-nail, that this person deserves the promotion (and I did that multiple times). An Endorse means I fully endorse the promotion, simple as that. I quickly learned Google “Strong Endorse” mapped to Amazon “Endorse,” which also meant a Google “Endorse” mapped to an Amazon “Weak Endorse” or really to a “I-kind-of-don’t-really-endorse-this-person-but-I-dont-want-to-be-rude.”   Ultimately, the way that both Amazon and Google operate in this regard maps to the DNA of the company. Amazonians have a healthy disregard for social cohesion, and a very direct and factual communication style which I admit can sometimes borderline being a jerk, whereas googlers fear being jerks, to the detriment of making hard and correct business decisions sometimes.   Ultimately, the challenge is to be both direct and polite. I think both companies could learn from each other, but as for me, I’m a lot more comfortable in a culture of directness and open discussion.

1,624

69 Comments

Hiring Manager rounds are the most undervalued ones. Many people take them lightly and give interviews without preparing for them. What do they end up getting up as a feedback: "Your examples were not strong and complex enough for the role, so we cannot proceed with this role but if you want we can proceed with <some lower> role". Hiring managers have their own expectations and goals they are evaluating you on. If you don't want to end up with some feedback like above, then give enough time for its preparation. Preparing for this round requires you to think about stories from your past experience and map them to the situations you navigated in those stories. You need to think about the stories that are complex enough for the role you are applying. If you need help preparing for such rounds or need a mock interview for this, connect with me via enginEBogie here: https://lnkd.in/dTqdUjwd Enginebogie has many other top-quality experts too, you can connect with them here: https://lnkd.in/g2hYi4UY

16

1 Comment

One of the most common mistakes engineers make while writing multi-threaded programs is assuming common operations are thread-safe ⚡ I went through the assembly code to understand why exactly "count++" is not thread-safe and compiled my learnings into a super interesting and highly practical video that covers - how threads compete for the shared resource - how to disassemble code to instructions - how code actually gets executed on the CPU give it a watch: youtu.be/kBHd7kn_1EU ⚡ Admissions for my System Design August 2024 cohort are open, if you are SDE-2, SDE-3, and above and looking to build a rock-solid intuition to design any and every system, check out arpitbhayani.me/course #AsliEngineering #ConcurrencyInDepth

239

5 Comments

Tomorrow (8th May, 9 pm to 11 pm IST), we are going to solve the Ride Sharing Problem live as part of my LLD cohort. People who have already registered for my recorded cohort will get it for FREE (Thanks for showing your trust and purchasing my course). For others, if you also want to join, then you can register from the I have pasted at the end of this. You will also get access to the recordings of the past classes plus all the future monthly sessions in which we will keep solving more and more problems. Please complete most of the previous classes before joining this live since the concepts discussed in previous ones shall be used in this one. It will become difficult for you to follow if you directly join this new class. This is also an opportunity for those who have not joined any live class, to ask their doubts directly to me. In case you miss it, a recording will always be available. I will see you in the class! More live classes will follow after this. Learn LLD in-depth and from scratch: https://lnkd.in/ga5nvShy Learn HLD in-depth and from scratch: https://lnkd.in/g-VVEmSU Learn Multi-threading in-depth and from scratch: https://lnkd.in/gDdwUY6w Free top-quality system design videos covering many topics: https://lnkd.in/dyVEPVqf Connect 1:1 with me on @engineBogie: https://lnkd.in/dTqdUjwd

13

Lumikai turns 4 this month! In 2020, Lumikai Fund was essentially two of us. Fast forward to 2024: Four years in, we have seen one pandemic, two global wars, one bull run, one private market bear run, average 20%+ stock market returns, and surprise bans/prohibitions. Lumikai has grown up in times of uncertainty. The fund platform has expanded from two to eleven team members with over 2,000 deals evaluated. In a new market, we were calibrated and built a tribe of 16+ portfolio companies. Now, we have been recognised as a Top 10 VC fund in the interactive media and gaming space - the only Indian fund to make it to the list. Most importantly, we have seen the Lumikai tribe of founders achieve new heights, many times despite massive headwinds. We cheered for Eloelo when it hit India’s #1 social entertainment app, All Star Games (Deftouch) Games became India’s #2 cricket sports game, and Bombay Play won Google Play’s “Best Game in 2023” across APAC region, when Studio Sirah launched India-first mid-core IP in Kurukshetra: Ascension, AutoVRse won the business of a major US tech giant from competition, and Vobble brought Peppa Pig in audio to Indian kids. Like us, many in our portfolio have had to traverse uncharted territory without any playbooks or defined paths. Bringing light to an emergent asset class from an untapped region, which now has adequate global and domestic attention, has been a work in progress. Over the last four years, we have learnt: Lesson #1: Pioneering is not just about being “the one” but for everyone. It’s not just about winning, but about sharing knowledge, expertise, and giving back to an ecosystem. Lesson #2: Sometimes, when the odds are against you, all you can do is persist and control the inputs. Lesson #3: Being non-contrarian and right is a long, hard, and lonely journey of self-reflection and introspection. Lesson #4: You overestimate what you can achieve in a day and underestimate what you can achieve in a year. Lesson #5: VC success favors the hungry, the curious, and the lucky. And many times, in the business of venture, you have to create your own luck. So, as we crystal ball gaze, here’s to more years facing turmoil and uncertainty with resilience, impact, and excellence! 🎉 May the Lumikai light continue to shine bright. 🙏 Justin Shriram Keeling I Kislay Shashwat I Aditya Deshpande I Anant Seth I Asher Siddiqui I Tiasha Guha Neogy I Nidhi Bavishi I Anahita Singh I Raghav Ganesh I Namisha Rajgarhia #LumikaiTurns4 #VentureCapital #Growth #Resilience #Innovation

410

49 Comments

In my early years at Microsoft, the annual performance review was an enigma wrapped up in stress. As I ascended to managerial roles, unveiling the mystery behind 'stack ranking' and 'forced curve', I realized something crucial - transparency isn't just okay; it's essential. Once I started sharing the process with my team, the taboo around performance reviews vanished. It reinforced a key lesson for me: trust is forged in the crucible of transparency. Whether it's about compensation, challenges, or personal flaws, being open paves the way for growth and authenticity. So here's your call to action: Embrace transparency. It might feel risky, even scary, but the reward is an empowered, trusting team. Remember, trust opens doors! If you got value from this, please follow me! I write daily about engineering management, team dynamics, and processes needed to build engaged and successful teams. #Transparency #Trust #Management #Growth ----------------------------------- Check out the full article at https://lnkd.in/gRZi8aDt

14

2 Comments

Pub/Sub and queue look the same when you hear it for the first time.😱 Here is a simple explanation of what they are and the differences between them. In the illustration depicted below, 🔴 Messaging Queue: People come to the mall to shop. Before they check out, they must wait in the queue to complete the payment. Here, the mall is like a producer, producing people ( who have shopped and are ready to complete payment) who are sent to the queue to complete the payment. One by one, each person completes payment at the checkout counter. The checkout counter is like a consumer. As the queue grows ( more people join the line), more than one checkout ( one consumer ) is needed, and we will need to add more checkout counters ( more consumers). We have Producers, Queues, and Consumers. 🔴 Pub / Sub - Extending on the above example, after the checkout is completed, this customer data might be required by various services. For instance, in the image below, we have a real-time service that calculates how many people have completed the checkout. Hence, the checkout service has subscribed to receive this data. Likewise, marketing and inventory services might subscribe to receive this data. We have Producers, Topics, and Subscribers. 𝗣𝘂𝗯/𝘀𝘂𝗯:⏹️ The subscribers don’t have to reveal anything to the publishers. Messages are in the topic even after multiple subscribers consume it. Messages are sent to multiple subscribers simultaneously and asynchronously Message ordering and sending to various consumers is not guaranteed Producers and consumers have loose coupling 𝗤𝘂𝗲𝘂𝗲: ⏹️ Queues need to reveal who they are sending messages to Producers would put the messages in the queue, storing them until one of the consumers consumes them Producers put the messages in the queue, typically sent using some ordering and sequentially to a single receiver Message ordering and sending to consumers is guaranteed Producers and consumers have some sort of relationship #engineering #systemdesign #interview #productdevelopment #eventdrivenarchitecture

66

16 Comments

The Power of Knowing What You Don't Know Was talking to a person who constantly failing in system design round. After every failure, he is gaining more knowledge around design pattern, distributed system, cloud etc, still no luck. After a deep discussion I realized he is unable to understand the gaps he has. He is focusing on what he knows, this pattern, this framework, SQL/NOSql etc but he least bothered to identify what he “Does Not Know”. His confidence on his technical skill was making him blind fold to identify his own gaps. The book titled; “Think Again by Adam G” talks this phenomenon in detail. When someone think he/she knows everything, this the biggest mistake they make. Knowing what you don’t know allows you to keep an open mind. It puts you in a better position to adapt when you find practical knowledge. In above use case, my colleague failed to focus on scalability, cost, possible failure points etc. he could not “read between the lines” when interviewer was giving hints. He could only connect to this blind spot when I identified them. This concept works everywhere, retrospection in agile methodology serves this purpose only where you identify the unknow in a particular sprint. It could be a developer strength/weakness, infra issues, cross vertical integration etc. While developing a feature/project, we always think about happy path (known info) but if we focus more to identify the unknowns then we will have more reliable systems. Unknows like, what are the scenarios where my code will fail, gaps in upstream and downstream system etc. Top sports person excels in this ability, they never focused on known strength of opponent players, rather they work on unknowns like, shots the player is not comfortable with, mistakes he does while playing a specific shot, going way ahead from the crease while playing the shots etc. The ability to recognize what you don’t know is a skill that can be developed. It starts with knowing the right questions to ask. As Socrates said, "If knowledge is power, knowing what you don’t know is wisdom. Use your ignorance to your advantage." A gap in knowledge shouldn’t be seen negatively but as an opportunity to seek wisdom and ensure a better future. Thanks for reading. 

12