Experience matters a lot in software development. For example, after many years as a software developer this is one of the valuable lessons I've learned: Outs

Experience matters a lot in software development. For example, after many years as a software developer this is one of the valuable lessons I've learned: Outside data is ALWAYS dirty. There is no clean data. And part of the value of experience is knowing how to handle this. For example, assume I get a very large data set from a third party that is crucial to my project. Unfortunately, all the fields are strings. Years of experience have taught me the following ways to deal with this: 1) Implement a lot of logic and validation into the program itself or as a pre-process step. This is a lot of work, requires a lot of domain knowledge, you may still miss a few edge cases, it will force the deadline to be pushed back, and increase the budget. However, those are the disadvantages. The advantages are that this could become a competitive advantage. Maybe no one else knows or has tried to clean up this data before. Identifying and doing this work can lead to a whole new business. A classic example of this is GIS data. GIS data is very very dirty. Street, St, St., Highway, Hwy, Freeway, Fwy, all typical examples of how dirty the data is. Companies like Navteq are based on their ability to clean up GIS data. Apple's Map app was a disaster because Google is so good at and has years of experience cleaning up GIS data. 2) Push back on the third party to clean up their data. Doing this takes some finesse and social/business skills. You can't just say, the data is hard to handle. That sounds like whining and laziness. Your manager can't do anything to help you if that's all you have. Instead one tactic is to make a schema (this is what WSDL and XML Schemas were invented for) and tell the third party that their data must conform. Then it becomes their problem. They might push back, but now your manager has something to fight with. But if they agree to conform and the schema is bad, then it will become your fault (the third party will just say, hey it conforms to your schema so that's your problem). So you better make sure the schema is good (experience helps here). And it may require meetings with the third party's technical team to compromise or figure out what a good schema is (which requires social/business skills and experience). 3) Identifying these problems early. If these data issues are not considered or found early, they will lead to big issues later. This is why senior developers get their work done fast and go home early. They don't make mistakes like using floats for currency or assuming a string is always a certain format. Junior developers make those kinds of mistakes and spend 80 hour weeks fixing it in the code, rewriting unit tests, and running scripts against the database to carefully migrate the data. They may seem more "productive" but they created the work (like the broken windows fallacy).