Iterative (or incremental) development is what's meant by doing only a little bit of product development at a time so that you can learn from the work you've done. The idea is, the more quickly you can get your work into the hands of users, the quicker you can learn what works and what doesn't work. Then you can take what you've learned and apply it to the next iterative cycle you do. Customer feedback is often the best indication of what's working, and the sooner you can get it the better.
The biggest advantage to doing product development iteratively is that you'll avoid getting into long product development cycles where work can drag on indefinitely without user feedback. Even if you've adequately done customer development to determine what you should build, you won't have a real sense of the impact of your work until it's in the hands of users. Focus groups, polling and talking to users can only get you so far. Getting the real product out there is the most telling evidence you can have of how your development is going.
The biggest disadvantage of iterative development is that you're so focused on small and frequent improvements that you fail to look at the big picture. The thrill and feedback of pushing out code and design at a quick pace often leads to taking fewer product risks. Often times in the early days of a startup — before product market fit — startups iterate too often on what they have already when what they have needs a thorough rework to get them to the next stage. They're so focused on the day-to-day that they lose sight of the end goal. What is it that you want your product to be one year from now?
The problem lies in the fact that the best results come from a mix of small iterative changes alongside a larger reworking of the product — but admittedly that's an ideal goal that's almost impossible to achieve. Most small teams cannot afford to spend the necessary time on both and are left with a choice: do they spend their resources on small changes or do they take the time to overhaul the product in a larger way? And how much time, percentage-wise, does the team devote to each?
From my personal experience, the ideal product cycle evolves as follows: After the product has first been circulated to users, a solid one to three months of major rework followed by three months of iterative development. Then repeat. This requires you to have a lot of discipline and not to get caught up in the micro management of your web application. You need to know when it's time to "level up" — to use a gaming term — your product.
Leveling up your product will also give your current user base a jolt of encouragement in knowing that you're continuing to grow and refine your product. Often iterative changes (code rewrites, polish, small features, etc.) go unnoticed by users, as they don't have huge impact. Big changes can sometimes lead to user revolt, but often-times, as the history of the Web has shown, if the changes help to bring in a larger user base, then all will be well in the end.
However you currently do your product development — fast and iterative or slow and refined — I encourage you to look at it from both perspectives. As with many things in life, a balanced approach is often the best way forward.