But It's A Defect!

Yes, the story is a defect story, but that most definitely does not mean that it automatically has the highest priority of any story in the backlog.

Even defect stories have business value associated with them.  It may not even be worth the cost in man hours for the development and QA of the fix for the defect in terms of how long it will take before the business sees a return on investment for that fix.

Yes,  defect stories are defects because they mean that a previously played story is not behaving exactly as it was supposed to.  But, there is also a portion of the original story that is behaving exactly as it was supposed to.  That means that it is entirely possible (and somewhat probable) that the defect will affect such a small percentage of the interactions with the system that the code fix for the defect will be much more expensive for the business than any manual (or otherwise) work-around that the business can implement to deal with the defect.

I am not saying that all defect stories are low priority.  There are the occasional defect stories that are exceptionally high priority.  For example, if a defect brings down an entire warehouse.  That has a very real cost associated with it the longer that warehouse is still down, so the faster that defect gets fixed, the better.

But, those severe defects are few and far between.  If they're not, that's a warning that something isn't right with the way you are doing things.  Perhaps you're missing a level of testing (unit, system, integration, end-to-end, etc.) or an area of your system isn't as well tested as it should be.  Or, perhaps it is an indication that a portion of the system needs to be rewritten to be able to better handle certain types of situations that were thought to be rare edge cases but turned out to be much more common.

So, yes.  I am saying that just because you found a defect, it doesn't mean that you have to fix it.  Some defects just aren't worth fixing because you'll either never get a ROI on it, or some other work-around for the defect will provide a faster ROI for the business.

You'll know the critical defects that need to be addressed immediately when you see them -- and so will your Product Owner.  Just like any other story, let your Product Owner prioritize your defect stories.  If they are leaning toward "all defects are critical", help nudge them back toward looking at what the real business value is in fixing the defects.

"Defect" is just a label that we're applying to help categorize stories.  Don't let the choice of word used to describe the category influence your impression of the importance of the story.  The categorization of a story is distinct from its prioritization.