Which rows get thrown out of a market dataset before anyone studies what is left? This note lists the ordinary discards, using bad prints in JPMorgan on the public tape as the example, and explains why the discard list is part of the data.
A filter is a claim
Every study uses a subset. Some rows are dropped because they are duplicates, because a timestamp is impossible, because a price is zero, or because a trade condition says the print is not a regular way trade. Each drop is a claim that the row would mislead the question being asked. If the claim is not written down, a later reader cannot tell whether a quiet day was quiet or whether the filter removed it. The remaining file looks cleaner than the source. Cleanliness without a list is just absence.
The order of filters matters. A filter that removes canceled trades and a filter that removes prices far from the quote will remove different rows depending on which one runs first. A pipeline that applies them in an undocumented order cannot be rebuilt from the prose. The discard rule should be a short list a person can read: the condition, the reason, and whether the row is deleted or merely flagged. Flagging keeps the evidence. Deleting saves space and loses the argument.
A discard rule is the written list of rows removed or flagged before a study, including the condition, the reason, and the version of the tape the rule was applied to.
A bad print in a liquid name
JPMorgan is a liquid New York Stock Exchange name, which makes a bad print easier to notice and easier to mishandle. A regular day contains a tight quote and a stream of small and large prints. Occasionally a print appears at a price far from that quote, with a condition code that marks it as late, as an average, or as an error that the exchange later corrects. A study of ordinary trading that leaves the print in will show a jump that no continuous quote supports. A study that deletes every large move will also delete the real jumps.
The public example is the distinction, not a story about that bank. Suppose the quote is stable and one print lands several dollars away, then the next prints return to the quote. A rule that drops prints beyond a stated distance from the prevailing quote will drop that row. The rule needs the distance stated in advance, and it needs an exception for openings, reopenings, and halts, when the quote itself is not a fair reference. Applying the midday rule to the opening auction will throw out the auction. The auction was not a bad print. It was a different mechanism.
Corrections arrive later. An exchange can bust a trade. A point-in-time file shows the print on the day it appeared and the bust on the day the bust was published. A file that simply erases the print, as if it had never arrived, teaches the backtest a tape the desk did not have. Throwing the row out of a research extract can still be right for a question about regular trades. The extract must say that busted and corrected prints were removed, and as of which version of the tape. Otherwise the absence looks like a day on which nothing odd happened.
Keep the discarded rows addressable
A mature extract stores a count of what was removed, by reason, next to the rows that remain. If the bad-print rule removes a large share of volume, the rule is no longer a cleanup. It is the study. The JPMorgan check is to compute volume before and after the filter on a dull day and on a day with a known halt or auction. The dull day should change very little. The special day should change in a way the note predicts. If the dull day loses a lot of volume, the distance threshold is too tight.
Nothing in the filter says the remaining series should be traded, or that the discarded prints were worthless to every question. A study of error prints would keep exactly the rows this study drops. The point is to stop those two studies from sharing one unlabeled file. What gets thrown out is a decision. The decision stays attached to the dataset, in words, or the dataset is not yet a dataset a second person can use.
This piece does not claim that any particular print was erroneous, that a distance rule is the right rule, or that a filtered tape is the market. It says the rows that were thrown out have to remain describable.
