(12:12:21 PM) Mike Beckerle: I think I understand this. Let me explain why the 'backtrack" is ok.
(12:12:38 PM) Mike Beckerle: We have this DFA, but in our pictures there's this PTERM state.
(12:12:56 PM) Mike Beckerle: That PTERM state is a "macro" for a much more complicated DFA.
(12:13:54 PM) Mike Beckerle: The upshot is this "macro" accepts - having consumed a bunch of input, or it fails, and we are supposed to be back at the start of the delimiter.
(12:15:06 PM) Mike Beckerle: That "macro" can be expressed by exploding it into a big DFA, or by some claver logic that orchestrates part-specific matchers.
(12:16:46 PM) Mike Beckerle: The fastest thing would be to explode it into a big DFA, but pragmatically this might not be noticably faster than the other way.
(12:17:38 PM) Taylor: Why I was wondering if we should have two DFA types. Field and delimiter. Where the 'Field' can be 'paused' while we check the delimiters.
(12:17:47 PM) Taylor: If the delimiter match succeeds, we're done.
(12:18:06 PM) Taylor: if it fails, resume the 'field' DFA with the r.data0 as a char.
(12:18:37 PM) Taylor: all the field is doing is handling escape schemes if any
(12:18:39 PM) Mike Beckerle: That is, in principle, what the transition to the "macro" PTERM state is. It's suspending current state. So yeah, I'm good with that.
(12:18:46 PM) Taylor: ok cool :D
(12:19:13 PM) Mike Beckerle: Cut paste this dialog into a comment in the code somewhere.
(12:12:21 PM) Mike Beckerle: I think I understand this. Let me explain why the 'backtrack" is ok. (12:12:38 PM) Mike Beckerle: We have this DFA, but in our pictures there's this PTERM state. (12:12:56 PM) Mike Beckerle: That PTERM state is a "macro" for a much more complicated DFA. (12:13:54 PM) Mike Beckerle: The upshot is this "macro" accepts - having consumed a bunch of input, or it fails, and we are supposed to be back at the start of the delimiter. (12:15:06 PM) Mike Beckerle: That "macro" can be expressed by exploding it into a big DFA, or by some claver logic that orchestrates part-specific matchers. (12:16:46 PM) Mike Beckerle: The fastest thing would be to explode it into a big DFA, but pragmatically this might not be noticably faster than the other way. (12:17:38 PM) Taylor: Why I was wondering if we should have two DFA types. Field and delimiter. Where the 'Field' can be 'paused' while we check the delimiters. (12:17:47 PM) Taylor: If the delimiter match succeeds, we're done. (12:18:06 PM) Taylor: if it fails, resume the 'field' DFA with the r.data0 as a char. (12:18:37 PM) Taylor: all the field is doing is handling escape schemes if any (12:18:39 PM) Mike Beckerle: That is, in principle, what the transition to the "macro" PTERM state is. It's suspending current state. So yeah, I'm good with that. (12:18:46 PM) Taylor: ok cool :D (12:19:13 PM) Mike Beckerle: Cut paste this dialog into a comment in the code somewhere.