Add split with limit operator (to protect against long input without delimiters)#34
Merged
davidmoten merged 3 commits intomasterfrom Apr 7, 2020
Merged
Add split with limit operator (to protect against long input without delimiters)#34davidmoten merged 3 commits intomasterfrom
davidmoten merged 3 commits intomasterfrom
Conversation
…h to avoid OutOfMemoryException when items very long
davidmoten
force-pushed
the
protected-split
branch
from
139f39f to
c60079a
Compare
Codecov Report
@@ Coverage Diff @@
## master #34 +/- ##
============================================
- Coverage 68.74% 68.68% -0.07%
- Complexity 417 427 +10
============================================
Files 69 69
Lines 3558 3602 +44
Branches 390 402 +12
============================================
+ Hits 2446 2474 +28
- Misses 988 996 +8
- Partials 124 132 +8 Continue to review full report at Codecov.
|
stephenohair
approved these changes
Apr 7, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
Transformers.split(maxItemLength, pattern, maxPatternLength)operator.This is a useful operator in the circumstance that the input is not neccessarily controlled and there exists the possibility of a really long stream of strings arriving without a delimiter. This may cause the normal split operator to throw an OutOfMemoryException as it accumulates a really long string waiting for the delimiter.
Unit test coverage of new operator is complete but for some
isUnsubscribed()checks. These are painful to write tests for because buffering is occurring in the state-machine operator so I've left them out.See javadoc of new operator for more detail.
In theory there is extra allocation overhead using this operator but benchmarks show no significant difference.