6 ms·
Removing the wrapping newline from the FASTA/FASTQ convention also dramatically improves parsing perf when you don't have to do as much lookahead to find record
by totalperspectiv 1y ago
Removing the wrapping newline from the FASTA/FASTQ convention also dramatically improves parsing perf when you don't have to do as much lookahead to find record ends.
- Gethsemane 1y agoUnfortunately, when you write a program that doesn't wrap output FASTAs, you have a bunch of people telling you off because SOME programs (cough bioperl cough) have hard limits on line length :)
- o11c 1y agoYou can use content-defined chunking to wrap at a predictable place so that compression still works.
- sharedptr 1y agoIs BioPerl still standard, did people move to BioPython? When I was shown BioPerl I was tempted to write a better, C++ version, but was overwhelmed by other university stuff and let it go.
- bede 1y agoThanks for reminding me to benchmark this!
- totalperspectiv 1y agoI've only tested this when writing my own parser where I could skip the record end checks, so idk if this improves perf on a existing parser. Excited to see what you find!