Skip to content

Conversation

@valerybokov
Copy link

Current algorithm is:
1 if charset is UTF8 then read 3 bytes.
2 if these 3 bytes have expected values then mark a hasUtf8BOM variable as true
3 close stream
4 open new stream of the same file
5 if the variable hasUtf8BOM is true then skip 3 bytes.
6 if couldn't skip 3 bytes then throw an exception
7 If bytes were skipped or there is no need to skip them, the rest of the file should be read.

These are questions not for you, but for the algorithm:
1 why we need read the file twice when it increases the likelihood that we won't succeed the second time?
2 Opening a stream twice is slower than opening it once.
3 According to the code, there's a possibility that we couldn't read the file a second time. Then why isn't there a check to see if the file is corrupted? That is, it's UTF-8 encoding, but what if one or two of these three bytes are different. Perhaps the format has such combinations that this can't be verified.

At this point, I propose making one stream instead of two. There are no other changes.

@THausherr
Copy link
Contributor

This one looked good but then I realized there's a problem, there are files that have UTF8 content but no BOM, and that's what the original code intended. A better solution would be to try with a BufferedInputStream, use mark / reset and then pass to the reader.

@valerybokov
Copy link
Author

This one looked good but then I realized there's a problem, there are files that have UTF8 content but no BOM, and that's what the original code intended. A better solution would be to try with a BufferedInputStream, use mark / reset and then pass to the reader.

I didn't quite understand why this code supports UTF-8 but not supports UTF-16 etc.

@THausherr
Copy link
Contributor

Thanks... lets wait if somebody needs it 😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants