Commit 1b72824
fix: binary file corruption (#230)
* fix: image commits leads to invalid images
**Location:** `src/blob.ts`, line 28
The code was reading files with UTF-8 text encoding:
```typescript
fs.createReadStream(this.absolutePath, { encoding: 'utf8' })
```
Images are **binary files** containing arbitrary byte values (0-255). The UTF-8 encoding option tells Node.js to:
1. Interpret the binary data as UTF-8 text
2. Drop any byte sequences that are not valid UTF-8
This causes data loss because:
- Valid UTF-8 sequences are 1-4 bytes depending on the character
- Many byte combinations in binary files are invalid UTF-8 sequences
- These invalid bytes are silently dropped by the UTF-8 decoder
- The resulting file is corrupted (missing bytes)
* fix: prevent binary file corruption when encoding to base64
The base64-encoder was converting base64 strings back to buffers before
pushing them to the stream, which caused them to be re-encoded as UTF-8
bytes. This corrupted binary files (images, PDFs, etc.) when they were
committed and pushed to GitHub.
Changes:
- base64-encoder.ts: Push base64 strings directly instead of converting
them back to buffers. This prevents the double-encoding that was
corrupting binary data.
- blob.ts: Remove UTF-8 encoding from file stream reading. Files are now
read as raw buffers, which is necessary for the base64 encoder to work
correctly with both text and binary files.
The fix ensures that:
- Binary files (PNG, JPG, PDF, etc.) are encoded without corruption
- Text files continue to work correctly
- The action properly base64 encodes file content for GitHub API
Fixes image corruption when committing and pushing images to GitHub
* fix: add binary regression tests to Base64Encoder and Blob.load() using a fixture with non-UTF-8 byte sequences to guard against silent data corruption
* chore: update bundle dist
---------
Co-authored-by: Joris Conijn <joris@conijnonline.nl>1 parent 00de5a7 commit 1b72824
7 files changed
Lines changed: 31 additions & 8 deletions
File tree
- __tests__
- fixtures
- stream
- dist
- src
- stream
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
75 | 85 | | |
76 | 86 | | |
77 | 87 | | |
| |||
Binary file not shown.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
18 | 32 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23957 | 23957 | | |
23958 | 23958 | | |
23959 | 23959 | | |
23960 | | - | |
| 23960 | + | |
23961 | 23961 | | |
23962 | 23962 | | |
23963 | 23963 | | |
23964 | 23964 | | |
23965 | | - | |
| 23965 | + | |
23966 | 23966 | | |
23967 | 23967 | | |
23968 | 23968 | | |
| |||
23979 | 23979 | | |
23980 | 23980 | | |
23981 | 23981 | | |
23982 | | - | |
| 23982 | + | |
23983 | 23983 | | |
23984 | 23984 | | |
23985 | 23985 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
30 | | - | |
31 | | - | |
| 29 | + | |
32 | 30 | | |
33 | 31 | | |
34 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
0 commit comments