MagicBytesValidator

Recognize file types from Streams or IFormFiles using MIME types or file extensions and validate them against the magic bytes according to the file types. The existing file types can be expanded in various ways.

How to install

• Install nuget package into your project:

Install-Package MagicBytesValidator -Version 2.2.0

dotnet add package MagicBytesValidator --version 2.2.0

• Reference in your csproj:

<PackageReference Include="MagicBytesValidator" Version="2.2.0" />

How to use

• Check if a stream content matches a file type:

  var validator = new MagicBytesValidator.Services.Validator();
  var pngFileType = validator.Mapping.FindByExtension("png");
  
  var isValidPng = await validator.IsValidAsync(memoryStream, pngFileType, CancellationToken.None);

• Check a file with its stream and file type:

var isValid = await validator.IsValidAsync(memoryStream, fileType, CancellationToken.None);

Validation strictness (FileByteType)

Some formats support multiple validation strategies (e.g. strict vs. lazy rules). For this purpose, the library exposes FileByteType:

• FileByteType.Strict (default)
• FileByteType.Lazy (optional relaxed/lazy rules for certain formats)

Validate an uploaded IFormFile with a specific validation type

The form file provider accepts an optional validationType parameter:

var fileType = await formFileTypeProvider.FindValidatedTypeAsync(
    formFile,
    null,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Lazy
);

Example using Lazy (lazy rules if the format supports it):

var fileType = await formFileTypeProvider.FindValidatedTypeAsync(
    formFile,
    null,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Lazy
);

Validate a stream with a specific validation type

The validator also accepts an optional validationType parameter:

var isValid = await validator.IsValidAsync(
    memoryStream,
    fileType,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Strict
);

Example using Lazy:

var isValid = await validator.IsValidAsync(
    memoryStream,
    fileType,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Lazy
);

Note: If a format does not define any Lazy-specific checks, Lazy behaves like “global checks only”. This keeps existing formats unchanged unless they opt into mode-specific rules.

Example: Lazy ("relaxed") PDF validation

Some PDFs contain additional trailing bytes after the %%EOF marker. While strict validation may require the file to end with %%EOF, lazy validation can accept the %%EOF marker anywhere within the last 1024 bytes of the file (behaviour tolerated by common PDF viewers).

Expand the file type mapping

• Get mapping:

// use the validator:
var mapping = validator.Mapping;

// use the formFileTypeProvider:
var mapping = formFileTypeProvider.Mapping;

// or create a new instance of the mapping:
var mapping = new MagicBytesValidator.Services.Mapping();

• Register a single FileByteFilter:

mapping.Register(
    new FileByteFilter(
        "traperto/trp", // MIME type
        new[] { "trp" } // file extensions
    ) {
        // magic byte sequences
        StartsWith(new byte?[]
        {
            0x78, 0x6c, 0x2f, 0x5f, 0x72, 0x65
        })
        .EndsWith(new byte?[]
        {
            0xFF, 0xFF
        })
    }
);

• FileByteFilters with specific offset checks:

mapping.Register(
    new FileByteFilter(
        "traperto/trp", // MIME type
        new[] { "trp" } // file extensions
    ) {
        // magic byte sequences
        Specific(new ByteCheck(512, new byte?[] { 0xFD }));
    }
);

ByteCheck allows for negative offset values to look for a specific offset counting from the end of file.

Optional: register mode-specific magic byte checks (Strict/Lazy)

When configuring a FileByteFilter, fluent methods accept an optional FileByteType parameter. If omitted, the check is global (applies to all validation types). If specified, the check applies only to that validation type.

Example:

StartsWith(new byte?[] { 0x25, 0x50, 0x44, 0x46, 0x2D }) // global
    .EndsWithAnyOf(new[]
    {
        new byte?[] { 0x25, 0x25, 0x45, 0x4F, 0x46 }
    }, MagicBytesValidator.Models.FileByteType.Strict)      // strict only
    .TailContains(1024, new byte?[] { 0x25, 0x25, 0x45, 0x4F, 0x46 },
        MagicBytesValidator.Models.FileByteType.Lazy);   // lazy only

var formFileTypeProvider = new MagicBytesValidator.Services.Http.FormFileTypeProvider();

try {
    var fileType = await formFileTypeProvider.FindValidatedTypeAsync(formFile, null, CancellationToken.None);

    if (fileType is not null) {
        // Further code
    } {
        // Can't determine type
    }
} catch(MagicBytesValidator.Exceptions.Http.MimeTypeMismatchException) {
    // Content and given MIME type / extension don't match.
}

// or create a new instance of the mapping: var mapping = new MagicBytesValidator.Services.Mapping();

  ```c#
  var streamFileTypeProvider = new MagicBytesValidator.Services.Streams.StreamFileTypeProvider();
  var fileType = await streamFileTypeProvider.TryFindUnambiguousAsync(fileStream, CancellationToken.None);

  if (fileType is not null) {
      // Further code
  } else {
      // Can't determine unambiguous type
  }

Add custom file types

• Register a custom type with filters:

  public class CustomType : MagicBytesValidator.Models.FileByteFilter
  {
      public CustomType() : base(
          ["traperto/trp"], // mime types
          ["trp"] // extensions
      )
      {
          // defined magic byte sequences
          StartsWith([
              0x78, 0x6c, 0x2f, 0x5f, 0x72, 0x65
          ])
          .EndsWith([
              0xFF, 0xFF
          ])
          .Specific(new ByteCheck(512, [0xFD])); // offset: 512 bytes, negative offset looks for a specific offset from the end of file
      }
  }
  
  var mapping = new MagicBytesValidator.Services.Mapping();
  mapping.Register(new CustomType());
  mapping.Register(new[] { new CustomType() }); // Add multiple types
  
  // Registering all `IFileType`s of the given assembly that are also not abstract and have an empty constructor.
  _mapping.Register(typeof(CustomType).Assembly);

CLI tool

There's a CLI tool (MagicBytesValidator.CLI) which can be used to determine MIME types for a local file by calling the following command:

dotnet run --project MagicBytesValidator.CLI -- [PATH]

This can be useful when debugging or validating newly added FileTypes.

List of file types

FileType	Extensions	MIME Types
AIF	aif, aiff, aifc	audio/x-aiff
BIN	bin, file, com, class, ini	application/octet-stream
BMP	bmp	image/bmp
CAB	cab	application/vnd.ms-cab-compressed, application/x-cab-compressed
DOC	doc, dot	application/msword
DOCX	docx	application/vnd.openxmlformats-officedocument.wordprocessingml.document
DXR	dxr, dcr, dir	application/x-director
EXE	exe, com, dll, drv, pif, qts, qtx , sys, acm, ax, cpl, fon, ocx, olb, scr, vbx, vxd, mui, iec, ime, rs, tsp, efi	application/x-dosexec, application/x-msdos-program
GIF	gif	image/gif
GZ	gz	application/gzip
HEIC	heic, heif	image/heic, image/heif
ICO	ico	image/x-icon
JPG	jpg, jpeg, jpe, jif, jfif, jfi	image/jpeg
MIDI	midi, mid	audio/x-midi
MP3	mp3	audio/mpeg
MP4	mp4	video/mp4
MPG	mpg, mpeg, mpe, m2p, vob	video/mpeg
ODP	odp	application/vnd.oasis.opendocument.presentation
ODS	ods	application/vnd.oasis.opendocument.spreadsheet
ODT	odt	application/vnd.oasis.opendocument.text
OGV	ogv, ogg, oga	video/ogg
PBM	pbm	image/x-portable-bitmap
PDF	pdf	application/pdf
PGM	pgm	image/x-portable-graymap
PNG	png	image/png
PPM	ppm	image/x-portable-pixmap
PPT	ppt, ppz, pps, pot	application/mspowerpoint, application/vnd.ms-powerpoint
PPTX	pptx	application/vnd.openxmlformats-officedocument.presentationml.presentation
RAR	rar	application/vnd.rar, application/x-rar-compressed
RPM	rpm	application/x-rpm, application/x-redhat-package-manager
RTF	rtf	application/rtf
SND	snd, au	audio/basic
SVG	svg, svgz	image/svg+xml
SWF	swf	application/x-shockwave-flash
3GP	3gp	video/3gpp
TIF	tif, tiff	image/tiff
TSV	ts, tsv, tsa, mpg, mpeg	video/mp2t
TXT	txt	text/plain
WEBM	mkv, mka, mks, mk3d, webm	video/webm
XLS	xls, xla	application/msexcel
XLSX	xlsx	application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
XML	xml	application/xml, text/xml
Z	z	application/x-compress
ZIP	zip	application/zip, application/x-zip-compressed

License

MIT License

▓▓  ▓▓▓▓▓▓▓▓▓
▓▓         ▓▓
▓▓▓▓▓▓▓▓▓  ▓▓
▓▓         ▓▓            traperto GmbH
▓▓  ▓▓▓▓▓▓▓▓▓
▓▓
▓▓▓▓▓▓▓▓▓   ▓▓