Skip to content

Unify catalog driver scrapers into a shared module + data tables #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jm2 merged 1 commit into from
Jun 15, 2026
Merged

Unify catalog driver scrapers into a shared module + data tables #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions Get-BroadcomEthernetDrivers.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#Requires -Version 7.0
<#
.SYNOPSIS
Fetches the latest Broadcom NetXtreme-E Ethernet drivers from the Microsoft Update Catalog.

.DESCRIPTION
Thin shim over catalogscrape\CatalogScrape.psm1, driven by catalogscrape\broadcom.psd1.
Covers the modern NetXtreme-E 10/25GbE controller (BCM57416, VEN_14E4&DEV_16D8), which the
other scrapers and build_win11pxe.ps1's in-box Broadcom services do NOT cover. Ported from
the gonefishin branch. Selection is highest-version-first.

.NOTES
Verify BCM57416 / DEV_16D8 is actually in your hardware scope before relying on this.
#>
[CmdletBinding()]
param(
[switch]$Install,
[string]$DownloadPath = 'C:\Temp',
[ValidateSet('x64', 'arm64', 'all')][string]$Architecture = 'x64'
)

$csDir = Join-Path $PSScriptRoot 'catalogscrape'
Import-Module (Join-Path $csDir 'CatalogScrape.psm1') -Force
$config = Import-PowerShellDataFile (Join-Path $csDir 'broadcom.psd1')
Invoke-DriverScrape -Config $config -Install:$Install -DownloadPath $DownloadPath -Architecture $Architecture
314 changes: 16 additions & 298 deletions Get-IntelEthernetDrivers.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,306 +4,24 @@
Fetches the latest Intel Ethernet drivers from the Microsoft Update Catalog.

.DESCRIPTION
Scrapes the Microsoft Update Catalog for specific Intel Ethernet hardware families
(e.g., I225-V, I219-V, X540, X550, X710, E810) to download targeted CAB packages
rather than downloading the monolithic multi-gigabyte Intel ZIP, ensuring a clean,
offline-ready set of INFs for DISM injection without downloading hundreds of CABs.
Thin shim over catalogscrape\CatalogScrape.psm1, driven by catalogscrape\intel-eth.psd1.
Targets specific Intel Ethernet families (I225/I226, I219/I210, X540/X550/X710, E810, IAVF)
and downloads the targeted CAB packages for DISM/pnputil injection rather than the
monolithic Intel ZIP. Selection is highest-version-first (catalog date as tiebreak).

.EXAMPLE
.\Get-IntelEthernetDrivers.ps1
.EXAMPLE
.\Get-IntelEthernetDrivers.ps1 -Install
#>

[CmdletBinding()]
param (
param(
[switch]$Install,
[string]$DownloadPath = 'C:\Temp\Intel_Ethernet',

[ValidateSet('x64','arm64','all')]
[string]$Architecture = 'x64'
)

$AcceptedArchs = switch ($Architecture) {
'x64' { @('AMD64') }
'arm64' { @('ARM64') }
'all' { @('AMD64','ARM64') }
}

if ($Install -and -not ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) {
Write-Error "The -Install flag requires Administrator privileges. Please run PowerShell as Administrator."
return
}

# ============================================================
# Helper: retry-with-backoff wrapper around Invoke-WebRequest
# ============================================================
# Defined locally so this script stays independently runnable. The Update Catalog
# throttles aggressively; retry up to 3 times with increasing delay, then re-throw so
# callers can try/catch and skip. NOTE: not available inside ForEach-Object -Parallel
# runspaces (those carry their own inline retry loop).
function Invoke-CatalogRequest {
param (
[Parameter(Mandatory)][hashtable]$Params,
[int]$MaxAttempts = 3
)
for ($attempt = 1; $attempt -le $MaxAttempts; $attempt++) {
try {
return Invoke-WebRequest @Params
}
catch {
if ($attempt -ge $MaxAttempts) { throw }
Start-Sleep -Seconds ($attempt * 2)
}
}
}

$Targets = @(
@{
Name = "Intel_2.5G_Family"
Devices = @(
@{ Prefix = "I225-V"; HWID = "VEN_8086&DEV_15F3"; FamilyName = "I225" },
@{ Prefix = "I226-V"; HWID = "VEN_8086&DEV_125C"; FamilyName = "I226" }
)
},
@{
Name = "Intel_1G_Family"
Devices = @(
# Intel's e1d driver is unified: the INF inside any single package covers ALL
# I219-V/LM generations (DEV_15B8, 15FA, 0DC8, 0D4F, 15D8, 15B3, etc.).
# Use DEV_15B8 (gen-2, most widely listed on the catalog) as the representative.
@{ Prefix = "I219-V"; HWID = "VEN_8086&DEV_15B8"; FamilyName = "I219" },
@{ Prefix = "I210"; HWID = "VEN_8086&DEV_1533"; FamilyName = "I210" }
)
},
@{
Name = "Intel_10G_Family"
Devices = @(
@{ Prefix = "X540"; HWID = "VEN_8086&DEV_1528"; FamilyName = "X540" },
@{ Prefix = "X550"; HWID = "VEN_8086&DEV_1563"; FamilyName = "X550" },
@{ Prefix = "X710"; HWID = "VEN_8086&DEV_1572"; FamilyName = "X710" }
)
},
@{
Name = "Intel_100G_Family"
Devices = @(
@{ Prefix = "E810"; HWID = "VEN_8086&DEV_1592"; FamilyName = "E810" }
)
},
@{
Name = "Intel_AVF_Family"
Devices = @(
@{ Prefix = "IAVF"; HWID = "VEN_8086&DEV_1889"; FamilyName = "IAVF" }
)
}
[string]$DownloadPath = 'C:\Temp',
[ValidateSet('x64', 'arm64', 'all')][string]$Architecture = 'x64'
)

if (-not (Test-Path $DownloadPath)) { New-Item -ItemType Directory -Path $DownloadPath -Force | Out-Null }

# Acquisition manifest — greppable ACQUIRED:/SKIPPED: lines emitted at the end so a
# throttled/partial parallel run is auditable.
$Manifest = [System.Collections.Generic.List[string]]::new()

foreach ($Target in $Targets) {
Write-Host "`n=> Investigating Microsoft Update Catalog for $($Target.Name)..." -ForegroundColor Cyan

$AvailablePackages = @()

foreach ($Device in $Target.Devices) {
$Prefix = $Device.Prefix
$HWID = $Device.HWID
$FamilyName = $Device.FamilyName
$Query = "$HWID Windows 11"
Write-Host " -> Searching specific HWID for Prefix $Prefix ($Query)..."

# Reset per-iteration so a failed fetch can't silently reuse the PREVIOUS
# device's page (which would parse the wrong update IDs for this device).
$SearchPage = $null
try {
$SearchUrl = "https://www.catalog.update.microsoft.com/Search.aspx?q=$([uri]::EscapeDataString($Query))"
$SearchPage = Invoke-CatalogRequest -Params @{ Uri = $SearchUrl; UseBasicParsing = $true }
}
catch {
Write-Warning "Search request failed for $Prefix ($Query): $_"
$Manifest.Add("SKIPPED: $FamilyName (search request failed)")
continue
}

# LIMITATION: Search.aspx returns only the first 25 relevance-sorted rows; the
# "newest by date" pick below only sees those 25. Full pagination needs
# __EVENTTARGET POST-backs (invasive). Parse the "1 - N of M" total and warn if M > 25.
if ($SearchPage.Content -match '(\d+)\s*-\s*(\d+)\s+of\s+(\d+)') {
$totalResults = [int]$matches[3]
if ($totalResults -gt 25) {
Write-Warning "Catalog reports $totalResults results for '$Query' but only the first 25 are parsed (pagination not implemented)."
}
}

$UpdateIds = [regex]::Matches($SearchPage.Content, "goToDetails\(['""]([a-f0-9\-]+)['""]\)") |
ForEach-Object { $_.Groups[1].Value } |
Select-Object -Unique

if (-not $UpdateIds) {
Write-Host " [!] No candidates found." -ForegroundColor Yellow
$Manifest.Add("SKIPPED: $FamilyName (no catalog candidates)")
continue
}

Write-Host " -> Found $($UpdateIds.Count) packages. Fetching deep versions..." -NoNewline

# Parallel detail page fetches — ~10x faster than sequential
$DetailResults = $UpdateIds | ForEach-Object -Parallel {
$Id = $_
$DetailsUrl = "https://www.catalog.update.microsoft.com/ScopedViewInline.aspx?updateid=$Id"
try {
# Inline retry-with-backoff (the script-scope helper is not visible here).
$DetailsPage = $null
for ($attempt = 1; $attempt -le 3; $attempt++) {
try { $DetailsPage = Invoke-WebRequest -Uri $DetailsUrl -UseBasicParsing; break }
catch { if ($attempt -ge 3) { throw } else { Start-Sleep -Seconds ($attempt * 2) } }
}
$DateString = if ($DetailsPage.Content -match 'id="ScopedViewHandler_versionDate">([^<]+)') { $matches[1].Trim() }
$Version = if ($DetailsPage.Content -match 'id="ScopedViewHandler_version">([^<]+)') { $matches[1].Trim() }

if ($Version -and $DateString) {
$DateObj = [datetime]::Parse($DateString)
$Arch = if ($DetailsPage.Content -match "ARM64") { "ARM64" } elseif ($DetailsPage.Content -match "AMD64|x64|amd64") { "AMD64" } else { "x86" }
[PSCustomObject]@{
Version = $Version
DateObj = $DateObj
Id = $Id
Arch = $Arch
}
}
}
catch {
Write-Warning "Detail fetch/parse failed for update $Id : $($_.Exception.Message)"
}
} -ThrottleLimit 8

Write-Host " Done."

foreach ($result in $DetailResults) {
if ($result -and $result.Arch -in $AcceptedArchs) {
$AvailablePackages += [PSCustomObject]@{
Prefix = $Prefix
FamilyName = $FamilyName
Version = $result.Version
DateObj = $result.DateObj
Id = $result.Id
Arch = $result.Arch
}
}
}
}

if (-not $AvailablePackages) {
Write-Host " [!] No matching prefixes found within candidate packages." -ForegroundColor Yellow
$Manifest.Add("SKIPPED: $($Target.Name) (no packages matched accepted architectures)")
continue
}

$GroupedPackages = $AvailablePackages | Group-Object Prefix, Arch

foreach ($Group in $GroupedPackages) {
$FirstObj = $Group.Group[0]
$Prefix = $FirstObj.Prefix
$Arch = $FirstObj.Arch
$FamilyName = $FirstObj.FamilyName

$BestPackage = $Group.Group | Sort-Object DateObj -Descending | Select-Object -First 1
Write-Host " -> Prefix $($Prefix) [$Arch]: Selected $($BestPackage.Version) (Update ID: $($BestPackage.Id))" -ForegroundColor Green

$DownloadPage = $null
$PostData = "[{`"size`":0,`"updateID`":`"$($BestPackage.Id)`",`"uidInfo`":`"$($BestPackage.Id)`"}]"
try {
$DownloadPage = Invoke-CatalogRequest -Params @{ Uri = "https://www.catalog.update.microsoft.com/DownloadDialog.aspx"; Method = 'Post'; Body = @{updateIDs = $PostData }; UseBasicParsing = $true }
}
catch {
Write-Warning "Download dialog request failed for $FamilyName [$Arch]: $_"
$Manifest.Add("SKIPPED: $FamilyName [$Arch] (download dialog failed)")
continue
}

$CabUrl = [regex]::Match($DownloadPage.Content, 'https://[^''\"<]+\.cab').Value

if (-not $CabUrl) {
Write-Host " [!] Could not extract .cab URL from payload." -ForegroundColor Red
$Manifest.Add("SKIPPED: $FamilyName [$Arch] (no .cab URL in payload)")
continue
}

$CabFile = Join-Path $DownloadPath "$($Target.Name)_$($Prefix)_$($Arch).cab"
$ExtractDir = Join-Path $DownloadPath "$($Target.Name)\$FamilyName\$Arch"

Write-Host " -> Downloading raw $Arch driver package..."
try {
Invoke-CatalogRequest -Params @{ Uri = $CabUrl; OutFile = $CabFile; UseBasicParsing = $true } | Out-Null
}
catch {
Write-Warning "CAB download failed for $FamilyName [$Arch]: $_"
$Manifest.Add("SKIPPED: $FamilyName [$Arch] (download failed)")
continue
}

# AUTHENTICITY: this CAB is extracted into boot-start kernel drivers. Verify the
# publisher's Authenticode signature and SKIP (do not extract/inject) anything
# that is not 'Valid', rather than silently trusting it.
$sig = Get-AuthenticodeSignature -FilePath $CabFile
if ($sig.Status -ne 'Valid') {
Write-Warning "Authenticode signature for $FamilyName [$Arch] CAB is '$($sig.Status)' (not Valid) — skipping extraction/injection."
$Manifest.Add("SKIPPED: $FamilyName [$Arch] (signature $($sig.Status))")
Remove-Item $CabFile -Force -ErrorAction SilentlyContinue
continue
}

Write-Host " -> Extracting payload using expand.exe..."
if (-not (Test-Path $ExtractDir)) { New-Item -ItemType Directory -Path $ExtractDir -Force | Out-Null }

# Capture expand.exe's exit code. On failure KEEP the source CAB for retry/inspection.
$expandProc = Start-Process "expand.exe" -ArgumentList "-F:* `"$CabFile`" `"$ExtractDir`"" -NoNewWindow -PassThru
$expandProc.WaitForExit()
if ($expandProc.ExitCode -ne 0) {
Write-Warning "expand.exe exited with code $($expandProc.ExitCode) for $FamilyName [$Arch] — keeping source CAB '$CabFile' for retry/inspection."
$Manifest.Add("SKIPPED: $FamilyName [$Arch] (expand.exe exit $($expandProc.ExitCode))")
continue
}
Remove-Item $CabFile -Force

# Manifest: confirm an actual .inf + .sys landed (an empty/partial extract is a skip).
$infCount = @(Get-ChildItem -Path $ExtractDir -Recurse -Filter *.inf -ErrorAction SilentlyContinue).Count
$sysCount = @(Get-ChildItem -Path $ExtractDir -Recurse -Filter *.sys -ErrorAction SilentlyContinue).Count
if ($infCount -gt 0 -and $sysCount -gt 0) {
$Manifest.Add("ACQUIRED: $FamilyName [$Arch]")
}
else {
$Manifest.Add("SKIPPED: $FamilyName [$Arch] (no .inf/.sys after extract)")
}

if ($Install) {
$SysArch = $env:PROCESSOR_ARCHITECTURE
if ($SysArch -eq $Arch) {
Write-Host " -> System is $SysArch. Injecting $Arch driver into Driver Store via pnputil..." -ForegroundColor Green
pnputil.exe /add-driver "$ExtractDir\*.inf" /install | Out-Null
Write-Host " -> Injection complete." -ForegroundColor Green
}
else {
Write-Host " -> System is $SysArch. Skipping $Arch driver installation." -ForegroundColor DarkGray
}
}
else {
Write-Host " -> Extracted to: $ExtractDir (Skipping installation)" -ForegroundColor DarkGray
}
}
}

# ============================================================
# Acquisition manifest (greppable)
# ============================================================
Write-Host "`n=> Acquisition manifest:" -ForegroundColor Cyan
if ($Manifest.Count -eq 0) {
Write-Host " (no device families processed)" -ForegroundColor DarkGray
}
else {
foreach ($line in $Manifest) {
if ($line -like 'ACQUIRED:*') { Write-Host " $line" -ForegroundColor Green }
else { Write-Host " $line" -ForegroundColor Yellow }
}
}

Write-Host "`nProcess complete." -ForegroundColor Cyan
$csDir = Join-Path $PSScriptRoot 'catalogscrape'
Import-Module (Join-Path $csDir 'CatalogScrape.psm1') -Force
$config = Import-PowerShellDataFile (Join-Path $csDir 'intel-eth.psd1')
Invoke-DriverScrape -Config $config -Install:$Install -DownloadPath $DownloadPath -Architecture $Architecture
Loading
Loading