* Windows 11 appears to be a lot more proactive in locking system partitions (ESPs, MSRs)
than previous versions of Windows were, resulting in format or access errors.
* Try to work around these by disabling exclusive drive locking as needed.
* Also use build number to detect Windows Server 2019 and Windows 11
since Microsoft are COMPLETE ASSES about their version reporting.
* Also fix a compilation warning.
* In replace_in_token_data() when looking for lines starting with a specific
token but finding lines containing a larger version of the token (e.g. looking
for 'linux' but finding 'linux16') we would forget to output the non matching
line as we rejected it.
* This produced issues such as the one described at:
https://ubuntuforums.org/showthread.php?t=2465291&page=10&p=14052629#post14052629
* Fix this by ensuring that we always output the lines that we reject.
* write_sector() should really only be used when writing single sectors as it
is way to slow for anything else => Switch to using WriteFileWithRetry().
* Also revert an unwarranted change from f0047986e7.
* ...that didn't get the memo about using UPPERCASE 11-chars max ISO labels.
* There's a reason why Arch labels its ISOs 'ARCH_YYYYMM', people!
* Anyway, EndeavourOS should now work in ISO mode when booted from UEFI.
* In their great wisdom, the openSUSE maintainers added a 'set linux=linux'
line to their grub.cfg, which means that their kernel option token is no
longer 'linux' but '$linux'... and we have to add a workaround for that.
* If users set the persistent size to max, we may run into a situation
where projected size (which is always a rough estimation) is too low.
* When persistence is in use, we increase the projected size by 10%, to
ensure that the above scenario cannot happen.
* Also work around potential issues with Windows APIs when the application
is launched from the root of a drive.
* While this is intended to solve the issue of saving GRUB/Syslinux files for the
App Store version, we apply this change globally, as it allows the user to move
the Rufus executable around while preserving access to existing downloads.
* Closes#1744
* This basically means that the script is validate *TWICE*, using two
completely independent signatures, before it is allowed to run, which
should add another mitigation layer against TOCTOU (which we already
friggin' mitigated against anyway) and other potential vectors of
attack.
* Also remove -DisableFirstRunCustomize option and the associated cookie
prompt monitoring, which the latest version of Fido no longer requires.
* Also update WDK version for signtool and flesh out PKI error messages.
* Trying to mount accessible partitions after writing an image may lead to the
creation of the infamous 'System Volume Information' folder on ESPs, which in
turn leads to checksum errors for Ubuntu's boot/grub/efi.img (that maps to the
Ubuntu ESP). So comment out that code.
* Also fix a missing CRLFs in the log after displaying write progress.
* Anaconda broke ISO compatibility, most likely with the following commit:
84529204fe
* However, Ret Hat, and its followers, have drunk the "DD only" kool aid, and
appear to be blissfully unaware of the very real drawbacks that enforcing a
"DD only" mode for ISOHybrid can actually place on distro users.
* Rather than spend another wasted effort trying get people, who appear to be
impervious to even remotely consider the idea that DD imaging can have flaws,
to look into the possibility that Red Hat might indeed have introduced a
regression, and given the downright hostility I have been subjected to from
trying to state this *very verifiable* fact, we'll just force DD mode for the
affected Red Hat and derivatives, whilst trusting that users will be smart
enough to compare their more limited installation experience against the ones
from other distros (such as Arch, Debian or Ubuntu, which, unlike Red Hat and
co., appear to fully understand that the whole ISOHybrid vs DD mode situation
is not all black and white), and see for themselves which distros do actually
place *their* interests first, rather than just the interests of the distro
maintainers...
* GRUB 2.0 maintainer think they're doing a fine job, even when there are
CRITICAL SECURITY FIXES that should warrant an immediate out of bound
release, and instead consider that waiting MONTHS or YEARS to release
anything is not a big deal at all.
* Ergo, distros, such as Ubuntu, start to pick whatever security patches
they see fit, since they can simply not RELY on the upstream project to
produce security releases in a timely manner. One such patch is:
https://lists.gnu.org/archive/html/grub-devel/2021-03/msg00012.html
* But since there is no new GRUB release per se, they still call their GRUB
version, onto which they applied patches that have come into existence
more than 2 years after the actual 2.04 release, "GRUB 2.04".
* Obviously, since GRUB 2.04 + literally hundreds of cherry picked patches
does deviate a lot from the last release, THINGS BREAK IN SPECTACULAR
FASHION, such as the recently released Ubuntu 21.04 failing to boot with
the error: grub_register_command_lockdown not found.
* Oh, and of course, regardless of all the above, if you ask anyone, they'll
tell you that there's nothing fundamentally wrong with the GRUB release
process (even if they should long have released 2.05, 2.05-1 and 2.05-2,
were their maintainer ready to acknowledge that delaying releases DOES
CREATES MAJOR ISSUES DOWSTREAM, as many people REPEATEDLY pointed to them
on the GRUB mailing list) or with the Ubuntu GRUB versioning process (that
really shouldn't be calling their version of GRUB "grub-2.04" but instead
something like "grub-2.04_ubuntu"). Oh no siree! Instead, the problem must
all be with Rufus and its maintainer, who should either spend their lives
pre-emptively figuring which breaking patch every other distro applied out
there, or limit media creation to DD mode, like any "sensible" person
would do, since DD mode is the ultimate panacea (Narrator: "It wasn't").
* So, once again, a massive thanks to all the people who have been involved
in the current GRUB 2.0 shit show, whose DIRECT result is to make end
users' lives miserable, while GRUB maintainers are hell bent on continuing
to pretend that everything's just peachy and are busy patting themselves
on the back on account that "Fedora recently dropped more than 100 of the
custom patches they had to apply to their GRUB fork" (sic). Nothing to see
here, it's just GRUB maintainer's Jedi business as usual. Besides, who the
hell cares about Windows users trying to transition to Linux in a friendly
manner anyway. I mean, as long as something doesn't affect existing Linux
users, it isn't a REAL problem, right?...
* Combined with the increase in buffer size from previous commits, this
should help us get close to a device's maximum write speed.
* Also add async write support to winio.h
* Also increase the buffer size for bad blocks check operations
* This is in preparation for async reads
* Also move open/close image operations to WriteDrive()
* Also increase DD buffer size to 32 MB to improve performance
* 2e1833e91e introduced issues with VDS since, despite what
Microsoft's documentation says, balancing CoInitialize with CoUninitialize
leads to VDS not properly relinquishing disk access.
* Of course, since Grub4DOS's grldr.mbr hasn't changed from previous releases
there's not much to update there, but then again, people like version bumps.
* InternetGetConnectedState() is next to useless and doesn't provide
coherent outcome on the ARM64 platform I'm testing with. This results
in Rufus declaring that Internet is unavailable on platforms that do
have actual Internet connectivity.
* Swicth to using INetworkListManager::GetConnectivity(), which actually
reports a dependable result.
* Closes#1691
* Also remove the mutex for uprintf(), which may produce thread lockout
and remove an unwanted double GetSignatureName() call on startup.
* Looks like executables installed from the Windows Store launch with a "/InvokerPRAID"
added parameter, which of course BREAKS apps that have a defined set of parameters
and don't except that Microsoft would gingerly add random unwanted stuff there...
* The provision of this extra parameter also appears to be tied to using one of:
- <TargetDeviceFamily Name="Windows.Universal" ...>
- <uap:SplashScreen ...>
- <Application EntryPoint="$targetentrypoint$" ...>
in the appxmanifest.
* This resulted in our argument processing loop to cause early exit on account that an
unexpected option was provided.
* Fix this by adding an explicit check for /InvokerPRAID and not exiting on unhandled
params and removing or altering the 3 appxmanifest options listed above.
* Also set an explicit Windows.FullTrustApplication and remove splash screen.
* Also update _pre-commit.sh to update appstore build number automatically.
* Also remove splash screen images, add store listing CSV and toggle App builds to manual.
* Closes#1690
Yes!!! We are finally *much* faster than 7-zip for SHA-256, even though
we are also computing MD5 and SHA-1 in parallel. Here are some averaged
comparative results, against the 5.71 GB Win10_20H2_EnglishInternational_x64.iso
(SHA-256 = 08535b6dd0a4311f562e301c3c344b4aefd2e69a82168426b9971d6f8cab35e1):
* Windows' PowerShell Get-FileHash: 48s
* 7-zip's SHA-256 : 31s
* Rufus (64-bit release version) : 23s
* Due to the partition gymnastic that is required by the hack that is ISOHybrid,
some ISOHybrid images that are written in DD mode, such as Ubuntu 20.10, may
result in Windows somehow "losing" the target disk from some of its listings.
* This "removal" can be seen for instance if you have diskpart already open and
issue 'list disk' after Rufus 3.13 completed its image writing.
* In the worst case scenario, Windows may flat out refuse to access the disk at
the sector level be it in diskpart or disk manager, which forces ones to clear
the partition tables on Linux or some other OS to be able to "recover" the disk.
* This appears to be mostly due to Windows VDS cache (which Microsoft assures
should be able to do a proper job of refreshing itself on its own, in the same
stride as they also feel the need to introduce IVdsService::Refresh whose sole
purpose appears to work around a limitation that Microsoft knows exists) not
being in sync with the actual disk layout.
* So we now add calls to VDS layout refresh where needed, to work around the issue.
* Also fix an ext2fs Coverity warning.
* For blank disks, GetVdsDiskInterface() may return success with a NULL pAdvancedDisk.
* Also silence the annoying "Failed to read label" error on ERROR_UNRECOGNIZED_VOLUME.
* When writing images such as tails, that contain a large ESP, Windows forcibly
removes the media while we are writing it, unless we lock the logical drive.
* Also fix a Bled Coverity warning.
* Remove early locking of logical volume (no longer necessary due to previous commits).
* Relax exclusive locking of physical drive when an ESP is created.
* This should help with #1637 and #1640
* Also add an extra check for sector size in WriteDrive()
* Factorize drive letter removal into a RemoveDriveLetters() call.
* Improve MountVolume() and RemountVolume() calls.
* Also bump Rufus version to 3.13
* Make sure that instantiated objects are released.
* Factorize the instantiating of disk interfaces.
* Allow the provision of an offset to delete a single partition.
* Add a ListVdsVolumes() call (which is pointless since Microsoft *CRIPPLED* its VDS implementation).
* SetAutoMount()/GetAutoMount() should check for INVALID_HANDLE_VALUE and not NULL.
Also we don't actually need to open MOUNTMGR_DOS_DEVICE_NAME rw to issue an IOCTL.
* ToggleEsp() failed to exit properly when an ESP offset was specified.
* Introduce PI_MAX to explicitly set the size of the partition_information table.
* write_sectors() has write retry, so there's no need to perform one on top of it.
* When we exit FormatThread(), GetLogicalName() should attempt to look for the the
main partition and be silent.
* Make sure that if we skip a deep directory during scan, we count at
least one block of data.
* Also produce a note about deep directory long scan times and improve
the formatting of some messages.
* Ubuntu switched to using GRUB for BIOS, so our update_md5sum() code was not being called.
* Move update_md5sum() to being called unconditionally to fix this.
* Closes#1616 (again...)
* GRUB have cherry-picked patches from the "BootHole" vulnerability fix at
https://lists.gnu.org/archive/html/grub-devel/2020-07/msg00034.html and
have applied them to their 2.04 GRUB loader.
* This results in breakage with "error: symbol 'grub_calloc' not found" when
using the release GRUB 2.04 version of core.img.
* Therefore, we too cherry-picked some patches to apply on top of GRUB 2.04
release to make our core.img compatible with Ubuntu 20.10.
* Closes#1616
* Also increase the maximum write stride for ms-sys to 64 KB (required to
write the GRUB 2.05 bootloader which is larger than 32 KB) and update hash DB.
* The presence of a > 4 GB file forces the use of NTFS which is incompatible with
SysLinux 4.x or earlier. As such, if an image uses SysLinux only, there's no
point in enabling MBR as SysLinux won't boot.
* Required for ISOs such as securityonion-2.0.1-rc1.iso.
* Commit 77d319267f broke lookup of ISO filenames since iso9660_open()
enabled the Rock Ridge extensions by default, despite using ISO_EXTENSION_NONE
for the internal call, and we addressed a FIXME related to this.
* This resulted in Rufus not being able to lookup 'boot/grub/i386-pc/normal.mod' to parse GRUB's
version, since without Rock Ridge, 'i386-pc/' is unable to match the ISO-9660 'I386_PC/' dir.
* Closes#1573 and addresses part of #1616.
* Also fix a MinGW compilation warning.
* These bootloaders will require LFN support. Since we don't expect that
many people to create bootable media for RISC-V derived from bootloaders
contained in a 'efi.img`, we simply ignore these for now.
* Don't use hDrive handle for longer than necessary
* Move all the popcount() function calls into missing.h
* Ensure that the thread_affinity[] array is properly sized
* Improve timeouts for conflicting process search
* A user is reporting that, on one of their platforms, Rufus is writing to the wrong target during the file-copy
phase and using their existing Y: local drive instead of the drive associated to the USB, despite the fact
that Rufus is passing the right volume name to GetVolumePathNamesForVolumeName().
* Here's the PowerShell wmic output, confirming that the volume GUID obtained by Rufus is the right one:
DriveLetter : Y:
DeviceId : \\?\Volume{000349b1-17d0-69f6-c13f-f31162930600}\
Capacity : 118540464128
FileSystem : NTFS
Label : Y-DISK
DriveLetter : H:
DeviceId : \\?\Volume{b150ff4a-d62b-11ea-86e3-f49634660e54}\
Capacity : 15791824896
FileSystem : FAT32
Label : ADATA16GB
* And here's the Rufus log demonstrating that GetVolumePathNamesForVolumeName() is returning the *WRONG* letter:
Found volume \\?\Volume{b150ff4a-d62b-11ea-86e3-f49634660e54}\
\\?\Volume{b150ff4a-d62b-11ea-86e3-f49634660e54}\ is already mounted as Y: instead of H: - Will now use this target instead...
* The last line shows, without the shadow of a doubt, that we did feed "\\?\Volume{b150ff4a-d62b-11ea-86e3-f49634660e54}\" to
GetVolumePathNamesForVolumeName() and that this API call was successful (returned a non zero size) but ultimately returned
the wrong letter (Y: instead of H:)...
* Therefore, Windows is BUGGY and the use of GetVolumePathNamesForVolumeName() must be avoided.
* ISOs with tons of Rock Ridge deep directory entries (such as OPNsense)
can be very slow to scan due to the nature of deep directory parsing,
which requires processing the whole ISO9660 fs, for each deep directory
file, in order to find the relevant LSN entry.
* Since we don't expect much of the content we care about to reside in a
deep directory entry, we amend the code to cut short the scan of any
directory that contains such elements.
* Note that this only applies for ISO scan and it does nothing to speed
up the ISO extraction process.
* Related to issue #1575
* Closes#1467
* Also ensure that previous element is set when repositioning a
control to preserve tabbing order
* Also fix x64 version being able to download x86 BETA
* This is mostly aimed at Debian 11 netinst on the Raspberry Pi 4
* Only available for regular UEFI ISOs if GPT and FAT are selected (no MBR ESPs).
* Also fix a MinGW warning in GetUnusedDriveLetter()
* Now use version 1.6 of the EfiFs drivers that enables firmwares that
don't provide EFI_DEVICE_PATH_TO_TEXT_PROTOCOL to mount NTFS partitions
regardless.
* Also use the latest version of UEFI:NTFS that displays additional info
about the system.
* Closes#1213.
* Also update version to Rufus-next.
* ASLR is enabled by default for Visual Studio builds but that isn't the case
for MinGW builds. Fix that and also add -Wformat-security while we're at it.
* Closes#1518
* Also ensure that we'll never write protective MBR message for non-bootable
GPT drives, even as we are not calling WriteSBR() for those anyway.
* Also fix SBR message not being written for bootable images
* Also add an extra partition refresh after deleting partitions to try
to further force Windows take its stinking paws off our drive.
* Fix RTL location of "ISO" in the "Copying ISO files" translation for Arabic and Persian
* Fix whitespace/message ending issues for various translations
* Sync all .po's with .loc to avoid another German progress update is missing issue
* We distractedly chose to populate the message from our protective MBR
for GPT/UEFI-only boot media into the 4KB that directly followed the
MBR, which of course is space that is being used by the primary GPT.
* This resulted on systems having to fall back to using the secondary
GPT, which not all appear to be designed to do.
* Alter the code to ensure the protective message is written at LBA 34,
after the primary GPT.
* Closes#1507
* If a converted label contains mostly underscore, the proposed
label is used for FAT32 instead. However this label still has
the KB/MB/GB symbols localized so it may be invalid.
* Ensure that we use a non-localized version of the size when
using such a label.
* Closes#1506.
* Also fix a VS2019 static analysis warning in net.c.
* The upcoming Ubuntu 20.04 comes with MD5 validation turned on by default.
* When creating persistent boot media, we may update some of the validated files
to add persistence, update the search labels, etc.
* Make sure that the files we modify get their MD5 updated where needed.
* Also add 'loopback.cfg' to the list of config files we can add persistence to.
* Part of #1499
* Among other nefarious things, ubuntu 20.04 added a $casper_flavour suffix
to their grub.cfg /casper/vmlinuz kernel option, so we can no longer rely
on 'persistent' being inserted in a proper location.
* Switch to latching on file=/cdrom/preseed and hope that it will work for
all of Ubuntu & derivatives.
* Part of #1499.
* Commit 4c5adf092e moved us away from using CreateFile()
when extracting a file on the target media, and as such the error code returned when
failing to create an 'autorun.inf' due to a security solution has shifted.
* Make sure we handle the new error and don't bail out on 'autorun.inf' creation.
* Also update the actual name of the RtlDosPathNameToNtPathNameXXX function we use.
* Closes#1496
* Recent versions of Windows can set the deafult locale to codepage 65001 (UTF-8).
* This produces an assert due to a missing entry in cp_hr_list[], so fix that.
* However, this fix alone is not enough, as a GetOEMCP() that returns 65001 means
that any systems set to UTF-8 will fall back to codepage 437 for DOS, which is
definitely not what we want => Add an extra call to determine the actual OEM
codepage when UTF-8 is detected.
* Closes#1468
* Commit [e522ef6c55] (PR #1426) regressed the '%s'
progress messages back to '%0.1f%%' which results in the percentage remaining at
zero when the UI is in German.
* Surround macro params to ensure expected results
* Fix copy-paste errors
* Fix a potential buffer overflow in SetSectionHeaders()
* Add const modifier where relevant
* Use GetWindowLongPtr() everywhere
* Use proper sprintf format for unsigned int
* Use %s for printf-like funcs (https://www.viva64.com/en/w/v618/print/)
* Closes#1464
* Status code assignation was removed when the original code
was altered to use pfNtFsControlFile(). Fix that and also
make the code more similar to other calls.
* Closes#1459
* msg.S now reads an ASCII message (with escaped colour sequences)
from the following blocks, which is both more flexible and allows
for more content to be displayed.
* Also adds Bochs testing to the MBR build facility
* Hopefully using DICS_FLAG_CONFIGSPECIFIC instead of DICS_FLAG_GLOBAL is all that was needed
to get device disabling/re-enabling work without creating zombie devices, because we sure
need to force Windows' hand when it comes to detecting logical volumes...
* Implement CreatePreallocatedFile() which uses NtCreateFile() to create files with preallocated sizes.
This is used during ISO extraction to improve performance.
* Remove now-unused preallocate_filesize which was called after CreateFileU().
* Closes#1445
* ClearMBRGPT() attempts to write WRITE_RETRIES times, even if all those times succeed.
* Instead, skip the remaining retries on success.
* Also improve code readability.
* Closes#1454
* So, as it happens, when assigning the product of two 32-bit variables into a 64-bit one,
compilers default to being *DUMB* and, against all reasonable expectations, do not perform
that multiplication as a 64-bit operation (even when the code is compiled as x64). Wow,
that's really great decision making by compiler designers if I ever saw some... Whoever
decided that C developers would much rather want truncation and 32-bit overflows, instead
of the expected *LOGICAL* behaviour of conducting arithmetic operations as 64-bit when the
result will be assigned to a 64-bit variable, need to be condemned to a lifetime of trying
to help elderly folks trying to conduct simple computing tasks as a punishment...
Anyhoo, nt_write_blk()'s offset.QuadPart = block * channel->block_size + nt_data->offset
was overflowing 32-bit as soon as block * channel->block_size went over the 4 GB mark,
with the disastrous results one can expect. Considering that this is code we practically
lifted verbatim from e2fsprogs, I guess e2fsprogs' NT I/O manager was never properly
tested with anything larger than a 4 GB. Awesome!
* We fix the above by doing what unix_io.c does and setting the 32-bit read/write_blk()
calls to be wrappers around their 64-bit counterpart (since, once you deal with a 64-bit
block variable, the computation is conducted as 64-bit).
* Also remove a bunch of stuff we don't need from config.h
* Closes#1396
* Fix use of EXT2_BLOCK_SIZE() instead of EXT2_INODE_SIZE() during inode
initialization, that made us zero way many more blocks than was needed.
* Also disable sparse_super feature and improve block setup.
* Also explicitly use IS_POWER_OF_2 macro where required.
* Most distros (Debian, Ubuntu) have moved to using Sylinus 6.04 even
as it has NOT officially been released, so we want our fallback to
work against this too.
* pre1 since the Syslinux folks advise against using pre2 or later...
* Closes#1444
* Only applies for blank UEFI:NTFS drives for now. UEFI:NTFS Windows drives are
still set to use NTFS only (since Windows 7 doesn't support UEFI exFAT boot).
* While compressed EFI bootloaders are not an issue for UEFI:NTFS, some UEFI firmwares
embed an NTFS driver that doesn't support NTFS compression.
To address that, also uncompress the EFI bootloaders on NTFS.
* Closes#1424
Yet another link in the long chain of Microsoft making it UNFATHOMABLY DIFFICULT
to figure out what version of Windows an application is actually running on...
* When using compressed NTFS, having a compressed bootmgr prevents BIOS boot, so we
now call `compress -u` where needed to leave the relevant bootmgr files uncompressed.
* Closes#1381
* Also fix a minor warning in ext2fs
* Windows platforms prior to Windows 10 1703 cannot access any logical partition besides the
first one (we don't even get a volume for those).
* This fix enables the use of physical + offset for ext# formatting to work around this,
which is file since we don't actually need to mount the partition.
* Also fix ext2fs_open2() not handling normalized versions of Windows drive paths ("\\?\...")
* Also fix an issue where we would make the drive letter unavailable after formatting a
standalone partition to ext#.
* Also ensure that we return an error if the drive we attempt to locate a partition on
through an offset does not match the currently selected one.
* Also remove some unused calls in drive.c.
* Closes#1374
* While we need to detect that 'txt.cfg' is a Syslinux config file, so that
we can alter it for persistence, it should never be used as a main config
file, such as the one we link to when we create /syslinux.cfg.
* Closes#1375
