TLDR; Don't use OneDrive to sync your development project directories
So I have a development projects directory that I wanted to keep backed up in case I lost my laptop. So I copied the entire thing into my OneDrive
I use CakePHP and React for programming mainly so you get massive vendor and node_modules directories in each project folder
What a nightmare using OneDrive to sync files - it's incredibly patchy and bug ridden. You can't easily sync the files between MacOS and Windows and Linux as the OneDrive client doesn't perform on MacOS and doesn't exist on Linux (I use Insync but meh) and you have to then selectively sync folders so that the MacOS and Linux clients don't have too much to do
I think Dropbox for business is a way better product
How to lessen the pain
- Remove the vendor and node_modules folders to reduce what OneDrive is syncing
- Create a development folder outside of your onedrive tree and then use git to clone your onedrive git repos
- Commit your changes back to the onedrive hosted repositories on a schedule
Make a dev directory outside Onedrive
mkdir c:\dev
cd C:\dev
git clone C:\Users\Rupert\Onedrive\Sites\repo1
mkdir c:\dev\bin
Create a sync script to run as a scheduled task to commit your local changes automatically to onedrive sans the node_modules or vendor dirs (add them to .gitignore if not already)
#!/bin/sh
# put this in c:\dev\bin\onedrive-repo-backup.sh
COMMIT_DATE=`date +"%Y-%m-%d"`
cd /c/dev/get-current-crypto-aud
# git add . -A
# just commit what is already commited
git commit -m "Changes committed ${COMMIT_DATE}"
# push to the backup branch
git push onedrive master:backup
Create a scheduled task to call the above sync script for your repositories. Add to the code to sync multiple repos.
Removing a heap of folders from your local OneDrive directories programmatically
You might think that you just run remove-item recurse but it won't remove the special NTFS link files that are "online only" pointers to the actual files stored in your onedrive so the following removed the vendor and node_modules folder
function Remove-FileSystemItem {
<#
.SYNOPSIS
Removes files or directories reliably and synchronously.
.DESCRIPTION
Removes files and directories, ensuring reliable and synchronous
behavior across all supported platforms.
The syntax is a subset of what Remove-Item supports; notably,
-Include / -Exclude and -Force are NOT supported; -Force is implied.
As with Remove-Item, passing -Recurse is required to avoid a prompt when
deleting a non-empty directory.
IMPORTANT:
* On Unix platforms, this function is merely a wrapper for Remove-Item,
where the latter works reliably and synchronously, but on Windows a
custom implementation must be used to ensure reliable and synchronous
behavior. See https://github.com/PowerShell/PowerShell/issues/8211
* On Windows:
* The *parent directory* of a directory being removed must be
*writable* for the synchronous custom implementation to work.
* The custom implementation is also applied when deleting
directories on *network drives*.
* If an indefinitely *locked* file or directory is encountered, removal is aborted.
By contrast, files opened with FILE_SHARE_DELETE /
[System.IO.FileShare]::Delete on Windows do NOT prevent removal,
though they do live on under a temporary name in the parent directory
until the last handle to them is closed.
* Hidden files and files with the read-only attribute:
* These are *quietly removed*; in other words: this function invariably
behaves like `Remove-Item -Force`.
* Note, however, that in order to target hidden files / directories
as *input*, you must specify them as a *literal* path, because they
won't be found via a wildcard expression.
* The reliable custom implementation on Windows comes at the cost of
decreased performance.
.EXAMPLE
Remove-FileSystemItem C:\tmp -Recurse
Synchronously removes directory C:\tmp and all its content.
#>
[CmdletBinding(SupportsShouldProcess, ConfirmImpact = 'Medium', DefaultParameterSetName = 'Path', PositionalBinding = $false)]
param(
[Parameter(ParameterSetName = 'Path', Mandatory, Position = 0, ValueFromPipeline, ValueFromPipelineByPropertyName)]
[string[]] $Path
,
[Parameter(ParameterSetName = 'Literalpath', ValueFromPipelineByPropertyName)]
[Alias('PSPath')]
[string[]] $LiteralPath
,
[switch] $Recurse
)
begin {
# !! Workaround for https://github.com/PowerShell/PowerShell/issues/1759
if ($ErrorActionPreference -eq [System.Management.Automation.ActionPreference]::Ignore) { $ErrorActionPreference = 'Ignore' }
$targetPath = ''
$yesToAll = $noToAll = $false
function trimTrailingPathSep([string] $itemPath) {
if ($itemPath[-1] -in '\', '/') {
# Trim the trailing separator, unless the path is a root path such as '/' or 'c:\'
if ($itemPath.Length -gt 1 -and $itemPath -notmatch '^[^:\\/]+:.$') {
$itemPath = $itemPath.Substring(0, $itemPath.Length - 1)
}
}
$itemPath
}
function getTempPathOnSameVolume([string] $itemPath, [string] $tempDir) {
if (-not $tempDir) { $tempDir = [IO.Path]::GetDirectoryName($itemPath) }
[IO.Path]::Combine($tempDir, [IO.Path]::GetRandomFileName())
}
function syncRemoveFile([string] $filePath, [string] $tempDir) {
# Clear the ReadOnly attribute, if present.
if (($attribs = [IO.File]::GetAttributes($filePath)) -band [System.IO.FileAttributes]::ReadOnly) {
[IO.File]::SetAttributes($filePath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
}
$tempPath = getTempPathOnSameVolume $filePath $tempDir
[IO.File]::Move($filePath, $tempPath)
[IO.File]::Delete($tempPath)
}
function syncRemoveDir([string] $dirPath, [switch] $recursing) {
if (-not $recursing) { $dirPathParent = [IO.Path]::GetDirectoryName($dirPath) }
# Clear the ReadOnly attribute, if present.
# Note: [IO.File]::*Attributes() is also used for *directories*; [IO.Directory] doesn't have attribute-related methods.
if (($attribs = [IO.File]::GetAttributes($dirPath)) -band [System.IO.FileAttributes]::ReadOnly) {
[IO.File]::SetAttributes($dirPath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
}
# Remove all children synchronously.
$isFirstChild = $true
foreach ($item in [IO.directory]::EnumerateFileSystemEntries($dirPath)) {
if (-not $recursing -and -not $Recurse -and $isFirstChild) {
# If -Recurse wasn't specified, prompt for nonempty dirs.
$isFirstChild = $false
# Note: If -Confirm was also passed, this prompt is displayed *in addition*, after the standard $PSCmdlet.ShouldProcess() prompt.
# While Remove-Item also prompts twice in this scenario, it shows the has-children prompt *first*.
if (-not $PSCmdlet.ShouldContinue("The item at '$dirPath' has children and the -Recurse switch was not specified. If you continue, all children will be removed with the item. Are you sure you want to continue?", 'Confirm', ([ref] $yesToAll), ([ref] $noToAll))) { return }
}
$itemPath = [IO.Path]::Combine($dirPath, $item)
([ref] $targetPath).Value = $itemPath
if ([IO.Directory]::Exists($itemPath)) {
syncremoveDir $itemPath -recursing
}
else {
syncremoveFile $itemPath $dirPathParent
}
}
# Finally, remove the directory itself synchronously.
([ref] $targetPath).Value = $dirPath
$tempPath = getTempPathOnSameVolume $dirPath $dirPathParent
[IO.Directory]::Move($dirPath, $tempPath)
[IO.Directory]::Delete($tempPath)
}
}
process {
$isLiteral = $PSCmdlet.ParameterSetName -eq 'LiteralPath'
if ($env:OS -ne 'Windows_NT') {
# Unix: simply pass through to Remove-Item, which on Unix works reliably and synchronously
Remove-Item @PSBoundParameters
}
else {
# Windows: use synchronous custom implementation
foreach ($rawPath in ($Path, $LiteralPath)[$isLiteral]) {
# Resolve the paths to full, filesystem-native paths.
try {
# !! Convert-Path does find hidden items via *literal* paths, but not via *wildcards* - and it has no -Force switch (yet)
# !! See https://github.com/PowerShell/PowerShell/issues/6501
$resolvedPaths = if ($isLiteral) { Convert-Path -ErrorAction Stop -LiteralPath $rawPath } else { Convert-Path -ErrorAction Stop -path $rawPath }
}
catch {
Write-Error $_ # relay error, but in the name of this function
continue
}
try {
$isDir = $false
foreach ($resolvedPath in $resolvedPaths) {
# -WhatIf and -Confirm support.
if (-not $PSCmdlet.ShouldProcess($resolvedPath)) { continue }
if ($isDir = [IO.Directory]::Exists($resolvedPath)) {
# dir.
# !! A trailing '\' or '/' causes directory removal to fail ("in use"), so we trim it first.
syncRemoveDir (trimTrailingPathSep $resolvedPath)
}
elseif ([IO.File]::Exists($resolvedPath)) {
# file
syncRemoveFile $resolvedPath
}
else {
Throw "Not a file-system path or no longer extant: $resolvedPath"
}
}
}
catch {
if ($isDir) {
$exc = $_.Exception
if ($exc.InnerException) { $exc = $exc.InnerException }
if ($targetPath -eq $resolvedPath) {
Write-Error "Removal of directory '$resolvedPath' failed: $exc"
}
else {
Write-Error "Removal of directory '$resolvedPath' failed, because its content could not be (fully) removed: $targetPath`: $exc"
}
}
else {
Write-Error $_ # relay error, but in the name of this function
}
continue
}
}
}
}
}
Get-ChildItem -Recurse -Directory | `
Select-Object -ExpandProperty FullName -Property FullName | Where-Object {
(
( $_.FullName -match '\\vendor$' -and $_.FullName -notmatch '\\vendor\\' ) -or
( $_.FullName -match '\\vendors$' -and $_.FullName -notmatch '\\vendors\\' ) -or
( $_.FullName -match '\\node_modules$' -and $_.FullName -notmatch '\\node_modules\\')
)
} | Where-Object {
# don't match these
# wordpress
$_.FullName -notmatch '\\node_modules\\vendors$'
} | Where-Object {
$_.FullName -notmatch 'wp-content'
} | ForEach-Object {
Write-Host "Removing $($_.FullName)"
Remove-FileSystemItem -Path $_.FullName -Recurse
}
0 Comments