Using OneDrive to sync development directories

by | Aug 23, 2021 | IT Tips, Wordpress | 0 comments

TLDR; Don't use OneDrive to sync your development project directories

So I have a development projects directory that I wanted to keep backed up in case I lost my laptop. So I copied the entire thing into my OneDrive

I use CakePHP and React for programming mainly so you get massive vendor and node_modules directories in each project folder

What a nightmare using OneDrive to sync files - it's incredibly patchy and bug ridden. You can't easily sync the files between MacOS and Windows and Linux as the OneDrive client doesn't perform on MacOS and doesn't exist on Linux (I use Insync but meh) and you have to then selectively sync folders so that the MacOS and Linux clients don't have too much to do

I think Dropbox for business is a way better product

How to lessen the pain

  1. Remove the vendor and node_modules folders to reduce what OneDrive is syncing
  2. Create a development folder outside of your onedrive tree and then use git to clone your onedrive git repos
  3. Commit your changes back to the onedrive hosted repositories on a schedule

Make a dev directory outside Onedrive

?
1
2
3
4
mkdir c:\dev
cd C:\dev
git clone C:\Users\Rupert\Onedrive\Sites\repo1
mkdir c:\dev\bin

Create a sync script to run as a scheduled task to commit your local changes automatically to onedrive sans the node_modules or vendor dirs (add them to .gitignore if not already)

?
1
2
3
4
5
6
7
8
9
10
11
#!/bin/sh
 
# put this in c:\dev\bin\onedrive-repo-backup.sh
 
COMMIT_DATE=`date +"%Y-%m-%d"`
cd /c/dev/get-current-crypto-aud
# git add . -A
# just commit what is already commited
git commit -m "Changes committed ${COMMIT_DATE}"
# push to the backup branch
git push onedrive master:backup

Create a scheduled task to call the above sync script for your repositories. Add to the code to sync multiple repos.

Removing a heap of folders from your local OneDrive directories programmatically

You might think that you just run remove-item recurse but it won't remove the special NTFS link files that are "online only" pointers to the actual files stored in your onedrive so the following removed the vendor and node_modules folder

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
function Remove-FileSystemItem {
    <#
  .SYNOPSIS
    Removes files or directories reliably and synchronously.
 
  .DESCRIPTION
    Removes files and directories, ensuring reliable and synchronous
    behavior across all supported platforms.
 
    The syntax is a subset of what Remove-Item supports; notably,
    -Include / -Exclude and -Force are NOT supported; -Force is implied.
     
    As with Remove-Item, passing -Recurse is required to avoid a prompt when
    deleting a non-empty directory.
 
    IMPORTANT:
      * On Unix platforms, this function is merely a wrapper for Remove-Item,
        where the latter works reliably and synchronously, but on Windows a
        custom implementation must be used to ensure reliable and synchronous
        behavior. See https://github.com/PowerShell/PowerShell/issues/8211
 
    * On Windows:
      * The *parent directory* of a directory being removed must be
        *writable* for the synchronous custom implementation to work.
      * The custom implementation is also applied when deleting
         directories on *network drives*.
 
    * If an indefinitely *locked* file or directory is encountered, removal is aborted.
      By contrast, files opened with FILE_SHARE_DELETE /
      [System.IO.FileShare]::Delete on Windows do NOT prevent removal,
      though they do live on under a temporary name in the parent directory
      until the last handle to them is closed.
 
    * Hidden files and files with the read-only attribute:
      * These are *quietly removed*; in other words: this function invariably
        behaves like `Remove-Item -Force`.
      * Note, however, that in order to target hidden files / directories
        as *input*, you must specify them as a *literal* path, because they
        won't be found via a wildcard expression.
 
    * The reliable custom implementation on Windows comes at the cost of
      decreased performance.
 
  .EXAMPLE
    Remove-FileSystemItem C:\tmp -Recurse
 
    Synchronously removes directory C:\tmp and all its content.
  #>
    [CmdletBinding(SupportsShouldProcess, ConfirmImpact = 'Medium', DefaultParameterSetName = 'Path', PositionalBinding = $false)]
    param(
        [Parameter(ParameterSetName = 'Path', Mandatory, Position = 0, ValueFromPipeline, ValueFromPipelineByPropertyName)]
        [string[]] $Path
        ,
        [Parameter(ParameterSetName = 'Literalpath', ValueFromPipelineByPropertyName)]
        [Alias('PSPath')]
        [string[]] $LiteralPath
        ,
        [switch] $Recurse
    )
    begin {
        # !! Workaround for https://github.com/PowerShell/PowerShell/issues/1759
        if ($ErrorActionPreference -eq [System.Management.Automation.ActionPreference]::Ignore) { $ErrorActionPreference = 'Ignore' }
        $targetPath = ''
        $yesToAll = $noToAll = $false
        function trimTrailingPathSep([string] $itemPath) {
            if ($itemPath[-1] -in '\', '/') {
                # Trim the trailing separator, unless the path is a root path such as '/' or 'c:\'
                if ($itemPath.Length -gt 1 -and $itemPath -notmatch '^[^:\\/]+:.$') {
                    $itemPath = $itemPath.Substring(0, $itemPath.Length - 1)
                }
            }
            $itemPath
        }
        function getTempPathOnSameVolume([string] $itemPath, [string] $tempDir) {
            if (-not $tempDir) { $tempDir = [IO.Path]::GetDirectoryName($itemPath) }
            [IO.Path]::Combine($tempDir, [IO.Path]::GetRandomFileName())
        }
        function syncRemoveFile([string] $filePath, [string] $tempDir) {
            # Clear the ReadOnly attribute, if present.
            if (($attribs = [IO.File]::GetAttributes($filePath)) -band [System.IO.FileAttributes]::ReadOnly) {
                [IO.File]::SetAttributes($filePath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
            }
            $tempPath = getTempPathOnSameVolume $filePath $tempDir
            [IO.File]::Move($filePath, $tempPath)
            [IO.File]::Delete($tempPath)
        }
        function syncRemoveDir([string] $dirPath, [switch] $recursing) {
            if (-not $recursing) { $dirPathParent = [IO.Path]::GetDirectoryName($dirPath) }
            # Clear the ReadOnly attribute, if present.
            # Note: [IO.File]::*Attributes() is also used for *directories*; [IO.Directory] doesn't have attribute-related methods.
            if (($attribs = [IO.File]::GetAttributes($dirPath)) -band [System.IO.FileAttributes]::ReadOnly) {
                [IO.File]::SetAttributes($dirPath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
            }
            # Remove all children synchronously.
            $isFirstChild = $true
            foreach ($item in [IO.directory]::EnumerateFileSystemEntries($dirPath)) {
                if (-not $recursing -and -not $Recurse -and $isFirstChild) {
                    # If -Recurse wasn't specified, prompt for nonempty dirs.
                    $isFirstChild = $false
                    # Note: If -Confirm was also passed, this prompt is displayed *in addition*, after the standard $PSCmdlet.ShouldProcess() prompt.
                    #       While Remove-Item also prompts twice in this scenario, it shows the has-children prompt *first*.
                    if (-not $PSCmdlet.ShouldContinue("The item at '$dirPath' has children and the -Recurse switch was not specified. If you continue, all children will be removed with the item. Are you sure you want to continue?", 'Confirm', ([ref] $yesToAll), ([ref] $noToAll))) { return }
                }
                $itemPath = [IO.Path]::Combine($dirPath, $item)
                ([ref] $targetPath).Value = $itemPath
                if ([IO.Directory]::Exists($itemPath)) {
                    syncremoveDir $itemPath -recursing
                }
                else {
                    syncremoveFile $itemPath $dirPathParent
                }
            }
            # Finally, remove the directory itself synchronously.
            ([ref] $targetPath).Value = $dirPath
            $tempPath = getTempPathOnSameVolume $dirPath $dirPathParent
            [IO.Directory]::Move($dirPath, $tempPath)
            [IO.Directory]::Delete($tempPath)
        }
    }
 
    process {
        $isLiteral = $PSCmdlet.ParameterSetName -eq 'LiteralPath'
        if ($env:OS -ne 'Windows_NT') {
            # Unix: simply pass through to Remove-Item, which on Unix works reliably and synchronously
            Remove-Item @PSBoundParameters
        }
        else {
            # Windows: use synchronous custom implementation
            foreach ($rawPath in ($Path, $LiteralPath)[$isLiteral]) {
                # Resolve the paths to full, filesystem-native paths.
                try {
                    # !! Convert-Path does find hidden items via *literal* paths, but not via *wildcards* - and it has no -Force switch (yet)
                    # !! See https://github.com/PowerShell/PowerShell/issues/6501
                    $resolvedPaths = if ($isLiteral) { Convert-Path -ErrorAction Stop -LiteralPath $rawPath } else { Convert-Path -ErrorAction Stop -path $rawPath }
                }
                catch {
                    Write-Error $_ # relay error, but in the name of this function
                    continue
                }
                try {
                    $isDir = $false
                    foreach ($resolvedPath in $resolvedPaths) {
                        # -WhatIf and -Confirm support.
                        if (-not $PSCmdlet.ShouldProcess($resolvedPath)) { continue }
                        if ($isDir = [IO.Directory]::Exists($resolvedPath)) {
                            # dir.
                            # !! A trailing '\' or '/' causes directory removal to fail ("in use"), so we trim it first.
                            syncRemoveDir (trimTrailingPathSep $resolvedPath)
                        }
                        elseif ([IO.File]::Exists($resolvedPath)) {
                            # file
                            syncRemoveFile $resolvedPath
                        }
                        else {
                            Throw "Not a file-system path or no longer extant: $resolvedPath"
                        }
                    }
                }
                catch {
                    if ($isDir) {
                        $exc = $_.Exception
                        if ($exc.InnerException) { $exc = $exc.InnerException }
                        if ($targetPath -eq $resolvedPath) {
                            Write-Error "Removal of directory '$resolvedPath' failed: $exc"
                        }
                        else {
                            Write-Error "Removal of directory '$resolvedPath' failed, because its content could not be (fully) removed: $targetPath`: $exc"
                        }
                    }
                    else {
                        Write-Error $_  # relay error, but in the name of this function
                    }
                    continue
                }
            }
        }
    }
}
 
Get-ChildItem -Recurse -Directory | `
    Select-Object -ExpandProperty FullName -Property FullName |  Where-Object {
    (
 
        ( $_.FullName -match '\\vendor$' -and $_.FullName -notmatch '\\vendor\\' ) -or
 
        ( $_.FullName -match '\\vendors$' -and $_.FullName -notmatch '\\vendors\\' ) -or
        ( $_.FullName -match '\\node_modules$' -and $_.FullName -notmatch '\\node_modules\\')
    )
} | Where-Object {
    # don't match these
    # wordpress
    $_.FullName -notmatch '\\node_modules\\vendors$'
} | Where-Object {
    $_.FullName -notmatch 'wp-content'
} | ForEach-Object {
  Write-Host "Removing $($_.FullName)"
  Remove-FileSystemItem -Path $_.FullName -Recurse
 }

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The reCAPTCHA verification period has expired. Please reload the page.