text:
failing to copy large files to acd (Reset without error count inc)

Hey,
I'm getting a an issue where the upload is getting reset on large files 15gb+ it looks like a connection reset. Any setting I need to add to keep the connection open due to large file size?
rclone version: rclone v1.35
my os is freenas which is on FreeBSD 9.3-RELEASE-p25 amd64
I'm using Amazon Cloud Drive and trying to run the below
rclone -v --transfers 1 --ignore-existing copy /mnt/all/UHD_Media/ acd:UHD_Media/ > & /var/log/rcloneUHD.log &

Logs
2017/03/19 22:13:41
Transferred:   51.265 GBytes (1.767 MBytes/s)
Errors:                 0
Checks:                21
Transferred:            2
Elapsed time:   8h15m1.4s
Transferring:
 * ...erstellar (2014)/Interstellar(BR-Disk).mkv: 63% done, 893.665 kBytes/s, ETA: 3h55m23s

2017/03/19 22:13:44 Movies/Interstellar (2014)/Interstellar(BR-Disk).mkv: Upload error detected but didn't finish upload: Post https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication: read tcp 10.56.12.0:18627->52.87.95.159:443: read: connection reset by peer ("HTTP status UNKNOWN")
2017/03/19 22:13:44 pacer: Rate limited, sleeping for 476.771718ms (1 consecutive low level retries)
 2017/03/19 22:13:44 pacer: low level retry 1/1 (error Post https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication: read tcp 10.56.12.0:18627->52.87.95.159:443: read: connection reset by peer)
2017/03/19 22:13:44 Movies/Interstellar (2014)/Interstellar(BR-Disk).mkv: Received error: Post https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication: read tcp 10.56.12.0:18627->52.87.95.159:443: read: connection reset by peer - low level retry 1/10
2017/03/19 22:13:44 pacer: Resetting sleep to minimum 20ms on success
2017/03/19 22:14:41
Transferred:   51.331 GBytes (1.766 MBytes/s)
Errors:                 0
Checks:                21
Transferred:            2
Elapsed time:   8h16m1.4s
Transferring:
 * ...erstellar (2014)/Interstellar(BR-Disk).mkv:  0% done, 1.060 MBytes/s, ETA: 8h54m23s

First question, how big is the file? ACD has a file size limit of 50GB.
Second, did your internet connection dropped during the transfer? There could be several reasons for this: the router couldn't handle the transfer load, or your internet provider forced a reset of your connection and so on...

@mazeprince The files are ~25gb max and I'm running pfsense router is able to handle the load and I've had no connection drops for weeks. no ip renew either so and connection didn't get reset so really not sure why this is happening.
Aslo today I noticed some files were at 100% but timed out waitung for object to be found, any ideas why this happens as well
Transferred:   332.631 GBytes (1.697 MBytes/s)
Errors:                 0
Checks:                26
Transferred:           11
Elapsed time:  55h46m1.4s
Transferring:
 * ... (2012)/The Dark Knight Rises(BR-Disk).mkv: 100% done, 0 Bytes/s, ETA: 0s

2017/03/21 21:44:41 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1153/1165)
2017/03/21 21:44:46 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1154/1165)
2017/03/21 21:44:52 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1155/1165)
2017/03/21 21:44:57 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1156/1165)
2017/03/21 21:45:02 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1157/1165)
2017/03/21 21:45:07 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1158/1165)
2017/03/21 21:45:12 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1159/1165)
2017/03/21 21:45:17 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1160/1165)
2017/03/21 21:45:22 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1161/1165)
2017/03/21 21:45:27 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1162/1165)
2017/03/21 21:45:33 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1163/1165)
2017/03/21 21:45:38 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1164/1165)
2017/03/21 21:45:41
Transferred:   332.631 GBytes (1.696 MBytes/s)
Errors:                 0
Checks:                26
Transferred:           11
Elapsed time:  55h47m1.4s
Transferring:
 * ... (2012)/The Dark Knight Rises(BR-Disk).mkv: 100% done, 0 Bytes/s, ETA: 0s

2017/03/21 21:45:53 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Object not found - waiting (1165/1165)
2017/03/21 21:45:58 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Giving up waiting for object - returning original error: HTTP code 504: "504 GATEWAY_TIMEOUT": no response body ("504 GATEWAY_TIMEOUT")
2017/03/21 21:45:58 pacer: Rate limited, sleeping for 762.776516ms (1 consecutive low level retries)
2017/03/21 21:45:58 pacer: low level retry 1/1 (error HTTP code 504: "504 GATEWAY_TIMEOUT": no response body)
2017/03/21 21:45:58 Movies/Dark Knight Rises, The (2012)/The Dark Knight Rises(BR-Disk).mkv: Received error: HTTP code 504: "504 GATEWAY_TIMEOUT": no response body - low level retry 1/10
2017/03/21 21:45:58 pacer: Resetting sleep to minimum 20ms on success
2017/03/21 21:46:41
Transferred:   332.720 GBytes (1.696 MBytes/s)
Errors:                 0
Checks:                26
Transferred:           11
Elapsed time:  55h48m1.4s
Transferring:
 * ... (2012)/The Dark Knight Rises(BR-Disk).mkv:  0% done, 2.017 MBytes/s, ETA: 4h32m56s

Unfortunately i have no other ideas for your initial problem.
As for your timed out 100% files: It looks like ACD has sometimes a lag with large uploaded files, thats why there is the --acd-upload-wait-per-gb option. Default is 3 min per GB and for me that was always more than enough, but maybe it helps in your case to adjust it. Although with default 3 min it rclone waits 60 minutes for a 20GB file... that should be more than enough in my opinion.
Sorry i cant really help you any further.

@mazeprince Thanks for taht, think i might move to transfer 2 files at a time and extend the wait.

I'm getting a an issue where the upload is getting reset on large files 15gb+ it looks like a connection reset. Any setting I need to add to keep the connection open due to large file size?

Connection resets can be caused by Amazon, your ISP, your router or any bit of networking equipment.
Most likely suspect is your router, though pfsense is very reliable I hear.  Is there another box the pfsense router connects to?
Some ISPs have been known to kill long running TCP connections...

First question, how big is the file? ACD has a file size limit of 50GB.
This has been discussed extensively in other threads, and yet rclone still will sit there and uselessly upload files well in excess of the limit. I sent ~700GB to Amazon (Two runs, three attempts each, of a 115GB file) before I finally found the bug reports in here - rclone gave no errors and spent a couple days on those useless transfers. I thought rclone was designed to throw an error if the file being uploaded was in excess of the 50GB limit?

The files are ~25gb max
The largest file is actually 32gb on disk so well under the 50gb limit, I have about 1TB of large files so just gonna put them on a back drive instead of ACD

@diamondsw BTW rclone writes this warning if it detects a too big file
"Warning: file %q may fail because it is too big. Use --max-size=%dM to skip large files.

I'm going to close this since rclone no longer has any ACD keys and Amazon aren't issuing new ones.

authors: ADeane6 diamondsw mazeprince ncw

repo: ncw/rclone


text:
[enhancement] Option to proceed after running with --dry-run

I think it'd be an improvement if after a dry-run is completed, rclone keeps the changes that would've been done in memory, and then has a Proceed [Y/N] option after completion. With this, it wouldn't have to run all the checks twice (assuming that the user then runs again without the dry-run flag) saving API calls against whatever service is being used.

Not a bad idea!  It would probably need to be some other flag used together with --dry-run or instead of it.

Sure, I was thinking that too, just that dry-run is the thing I typically run first to make sure I'm not making any unintended changes. Perhaps a new flag like --require-confirmation ?

You know as soon as you do this someone will want a confirmation after each file as well the whole set.  Perhaps
--confirm each
--confirm all
You could default to 'all' if someone just types 'confirm'.  I would think that dry-run and this should be mutually exclusive (instead of).   Just my 1 penny.

source: gharchive/issue

authors: calisro mttlmy ncw

license: MIT

license_type: permissive

license_source: github-api


text:
Make: Build Fail

What is the problem you are having with rclone?
when running make i get a go.mod: checksum mismatch.
this is an issue with the gobuffalo/packr dependency (resolved by updating) documented here: https://github.com/gobuffalo/packr/issues/213
this was introduced in #246
also #246 doubles the amount of dependencies in go.mod and go.sum for one extra library, was that expected?
What is your rclone version (output from rclone version)
master
Which OS you are using and how many bits (eg Windows 7, 64 bit)
linux 64bit
Which cloud storage system are you using? (eg Google Drive)
any
The command you were trying to run (eg rclone copy /tmp remote:tmp)
make
A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)
verifying github.com/gobuffalo/packr/v2@v2.4.0/go.mod: checksum mismatch
	downloaded: h1:ra341gygw9/61nSjAbfwcwh8IrYL4WmR4IsPkPBhQiY=
	go.sum:     h1:tj7r/7WvgPz0qgtdtKjlbAiGglcdbM02mq/Bv25LsOA=

What is the problem you are having with rclone?
when running make i get a go.mod: checksum mismatch.
this is an issue with the gobuffalo/packr dependency (resolved by updating) documented here: gobuffalo/packr#213
this was introduced in #246
also #246 doubles the amount of dependencies in go.mod and go.sum for one extra library, was that expected?

There isn't an issue #246 which one did you mean?
I'm interested in reducing the dependencies if we can!

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)
verifying github.com/gobuffalo/packr/v2@v2.4.0/go.mod: checksum mismatch
	downloaded: h1:ra341gygw9/61nSjAbfwcwh8IrYL4WmR4IsPkPBhQiY=
	go.sum:     h1:tj7r/7WvgPz0qgtdtKjlbAiGglcdbM02mq/Bv25LsOA=

Ah, this is the real problem: https://github.com/gobuffalo/packr/issues/213#issuecomment-505620338

Ok, the problem should be solved now. It appears that the v2.4.0 got shifted to a different commit somehow.
If you still get the problem, it's because Go is now caching the wrong commit.
$ (sudo) rm -rfv $GOPATH/pkg/mod/github.com/gobuffalo/packr
Should clear it for you.
Sorry for the problems folks!

So it looks like the commit has been moved back so clearing the cache should fix it.

In fact I have no idea why gobuffalo is even in the go.mod
Given that rclone vendors its dependencies, there are no references to it anywhere in the code
$ git grep -l github.com/gobuffalo
go.mod
go.sum

I deleted go.sum and the vendor directory, ran go mod tidy and go mod vendor and I seem to have purged gobuffalo from the system.
I don't understand why go mod tidy didn't do that when I ran it a week or so ago as part of d61ba7ef78966b0903b623d18d258fbba923f712
In fact go modules is mostly a mystery to me!
I've pushed this to master and checked this builds with and without modules - can you confirm it is working for you?

it works now,
sorry I meant 2d0e9885bd78f23f0957f36a81b065afda2fc714 however on the github website it has a 246 next to it

I found https://github.com/golang/go/wiki/Modules helpful in explaining modules especially:

By default, go commands like go build ignore the vendor directory when in module mode.
The -mod=vendor flag (e.g., go build -mod=vendor) instructs the go commands to use the main module's top-level vendor directory to satisfy dependencies. The go commands in this mode therefore ignore the dependency descriptions in go.mod and assume that the vendor directory holds the correct copies of dependencies. Note that only the main module's top-level vendor directory is used; vendor directories in other locations are still ignored.
Some people will want to routinely opt-in to vendoring by setting a GOFLAGS=-mod=vendor environment variable.

which is why by default (from go 1.11) make downloads all the modules instead of relying on the vendor folder, if you want to change this add GOFLAGS=-mod=vendor  to the makefile
also from the faq:

Why does 'go mod tidy' record indirect and test dependencies in my 'go.mod'?
The modules system records precise dependency requirements in your go.mod. (For more details, see the go.mod concepts section above or the go.mod tip documentation).
go mod tidy updates your current go.mod to include the dependencies needed for tests in your module — if a test fails, we must know which dependencies were used in order to reproduce the failure.
go mod tidy also ensures your current go.mod reflects the dependency requirements for all possible combinations of OS, architecture, and build tags (as described here). In contrast, other commands like go build and go test only update go.mod to provide the packages imported by the requested packages under the current GOOS, GOARCH, and build tags (which is one reason go mod tidy might add requirements that were not added by go build or similar).
If a dependency of your module does not itself have a go.mod (e.g., because the dependency has not yet opted in to modules itself), or if its go.mod file is missing one or more of its dependencies (e.g., because the module author did not run go mod tidy), then the missing transitive dependencies will be added to your module's requirements, along with an // indirect comment to indicate that the dependency is not from a direct import within your module.
Note that this also means that any missing test dependencies from your direct or indirect dependencies will also be recorded in your go.mod. (An example of when this is important: go test all runs the tests of all direct and indirect dependencies of your module, which is one way to validate that your current combination of versions work together. If a test fails in one of your dependencies when you run go test all, it is important to have a complete set of test dependency information recorded so that you have reproducible go test all behavior).
Another reason you might have // indirect dependencies in your go.mod file is if you have upgraded (or downgraded) one of your indirect dependencies beyond what is required by your direct dependencies, such as if you ran go get -u or go get foo@1.2.3. The go tooling needs a place to record those new versions, and it does so in your go.mod file (and it does not reach down into your dependencies to modify their go.mod files).
In general, the behaviors described above are part of how modules provide 100% reproducible builds and tests by recording precise dependency information.
If you are curious as to why a particular module is showing up in your go.mod, you can run go mod why -m  to answer that question. Other useful tools for inspecting requirements and versions include go mod graph and go list -m all

which is why go get -u & go mod tidy is not always the best.

it works now,
sorry I meant 2d0e988 however on the github website it has a 246 next to it

Thanks - don't think I reviewed that pull request well enough!
I think this was the root of the problem since I merged that after I last did go mod tidy.
Note that after I rebuilt the vendor directory there were no files added, removed or changed by it, so I think this was entirely a go.sum problem caused by that PR.
I'm attempting to have rclone buildable with and without modules.  Normally I build it without modules.

which is why by default (from go 1.11) make downloads all the modules instead of relying on the vendor folder, if you want to change this add GOFLAGS=-mod=vendor to the makefile

I think that is fine if someone wants to use modules, or do you think I should change that so if someone invokes make not in GOPATH it uses the vendored modules?  However if they just invoke go build then they will get the existing behaviour which will make it inconsistent.

which is why go get -u & go mod tidy is not always the best.

The procedure I use for updating the dependencies in in the Makefile.  I do this with go mod and vendor all the modules.
# Update dependencies
update:
	GO111MODULE=on go get -u ./...
	GO111MODULE=on go mod tidy
	GO111MODULE=on go mod vendor

Do you think that is correct?

i am new to go, so am not a reliable source on the matter, but

I'm attempting to have rclone buildable with and without modules. Normally I build it without modules.

which is why by default (from go 1.11) make downloads all the modules instead of relying on the vendor folder, if you want to change this add GOFLAGS=-mod=vendor to the makefile

I think that is fine if someone wants to use modules, or do you think I should change that so if someone invokes make not in GOPATH it uses the vendored modules? However if they just invoke go build then they will get the existing behaviour which will make it inconsistent.

i think it is fine to have the makefile do either or by default, however whatever the default is should be tested by the CI also which would have caught this bug.

which is why go get -u & go mod tidy is not always the best.

The procedure I use for updating the dependencies in in the Makefile. I do this with go mod and vendor all the modules.
# Update dependencies
update:
	GO111MODULE=on go get -u ./...
	GO111MODULE=on go mod tidy
	GO111MODULE=on go mod vendor

Do you think that is correct?

i was just quoting the wiki before, i really am not an expert and don't know, sorry

i am new to go, so am not a reliable source on the matter, but

I think the problem is that we are all new to modules!

i think it is fine to have the makefile do either or by default, however whatever the default is should be tested by the CI also which would have caught this bug.

Yes this occurred to me too!  I have added a with modules builder now dcaee0016a73b1759af2d75b9c56f4caad077275 which effectively does a make in a non modules environment.

perfect

i am new to go, so am not a reliable source on the matter, but

I think the problem is that we are all new to modules!

i think it is fine to have the makefile do either or by default, however whatever the default is should be tested by the CI also which would have caught this bug.

Yes this occurred to me too! I have added a with modules builder now dcaee00 which effectively does a make in a non modules environment.

perfect

authors: ncw yparitcher


text:
move not removing local files

When filing an issue, please include the following information if
possible as well as a description of the problem.  Make sure you are
using the latest version of rclone.

What is your rclone version (eg output from rclone -V)
rclone v1.33-13-g7be4c3a-fuse-flagsβ

Which OS you are using and how many bits (eg Windows 7, 64 bit)
Ubuntu 16.04
Which cloud storage system are you using? (eg Google Drive)
Encrypted Amazon Cloud Drive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone move /tv_shows/ eacd:media/tv_shows -v

A log from the command with the -v flag (eg output from rclone -v copy /tmp remote:tmp)

2016/09/29 16:45:11 Encrypted amazon drive root '40vr5l7bbib53ql2gesinnck6c/lnhbggu7s6ai3oora37epq5qmk': Waiting for checks to finish
2016/09/29 16:45:11 Ink Master/folder.jpg: Sizes identical
2016/09/29 16:45:11 Ink Master/folder.jpg: Unchanged skipping
2016/09/29 16:45:11 Ink Master/logo.png: Sizes identical
2016/09/29 16:45:11 Ink Master/landscape.jpg: Sizes identical
2016/09/29 16:45:11 Ink Master/landscape.jpg: Unchanged skipping
2016/09/29 16:45:11 Ink Master/fanart.jpg: Sizes identical
2016/09/29 16:45:11 Ink Master/tvshow.nfo: Sizes identical
2016/09/29 16:45:11 Ink Master/tvshow.nfo: Unchanged skipping
2016/09/29 16:45:11 Ink Master/fanart.jpg: Unchanged skipping
2016/09/29 16:45:11 Ink Master/logo.png: Unchanged skipping
2016/09/29 16:45:11 Ink Master/S8/folder.jpg: Sizes identical
2016/09/29 16:45:11 Ink Master/S8/folder.jpg: Unchanged skipping
2016/09/29 16:45:11 Ink Master/clearart.png: Sizes identical
2016/09/29 16:45:11 Ink Master/clearart.png: Unchanged skipping
2016/09/29 16:45:11 Ink Master/S8/poster.jpg: Sizes identical
2016/09/29 16:45:11 Ink Master/S8/poster.jpg: Unchanged skipping
2016/09/29 16:45:11 Ink Master/poster.jpg: Sizes identical
2016/09/29 16:45:11 Ink Master/poster.jpg: Unchanged skipping
2016/09/29 16:45:11 Encrypted amazon drive root '40vr5l7bbib53ql2gesinnck6c/lnhbggu7s6ai3oora37epq5qmk': Waiting for transfers to finish
2016/09/29 16:45:11
Transferred:      0 Bytes (0 Bytes/s)
Errors:                 0
Checks:                 9
Transferred:            0
Elapsed time:        7.8s
2016/09/29 16:45:11 Go routines at exit 33
It had issues with uploading some of these initially but they seem to have uploaded but it will not remove them from the local directory on a second run. I noticed this on other files as well, if I did a 'move' it would find the ones in ACD and then say the size was identical and do the unchanged skipping but then leave them in the local directory.
I expected that it would remove the local copies since I did the 'move' option. Is that not the case?
└── tv_shows
└── Ink Master
├── clearart.png
├── fanart.jpg
├── folder.jpg
├── landscape.jpg
├── logo.png
├── poster.jpg
├── S8
│   ├── folder.jpg
│   └── poster.jpg
└── tvshow.nfo
Thank you!
Edit: adding -I to the command uploaded the files again and deleted the local copies.

I noticed the same behavior and believe it is intentional but undocumented that a move that doesn't actually involve a copy isn't destructive.
http://rclone.org/commands/rclone_move/
Notes that

If possible a server side move will be used, otherwise it will copy it (server side if possible) into dest:path then delete the original (if no errors on copy) in source:path.

I believe this should be

If possible a server side move will be used, otherwise it will copy it (server side if possible) into dest:path then delete the original (if the file was moved) in source:path.

To cover the case where you skip copying of the local copy because it matches the already uploaded file.

It seems like odd behavior because it's a 'move'. If it encounters the same file on the destination(size, etc match), then the intent of the 'move' should remove that file from the source.  I don't want to have to use the -I all of the time to get the behavior I need because it's forcing the re-transferring of data that is already good. It's wasting resources. Maybe we need another option that is destructive if this was done intentionally.

I don't actually see the -l argument documented either. What is the command supposed to do?
I think the closest analog is rsync with "--remove-source-files" which does function as would be expected - if the source files already exist at the destination they are removed, even if they don't need to be copied.

I found it in the rclone --help.
-I, --ignore-times                    Don't skip files that match size and time - transfer all files
so its transferring  the files again. This isnt a big deal on small jpegs but it is a big deal on Multi-GB files. This will transfer the file again and clean it up on the source.
I know the intended behavior is to leave directories behind so I clean those up with this after the move.
find /path/to/source -empty -type d -delete
I noticed it wasnt cleaning up the directories and thats when I saw that there were files left behind.

#754 updates the documentation to reflect this behavior.

I think this is a bug - move should move the files and if an identical file exists in the destination then the source file should be deleted.  This is what would happen if it was copied
Here is a demo of the problem.
$ mkdir /tmp/a /tmp/b
$ echo hellp >/tmp/a/test.txt
$ rclone -q copy /tmp/a /tmp/b
$ rclone move -v /tmp/a /tmp/b
2016/10/01 17:08:33 rclone: Version "v1.33-DEV" starting with parameters ["rclone" "move" "-v" "/tmp/a" "/tmp/b"]
2016/10/01 17:08:33 Local file system at /tmp/b: Modify window is 1ns
2016/10/01 17:08:33 Local file system at /tmp/b: Using server side directory move
2016/10/01 17:08:33 Local file system at /tmp/b: Server side directory move failed - fallback to file moves: can't copy directory - destination already exists
2016/10/01 17:08:33 Local file system at /tmp/b: Waiting for checks to finish
2016/10/01 17:08:33 test.txt: Size and modification time the same (differ by 0s, within tolerance 1ns)
2016/10/01 17:08:33 test.txt: Unchanged skipping
2016/10/01 17:08:33 Local file system at /tmp/b: Waiting for transfers to finish
2016/10/01 17:08:33
Transferred:      0 Bytes (0 Bytes/s)
Errors:                 0
Checks:                 1
Transferred:            0
Elapsed time:          0s
2016/10/01 17:08:33 Go routines at exit 3
$ rclone -q ls /tmp/a
        6 test.txt

I think the behavior should be changed as well.
At a minimum, if it is not changed, the documentation needs to be updated.
On Sat, Oct 1, 2016 at 9:10 AM Nick Craig-Wood notifications@github.com
wrote:

I think this is a bug - move should move the files and if an identical
file exists in the destination then the source file should be deleted. This
is what would happen if it was copied
Here is a demo of the problem.
$ mkdir /tmp/a /tmp/b
$ echo hellp >/tmp/a/test.txt
$ rclone -q copy /tmp/a /tmp/b
$ rclone move -v /tmp/a /tmp/b
2016/10/01 17:08:33 rclone: Version "v1.33-DEV" starting with parameters ["rclone" "move" "-v" "/tmp/a" "/tmp/b"]
2016/10/01 17:08:33 Local file system at /tmp/b: Modify window is 1ns
2016/10/01 17:08:33 Local file system at /tmp/b: Using server side directory move
2016/10/01 17:08:33 Local file system at /tmp/b: Server side directory move failed - fallback to file moves: can't copy directory - destination already exists
2016/10/01 17:08:33 Local file system at /tmp/b: Waiting for checks to finish
2016/10/01 17:08:33 test.txt: Size and modification time the same (differ by 0s, within tolerance 1ns)
2016/10/01 17:08:33 test.txt: Unchanged skipping
2016/10/01 17:08:33 Local file system at /tmp/b: Waiting for transfers to finish
2016/10/01 17:08:33
Transferred:      0 Bytes (0 Bytes/s)
Errors:                 0
Checks:                 1
Transferred:            0
Elapsed time:          0s
2016/10/01 17:08:33 Go routines at exit 3
$ rclone -q ls /tmp/a
6 test.txt
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ncw/rclone/issues/751#issuecomment-250920675, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC-KdW4HOjRxx7H-mw47Mmy1nxnooYEks5qvoYEgaJpZM4KKhjy
.

I've fixed this in this beta: http://pub.rclone.org/v1.33-30-g945f49aβ/
If you find any problems then please re-open the ticket!

Arg - don't use that beta - I brown paper bagged it!  I'll post another in a moment

http://pub.rclone.org/v1.33-31-g9ea20baβ/

authors: RXWatcher1 ncw xor-zz


text:
EPN-neoDapp

你好：
请您帮忙审核一下我们基于事例做的neo-dapp是否符合规范。
谢谢。

修改的不对
1、除了上传图片，和修改 index.html 以外，其余的文件不要更改。
2、index.html 中需要把模板的代码复制一份，删掉 d-none CSS，然后修改你的 dApp 信息，而不是直接改模板。 @nerodeveloper

source: gharchive/pull-request

authors: chenzhitong nerodeveloper

repo: ndapp/WebSite


text:
Add alias for "git pull --rebase"

:+1:

:+1:

Any concerns or why is this pull request still not merged?

Sorry, the only reason is that I don't really enjoy maintaining this project anymore. But I'll merge this one now, hope that helps!

@ndbroadbent thx for merging. Yes OK I saw see hint on the frontage. What is about a owner / maintainer change?

@ndbroadbent I'll throw myself in as a candidate for maintaining scm breeze.

authors: Tr4pSt3R ghthor jawshooah jeffbyrnes ndbroadbent tuxinaut

license: mit

license_source: bigquery


text:
Warning: Didn't find expected autogen file:

I want to start ghcid in vscode using stack. The project contains still some errors, but ghcid should start to help me correct these. I get
`Warning: Build failed, but trying to launch GHCi anyway
Configuring GHCi with the following packages: CatalogQuery, TreeTagSets, litFoundation, uniform-algebras, uniform-convenience, uniform-error, uniform-fileio, uniform-http, uniform-json, uniform-strings, uniform-time
Warning: Didn't find expected autogen file:
/home/frank/Workspace8/LitTextWork/CatalogQuery/.stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/CatalogQuery/autogen/cabal_macros.h`
When I check the directory, I find that the file missing is indeed missing. Seemingly it is only produced after a clean compile? Is it the case that ghcid can only start when the compilation is clean? if so, perhaps a warning in the read.me file would be helpful.

I've had this happen when attempting to only load a test target after a clean.
e.g. with the following I've seen Didn't find expected autopen file
ghcid --command='stack ghci myproj:test:myproj-test' --run=':main'

Changing the invocation to include the library along with the test target fixed the issue:
e.g.
ghcid --command='stack ghci myproj:lib myproj:test:myproj-test' --run=':main'

Note: you can find the list of targets with stack ide target

@andrewufrank I think if you'd ever had a clean compile it should be sufficient to start. An old cabal_macros.h is probably just fine. However, I note that all those messages seem to be coming from stack, not ghcid - since ghcid has no idea what is autogenerated and what isn't.

thank you - the issue was that I never had a clean compile and thus could not start. the workaround to produce a first compile with the empty .hs files is workable - but not very nice. Obviously the interaction between the different services is not perfect.

I think in this case it's only Stack that can do better, since the issues are coming before ghcid gets involved...

authors: andrewufrank joncfoo ndmitchell

repo: ndmitchell/ghcid


text:
Incorrect suggestion for -fglasgow-exts

{-# OPTIONS_GHC -fglasgow-exts #-}

Produces
Found:
  {-# OPTIONS_GHC -fglasgow-exts  #-}

Why not:
  {-# LANGUAGE EnableExtension ForeignFunctionInterface,
    EnableExtension UnliftedFFITypes, EnableExtension GADTs,
    EnableExtension ImplicitParams,
    EnableExtension ScopedTypeVariables, EnableExtension UnboxedTuples,
    EnableExtension TypeSynonymInstances,
    EnableExtension StandaloneDeriving,
    EnableExtension DeriveDataTypeable,
    EnableExtension FlexibleContexts,
    EnableExtension FlexibleInstances,
    EnableExtension ConstrainedClassMethods,
    EnableExtension MultiParamTypeClasses,
    EnableExtension FunctionalDependencies, EnableExtension MagicHash,
    EnableExtension PolymorphicComponents,
    EnableExtension ExistentialQuantification,
    EnableExtension UnicodeSyntax, EnableExtension PostfixOperators,
    EnableExtension PatternGuards, EnableExtension LiberalTypeSynonyms,
    EnableExtension RankNTypes, EnableExtension ImpredicativeTypes,
    EnableExtension TypeOperators, EnableExtension RecursiveDo,
    EnableExtension ParallelListComp, EnableExtension EmptyDataDecls,
    EnableExtension KindSignatures,
    EnableExtension GeneralizedNewtypeDeriving,
    EnableExtension TypeFamilies #-}

I will fix this.

Thanks a lot! Good catch and a very simple fix - perfect.

authors: mpickering ndmitchell

repo: ndmitchell/hlint


text:
Link to skills matter 2015 talk

Somewhere - not sure where. Perhaps I should have a list of Shake talks?

https://skillsmatter.com/skillscasts/6548-defining-your-own-build-system-with-shake

Shake, like Make, is a tool for writing build systems. However, unlike Make, Shake features monadic dependencies (your dependencies themselves can depend on the results of previous dependencies), polymorphic dependencies (your dependencies don't have to be files) and stable dependencies (if something rebuilds but doesn't change things that depend on it don't have to rebuild). These features make it much easier to define build systems that have accurate dependencies. Additionally, these features let you define your own custom build system, and then implement a Shake-based interpreter for it. It is often said that the easiest way to solve a problem is to define the perfect language for solving the problem, then write an interpreter. In this talk we'll see how to use Shake to apply that advice to build systems.

Done, and the FP Days talk as well.

repo: ndmitchell/shake


text:
Can't get AudioNodes editor - Nodes doesn't exist in example file

Hello @neXyon
Unfortunately, I get errors when opening your example file, and despite your addon loads fine, with no errors, I can't have nodes in the example file you provided.
Here is a screenshot:

Please, help me. I'd really like to try your nodes =)

I pushed a fix, should work now! Please reopen if it still doesn't.

Yes, it now works, thanks a lot.

repo: neXyon/audionodes


text:
build: ignore Dokia dashboard 404

It looks like this website always returns a 404, even though the page loads: https://staking.dokia.cloud/staking/near/validators
Let's ignore this error in our broken link checker

@thisisjoshford any idea why it's still failing, saying that the newly-excluded URL is broken?

superseded by #511

repo: near/docs


text:
Introduce AccountIdRef

Closes #944
Description
This PR introduces the AccountIdRef type for dealing with borrowed account IDs without unnecessarily allocating new strings.
The new type acts a lot like Path in the standard library. It wraps str and is unsized, meaning you can't create a value of type AccountIdRef, but you can create useful references: &AccountIdRef and &mut AccountIdRef.
The two types can also now be used with Cow<'_, AccountIdRef>.
&AccountIdRef supports zero-copy deserialization with serde.
Extra changes
There are some extra improvements to the account ID API:

idiomatic fallible new constructors,
constructors accept a range of "stringy" types via generics,
PartialEq and PartialOrd implementations across account id and string types.

TODO: review existing contracts and try to switch to &AccountIdRef where applicable.

[ ] examples
[ ] standards

Oh, before it is too late, let's extract near-account-id from nearcore into a standalone repo, and port all the changes there, and use it here instead of maintaining 3 copies of AccountId (the third one was in workspaces-rs, and fortunately it actually moved to use the nearcore's near-account-id, but it suffers from lack of proper semver versioning)

@frol Sounds good to me. #944 also suggested enabling static address validation, which would require us to extract some of this logic to a standalone crate anyway.

@uint I have created the repo: https://github.com/near/near-account-id, and invited you to be the maintainer. Please, set it up there (I will take care of publishing pipeline, don't worry about that part)

@uint I have created the repo: https://github.com/near/near-account-id, and invited you to be the maintainer. Please, set it up there (I will take care of publishing pipeline, don't worry about that part)

@frol Are there other types like this we'd want to pull out and make this repo a near-common one?

Closing this in favor of https://github.com/near/near-account-id/pull/2

authors: frol uint

repo: near/near-sdk-rs


text:
Implicit addresses as default for first-time account creation

This PR enables accounts creation with implicit account by default.
Exceptions include:

User is already logged in to at least one account
User is creating an account with linkdrop

The feature is currently only enabled in a mainnet_STAGING environment using the feature flag: CREATE_IMPLICIT_ACCOUNT.
More information about the feature: https://github.com/near/near-wallet/issues/2284
The flow is available for testing on the PR staging link: https://deploy-preview-2313--near-wallet-staging.netlify.app/

Looking good! Here are a few things I picked up on or have questions about:

Let's make sure copy across the board is updated to match the new designs. Right now "Protect Your Account" should read "Choose a Security Method".

@corwinharrell will do 👍

I noticed the addition of a "Start Over" button on the seed phrase confirmation screen. I don't recall if we talked about this but could you give some context on the addition?

It would be best to add back the 'start over' button because the verify seed phrase step is on the same route (state/view), so if the user clicks on the browser back button, the user will actually go back two steps instead of the beginning of the seed phrase flow. There are ways around it but thought this would be the best solution, also personally like being able to clearly start over. Can figure out a different way if you really don't like the button though.

Once the user has complete the funding step, if they have enough funds in order to pay for the txn to reserve a custom address, they should see a modal once logged in for "Add a custom address". This more actively encourages them to do this once they have sufficient funding, vs the more passive banner that sits on the side of the page atm.

It's currently set to only show the 'Add a custom address' modal if the user has funded their account with at least 0.2N - to ensure there's enough balance to create additional accounts, since the first account will be funded with 0.1N.

It would be great to begin concatenating implicit address at this point (aw34f....afw34). Makes it easier to confirm an address if the user can see the first and last few characters, and this has quickly become a standard crypto UX pattern.

Yep I agree 👍. Let's do this in a separate PR since it'll affect a lot of places.

It's currently set to only show the 'Add a custom address' modal if the user has funded their account with at least 0.2N - to ensure there's enough balance to create additional accounts, since the first account will be funded with 0.1N.

Shouldn't the minimum be 0.1N + the estimated cost of the single TXN necessary to create an additional account? Does it need to be 0.2N or can it be less?

It's currently set to only show the 'Add a custom address' modal if the user has funded their account with at least 0.2N - to ensure there's enough balance to create additional accounts, since the first account will be funded with 0.1N.

Shouldn't the minimum be 0.1N + the estimated cost of the single TXN necessary to create an additional account? Does it need to be 0.2N or can it be less?

Yes I can lower it to 0.17N:
Initial funding: 0.2N
Reserved for storage: -0.003N
Reserved for transactions: -0.05N
Gas cost to create first account: -0.002N (likely much less)
Available balance: 0.145N - which is enough to create a named account.
We'd want to give it a bit of padding to be on the safe side, but lowering the amount required to show the 'add custom address' modal to 0.17N should be safe as the user will end up with ~0.115N in available balance. I'll update it to that amount. We could also check the available balance after the first account has been created, but doing those calculations can sometimes be a bit slow, and the modal may not show right away when the user lands on the dashboard, which is suboptimal UX.

repo: near/near-wallet


text:
Fix NODE_PATH for legacy projects

Seems this doesn't matter anymore
https://github.com/nearform/node-clinic-doctor/pull/132
But ths still relates to current state
https://github.com/nearform/node-clinic/issues/148
So there might be changes done to fix NODE_PATH back, for legacy projects.

hmm it looks like the PR that supports spaces in option values in Node was only landed 6 days ago as a semver major change: https://github.com/nodejs/node/pull/24065
is the real problem here that we're overwriting NODE_PATH? we should do similar concatenation there as we're doing with NODE_OPTIONS, i think, so both user paths and our internal paths are supported.

Hi!

is the real problem here that we're overwriting NODE_PATH?

Yes, for me this was problem. Our legacy project is full of this NODE_PATH, and I was unable to run doctor on it. So the problem is overriding.

we should do similar concatenation there as we're doing with NODE_OPTIONS, i think, so both user paths and our internal paths are supported.

I think this is not about this PR, cause it dropps NODE_PATH overrdes.

Right, but we need our custom NODE_PATH for file paths with spaces in them :smile: We can support both cases if we properly concatenate them. I believe it's something like this:
let nodePath = path.join(__dirname, 'injects')
if (process.env.NODE_PATH) {
  if (process.platform === 'win32') {
    nodePath += `;${process.env.NODE_PATH}`
  } else {
    nodePath += `:${process.env.NODE_PATH}`
  }
}

Right ... custom NODE_PATH .. something like this ...

@goto-bus-stop great thanks! It works!
Related PR for bubbleprof #319.

@goto-bus-stop it is strange, same changes for bubbleprof passed all tests.
And this test fails for sampling interval checker. Moreover, it has unstable behaviour, it fails only on Mac bundles and may fail or not. As I logged there for
test('cmd - collect - data files have content ...
seems checking on direct value of not more than 20ms is not so stable check and may depend on something else there, in a cloud.

Yeah sorry, that test is flaky. You can ignore those failures. I have a PR to loosen the check in #237. Could you revert the changes to the test? Rest LGTM :D

Yeah sorry, that test is flaky. You can ignore those failures. I have a PR to loosen the check in #237. Could you revert the changes to the test? Rest LGTM :D

I'll try, thank you for explanations ^)

Published as :package: 4.0.4
Thanks!

authors: goto-bus-stop wentout

repo: nearform/node-clinic-doctor


text:
[DOC] Explain how to access the storage object

Is your feature request related to a problem? Please describe.
In the documentation on Assembly Script: https://docs.nearprotocol.com/docs/development/writing-smart-contracts we mention that all the interactions with the storage should happen via storage object, and link to the storage API docs:

Like with web servers, function calls are stateless. Any state that you want to save to the blockchain needs to be explicitly saved by interacting with the storage object.
This object provides an interface to the blockchain storage. It is a standard key-value store where keys are strings and the values can be multiple types including string, bytes, u64. Anything else needs to be first converted into these types.
See the storage docs and collections for the full reference.

We need to explain how to actually import and access the storage object. A simple example would be great.
Describe the solution you'd like
Add a simple example of a smart contract that uses raw storage (as opposed to persistent collections) and shows how to import storage and how to interface with it.

would it be sufficient to link existing examples in the docs?  or do we need a snippet of code inline here?
all of the following examples use storage, although limited to setString / getString and setBytes / getBytes

NEAR Wallet Integration
Implement an Oracle
Testing Smart Contracts

authors: SkidanovAlex amgando

repo: nearprotocol/docs


text:
Contract arguments are displayed broken

Here is the example: https://explorer.nearprotocol.com/transactions/5dxrDYuPnrNnMUkZpKfRSHiKsvcAodaziQeoDv2oy971

Called method: send_message with args {"receiver_id","app_id","message","subject","content"} in contract: @sheeet

On the dashboard and the transactions list it should be in a short form:

Called method: send_message in contract: @sheeet

and on the transaction details page:

Called method: send_message in contract: @sheeet
{base64-decoded args}

The main motivation is that the args can get really logs.
NOTE: there is no need to do anything but base64 decoding!

@frol What are your thoughts on @vgrichina idea to use hexdump to parse binary?
"makes sense to do hexdump (e.g. using https://www.npmjs.com/package/hexdump-nodejs) when args cannot be parsed as JSON"

@k06a that is totally a good idea, let's have a separate issue addressing this.

@frol me?

@k06a oops, sorry :)
@kcole16 ^^^

repo: nearprotocol/near-explorer


text:
Update speedster/README.md

Github -> GitHub

Hello @eltociear! Thanks for fixing the typo!

authors: diegofiori eltociear

repo: nebuly-ai/nebullvm


text:
Defaults for template_extensions override .coveragerc "omit"

We have "*.txt" in our .coveragerc "omit" section, but .txt files are still being included in coverage reports. Started when the template_extensions configuration option was added. We aren't using that configuration option, so it looks like the default values for it ignore "omit" in .coveragerc. I'd expect custom values to override it, but not the defaults.

Or if that's supposed to be the new behavior then the docs should make it clear.

The Django template plugin uses some existing settings from your .coveragerc file. The source=, include=, and omit= options control what template files are included in the report.

No, you are right, it should not be doing that.  Is there any chance you could give me a repo to try it with?

The repo we are using it in is private but I can make a simple version sometime this week.

Oops, sorry. Looks like we are overriding our base .coveragerc in a few of our repos, and those do not have *.txt in the omit section. So I think everything is working correctly.

authors: katylava nedbat


text:
Filter eq with reserved characters does not seem to need %22

I stumbled across a weird behavior where I could not use a filter with eq to filter by an IP-Address (e.g. 10.0.0.1).
The generated url for From("Foo").Select("*").Eq("ip", "10.0.0.1").Execute() is /rest/v1/Foo?select=%2A&vpn_ip=eq.%2210.0.0.1%22. As the PostgrestAPI states, when using reserved characters we have to use percent encoded double quotes %22 for correct processing (Source).
This library correctly identifies the reserved characters and does as the documentation states. Unfortunately the result is always empty.
Using the generated URL in a curl request confirms that an empty result is returned.
Interestingly it is possible to perform following curl request and receiving the expected single result:
/rest/v1/Foo?select=%2A&vpn_ip=eq.10.0.0.1

Note the missing " or %22 in the request above.
With some further research I could identify a different behaviour in the official postgrest-js. It appears that not all filters must recognize reserved chars. On first glance it mainly is the in filter.
If you could confirm this @nedpals I could prepare a pull request until next week.
Edit: I would open this issue in the corresponding repository if issues were possible (nedpals/postgrest-go)

hi! i haven't used supabase for a while now so i won't be able to confirm this behavior on my side (sorry about that)
feel free to send a PR if you think this is a problem that should be fixed.

@Fritte795 Found the issue exist in this file on function SanitizeParam. It originates from this file on line 208:
// Eq adds an equality filter condition to the request.
func (b *FilterRequestBuilder) Eq(column, value string) *FilterRequestBuilder {
	return b.Filter(column, "eq", SanitizeParam(value))
}

If you don't have time to get to it I can give it a shot cause I think I'll be running into this issue in the coming weeks

authors: Fritte795 nedpals whoiscarlo

repo: nedpals/supabase-go


text:
Add Class to noty element / change width of noty

Hey, I was looking for a simple way of adding some CSS style to an entire noty element, for example, to change the default width of a single noty. I did not find a very easy way to go about doing this.
My suggestion would be to create an attribute for noty's to be able to addClass (like there is for buttons) or a css attribute.
Likewise, is there an easier way to go about changing just the width of a noty?

Hi guys.  Was this ever resolved or changed?  I'm finding that I need several notification containers of different sizes.  Being able to adjust appearance via css classes would be great.  Creating a new layout each time just to change the width of the noty container seems like overkill.
Great plugin, otherwise.  Thanks.

I know this is a super old question, but I ran into the same need (expand notification window dimensions), and was able to easily solve it by adding a bit of jQuery right after Noty.show().
var n = new Noty({
   ...
   ...
   ...
});

n.show();

$('#' + n.id).css({ width: '600px' });

Hope that helps!

I know this is a super old question, but I ran into the same need (expand notification window dimensions), and was able to easily solve it by adding a bit of jQuery right after Noty.show().
var n = new Noty({
   ...
   ...
   ...
});

n.show();

$('#' + n.id).css({ width: '600px' });

Hope that helps!

Hey! I tried this but it is not helping me. Does it work with the default themes ?

It's working for me.  I'm using the bootstrap-v3 theme.
For instance, I have:-
`var n = new Noty({
theme: 'bootstrap-v3',
text: 'Are you sure you want to cancel this booking?',
layout: 'center',
animation: {
open: 'animated bounceInDown', // Animate.css class names
close: 'animated bounceOutRight' // Animate.css class names
},
modal: true,
buttons: [
Noty.button('YES', 'btn btn-success', function () {
// console.log('button 1 clicked');
n.close();
                                                                            				// Do the Cancel button postback
                                                                                // alert('hello');
                                                                                __doPostBack($('[id*=btnCancel]')[0].id, '');
                                                                                }, {id: 'button1', 'data-status': 'ok'}),

                                                                                Noty.button('NO', 'btn btn-error', function () {
                                                                                // console.log('button 2 clicked');
                                                                                n.close();
                                                                                })
                                                                                ]
                                                                                });
                                                                                n.show();

                                                                                $('#' + n.id).css({ width: '600px' });
                                                                                $('#' + n.id + ' .noty_buttons').addClass('pull-right');`

This increases the width of the confirmation prompt and also pulls the Yes/No buttons to the right-hand side.

authors: BackTrak alexington arctistic plcrewe ryanboss-iq

repo: needim/noty


text:
Cant download the datasets

The progress has always been 0%，my env is：

Do you have any suggestions？Thanks a lot！

Same problem here, Just followed the instruction as in the readme also tried with a different python version 3.9. But since @9Cirno9 seems to have the same problem I don´t think it's related to python version. After a while in 0% in just ends up with the error:

Found 489 cases to download
Dowloading case_00000...:   0%|                                                                                | 0/489 [00:00<?, ?it/s]
Failed to download case_00000. Retrying...
Dowloading case_00000...:   0%|                                                                                | 0/489 [10:36<?, ?it/s]
Traceback (most recent call last):
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\download.py", line 38, in download_case
urllib.request.urlretrieve(url, str(tmp_pth))
File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 278, in urlretrieve
raise ContentTooShortError(
urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 225683164 out of 225959569 bytes>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\User\Desktop\minerva projects\kidney_dataset\venv\Scripts\kits23_download_data-script.py", line 33, in
sys.exit(load_entry_point('kits23', 'console_scripts', 'kits23_download_data')())
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\entrypoints.py", line 9, in download_data_entrypoint
download_dataset()
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\download.py", line 77, in download_dataset
download_case(case_num, pbar)
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\download.py", line 52, in download_case
download_case(case_num, pbar, retry=False)
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\download.py", line 56, in download_case
cleanup(tmp_pth, e)
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\download.py", line 29, in cleanup
raise(e)
File "c:\users\user\desktop\minerva projects\kidney_dataset\kits23\kits23\download.py", line 38, in download_case
urllib.request.urlretrieve(url, str(tmp_pth))
File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 278, in urlretrieve
raise ContentTooShortError(
urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 225720172 out of 225959569 bytes>

Any suggestions or alternatives appreciated
Thanks in advance.

Same problem. When they work, the downloads are very very slow, but most of them fail anyway after a long period of waiting. It also always seems like only a few bytes off the total fail to download.

@neheller Same question. Any suggestion?

Same here

authors: 9Cirno9 buni-alex dt-yuhui loijilai manuel-lincbiotech


text:
Getting "The given range isn't in document" error when inserting ""

Steps to reproduce:
On http://neilj.github.io/Squire/, in Chrome:

replace the iframe and the Squire-UI div with <textarea id="foo"></textarea> (i.e., reset the editor to get a handle on it)
in the console, run:

var s = new SquireUI({replace: 'textarea#foo', height: 300});
s.insertHTML('

')

Expected:

no error

Actual:

The given range isn't in document. is logged in the console.

Using

 doesn't raise an error.
This error comes from DOMSelection.addRange and is triggered (AFAICT) when the range's start and end containers aren't "connected", i.e., "if its shadow-including root is [not] a document."
On Squire's side, addRange is called in setSelection.
(the actual error-case was naturally more complex than that but that's what I was able to reduce it to)

It sounds like you might be trying to use Squire with an iframe, and trying to write to the document before the iframe is loaded?
Not sure; SquireUI itself is an unsupported demo integration which I didn't write. If you can give a reproduction path using Demo.html in the main Squire repo I'm happy to take a look.

I am indeed using Squire inside an iframe. But the document is definitely loaded at that time. Hmm… 🤔
I can't reproduce in Demo.html right now 😞  I'll close if I find a way. (seems unlikely)

authors: Timothee neilj

repo: neilj/Squire


text:
androidx支持

项目从support迁移到androidx之后插件不生效了。

项目从support迁移到androidx之后插件不生效了。

请问您这个问题解决了吗?

authors: BennyTian Ymmmsick

repo: nekocode/JarFilterPlugin


text:
Update sbt to 1.7.0

Updates org.scala-sbt:sbt from 1.6.2 to 1.7.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "org.scala-sbt", artifactId = "sbt" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #85.

repo: nemoo/play-slick-synchronous-example


text:
获取不到地址余额

getaccountstate：
curl --data-binary '{"jsonrpc": "2.0", "id":"2", "method": "getaccountstate", "params": ["AK8yMHrrT3gZAdqE9rTcY4qDBtEaKB54HB"] }' -H 'Content-Type:application/json;' http://127.0.0.1:10332
返回结果：
{"jsonrpc":"2.0","id":"2","result":{"version":0,"script_hash":"0x21cc5d4dcdad79809838b7d1efbd85629adfe524","frozen":false,"vo
tes":[],"balances":[]}}
invokefunction：
curl --data-binary '{"jsonrpc": "2.0", "id":"2", "method": "invokefunction", "params": ["c36aee199dbba6c3f439983657558cfb67629599","balanceOf",[{"type": "Hash160","value": "24e5df9a6285bdefd1b738988079adcd4d5dcc21"}]] }' -H 'Content-Type:application/json;' http://127.0.0.1:10332
返回的balances都是null。但是区块浏览器上查询显示该地址有余额。
neo-cli没有打开钱包，接口文档上说明这两个接口不需要打开钱包。
版本：2.9

不好意思，知道问题原因了

repo: neo-project/neo-cli


text:
NEO3 neo-gui : which UI framework should we use?

Based on our discussion during the Community Assembly in Shanghai, we have reach the consensus about few things for neo-gui. And we also left some points to discuss further here on Github.
Things we have all agreed:

Targeted Users:  mainly developers & BlockChain purist
neo-gui need to be cross-platform
neo-gui should have all functions as neo-cli, and being updated along with neo-cli
UI/UX: maybe a little improvement
Tech Stack: Cross Platform UI framework + .net Core 3.0

Avalonia +  .net Core 3.0: NGD proposed it since it is close to WPF
Electron +  .net Core 3.0 :  most ppls' favorite during the NCA Discussion
ELF + .net Core 3.0: NSPCC proposed this
Wait few months for MS official cross-platform UI framework: Shargon & Belane proposed

The discussion was left are:

Which Cross Platform UI framework should we use ?
Any more proposals or suggestions?

Thank you all for all the useful inputs and discussion. @devhawk @vncoelho @belane @shargon @lock9 @Lichen9618 @Ashuaidehao @lightszero.
Let's discuss here for the remaining questions. Hopefully we can have a discussion soon so that we can move forward to develop gui for NEO3.

we can mix neogui and neocli together
maybe can open gui with "neocli --withgui"
or in neocli cmdline type "show gui"
I try to write something with electron
https://github.com/lightszero/webwindow.netcore

Vote for Electron

Vote for wait until official MS framework

Mix GUI and CLI is a good direction. We can wait for .NET Core 3.0.

What is "BlockChain purist"?

I think Blockchain users who strongly care about privacy & safety, want to owner their own data, such as wallet keys, etc. who only trust wallets or client that they can control.

mix neogui and neocli together
@erikzhang @lightszero this is a very interesting proposal. We will look more into the repo.

neo-gui should have all functions as neo-cli, and being updated along with neo-cli
Hi @wanglongfei88, I think that what we had agreed is that neo-gui should have all neo features, to be used as a reference implementation of the possible things you can do in the platform.

Note: I'm often mistaken

@erikzhang @shargon  .NET Core 3 won't support cross-platform UIs
https://devblogs.microsoft.com/dotnet/net-core-3-and-support-for-windows-desktop-applications/
They will support WPF, on Windows only 😂

I strongly suspect that Xamarin Forms will be Microsoft's long term, cross platform UI story. Just as .NET Core 3.0 had a strong focus on enabling Windows Client apps running on .NET Framework via WPF and WinForms to move to .NET Core, I suspect .NET 5 will have a strong focus on enabling managed Mobile apps running on Mono via Xamarin to move to .NET Core. That way, when .NET 5 ships, MSFT will be down to a single CLR runtime and library going forward (though .NET Framework and Mono will probably remain in long term support essentially forever - they just won't get new features).
Xamarin Forms already supports iOS, Android and Windows UWP. There is preview support for Linux (via GTK#), Mac and Windows Desktop (via WPF): https://docs.microsoft.com/en-us/xamarin/xamarin-forms/platform/other/
I don't think we have enough information to make a decision right now. I recommend we get someone to prototype building an xplat Desktop app w/ Xamarin Forms that has similar look and feel to NEO GUI and see what the experience is like.

We are going to do some research on some of recommend UI frameworks and hopefully will have a thorough compare among them early next week. We will compare the following:

Electron
Avalonia
Xamarin Forms
uno platform

@wanglongfei88 Are all of these multiplatform?

@shargon they are. But some of them only support limited platforms. We will deliver a comparison report soon .

We have done some research about the proposed UI framwork. Overall,

Xamarin and Uno are not good options for cross platform PC application.
Avalonia & Electron are better options

Avalonia has the advantage of C# for both front and back end.
Electron is more mature and better support for cross-platform PC application. But JS front-end and C# back-end may cause performance or integration issues.

@neo-project/core @devhawk @lightszero @steven1227 , what do you think ?

I think we can use Avalonia. What was the problem with Xamarin? It may need some tweaks sometimes.
I’m not in favor of using electron because we need an implementation using the neo library. I also don’t like the idea to expose the wallet using RPC because this may allow any application in the host to steal user funds.

Avalonia ++

Thanks for the research @wanglongfei88!
I still think Xamarin Forms is the best choice because of official MSFT support. But I don't have that strong an opinion - I avoid any kind of UI development if I can help it. So I'd be fine with Avalonia

https://github.com/xamarin/Xamarin.Forms/wiki/Platform-Support
As far as I know, the Xamarin Linux and WPF what we required to develop PC applications, were supported by community not MS.  So whether Xamarin or Avalonia is same for us now.

Microsoft is decoupling their WinUI stack - i.e. the native XAML stack in Windows - from the UWP SDK. Alpha release arrives next week.
I suspect this means that the long term Xamarin Forms story on Windows will be WinUI for desktop and UWP apps. It probably won't be done before Neo 3 ships - they are planning to ship a "more complete preview" in the first half of next year. However, given the audience for neo-gui, I suspect that would not be a blocker.

Avalonia+
We also think Avalonia will be a good fit for neo-gui and the whole tech stack with C# node as back-end.
If we choose Electron, JS front end will need to communicate with node via RPC which makes no different with other wallet like O3 and neon. And it is NOT a full node with User Interface as we would like it to be.

Point 1:
Avalonia is too young choose it will be dangerous。
Electron is used in many project.
we have no need to use rpc,we can make a specpipeline for communicate,like websocket.
and here is a sample:
https://github.com/lightszero/webwindow.netcore
Point  2:
Html is a good ui solution.
Avalonia is like wpf,it is good too,but avalonia do not have good toolkit for edit it.
Point 3:
write all code by C#
write code by c# and typescript
It is samething for me.

Hi @lightszero,
I really think we should use only C#, I think the most basic requirement is to make it a use case reference for neo.
Xamarin may be good because:

More mature;
Mobile compatibility;
Microsoft seems to be friendly with us ;
It is open source too so we can fix problems;

Avalonia is good because:

'100%' open source;
It works on all platforms (I have used it in the past for desktop development)
It has a simple architecture, but less UI components  (at least I have used before and it lacked some few UI customizations options)

Based on this, I think we should go for Xamarin. Microsoft can be a good partner in the future, it is a very big company, there should be many opportunities for a project like ours.

Hi @lock9,
Xamarin only runs at windows or mobile platforms now. I think the PC users are neo-gui's majority purpose, so we need choose a PC friendly tech where Xamarin is not good for it.
As discussed before, I think Avalonia is the most feasible choice with C# for the moment, Electron is the best choice without C#.

@Ashuaidehao  lets go with Avalonia them, it is ok for me

After discussion with @lightszero @wanglongfei88 @Ashuaidehao
We have developed two simple demos with Electron and Avalonia,
here is experience for your reference:

Electron
Advantage:
1.UI is much easier to implement and maintain
2.Mature framework & abundant UI elements and references
3.Front-end and back-end development can go parallelly
Disadvantage
1.Much complex structure for front-end.
2.Frond-end and back-end interaction performance is worse than Avalonia.
3.Hard to release mixing JS & C#.
4.Better to implement WebSocket server on neo system.	 (To ensure RPC security)

Avalonia
Advantage:
1.Much easier to implement plugins.
2.Frond-end and back-end interaction performance is good.
3.Less workload for back-end.
Disadvantage
1.Bad UI tools support and development-experience.
2.Not a popular framework, high barrier for community developers to join. And less tutorial and docs.
3.	Immature framework with potential bugs.

Overall, in order to implement a much better neo-gui, and get more community developers involved, we think electron will be a better choice. In addition, we can add WebSocket support for neo which is a plus.

repo: neo-project/neo-gui


text:
ActiveNode/ActiveRel helpers for moving relationships

Neo4j doesn't let you simply change the start/end nodes of a relationship.  To do this you need to delete the old one and create a new one (propagating any data stored on the relationship).  We should be able to make this easy from the Active(Node|Rel) APIs

Well, this is exactly what I need today ;-)
When do you think to implement and release it?
TIA

Good question ;)  Right now @subvertallchris and I have been focusing on a couple of big refactors of the gems for the next major release.  I would like to see this in the near future, though.
If you have time it might not even be that hard since it would be all new functionality.  Perhaps you could propose an API and we could discuss it

My dime:
IMHO the API must adhere to the KISS principle, then, starting with
rel = EnrolledIn.new(from_node: student, to_node: lesson)
#or
rel = EnrolledIn.create(from_node: student, to_node: lesson)

a solution could be
rel.redirect!(:start, new_start_node)
rel.redirect!(:end, new_end_node)

an alternate better name could be reorient
and a less misleading solution coul then be
rel.reorient_start_to!(new_start_node)
rel.reorient_end_to!(new_end_node)

Maybe end node could be also a collection, so letting the user to easily duplicate the relationships.
A good solution should be able to solve the most general problem of reorienting all the 'same kind' relationship belonging to the Current node to a new node
Imagine a property transfer: I own several different things but I want transfer all my ownerships to another person or persons or (persons and pets).
CurrentNode.[rel:RelationKind].reorient(:to|:from).aNewNode

I like that, though I might propose rewire as an alternative.  Though I might still like redirect better.
Keeping it super simple, we might just add a start_node= method to compliment the start_node method.  What do you think?
As far as making duplicates, I thing the SRP would say that it should be a separate method.  Perhaps you can call clone / dup on an ActiveRel object in a similar manner to ActiveRecord (see #clone / #dup).  That makes me realize that we should probably copy ActiveRecord in this respect for ActiveNode as well.

authors: cheerfulstoic pdipietro


text:
informant: Fix panic on HealthCheck for missing agent

Without this, got the following in the logs, while doing other cgroup-related testing.
I0423 21:58:10.558353     142 main.go:165] Running vm-informant with args [-cgroup neon-test]
I0423 21:58:10.669739    4505 main.go:41] buildInfo.GitInfo:   v0.6.0-30-g3e7297f
I0423 21:58:10.670666    4505 main.go:42] buildInfo.GoVersion: go1.20.3
I0423 21:58:10.671451    4505 main.go:95] Selected cgroup "neon-test", starting handler with config {OOMBufferBytes:209715200 MemoryHighBufferBytes:104857600 MaxUpscaleWaitMillis:20}
I0423 21:58:10.673234    4505 cgroupmanager.go:124] Initial memory.events: {Low:0 High:15140 Max:0 OOM:0 OOMKill:0}
I0423 21:58:10.674406    4505 main.go:111] No postgres file cache selected
I0423 21:58:10.676361    4505 cgroup.go:74] Total memory available for cgroup is 968151040 bytes (923.30078125 MiB). Setting cgroup memory.high to 758435840 bytes (723.30078125 MiB)
I0423 21:58:10.678046    4505 cgroupmanager.go:247] Updating cgroup memory.high to 723.30078125 MiB (758435840 bytes)
I0423 21:58:10.679988    4505 cgroup.go:85] Successfully set cgroup memory.high
I0423 21:58:10.680859    4505 main.go:129] Starting server at 0.0.0.0:10301
I0423 21:58:11.154242    4505 cgroupmanager.go:100] New memory.events: {Low:0 High:15157 Max:0 OOM:0 OOMKill:0}
W0423 21:58:11.155349    4505 cgroupmanager.go:81] Respecting minimum wait of 1s before restarting memory.events listener
I0423 21:58:11.156609    4505 cgroup.go:147] Received memory high event. Freezing cgroup
I0423 21:58:11.157575    4505 cgroup.go:160] Sending request for immediate upscaling, waiting for at most 20ms
W0423 21:58:11.177870    4505 cgroup.go:170] Timed out after 20.301145ms waiting for upscale. Thawing cgroup
I0423 21:58:11.679928    4505 cgroupmanager.go:87] Restarting memory.events listener
I0423 21:58:11.800184    4505 cgroupmanager.go:100] New memory.events: {Low:0 High:15193 Max:0 OOM:0 OOMKill:0}
W0423 21:58:11.801183    4505 cgroupmanager.go:81] Respecting minimum wait of 1s before restarting memory.events listener
I0423 21:58:11.802446    4505 cgroup.go:147] Received memory high event. Freezing cgroup
I0423 21:58:11.803461    4505 cgroup.go:160] Sending request for immediate upscaling, waiting for at most 20ms
W0423 21:58:11.823512    4505 cgroup.go:170] Timed out after 20.055166ms waiting for upscale. Thawing cgroup
I0423 21:58:12.681616    4505 cgroupmanager.go:87] Restarting memory.events listener
I0423 21:58:13.075906    4505 cgroupmanager.go:100] New memory.events: {Low:0 High:15248 Max:0 OOM:0 OOMKill:0}
W0423 21:58:13.077121    4505 cgroupmanager.go:81] Respecting minimum wait of 1s before restarting memory.events listener
I0423 21:58:13.078664    4505 cgroup.go:147] Received memory high event. Freezing cgroup
I0423 21:58:13.079847    4505 cgroup.go:160] Sending request for immediate upscaling, waiting for at most 20ms
W0423 21:58:13.100203    4505 cgroup.go:170] Timed out after 20.378035ms waiting for upscale. Thawing cgroup
I0423 21:58:13.216120    4505 handle.go:46] Received request on /health-check (client = 10.244.162.132:57170) {AgentID:8de14343-cc1e-4187-b450-56d4bf655cc4}
2023/04/23 21:58:13 http: panic serving 10.244.162.132:57170: runtime error: invalid memory address or nil pointer dereference
goroutine 64 [running]:
net/http.(*conn).serve.func1()
	/usr/local/go/src/net/http/server.go:1854 +0xbf
panic({0x1519580, 0x237b870})
	/usr/local/go/src/runtime/panic.go:890 +0x263
github.com/neondatabase/autoscaling/pkg/informant.(*State).HealthCheck(0x1?, {0x1bf3b29?, 0xc0000d90a0?}, 0xc00047bbf0?)
	/workspace/pkg/informant/endpoints.go:230 +0xb1
github.com/neondatabase/autoscaling/pkg/util.AddHandler[...].func1(0xc000417100)
	/workspace/pkg/util/handle.go:48 +0x5d2
net/http.HandlerFunc.ServeHTTP(0xc000113ce0?, {0x19018f0?, 0xc000113ce0?}, 0x16d1ed8?)
	/usr/local/go/src/net/http/server.go:2122 +0x2f
net/http.(*ServeMux).ServeHTTP(0x0?, {0x19018f0, 0xc000113ce0}, 0xc000417100)
	/usr/local/go/src/net/http/server.go:2500 +0x149
net/http.serverHandler.ServeHTTP({0x18f5538?}, {0x19018f0, 0xc000113ce0}, 0xc000417100)
	/usr/local/go/src/net/http/server.go:2936 +0x316
net/http.(*conn).serve(0xc000108630, {0x1902520, 0xc0004a4840})
	/usr/local/go/src/net/http/server.go:1995 +0x612
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3089 +0x5ed

e2e failure is due to docker rate limit. The change is correct, self-validated that it fixes the problem, etc etc. Merging anyways.

repo: neondatabase/autoscaling


text:
Optimize importing a physical backup

Before this patch, importing a physical backup followed the same path
as ingesting any WAL records:

All the data pages from the backup are first collected in the
DatadirModification object.
Then, they are "committed" to the Repository. They are written to
the in-memory layer
Finally, the in-memory layer is frozen, and flushed to disk as a
L0 delta layer file.

This was pretty inefficient. In step 1, the whole physical backup was
held in memory. If the backup is large, you simply run out of
memory. And in step 3, the resulting L0 delta layer file is large,
holding all the data again. That's a problem if the backup is larger
than 5 GB: Amazon S3 doesn't allow uploading files larger than 5 GB
(without using multi-part upload, see github issue #1910). So we want
to avoid that.
To alleviate those problems, optimize the codepath for importing a
physical backup. The basic flow is the same as before, but step 1
is optimized so that it doesn't accumulate all the data in memory,
and step 3 writes the data in image layers instead of one large delta
layer.

To test this, I used the test_import_from_vanilla test from test_runner/batch_others/test_import.py, just modified to have a lot more rows, so that the backup is > 5 GB in size. Works now, and doesn't consume a ridiculous amount of memory. There's a lot more optimization that we could do, but I think with this we will survive for now.

See https://github.com/neondatabase/neon/issues/2020. This doesn't add a new test case because generating and importing a large backup takes a long time, however.

repo: neondatabase/neon


text:
Strip useless debug sections from binaries, to make them smaller.

The rust compiler or linker (I'm not sure which) emits .debug_pubnames and .debug_pubtypes sections in the binaries. They are supposed to speed up launching a debugger, but they're obsolete. They've been superseded by .debug_names section in more recent DWARF spec and debuggers, and gdb and lldb just ignores them.
I could not find any way to prevent rustc / ldb / mold from emitting these sections in the first place. So this commit adds a hack to strip them off afterwards. This makes the binaries about 30% smaller, for no apparent downside.

The strip-step takes about 90 seconds, which is more than I hoped. Might still be worthwhile. Or maybe do this only for the larger binaries or some other such compromise.

I wasn't happy with the added 1.5 minutes, so I spent a little more time on this. I replaced the "find | xargs" commands with a little python script. It understands hard links and knows not to invoke two objcopy's on the same hardlinked file, which allows more processes to run in parallel. This runs in 30 s on my laptop now, let's see how fast it is in the CI.

1m 46s, most of the time seems to be spent on async-trait?

Related: https://github.com/neondatabase/neon/pull/2803

As discussed moving to draft

There's a new proposal on rustc to stop emitting these: https://github.com/rust-lang/compiler-team/issues/688

This was fixed in rustc 1.76.0, it no longer generates the useless .debug_pubnames and .debug_pubtypes sections (see https://github.com/rust-lang/rust/issues/48762). Closing.
(Our binaries are still huge, but for different reasons)

authors: LizardWizzard SomeoneToIgnore hlinnaka


text:
Update Rust to 1.82.0 and mold to 2.34.0

We keep the practice of keeping the compiler up to date, pointing to the latest release. This is done by many other projects in the Rust ecosystem as well.
Release notes.
Also update mold. release notes for 2.34.0, release notes for 2.34.1.
Prior update was in #8939.

We also need to bump LLVM_VERSION to 19

strange, I ran rustc --version --verbose and it showed me 18... but indeed now it shows me 19.


text:
cert-manager deployment is not in the cert-manager namespace

When the helm chart is run with --dry-run, one can see that the Deployment yaml looks like this:
# Source: cert-manager-webhook-dnsimple/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cert-manager-webhook-dnsimple
  labels:
    app: cert-manager-webhook-dnsimple
    chart: cert-manager-webhook-dnsimple-0.1.0
    release: cert-manager-webhook-dnsimple
    heritage: Helm
spec:
  replicas:
  selector:
    matchLabels:
      app: cert-manager-webhook-dnsimple
      release: cert-manager-webhook-dnsimple
  template:
    metadata:
      labels:
        app: cert-manager-webhook-dnsimple
        release: cert-manager-webhook-dnsimple
    spec:
      serviceAccountName: cert-manager-webhook-dnsimple
      containers:
        - name: cert-manager-webhook-dnsimple
          image: "neoskop/cert-manager-webhook-dnsimple:0.1.0"
          imagePullPolicy: IfNotPresent
          args:
            - --tls-cert-file=/tls/tls.crt
            - --tls-private-key-file=/tls/tls.key
          env:
            - name: GROUP_NAME
              value: "your.group.name"
          ports:
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /healthz
              port: https
          readinessProbe:
            httpGet:
              scheme: HTTPS
              path: /healthz
              port: https
          volumeMounts:
            - name: certs
              mountPath: /tls
              readOnly: true
          resources:
            {}
      volumes:
        - name: certs
          secret:
            secretName: cert-manager-webhook-dnsimple-webhook-tls

Importantly, the volume is attempting to mount from the secret named cert-manager-webhook-dnsimple-webhook-tls. This automatically gets created according to the documentation from the Certificate's secretName that's also in the yaml generated by the helm chart in --dry-run mode.
However, when I install everything, the pod remains in ContainerCreating forever, and the description of the pod says:
Events:
  Type     Reason       Age               From               Message
  ----     ------       ----              ----               -------
  Normal   Scheduled    24s               default-scheduler  Successfully assigned default/cert-manager-webhook-dnsimple-5df7648688-28h8r to kn1
  Warning  FailedMount  9s (x6 over 24s)  kubelet            MountVolume.SetUp failed for volume "certs" : secret "cert-manager-webhook-dnsimple-webhook-tls" not found

It turns out that everything in the yaml file (where appropriate) gets created in the cert-manager namespace except for the Deployment, and so the deployment can't mount that volume because the secret is in a different namespace.

After I manually edited the yaml to put the Deployment into the cert-manager namespace, the ReplicaSet wasn't able to create the pod because the service account also was not in the cert-manager namespace:
Events:
  Type     Reason        Age                From                   Message
  ----     ------        ----               ----                   -------
  Warning  FailedCreate  9s (x14 over 50s)  replicaset-controller  Error creating: pods "cert-manager-webhook-dnsimple-5df7648688-" is forbidden: error looking up service account cert-manager/cert-manager-webhook-dnsimple: serviceaccount "cert-manager-webhook-dnsimple" not found

Once I also put the service account into the cert-manager, everything started.

You might be able to get it working by overriding the chart value for the namespace as well. I think it's because here it's referenced in a lot of places: https://github.com/neoskop/cert-manager-webhook-dnsimple/blob/master/deploy/dnsimple/values.yaml#L11.
So when I tried to deploy on another namespace other  than cert-manager, I have to also override the Chart value like this (see lines 2 and 4):
helm install cert-manager-webhook-dnsimple \
    --namespace cert-manager-example \
    --set dnsimple.token=$DNSSIMPLE_TOKEN \
    --set certManager.namespace=cert-manager-example \
    --set clusterIssuer.production.enabled=true \
    --set clusterIssuer.staging.enabled=true \
    --set clusterIssuer.email=$EMAIL \
    neoskop/cert-manager-webhook-dnsimple

It's a little redundant so it would be nice if there was a way to let the Helm chart know to just use the --namespace value for namespace everywhere.

repo: neoskop/cert-manager-webhook-dnsimple


text:
Add metadata to setup.cfg for pypi

As it currently stands the repo's page on pypi doesn't include any metadata which would make it harder to find the project as well as displaying no information about it on the website.
This adds classifiers and keywords to help the project be easier to find as well as adding the README to it's description.
I also added some whitespace to one of the docstrings because building the docs were still generating a few errors, and a few simple additions to the gitignore that I needed while working on this.

Why was this closed? It doesn't look resolved to me.

https://github.com/neovim/pynvim/pull/459
Reopened in a different pull request

authors: bfredl farisachugthai

repo: neovim/pynvim


text:
Cannot use # in command name definition

Example on Python 3:
import neovim

@neovim.plugin
class MyPlugin:
    _prefix = 'MyPlugin#'
    def __init__(self, vim):
        self.vim = vim

    @neovim.command(_prefix + 'myCommand', range='', nargs='*', sync=True)
    def my_command(self):
        pass

On a related note, _prefix = 'MyPlugin"#"' Makes the command to be defined as MyPlugin.

but is it really a valid command name? :command MyPlugin#myCommand gives an error in nvim.

OK, I know what has confused me. I was reading the omnisharp-vim README and there was a line vnoremap <leader><space> :call OmniSharp#GetCodeActions('visual')<cr> and I got mixed up.
If I understand correctly, this is calling a function and my question then would be how to define such function?.
Thanks for pointing it out and sorry for the noise.

authors: aignas bfredl

repo: neovim/python-client


text:
Validate fn added to support the interface lib

Added a set of function to add the validation of the interface objects. Used in the interface lib as part of the arm functions

/approve
/lgtm

authors: henderiw johnbelamaric

repo: nephio-project/api


text:
Please add the trained model to the fashion mnist problem.

as above

Added

repo: neptune-ml/minerva


text:
permission denied for /tmp/config.yml

Hello There,
I got into a weird issue. While running the below command.
docker run -d --rm -v $PWD/credentials:/exporter/.aws/credentials -v $PWD/config.yml:/tmp/config.yml  -p 5000:5000 --name yace quay.io/invisionag/yet-another-cloudwatch-exporter:v0.25.0-alpha -debug -config.file /tmp/config.yml
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. Trying to pull quay.io/invisionag/yet-another-cloudwatch-exporter:v0.25.0-alpha... Getting image source signatures Copying blob 1d7f164cde91 done   Copying blob 1e044a4e9ff2 done   Copying blob d10629ff8eaa done   Copying blob 801bfaa63ef2 done   Copying config e379306e35 done   Writing manifest to image destination Storing signatures 13cb194fade9830ec2e3774be96af15a58dc3c644295331b820967ffcd1fb235
While running the docker
docker run e379306e3598 Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. {"level":"info","msg":"Parse config..","time":"2021-03-08T12:01:17Z"} {"level":"fatal","msg":"Couldn't read /tmp/config.yml: open /tmp/config.yml: no such file or directory","time":"2021-03-08T12:01:17Z"}
But I already have a file named config.yml in /tmp
ll /tmp/config.yml  -rw-r--r--. 1 root root 210 Mar  8 12:55 /tmp/config.yml
The other error I'm facing is
docker run -ti -p 5000:5000 -v /home/ec2-user/git/yet-another-cloudwatch-exporter/credentials:/exporter/.aws/credentials -v /home/ec2-user/git/yet-another-cloudwatch-exporter/yace-config.yml:/tmp/config.yml e379306e3598 Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. {"level":"info","msg":"Parse config..","time":"2021-03-08T11:57:07Z"} {"level":"fatal","msg":"Couldn't read /tmp/config.yml: open /tmp/config.yml: permission denied","time":"2021-03-08T11:57:07Z"}
Please help me resolving this issue. Thanks

My bad, wrong config path, working fine

I am trying to run ECS task with YACE image. I also get same error. I am mounting EFS share which has config.yaml file with permission 777. I tried to change path to /mnt too but same permission denied error. Could you please let me know what path should be used to mount?

My bad, wrong config path, working fine
@akhilrepo  I am trying to run ECS task with YACE image. I also get same error. I am mounting EFS share which has config.yaml file with permission 777. I tried to change path to /mnt too but same permission denied error. Could you please let me know what path should be used to mount?


text:
How to use gaussian splatting code

I'm trying to run the gaussian-splatting branch with the mip360 data and the following command:
ns-train gaussian-splatting --vis wandb --experiment-name gsplats colmap --colmap_path sparse/0 --data data/mip360/bicycle

but I hit the error:
File "/home/ubuntu/nerfstudio/nerfstudio/models/gaussian_splatting.py", line 475, in get_outputs
    assert camera.shape[0] == 1, "Only one camera at a time"

Could you please provide an example how to use the code?

Currently the eval pipeline isn't supported (wandb/tensorboard), only the live viewer. If you try with the viewer does it work? We're working on integrating the rest of the eval pipelines still.

thanks for the quick reply
with your suggestion I get farther but I get an other error:
140 (0.47%)         21.996 ms            10 m, 56 s           31.26 M
150 (0.50%)         16.357 ms            8 m, 8 s             40.02 M
160 (0.53%)         15.673 ms            7 m, 47 s            41.53 M
170 (0.57%)         15.583 ms            7 m, 44 s            41.86 M
180 (0.60%)         15.556 ms            7 m, 43 s            41.93 M
190 (0.63%)         17.369 ms            8 m, 37 s            38.20 M
----------------------------------------------------------------------------------------------------
Viewer Beta at: https://viewer.nerf.studio/versions/23-05-15-1/?websocket_url=ws://localhost:7007
Printing profiling stats, from longest to shortest duration in seconds
Trainer.train_iteration: 0.0223
VanillaPipeline.get_train_loss_dict: 0.0146
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/bin/ns-train", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/ubuntu/nerfstudio/nerfstudio/scripts/train.py", line 262, in entrypoint
    main(
  File "/home/ubuntu/nerfstudio/nerfstudio/scripts/train.py", line 247, in main
    launch(
  File "/home/ubuntu/nerfstudio/nerfstudio/scripts/train.py", line 189, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "/home/ubuntu/nerfstudio/nerfstudio/scripts/train.py", line 100, in train_loop
    trainer.train()
  File "/home/ubuntu/nerfstudio/nerfstudio/engine/trainer.py", line 252, in train
    loss, loss_dict, metrics_dict = self.train_iteration(step)
  File "/home/ubuntu/nerfstudio/nerfstudio/utils/profiler.py", line 127, in inner
    out = func(*args, **kwargs)
  File "/home/ubuntu/nerfstudio/nerfstudio/engine/trainer.py", line 472, in train_iteration
    _, loss_dict, metrics_dict = self.pipeline.get_train_loss_dict(step=step)
  File "/home/ubuntu/nerfstudio/nerfstudio/utils/profiler.py", line 127, in inner
    out = func(*args, **kwargs)
  File "/home/ubuntu/nerfstudio/nerfstudio/pipelines/base_pipeline.py", line 301, in get_train_loss_dict
    loss_dict = self.model.get_loss_dict(model_outputs, batch, metrics_dict)
  File "/home/ubuntu/nerfstudio/nerfstudio/models/gaussian_splatting.py", line 603, in get_loss_dict
    Ll1 = torch.abs(gt_img - outputs["rgb"]).mean()
RuntimeError: The size of tensor a (411) must match the size of tensor b (410) at non-singleton dimension 0

Try pulling from the repo, there's a commit that might fix this there.

cool, it works now (you are working around the clock, I did pull 1h ago also) :)

Great thanks!

stupid question: the training is complete, I only have a check point file, how do I see the GS model?

Hi @cdcseacave @kerrj
I wonder if there is anyway that I can render a circular video without the online viewer to create camera-path?

stupid question: the training is complete, I only have a check point file, how do I see the GS model?

You probably should load the checkpoint file and parse its contents in a .ply file. Ckpt contains all necessary information about gaussians (mean, covariances, SHs, opacity)

stupid question: the training is complete, I only have a check point file, how do I see the GS model?

You probably should load the checkpoint file and parse its contents in a .ply file. Ckpt contains all necessary information about gaussians (mean, covariances, SHs, opacity)

Btw how to export it into Pointcloud ?

@hariharan1412 check the ns-export command and relevant code in the pr

Thanks I exported successfully, I have some doubts regarding the outputs, for the same sparse from the colmap the NERF output is very good but GS output is not, The GS output is yellow and green only. to add on the NERF output is 250MB but GS output is only 13Mb. Do you have any idea why this is happening ? ? It'll be very helpfull for me

@hariharan1412 check the source code for the exporter from nerf, also compare the number of points in both ply files. The nerf one most likely just exports more points than what the GS model has, so this results in a better looking pc.

authors: ChongjianGE antonzub99 cdcseacave hariharan1412 kerrj maturk

repo: nerfstudio-project/nerfstudio


text:
feat(rpc): use bigint instead of number in the interfaces of rpc methods

allow bigit and forbid number type
BREAKING CHANGE: use bigint instead of number in the interfaces of rpc methods

Codecov Report

Merging #357 into develop will increase coverage by <.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop     #357      +/-   ##
===========================================
+ Coverage    99.05%   99.05%   +<.01%
===========================================
  Files           16       16
  Lines          530      531       +1
  Branches       150      149       -1
===========================================
+ Hits           525      526       +1
  Misses           1        1
  Partials         4        4

Impacted Files
Coverage Δ

packages/ckb-sdk-rpc/src/paramsFormatter.ts
98.18% <100%> (+0.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02e07d2...c8d994b. Read the comment docs.

authors: Keith-CY codecov-io

repo: nervosnetwork/ckb-sdk-js


text:
perf: optimize molecule usage

The principle here is to try to avoid Entity::from_slice or Reader::to_entity if possible, because they need to allocate and copy memory.

Replace Box<[u8]> -> as_ref -> Reader::from_slice_should_be_ok -> to_entity with from_box_should_be_ok!, which is from_slice_should_be_ok + into -> new_unchecked.

Replace Entity::from_slice -> as_reader with Reader::from_slice.

Replace to_entity -> unpack with unpack.

Pls rebase PR

Pls rebase PR

Done.
Did you leave a comment? I force-pushed to the branch too fast and couldn't see it any more.

authors: jjyr sopium

repo: nervosnetwork/godwoken


text:
Sysbox Nesting (future support?)

I understand that sysbox nesting is currently a limitation.
Is this a feature that is ever planned on being supported in the future?

I'm aware of some solutions like this Y Combinator comment. However, this comes with it's drawbacks as mentioned in the well known "Using Docker-in-Docker for your CI or testing environment? Think twice." blog post.
I can see this being a common use case such as for a CI server:

You run your CI server in a docker container.
Each build job runs inside a container
The build jobs would like to do things like build, publish, etc.

For this case, 3 levels deep may be adequate, but I'm sure there are less common but valid use cases which would involve many more levels of nesting.

Thanks @ScottG489 for filing the issue.
I think I see your point here:

The CI server would run in a container deployed with Docker + Sysbox. This way the CI server can have a dedicated docker daemon yet be well isolated from the underlying host.

Inside that CI server, the jobs would run in containers. But some of those jobs may in turn need a docker daemon (e.g., to build, publish, run containers), so you want docker-in-docker inside the CI server, but want to avoid using privileged containers. That is, inside the CI server container, you want Docker + Sysbox installed.

Makes sense, and this would imply Sysbox itself running inside a Sysbox container (nested) as you suggested.
We don't have a Sysbox nesting in the roadmap right now. It's certainly something we've thought about, but it's unlikely we will support this at any time in the near future.
I'll label this issue as an epic though so we can track it here and see if we get more requests for this functionality.
Thanks again!

Great! Thanks for the quick reply, @ctalledo.
I believe I've found a temporary workaround. For my particular use case at least.
I'm mounting docker.sock on outer-most container. Inside this I run the containers using the sysbox runtime. All of these sysbox containers seem to be sandboxed from the outermost container as well as the host. Which all makes sense.
Do you know any potential problems or vulnerabilities this could cause? From my initial tests everything looked good.

As a side note, I'm having trouble running sysbox inside AWS. None of the AMI's I've been able to find have the shiftfs module. Do you happen to have any info on this? I'm happy to create a separate issue for this as it's bit off topic.

I'm mounting docker.sock on outer-most container. Inside this I run the containers using the sysbox runtime. All of these sysbox containers seem to be sandboxed from the outermost container as well as the host. Which all makes sense.

Just to make sure I get the config you are describing, is the following correct?

You have a host with Docker + Sysbox installed.
You are deploying the CI server is a regular container and mounting the host's docker socket into it.
The CI server is using the Docker + Sysbox on the host to deploy jobs.
The jobs are running in Sysbox containers (which isolates them from the host properly and allows them to run inner docker containers as needed).

By the way, you may have seen these already, but we have a couple of blog posts describing different CI configs with GitLab + Sysbox and Jenkins + Sysbox that may be helpful (even if they don't apply in your particular case):
https://blog.nestybox.com/2020/10/21/gitlab-dind.html
https://blog.nestybox.com/2019/09/29/jenkins.html

Regarding your first comment, yes that's all just about correct.
Just to make sure we're on the same page though, the jobs themselves are running from the CI container but since docker.sock is mounted on it from the host they essentially run on the host as you said.
I just took a look at the blog posts.
It looks like the Jenkins solution would be suffering from the same problem I'm trying to solve. Although the outer-most container is running in sysbox (so it can't interfere with the host), it's mounting docker.sock on all of the nested "job" containers. This means that these separate jobs can actually interfere with each other since they're all using the same docker daemon. Correct me if I'm wrong.

Finished reading the gitlab post. It seems that this post calls attention to the problem I mentioned above:

But there is a drawback:

For CI jobs that interact with Docker, the isolation boundary is at the system container level rather than at the job level. That is, such a CI job could easily gain control of the system container and thus compromise the GitLab runner environment, but not the underlying host.

In any case, my other problem still remains in that I'm having a hard time finding an AMI that has shiftfs so that I can use the sysbox runtime in EC2.

Just to make sure we're on the same page though, the jobs themselves are running from the CI container but since docker.sock is mounted on it from the host they essentially run on the host as you said.

Got it. I think your setup is fine, since Sysbox is giving you a strong isolation boundary between the CI jobs and the host.

In any case, my other problem still remains in that I'm having a hard time finding an AMI that has shiftfs so that I can use the sysbox runtime in EC2.

Let's move this thread to the sysbox discussion forum: https://github.com/nestybox/sysbox/discussions/121

Hi @ScottG489,

Does this functionality directly make progress on this issue?

Not really as it uses a privileged container and runs Sysbox inside. This issue calls for deploying a Sysbox container and running Sysbox inside, which is a significantly more difficult task (but one that we want to pursue in time).
Sysbox-in-Docker was meant to provide an easy way for folks to play around with Sysbox without having to install it on a host.

At least compared to using the docker.sock volume mount method, in terms of isolation it seems like it has the benefit of not having access to the hosts docker daemon, but all the drawbacks of using --privileged along with whatever the implications are of the other required mounts (e.g. -v /var/tmp/sysbox-var-lib-docker:/var/lib/docker, etc).

That's correct. The sysbox-in-docker image includes a Docker daemon in it, so no connection to the host Docker daemon is required. But it uses a privileged container, so it's not ideal.
Note however that inner containers launched with the inner Docker + Sysbox are strongly isolated: those use the Linux user-namespace plus all the other isolation features of Sysbox. Thus, you can run untrusted workloads inside the inner Sysbox containers, even if the outer container is privileged.
Finally, the Sysbox test framework uses the sysbox-in-docker approach (though with a dedicated test image), because it allows us to play with Sysbox easily without messing up the host.

repo: nestybox/sysbox