text:
Slack Invite CTA #281

Made CTA buttons blue on the homepage. Made Slack invitation request more prominent and above the fold on the Slack community page, created some CTA "buttons".

✅ By submitting this PR, I have verified the following

[✅ ] Checked to see if a similar PR has already been opened 🤔
[✅ ] Reviewed the contributing guidelines 🔍
[✅ ] Added my name to the bottom of the list under the Contributors section in the README.md with a link to my personal website or GitHub profile 👥

@fvcproductions  I couldn't get the css override working for the arrows after image links. But, finished everything else! Changes should be reflected in PR.

LGTM, we might replace the images to better reflect branding but this can be done later 👍
Thanks so much for your contributions! 💛

authors: KGmajor fvcproductions

repo: techqueria/website


text:
[1pt] Choose assets - group task

Choose images, text, icons, fonts.

Font: https://fonts.google.com/specimen/Lato
Icons: https://fonts.google.com/icons
Logo: https://cdn-website.partechpartners.com/media/images/Maze-logo-web.original.png (Credit: https://maze.co/).

source: gharchive/issue

repo: teckim/tv-maze

license: MIT

license_type: permissive

license_source: github-api


text:
Removed unnecessary unsafe fns.

add warn unsafe_code lint.

Is there a use-case for having the unsafe functions?
The use of unsafe keyword here is stretching it's intended use.
Using unsafe is intended to turn off memory safety checks.
But prae is using it to mean "turn off data integrity checks"

These functions are handy for performance  optimisations. If the user has a very frequent operation and can guarantee that the invariants of provided data are upheld, he can avoid the overhead of calling adjust and mutate. This was part of this discussion.
But the use of unsafe might violate its intended usage, as was also discussed in the discussion linked.  Given that some crate might have forbid(unsafe_code) and the functions clearly say that one must be cautious  using them, we can just remove unsafe keywords from them.

Perhaps they should be called _unchecked, but I'm not sure if it's clear that the functions not only don't call validate, but also don't call adjust. What do you think?

I can see someone wanting to use unchecked versions of the functions for optimization reasons. I would prefer that was behind a feature gate. As in, if I was a lead on a project and prae was included, I may want to disable the possibility of unchecked mutability.
Instead of removing the unchecked functions, let's move them into a separate trait that is behind a feature gate then if someone consciously chooses to use unchecked mutability, they need to bring the trait into scope.

Seems like a good solution. We can call the feature manual_access. Do you want to do that?

Yea I'll implement that :wink: I'm attending an all day birthday shindig today. But I'll take a break from it all at some stage and see what I can get done.

Super :)

I'd go with the name unchecked_access as that maps to the function names. manual implies that something was automatic and it now done by hand, which is a smooth fit in this case.

I added a new feature unchecked-access and an associated trait UncheckedAccess that contains the unchecked fns.
I also added tests for the unchecked fns.

Thank you!

source: gharchive/pull-request

authors: ObsceneGiraffe teenjuna

repo: teenjuna/prae

license: Unlicense


text:
Is this a rewrite of rumqtt?

We started using Rumqtt at work last year, and now I see that is has been archived and ownership is transferred here. Since this is in alpha state, is it a rewrite?

MQTT 5 is also quite interesting to us.
I'm currently trying to find the "right" MQTT client library in Rust and rumq looks promising! How production-ready do you think it is right now, or, in April? I know you have experience writing MQTT libs but I don't want to just build something on top of rumq if you can tell me right now it'll in an experimental phase for a while :)
I'm not sure how much time I'll have to contribute to the implementation, but I'll try. My alternative right now would be to update the mosquitto binding, fyi.

How production-ready do you think it is right now, or, in April

I'm not sure of the robustness as this library hasn't seen much use in production yet. But the design came out good IMO. We are going to deploy this for a few partners we are working with closely in April. I expect things to get stable by the end of May.
But the test suite is getting ready. And I'm willing to work closely with people trying to use this in production and fix issues on priority (If they let me know that they have a production use case :) )

authors: Progdrasil killercup tekjar

repo: tekjar/rumq


text:
Github secret missing causes error could not read Username for 'https://github.ibm.com': No such device or address in build step of the tekton pipelines.

Expected Behavior
need better error message when missing github secret.
Actual Behavior
When a user forgets to either miss to do below 3 things for the setup of Github secret prior to running any pipeline the build step in pipeline fails
Error snippet :
{"level":"warn","ts":1568057939.538,"logger":"fallback-logger","caller":"logging/config.go:69","msg":"Fetch GitHub commit ID from kodata failed: \"KO_DATA_PATH\" does not exist or is empty"}
{"level":"error","ts":1568057939.8545134,"logger":"fallback-logger","caller":"git/git.go:35","msg":"Error running git [fetch --depth=1 --recurse-submodules=yes origin fc7fe7fb8d87779dd5419b509dd2c91e63ba87b7]: exit status 128\nfatal: could not read Username for 'https://github.ibm.com': No such device or address\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:35\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:88\nmain.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:36\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}
{"level":"error","ts":1568057940.1771348,"logger":"fallback-logger","caller":"git/git.go:35","msg":"Error running git [pull --recurse-submodules=yes origin]: exit status 1\nfatal: could not read Username for 'https://github.ibm.com': No such device or address\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:35\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:91\nmain.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:36\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}
{"level":"warn","ts":1568057940.1772096,"logger":"fallback-logger","caller":"git/git.go:92","msg":"Failed to pull origin : exit status 1"}
{"level":"error","ts":1568057940.1798232,"logger":"fallback-logger","caller":"git/git.go:35","msg":"Error running git [checkout fc7fe7fb8d87779dd5419b509dd2c91e63ba87b7]: exit status 128\nfatal: reference is not a tree: fc7fe7fb8d87779dd5419b509dd2c91e63ba87b7\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:35\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:94\nmain.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:36\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}
{"level":"fatal","ts":1568057940.179904,"logger":"fallback-logger","caller":"git-init/main.go:37","msg":"Error fetching git repository: exit status 128","stacktrace":"main.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:37\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}

Step failed

This type of error can occur for a set of related reasons:

The user has created a secret for github but not patched that secret onto the service account used by the PipelineRun or TaskRun in question, or
The user has not created a secret for github, but has tried to patch the service account, or
The user has not created a secret for github and has not patched the service account

Steps to Reproduce the Problem

Additional Info
We were able to reproduce it when we have our project in github.ibm.com and we miss to setup github secret for it in the cluster before running the pipelinerun. If user miss to do that you get the error shown in description of this issue.
After investigation we found it comes from Tektoncd code.
Below is the direct link to the Tekton code from where the error is coming which we see after the failure of the pipeline.
https://github.com/tektoncd/pipeline/blob/master/pkg/git/git.go#L35
The problem is the error is not showing clear message that github secret is missing.

This issue was created as a output of other issue in pipelines repository (https://github.com/kabanero-io/kabanero-pipelines)
Below is the reference issue:
https://github.com/kabanero-io/kabanero-pipelines/issues/54

@aadeshpa
hi Ashish
so did you finally fixed this issue?
I got the same error ...
It seems that Tekton only support the secret for github.com gitlab.com , not support the private github

/kind documentation

@aadeshpa
hi Ashish
so did you finally fixed this issue?
I got the same error ...
It seems that Tekton only support the secret for github.com gitlab.com , not support the private github

It is definitely possible to clone git repos from github.ibm.com, this works fine for me:
apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  generateName: use-github-ibm-
spec:
  taskSpec:
    inputs:
      resources:
      - name: source
        type: git
    steps:
    - name: list-git
      image: busybox
      script: |
        #!/bin/sh
        set -ex
        ls $(inputs.resources.source.path)
  inputs:
    resources:
    - name: source
      resourceSpec:
        params:
        - name: url
          value: git@github.ibm.com:Andrea-Frittoli/beamer_template.git
        - name: revision
          value: master
        type: git

and

Looking at the initial log in this ticket, I see that the URL is an anonymous only one: https://github.ibm.com.  Could you try git@github.ibm.com or git+ssh://github.ibm.com/ instead?

@afrittoli : thank you for the details, however we were able to get it working once we created a secret for github.ibm.com with our account , however did not came back and updated this issue . Will mark this issue as closed

authors: BinWayne aadeshpa afrittoli sbwsg

repo: tektoncd/pipeline

license_source: bigquery


text:
variable substitution in finally section

Changes
Variables under params section were not substituted with the values for
tasks in finally section. Fixing the parameter interpolation for final tasks.
Fixes #2906
/cc @bobcatfish @vdemeester
/kind bug
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[x] Includes tests (if functionality changed/added)
[ ] Includes docs (if user facing)
[x] Commit messages follow commit message best practices
[x] Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
Variables from params section are now substituted with specified values for final tasks.

how come go coverage stats are missing? 🤔

/approve

/lgtm

@pritidesai Unfortunately it looks like the EasyCLA bot might require a kick to wake it up, which means making a null edit (adding an extra space or something to the commit message should work) and then a force-push.  Apologies for that.

how come go coverage stats are missing? 🤔

Our coverage tool is unreliable :( https://github.com/tektoncd/plumbing/issues/165 I think we should switch to using something like coveralls

/lgtm

/test pull-tekton-pipeline-build-tests

I have bitten myself in the foot a few times using append(). I know that the way @pritidesai initially wrote is safe. This may also work - but it may also introduce problems that is hard to debug since it may mutate p.Tasks
I don't have deep enough understanding about this, there are some articles that may elaborate about this, e.g. https://appliedgo.net/slices/

yup and that's one of the reasons its not directly ranging over p.Tasks instead its assigned to a new object/variable tasks:
tasks := p.Tasks
for i := range tasks {

I have been struggling to avoid append completely, will give it one more try.

yup and that's one of the reasons its not directly ranging over p.Tasks instead its assigned to a new object/variable tasks:
tasks := p.Tasks
for i := range tasks {

I have been struggling to avoid append completely, will give it one more try.

I mean, as you originally wrote:
tasks := append(tasks, p.Tasks...)
tasks := append(tasks, p.Finally...)
for i := range tasks {

is safe.
However, the optimization:
tasks := append(p.Tasks, p.Finally...)
for i := range tasks {

may (I am not sure - but I have struggled with this earlier) be problematic, as append() here "mutates" p.Tasks and even if you range tasks here, there may be problem in other places in the code - since p.Tasks is mutated now.
I don't know - but my Go experience tells me that this may be problematic.

may (I am not sure - but I have struggled with this earlier) be problematic, as append() here "mutates" p.Tasks and even if you range tasks here, there may be problem in other places in the code - since p.Tasks is mutated now.

My understanding of this is that p.Tasks will not be mutated - my understanding of slices comes and goes depending on how recently I've read https://blog.golang.org/slices but my understanding is that append returns a new slice and sometimes allocates new memory to do it, but the slice passed in as the first argument is not changed

@jlpettersson i didnt see the article you linked above (https://appliedgo.net/slices/) before replying but i think there's an important distinction:

Especially the second one can cause confusion, because after an append(), sometimes the original array has been changed, and sometimes a new array has been created, and the original one stays the same. If the original array was referenced by different parts of the code, one reference then may point to stale data.

The array afaik is the underlying storage; it might be mutated but that doesn't change the slice that was passed in as an arg.
Changes to slices can cause unexpected side effects to other slices, e.g.: https://play.golang.org/p/zY0WJztAt_z but afaik that append function could change the array underlying p.Tasks but wouldn't change the slice. (95% sure? 😅 )

I am still trying to understand as well if the original p.Tasks gets changed 😓
Wrote a sample program with three slices (slice1, slice2, and slice3) where slice3 := append(slice1, slice2...)  and this kind of append does not change the original slice slice1 but for some reason I am not 100% convinced and feel the original changes were safe 😭
https://play.golang.org/p/T28YUyMXQka

and build shows failed when its not finished 😭 :

but for some reason I am not 100% convinced and feel the original changes were safe 😭

one of the things that makes this hard is that the choice to allocate more memory or not will depend on the specifics of the machine it's being run on, so its hard to be certain about it.
afaik the underlying array can be mutated by append but not the slice (which is why append requires you to assign the new slice)
For example: https://play.golang.org/p/SmRt2_3eJVG
No matter how many times we append to the original slice, it doesn't change. The underlying memory might, because we append to it and the slices that are created as a result have to increase in size and eventually capacity, but the original is the same.

/lgtm

Couldn't resist, sorry for commenting on closed PR 🙏,  this is interesting article:
https://medium.com/@Jarema./golang-slice-append-gotcha-e9020ff37374
The changes here wouldn't be impacted since the original slice is not modified but sharing in general. It causes issues when more than one slices are created using append and one single original slice:

Results in:
initial slice:  [0 1 2 3 4 5 6 7 8 9 10]
i: [0 1 2 3 4 5 6 7 8 9 10]
j: [0 1 2 3 4 5 6 7 8 9 10 102]
g: [0 1 2 3 4 5 6 7 8 9 10 102]
h:[0 1 2 3 4 5 6 7 8 9 10 102]

All the slices j, g, and h has 102 as the last element even when j and g are appending 100 and 101 respectively.

Couldn't resist, sorry for commenting on closed PR 🙏, this is interesting article:
https://medium.com/@Jarema./golang-slice-append-gotcha-e9020ff37374

From the article

What’s the solution?

use append only to append new value to given slice, not to create new one:
someSlice = append(someSlice, newElement)

This is the pattern I follow, to avoid my earlier mistakes, and therefore suggested
tasks := append(tasks, p.Tasks...)
tasks := append(tasks, p.Finally...)

but as written now, probably works in this case - just be careful with that pattern in other situations.

well that is definitely not what i expected!! it makes sense after staring at it for a while but i think you're both right that there are more gotchas here than I realized!

It depends on how the original slice is initialized.

Specifically it depends on the size of the underlying array - when the array is initalized in this loop:
func create(iterations int) []int {
    a := make([]int, 0)
    for i := 0; i < iterations; i++ {
        a = append(a, i)
    }
    return a
}

The slice is created with size 0 so many of the append calls need to allocate more space in the array - if you look at the len and cap as it runs, in the playground the space is doubled every time more is needed, so when the loop returns, the slice has capacity 16, while len is 16.
When the slice is initialized with i := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10} both the size and the capacity are set to 11 immediately. This means that the append calls need to create a new array each time and so j, g and h are pointing at different arrays.
In the cases where the behavior is as expected, all the slices are pointing at different memory - in the cases where the behavior is not as expected, all the slices are pointing at the same memory.
https://play.golang.org/p/woTJPqN8E6Q
Also note that printing i, the original slice, continues to return the same data because the length and capacity of that slice are unchanged.
So appending to p.Tasks would not actually change p.Tasks - what it might change is memory beyond the end of the slice pointed at by p.Tasks.

If you tried to do something like:
go foo(append(p.Tasks, something))
go bar(append(p.Tasks, somethingElse))
go baz(append(p.Tasks, someOtherThing))

Then depending on the capacity of p.Tasks you might see something crazy! i.e. all 3 functions might be operating on a slice with p.Tasks and someOtherThing :O
Or if you did something like foo(append(p.Tasks, something)) and then saved the arg to some cache or something - by the time you get back to it it might have changed!

Yup I absolutely agree @bobcatfish @jlpettersson
And the way we initialize p.Tasks is through make 😞
https://github.com/tektoncd/pipeline/blob/568efa42e0482ec2991939e958ee86cfe8867459/pkg/apis/pipeline/v1beta1/zz_generated.deepcopy.go#L764-L769
ps.Tasks slice is created using make and length is set to the number of tasks without any capacity which by default sets the capacity to the length of tasks. And same for ps.Finally so append(ps.Tasks, ps.Finally...)  ends up creating new slice with new memory allocation. Making ps.Tasks using make works fine since the expectation is not updating it after its created except variable substitutions 🤔
I am trying to convert our observations here into tests to confirm pipeline is not impacted in general.
Wrote two tests with two pipelines:

one dag task and one final task in a pipeline
five dag tasks and one final tasks in a pipeline

https://github.com/tektoncd/pipeline/compare/master...pritidesai:append?expand=1#diff-c9627eaed950dc49af457b127fc6da6f
Without anything in the pipeline (which is not the ideal case), memory allocations of tasks and finally are

one dag task and on final task with Sizeof(v1beta1.PipelineTask) being 152 bytes.

DAG tasks: 0xc000454c90
Final Tasks:  0xc000454cc0
Appended slice: 0xc000431a40

five dag tasks and one final task

DAG tasks: 0xc000454e40 - 0xc000454f00
Final Tasks:   0xc000454f30
Appended slice: 0xc00042b800

Also, note the length and capacity of a new slice created using Append with 5 DAG tasks and 1 final task is set to - Length 6 and Capacity 10 (double of the length of original slice).
 go test ./pkg/apis/pipeline/v1beta1/... -tags=unit -run=TestPipeline_Copy -v
=== RUN   TestPipeline_Copy
    TestPipeline_Copy: pipeline_types_test.go:26: Tasks: [{dag-task-1 0xc000454c90 <nil> [] 0 [] <nil> [] [] nil}]
    TestPipeline_Copy: pipeline_types_test.go:27: Length of Tasks:  1
    TestPipeline_Copy: pipeline_types_test.go:28: Capacity of Tasks:  1
    TestPipeline_Copy: pipeline_types_test.go:30: Finally:  [{final-task-1 0xc000454cc0 <nil> [] 0 [] <nil> [] [] nil}]
    TestPipeline_Copy: pipeline_types_test.go:31: Length of Finally:  1
    TestPipeline_Copy: pipeline_types_test.go:32: Capacity of Finally:  1
    TestPipeline_Copy: pipeline_types_test.go:35:
    TestPipeline_Copy: pipeline_types_test.go:36: Append Tasks and Finally:  [{dag-task-1 0xc000454c90 <nil> [] 0 [] <nil> [] [] nil} {final-task-1 0xc000454cc0 <nil> [] 0 [] <nil> [] [] nil}]
    TestPipeline_Copy: pipeline_types_test.go:37: Address: 0xc000431a40
    TestPipeline_Copy: pipeline_types_test.go:38: Length:  2
    TestPipeline_Copy: pipeline_types_test.go:39: Capacity:  2
--- PASS: TestPipeline_Copy (0.00s)
=== RUN   TestPipeline_Copy_Muliple_Tasks
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:61: Tasks: [{dag-task-1 0xc000454e40 <nil> [] 0 [] <nil> [] [] nil} {dag-task-2 0xc000454e70 <nil> [] 0 [] <nil> [] [] nil} {dag-task-3 0xc000454ea0 <nil> [] 0 [] <nil> [] [] nil} {dag-task-4 0xc000454ed0 <nil> [] 0 [] <nil> [] [] nil} {dag-task-5 0xc000454f00 <nil> [] 0 [] <nil> [] [] nil}]
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:62: Length of Tasks:  5
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:63: Capacity of Tasks:  5
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:65: Finally:  [{final-task-1 0xc000454f30 <nil> [] 0 [] <nil> [] [] nil}]
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:66: Length of Finally:  1
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:67: Capacity of Finally:  1
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:70:
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:71: Append Tasks and Finally:  [{dag-task-1 0xc000454e40 <nil> [] 0 [] <nil> [] [] nil} {dag-task-2 0xc000454e70 <nil> [] 0 [] <nil> [] [] nil} {dag-task-3 0xc000454ea0 <nil> [] 0 [] <nil> [] [] nil} {dag-task-4 0xc000454ed0 <nil> [] 0 [] <nil> [] [] nil} {dag-task-5 0xc000454f00 <nil> [] 0 [] <nil> [] [] nil} {final-task-1 0xc000454f30 <nil> [] 0 [] <nil> [] [] nil}]
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:72: Address: 0xc00042b800
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:73: Length:  6
    TestPipeline_Copy_Muliple_Tasks: pipeline_types_test.go:74: Capacity:  10
--- PASS: TestPipeline_Copy_Muliple_Tasks (0.00s)
PASS
ok  	github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1	3.169s

I don't have any outcome of this research yet but wanted to share (1) the way ps.Tasks is being initialized and (2) how ps.Finally is created and allocated right after ps.Tasks (may be because the test pipeline doesn't have any thing beyond that). 🤔

Thanks for doing this research @pritidesai !
I would to be clear that I think the danger of any impact to our controllers in general is minimal - this is something that's worth keeping in mind but I think the circumstances that lead to this gotcha are unlikely. (famous last words haha XD)

I'd like to add a reassurance to everyone as well:
We can all try our best to write bug free code, but it's never going to be perfect. As much as possible we want everyone to feel safe to create contributions without having to agonize over edge cases - we have the test automation to support this. Of course we want to think things through, but we can only do so much.
I think being educated about this kind of behavior will be most useful if:

We discover a bug one day that is something like "hey it seems like something else is modifying this memory, I wonder what it could be"
If we start adding more go routines or global state. In the case of global state: we better have a really good reasons because the edge cases and problems it causes are numerous! In the case of go routines: this is something we always need to do with care anyway. No matter how easy go makes this, there is a lot of reasoning that needs to be done around accessing memory safely. (And side note, if there was anywhere to look for impacts caused by this kind of code, our existing go routines, such as our timeout handler, are probably the place to look - but in no way do I expect anyone on this PR to do that! Unless they feel like it :)

And the way we initialize p.Tasks is through make 😞

https://github.com/tektoncd/pipeline/blob/568efa42e0482ec2991939e958ee86cfe8867459/pkg/apis/pipeline/v1beta1/zz_generated.deepcopy.go#L764-L769
@pritidesai I think that might in fact be good news!
The problem with the example code above was that make was being used with an initial size of 0:

a := make([]int, 0)

In the call to make above, we're making the underlying array exactly the size that we need. I'm not sure we can guarantee it, but I think that makes the chances very good that any attempts to append to that memory will require creation of a new array. (Which is what we want, b/c changes to that new array will not impact the original one.)

Circling back here @jlpettersson you may be interested to hear that THIS VERY LINE DID END UP BITING US: https://github.com/tektoncd/pipeline/pull/3244 ( + @jerop @pritidesai )
On the upside 😅  I think we can now express why this is a problem, and it wasn't because of it mutating the underlying memory, it was because:

Sometimes append would return a new slice and sometimes it wouldn't; when it did, it would copy everything from the original to the new slice. So the resulting logic would sometimes operate on the original, and sometimes on a copy which led to inconsistent behavior - which was then hard to reason about due to slices within the object being copied such that the pointer would still point at the original memory @_@ so the resulting structure was like a hybrid (some attributes are copies and updating them does not update the original; some were pointers at the same memory and updating them DOES update the original)

whew!!!

authors: afrittoli bobcatfish jlpettersson pritidesai sbwsg


text:
Remove yaml tests

Prior to this commit, some of our examples tests were duplicated as yaml tests. The main difference between the two tests was that in examples tests, the yaml was applied via kubectl create, and in the yaml tests, the yaml was applied via ko create. However, the yaml tests no longer include PipelineRuns that use images built by Tekton, so they are redundant with the examples tests.
This commit removes yaml tests, speeding up e2e test execution.
/kind cleanup
closes https://github.com/tektoncd/pipeline/issues/6844
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

n/a Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
[x] Has Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
n/a Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
n/a Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

/lgtm

authors: JeromeJu lbernick


text:
[TEP-0137] Rework the events controller cache

Dependencies:

[ ] #6883
[ ] #6884
[ ] #6889

Once dependencies are merged, the PR will contain 1 commit only
Changes
Rework the events controller cache
Switch the cache implementation from a simple LRU to the bigcache
library which provides better performance and reduces the need
for garbage collection.
Further improve the cache by reducing the size of keys, by
adopting hashing of keys and thus storing smaller data in the cache.
Hashing also allows for detecting changes in a larger surface of
the data. While todays keys are based on the event type and
resource metadata, we can now include conditions in the equation
as well, to provide and enhanced logic for event sending.
Storing a resource metadata and its condition in the hashed keys
means we can detect immediately if no relevant change was made
in a resource, and quickly finish the reconcile cycle.
Additionally storing the event type along with the resource
metadata, means that we can detect if a specific message was
already sent for a specific resource.
To be able to rely on the cache to decide which event to send,
we need to ensure that reconcile cycles for the same resource
are never executed concurrently. To achieve this, the thread
number (or concurrency) is set to 1 in the controller.
Scalability needs can be addressed by scaling the events
controller horizontally. The sharding of resources across
instances of the reconciler is based on the resource key, so
the same resource will always be directed to the same controller
thus avoiding any cache miss.
Signed-off-by: Andrea Frittoli andrea.frittoli@uk.ibm.com
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[ ] Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
[x] Has Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[x] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

/kind misc

@afrittoli do you think we can get this in for this milestone? Or should I move it over to 0.57?

This PR has not made any progress since Pipelines v0.51 when it was initiated introduced. Clearing it from the milestone. Please add back as appropriate. Thanks!

authors: afrittoli chitrangpatel pritidesai


text:
Add frontmatter for api and watcher docs.

Part of #28

/lgtm

authors: ImJasonH wlynch

repo: tektoncd/results


text:
Implement support for rate limits from GitHub

GitHub rate limits may apply to some actions when using the API, If we hit a rate limit, it's expected that we back off from making requests and try again later.

When you have been limited, use the Retry-After response header to slow down. The value of the Retry-After header will always be an integer, representing the number of seconds you should wait before making requests again. For example, Retry-After: 30 means you should wait 30 seconds before sending more requests.
Requests that create content which triggers notifications, such as issues, comments and pull requests, may be further limited and will not include a Retry-After header in the response. Please create this content at a reasonable pace to avoid further limiting.

The GitHub documentation seems to have evolved a bit, introducing x-ratelimit-remaining and x-ratelimit-reset headers.
See: https://docs.github.com/en/rest/using-the-rest-api/best-practices-for-using-the-rest-api?apiVersion=2022-11-28#handle-rate-limit-errors-appropriately
So the proposed implementation in #238 may have to evolve. But in my own experience, I've never hit rate limits with GitHub API—Is this issue still relevant?
Anyway, now that the Cloud Foundry community is maintaining a fork at cloudfoundry-community/github-pr-resource, it would be nice that any feedback is posted there. Thanks!

authors: bgandon rickardl

repo: telia-oss/github-pr-resource


text:
Update ipc send data macro

Added updates for:
http://192.168.48.49:8080/browse/MATTER-530

Plese update N22 revision to 7f762324b576ce95c4ffdde107b9be6f0646750a

authors: andriy-bilynskyy serhiiSalamakha

repo: telink-semi/zephyr


text:
Update Readme

Links to documentation are broken (from keyword on discord)

I already added this in @brendaloya trello card, but will also link this issue to that she can reference it in the PR

authors: krasi-georgiev themandalore

repo: tellor-io/TellorMiner


text:
feat: inherit all HTML language server features

Should solve this issue https://github.com/a-h/templ/issues/498
We use this feature to allow the templ language to inherit HTML LSP features such as code completions, hovers, and outline.
I modified the package.json on my host and it's working

would be nice to have this :)

authors: Zenthae milanjrodd

repo: templ-go/templ-vscode


text:
🛑 Secret Site2 is down

In 0dc9f8c, Secret Site2 ($SECRET_SITE_2) was down:

HTTP code: 404
Response time: 112 ms

Resolved: Secret Site2 is back up in 08311a4 after 10 minutes.

repo: templain/mywatcher


text:
🛑 Secret Site2 is down

In d9c9438, Secret Site2 ($SECRET_SITE_2) was down:

HTTP code: 404
Response time: 50 ms

Resolved: Secret Site2 is back up in 1b5f97c after 10 minutes.


text:
🛑 OBOERU is down

In 33a5d9e, OBOERU ($SECRET_SITE_OBOERU) was down:

HTTP code: 0
Response time: 0 ms

Resolved: OBOERU is back up in 743c3d0 after 10 minutes.


text:
🛑 QRCODE VIEWER is down

In a61f0bc, QRCODE VIEWER (https://qrcode-viewer.onrender.com/qrcode-viewer/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: QRCODE VIEWER is back up in 781c90f after 9 minutes.


text:
Symfony\Component\Console\Exception\InvalidOptionException: The "--website_id" option does not exist.

Description

When trying to create a new tenant the application throws --website_id option does not exist.
When I was debugging this. initially it complaining about --website_id option and after clicking on step over it is throwing FileNotFoundException
Below are some lines of code from my application. Let me know if I made any mistakes here

<?php

    public static function createFrom($name, $email, $organization, $domain, $password = null, $order_id = null): Tenant
    {
        // associate the customer with a website
        try {
            $website = new Website;
            try {
                $website_db = app(WebsiteRepository::class)->create($website);
            } catch (\Exception $website_option_error) {
                dd($website_option_error);
            }
            $hostname = new Hostname;
            $baseUrl = config('app.url_base');
            $hostname->fqdn = "{$domain}.{$baseUrl}";
            $hostname_db = app(HostnameRepository::class)->attach($hostname, $website);

            $organization_db = app(OrganizationRepository::class)->create([
                'name' => $organization,
                'hostname' => $hostname_db->id,
                'website' => $website_db->id,
                'creator_name' => $name,
                'creator_email' => $email,
                'domain' => $domain,
            ]);

            // make hostname current
            app(Environment::class)->hostname($hostname);

            $admin = static::makeAdmin($website, $name, $email, $password ?: Str::random(), $gw_admin);

            return new Tenant($website, $hostname, $admin);
        } catch (\Exception $createErr) {
            dd($createErr);
            throw $createErr;
        }
    }

    ?>

Actual behavior
Unable to create a new tenant
Expected behavior
Should be able to create a new tenant.

Information

hyn/multi-tenant version: ^5.4
laravel version: ^7.0
database driver and version:
webserver software and version:  nginx version: nginx/1.16.1 (Ubuntu 19.04)
php version:

PHP 7.3.11-0ubuntu0.19.10.4 (cli) (built: Apr  8 2020 18:58:29) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.11, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.11-0ubuntu0.19.10.4, Copyright (c) 1999-2018, by Zend Technologies
    with Xdebug v2.7.2, Copyright (c) 2002-2019, by Derick Rethans

tenancy.php config
<?php

/*
 * This file is part of the hyn/multi-tenant package.
 *
 * (c) Daniël Klabbers <daniel@klabbers.email>
 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 *
 * @see https://laravel-tenancy.com
 * @see https://github.com/hyn/multi-tenant
 */

use Hyn\Tenancy\Database\Connection;

return [
    /**
     * Random key used for tenant database user password
     */
    'key' => env('TENANCY_KEY', env('APP_KEY')),

    'models' => [
        /**
         * Specify different models to be used for the global, system database
         * connection. These are also used in their relationships. Models
         * used have to implement their respective contracts and
         * either extend the SystemModel or use the trait
         * UsesSystemConnection.
         */

        // Must implement \Hyn\Tenancy\Contracts\Hostname
        'hostname' => \Hyn\Tenancy\Models\Hostname::class,

        // Must implement \Hyn\Tenancy\Contracts\Website
        'website' => \Hyn\Tenancy\Models\Website::class
    ],
    /**
     * The package middleware. Removing a middleware here will disable it.
     * You can of course extend/replace them or add your own.
     */
    'middleware' => [
        // The eager identification middleware.
        \Hyn\Tenancy\Middleware\EagerIdentification::class,

        // The hostname actions middleware (redirects, https, maintenance).
        \Hyn\Tenancy\Middleware\HostnameActions::class,
    ],
    'website' => [
        /**
         * Each website has a short random hash that identifies this entity
         * to the application. By default this id is randomized and fully
         * auto-generated. In case you want to force your own logic for
         * when you need to have a better overview of the complete
         * tenant folder structure, disable this and implement
         * your own id generation logic.
         */
        'disable-random-id' => false,

        /**
         * The random Id generator is responsible for creating the hash as mentioned
         * above. You can override what generator to use by modifying this value
         * in the configuration.
         *
         * @warn This won't work if disable-random-id is true.
         */
        'random-id-generator' => Hyn\Tenancy\Generators\Uuid\ShaGenerator::class,

        /**
         * Enable this flag in case you're using a driver that does not support
         * database username or database name with a length of more than 32 characters.
         *
         * This should be enabled for MySQL, but not for MariaDB and PostgreSQL.
         */
        'uuid-limit-length-to-32' => env('LIMIT_UUID_LENGTH_32', false),

        /**
         * Specify the disk you configured in the filesystems.php file where to store
         * the tenant specific files, including media, packages, routes and other
         * files for this particular website.
         *
         * @info If not set, will revert to the default filesystem.
         * @info If set to false will disable all tenant specific filesystem auto magic
         *       like the config, vendor overrides.
         */
        'disk' => null,

        /**
         * Automatically generate a tenant directory based on the random id of the
         * website. Uses the above disk to store files to override system-wide
         * files.
         *
         * @info set to false to disable.
         */
        'auto-create-tenant-directory' => true,

        /**
         * Automatically rename the tenant directory when the random id of the
         * website changes. This should not be too common, but in case it happens
         * we automatically want to move files accordingly.
         *
         * @info set to false to disable.
         */
        'auto-rename-tenant-directory' => true,

        /**
         * Automatically deletes the tenant specific directory and all files
         * contained within.
         *
         * @see
         * @info set to true to enable.
         */
        'auto-delete-tenant-directory' => env('AUTO_DELETE_TENANT_DIRECTORY', false),

        /**
         * Time to cache websites in minutes. Set to false to disable.
         */
        'cache' => 10,
    ],
    'hostname' => [
        /**
         * If you want the multi tenant application to fall back to a default
         * hostname/website in case the requested hostname was not found
         * in the database, complete in detail the default hostname.
         *
         * @warn this must be a FQDN, these have no protocol or path!
         */
        'default' => env('TENANCY_DEFAULT_HOSTNAME'),
        /**
         * The package is able to identify the requested hostname by itself,
         * disable to get full control (and responsibility) over hostname
         * identification. The hostname identification is needed to
         * set a specific website as currently active.
         *
         * @see src/Jobs/HostnameIdentification.php
         */
        'auto-identification' => env('TENANCY_AUTO_HOSTNAME_IDENTIFICATION', true),

        /**
         * In case you want to have the tenancy environment set up early,
         * enable this flag. This will run the tenant identification
         * inside a middleware. This will eager load tenancy.
         *
         * A good use case is when you have set "tenant" as the default
         * database connection.
         */
        'early-identification' => env('TENANCY_EARLY_IDENTIFICATION', true),

        /**
         * Abort application execution in case no hostname was identified. This will throw a
         * 404 not found in case the tenant hostname was not resolved.
         */
        'abort-without-identified-hostname' => env('TENANCY_ABORT_WITHOUT_HOSTNAME', false),

        /**
         * Time to cache hostnames in minutes. Set to false to disable.
         */
        'cache' => 10,

        /**
         * Automatically update the app.url configured inside Laravel to match
         * the tenant FQDN whenever a hostname/tenant was identified.
         *
         * This will resolve issues with password reset mails etc using the
         * correct domain.
         */
        'update-app-url' => false,
    ],
    'db' => [
        /**
         * The default connection to use; this overrules the Laravel database.default
         * configuration setting. In Laravel this is normally configured to 'mysql'.
         * You can set a environment variable to override the default database
         * connection to - for instance - the tenant connection 'tenant'.
         */
        'default' => env('TENANCY_DEFAULT_CONNECTION'),
        /**
         * Used to give names to the system and tenant database connections. By
         * default we configure 'system' and 'tenant'. The tenant connection
         * is set up automatically by this package.
         *
         * @see src/Database/Connection.php
         * @var system-connection-name The database connection name to use for the global/system database.
         * @var tenant-connection-name The database connection name to use for the tenant database.
         */
        'system-connection-name' => env('TENANCY_SYSTEM_CONNECTION_NAME', Connection::DEFAULT_SYSTEM_NAME),
        'tenant-connection-name' => env('TENANCY_TENANT_CONNECTION_NAME', Connection::DEFAULT_TENANT_NAME),

        /**
         * The tenant division mode specifies to what database websites will be
         * connecting. The default setup is to use a new database per tenant.
         * If using PostgreSQL, a new schema per tenant in the same database can
         * be setup, by optionally setting division mode to 'schema'.
         * In case you prefer to use the same database with a table prefix,
         * set the mode to 'prefix'.
         * To implement a custom division mode, set this to 'bypass'.
         *
         * @see src/Database/Connection.php
         */
        'tenant-division-mode' => env('TENANCY_DATABASE_DIVISION_MODE', 'database'),

        /**
         * The database password generator takes care of creating a valid hashed
         * string used for tenants to connect to the specific database. Do
         * note that this will only work in 'division modes' that set up
         * a connection to a separate database.
         */
        'password-generator' => Hyn\Tenancy\Generators\Database\DefaultPasswordGenerator::class,

        /**
         * The tenant migrations to be run during creation of a tenant. Specify a directory
         * to run the migrations from. If specified these migrations will be executed
         * whenever a new tenant is created.
         *
         * @info set to false to disable auto migrating.
         *
         * @warn this has to be an absolute path, feel free to use helper methods like
         * base_path() or database_path() to set this up.
         */
        'tenant-migrations-path' => database_path('migrations/tenant'),

        /**
         * The default Seeder class used on newly created databases and while
         * running artisan commands that fire seeding.
         *
         * @info requires tenant-migrations-path in order to seed newly created websites.
         * @info seeds stored in `database/seeds/tenants` need to be configured in your composer.json classmap.
         *
         * @warn specify a valid fully qualified class name.
         */
        'tenant-seed-class' => TenantDatabaseSeeder::class,
        //      eg an admin seeder under `app/Seeders/AdminSeeder.php`:
        //        'tenant-seed-class' => App\Seeders\AdminSeeder::class,

        /**
         * Automatically generate a tenant database based on the random id of the
         * website.
         *
         * @info set to false to disable.
         */
        'auto-create-tenant-database' => true,

        /**
         * Automatically generate the user needed to access the database.
         *
         * @info Useful in case you use root or another predefined user to access the
         *       tenant database.
         * @info Only creates in case tenant databases are set to be created.
         *
         * @info set to false to disable.
         */
        'auto-create-tenant-database-user' => true,

        /**
         * Set of database privileges to give to the tenant database user.
         *
         * @info Useful in case your database restricts the privileges you
         *       can set (for example AWS RDS).
         * @info These privileges are only used in case tenant database users
         *       are set to be created.
         *
         * @info null by default means "ALL PRIVILEGES". Override with a list
         *       of privileges as a string, e.g. 'SELECT, UPDATE'.
         */
        'tenant-database-user-privileges' => null,

        /**
         * Automatically rename the tenant database when the random id of the
         * website changes. This should not be too common, but in case it happens
         * we automatically want to move databases accordingly.
         *
         * @info set to false to disable.
         */
        'auto-rename-tenant-database' => true,

        /**
         * Automatically deletes the tenant specific database and all data
         * contained within.
         *
         * @info set to true to enable.
         */
        'auto-delete-tenant-database' => env('TENANCY_DATABASE_AUTO_DELETE', false),

        /**
         * Automatically delete the user needed to access the tenant database.
         *
         * @info Set to false to disable.
         * @info Only deletes in case tenant database is set to be deleted.
         */
        'auto-delete-tenant-database-user' => env('TENANCY_DATABASE_AUTO_DELETE_USER', false),

        /**
         * Define a list of classes that you wish to force onto the tenant or system connection.
         * The connection will be forced when the Model has booted.
         *
         * @info Useful for overriding the connection of third party packages.
         */
        'force-tenant-connection-of-models' => [
            //            App\User::class
        ],
        'force-system-connection-of-models' => [
            //            App\User::class
        ],
    ],

    /**
     * Global tenant specific routes.
     * Making it easier to distinguish between landing and tenant routing.
     *
     * @info only works with `tenancy.hostname.auto-identification` or identification happening
     *       before the application is booted (eg inside middleware or the register method of
     *       service providers).
     */
    'routes' => [
        /**
         * Routes file to load whenever a tenant was identified.
         *
         * @info Set to false or null to disable.
         */
        'path' => base_path('routes/tenants.php'),

        /**
         * Set to true to flush all global routes before setting the routes from the
         * tenants.php routes file.
         */
        'replace-global' => false,
    ],

    /**
     * Folders configuration specific per tenant.
     * The following section relates to configuration to files inside the tenancy/<uuid>
     * tenant directory.
     */
    'folders' => [
        'config' => [
            /**
             * Merge configuration files from the config directory
             * inside the tenant directory with the global configuration files.
             */
            'enabled' => true,

            /**
             * List of configuration files to ignore, preventing override of crucial
             * application configurations.
             */
            'blacklist' => ['database', 'tenancy', 'webserver'],
        ],
        'routes' => [
            /**
             * Allows adding and overriding URL routes inside the tenant directory.
             */
            'enabled' => true,

            /**
             * Prefix all tenant routes.
             */
            'prefix' => null,
        ],
        'trans' => [
            /**
             * Allows reading translation files from a trans directory inside
             * the tenant directory.
             */
            'enabled' => true,

            /**
             * Will override the global translations with the tenant translations.
             * This is done by overriding the laravel default translator with the new path.
             */
            'override-global' => true,

            /**
             * In case you disabled global override, specify a namespace here to load the
             * tenant translation files with.
             */
            'namespace' => 'tenant',
        ],
        'vendor' => [
            /**
             * Allows using a custom vendor (composer driven) folder inside
             * the tenant directory.
             */
            'enabled' => true,
        ],
        'media' => [
            /**
             * Mounts the assets directory with (static) files for public use.
             */
            'enabled' => true,
        ],
        'views' => [
            /**
             * Enables reading views from tenant directories.
             */
            'enabled' => true,

            /**
             * Specify a namespace to use with which to load the views.
             *
             * @eg setting `tenant` will allow you to use `tenant::some.blade.php`
             * @info set to null to add to the global namespace.
             */
            'namespace' => null,

            /**
             * If `namespace` is set to null (thus using the global namespace)
             * make it override the global views. Disable by setting to false.
             */
            'override-global' => true,
        ]
    ]
];

webserver.php config
<?php

/*
 * This file is part of the hyn/multi-tenant package.
 *
 * (c) Daniël Klabbers <daniel@klabbers.email>
 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 *
 * @see https://laravel-tenancy.com
 * @see https://github.com/hyn/multi-tenant
 */

return [
    /**
     * Apache2 is one of the most widely adopted webserver packages available.
     *
     * @see http://httpd.apache.org/docs/
     * @see https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu
     */
    'apache2' => [
        /**
         * Whether the integration with Apache2 is currently active.
         */
        'enabled' => false,

        /**
         * Define the ports of your Apache service.
         */
        'ports' => [
            /**
             * HTTP, non-SSL port.
             *
             * @default 80
             */
            'http' => 80,
            /**
             * HTTPS, SSL port.
             *
             * @default 443
             */
            'https' => 443
        ],

        /**
         * The generator taking care of hooking into the Apache services and files.
         */
        'generator' => \Hyn\Tenancy\Generators\Webserver\Vhost\ApacheGenerator::class,

        /**
         * The view that holds the vhost configuration template.
         */
        'view' => 'tenancy.generators::webserver.apache.vhost',

        /**
         * Specify the disk you configured in the filesystems.php file where to store
         * the tenant vhost configuration files.
         *
         * @info If not set, will revert to the default filesystem.
         */
        'disk' => null,

        'paths' => [
            /**
             * Location where vhost configuration files can be found.
             */
            'vhost-files' => [
                '/etc/apache2/sites-enabled/'
            ],

            /**
             * Actions to run to work with the Apache2 service.
             */
            'actions' => [
                /**
                 * Action that asserts Apache2 is installed.
                 */
                'exists' => '/etc/init.d/apache2',
                /**
                 * Action to run to test the apache configuration.
                 *
                 * @set to a boolean to force the response of the test command.
                 * @info true succeeds, false fails
                 */
                'test-config' => 'apache2ctl -t',
                /**
                 * Action to run to reload the apache service.
                 *
                 * @info set to null to disable reloading.
                 */
                'reload' => 'apache2ctl graceful'
            ]
        ]
    ],

    /**
     * Nginx webserver support.
     *
     * @see http://nginx.org
     */
    'nginx' => [
        /**
         * Whether the integration with nginx is currently active.
         */
        'enabled' => true,

        /**
         * The php sock to be used.
         */
        'php-sock' => 'unix:/var/run/php/php7.2-fpm.sock',

        /**
         * Define the ports of your nginx service.
         */
        'ports' => [
            /**
             * HTTP, non-SSL port.
             *
             * @default 80
             */
            'http' => 8080,
            /**
             * HTTPS, SSL port.
             *
             * @default 443
             */
            'https' => 443
        ],

        /**
         * The generator taking care of hooking into the nginx services and files.
         */
        'generator' => \Hyn\Tenancy\Generators\Webserver\Vhost\NginxGenerator::class,

        /**
         * The view that holds the vhost configuration template.
         */
        'view' => 'tenancy.generators::webserver.nginx.vhost',

        /**
         * Specify the disk you configured in the filesystems.php file where to store
         * the tenant vhost configuration files.
         *
         * @info If not set, will revert to the default filesystem.
         */
        'disk' => null,

        'paths' => [
            /**
             * Location where vhost configuration files can be found.
             */
            'vhost-files' => [
                '/etc/nginx/sites-enabled/'
            ],

            /**
             * Actions to run to work with the Nginx service.
             */
            'actions' => [
                /**
                 * Action that asserts nginx is installed.
                 */
                'exists' => '/etc/init.d/nginx',
                /**
                 * Action to run to test the nginx configuration.
                 *
                 * @info set to a boolean to force the response of the test command.
                 *  true succeeds, false fails
                 */
                // 'test-config' => '/etc/init.d/nginx configtest',
                'test-config' => 'sudo nginx -t',
                // 'test-config' => '',
                /**
                 * Action to run to reload the nginx service.
                 *
                 * @info set to null to disable reloading.
                 */
                // 'reload' => '/etc/init.d/nginx reload'
                'reload' => 'sudo systemctl nginx reload'
                // 'reload' => ''
            ]
        ]
    ]
];

Error log
Symfony\Component\Console\Exception\InvalidOptionException: The "--website_id" option does not exist.

Illuminate\Contracts\Filesystem\FileNotFoundException: File does not exist at path api/storage/framework/cache/data/85/5f/855f92484c8c414d36c1b25cb24876e30229cbbf

I don't see the code in your command that allows the website_id option to be passed.

@bkintanar I m not using a custom command to create a tenant. This error is happening from the multi-tenant library itself.

@bkintanar Attaching some of the screenshots taken at the time of debugging.
These might help you to understand more about this issue.

The code that has integration with mult-tenant WebsiteRepository

The tenancy:migrate command issued from the multi-tenant library itself.

Exception at Symfony Console Input

That's weird. can you show the entire app/Tenant.php file?

@bkintanar added sample repository here
Here is entire Tenant.php file

authors: bkintanar lakshmajim

repo: tenancy/multi-tenant


text:
Write essential documents for setting up, building, and executing

Bullet Points

[ ] add docs/Tutorials

[ ] init testnet with genesis setting for proposing budget(short voting period) and account balance for budget source accounts and collected budget accounts
[ ] Add some tax using ChangeParameterProposal
[ ] checking balance of a account of collected budget

[ ] update README.md
[ ] update TECHNICAL-SETUP.md

References

https://github.com/allinbits/cosmos-cash/blob/main/docs/Explanation/ADR/adr-002-docs-structure.md

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

I'm on it now 🚀

authors: dongsam kogisin

repo: tendermint/budget


text:
🛑 Beeldoverleg Website is down

In 1e7787a, Beeldoverleg Website (https://beeldoverleg.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Beeldoverleg Website is back up in 76f6c7b after 57 minutes.

repo: tendsgn/upptime


text:
error in dmrg

Hello, everyone. After i install the newest version of tenpy, i run the test, it returns an error, but i don't know what the matter. Is there anyone who also meet this problem?
PS: I have tried to replace the dmrg.py file by the older one, and it passes the test.
Also, there is  another problem that the dmrg doesn't work for BoseHubbardChain, the overlap between the initial state and the output of dmrg is 1, which means the dmrg doesn't update the wavefunction in the case of BoseHubbardChain.
Thanks,
Qicheng

thanks for your reply!
Yes, i didn't mention about the initial state, i thought the parameter "filling" is the parameter to set the total particle number, but i was wrong. Now the problem is solved.

repo: tenpy/tenpy


text:
Register

@JuliaRegistrator register

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/106384
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.6 -m "<description of version>" f6c4ec099608ff6cf30688dc901645377cfa3ac7
git push origin v0.9.6

Registration pull request updated: JuliaRegistries/General/106384
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.6 -m "<description of version>" f6c4ec099608ff6cf30688dc901645377cfa3ac7
git push origin v0.9.6

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/108018
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.7 -m "<description of version>" c618386d7585947f50acea6aa106d24cf0c2536c
git push origin v0.9.7

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/110323
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.8 -m "<description of version>" 1570cad8bfc7465c8749d2fb1e472034353a2755
git push origin v0.9.8

Registration pull request created: JuliaRegistries/General/110800
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.9 -m "<description of version>" 2627d0e300cea981ceefdcf860a59e9cef35bafc
git push origin v0.9.9

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/110852
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.10 -m "<description of version>" 97d1248ffa2df0f7783c4a6d8a8f7424372fb839
git push origin v0.9.10

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/113632
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.11 -m "<description of version>" 99c552a7072f7f913b52fe02a62362f4e6452f2f
git push origin v0.9.11

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/114526
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.12 -m "<description of version>" e42b599b7db6b0d3b949f383e123c1ef0756f5d8
git push origin v0.9.12

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/117152
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.13 -m "<description of version>" e138f2cd06dc2340887af0d46b5efeaee77978c5
git push origin v0.9.13

Registration pull request created: JuliaRegistries/General/119934
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.9.14 -m "<description of version>" 66430241a65665d50db02fa49ca0638d6c663463
git push origin v0.9.14

authors: JuliaRegistrator shinaoka


text:
Image transform operation is very slow

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
TensorFlow installed from (source or binary): Binary
TensorFlow version (use command below): 2.0.0-rc0
TensorFlow Addons installed from (source, PyPi): Pypi
TensorFlow Addons version: 0.5.0.dev20190824
Python version and type (eg. Anaconda Python, Stock Python as in Mac, or homebrew installed Python etc): Anaconda Python
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
Is GPU used? (yes/no): Yes (not required)
GPU model (if used): T4

Describe the bug
Function tensorflow_addons.image.transform_ops.transform is super slow, when compared to a ad-hoc written function in basic TF, I've obtained almost a 10-fold speedup (on CPU) ...
`@tf.function
def transform_img(images, ops):
images_channels = tf.shape(images)[-1]
images_size = tf.shape(images)[1]
batch_size = tf.shape(images)[0]
op_matrix = tf.reshape(tf.concat([ops, tf.ones((tf.shape(ops)[0], 1))], axis=1), (batch_size, 3,3))
batch_ind = tf.range(0, batch_size)
ind = tf.range(0, images_size, dtype=tf.float32) # square images
ind_x = tf.tile(ind, [images_size])
ind_y = tf.reshape(tf.tile(tf.expand_dims(ind, 1), [1, images_size]), (-1,))
indices = tf.stack([ind_y, ind_x, tf.ones_like(ind_x, dtype=tf.float32)], axis=1)
indices = tf.tile(tf.expand_dims(indices, 0), [batch_size, 1, 1])

indices_trans = tf.matmul(indices, op_matrix, transpose_b=True)[..., :2]
indices_trans = tf.round(indices_trans)
indices_trans = tf.cast(indices_trans, tf.int32)
indices_trans = tf.math.floormod(indices_trans, images_size)

indices_batch = tf.reshape(tf.tile(tf.reshape(batch_ind, (-1,1)), [1, images_size * images_size]), (-1,1))
indices_trans = tf.concat([indices_batch, tf.reshape(indices_trans, (-1,2))], axis=1)
return tf.reshape(tf.gather_nd(images, indices_trans), (batch_size, images_size, images_size, images_channels))`

Code to reproduce the issue
Link to colab reproducing the issue:
(https://colab.research.google.com/drive/1JMVU-Y58O49ltk16ZRbDySSMobzUR--m)
Other info / logs
Moreover, the above version has a further speedup when using GPU (particularly with larger batch size)
Sadly, I don't have any experience when it comes to custom C++ TF kernels, so probably can't fix it myself.

Hi @lukaszgrad, could you make the colab link shareable? Thanks!

Should work now, sorry!

Seems that it indeed speeds up a lot on colab with tfa-nightly. But when I build from source with -c opt flag, two implementations get similar performance (original kernel is a little bit faster in fact on both CPU/GPU).
@seanpmorgan, Sean, what's our compiling flag while build pip package? I see
https://github.com/tensorflow/addons/blob/master/tools/ci_build/builds/release_linux.sh, where we do not include any optimization flag. Please correct me if I'm wrong.
@lukaszgrad would you mind building from source with optimization flag and comparing it with your implementation again? Thanks.
My benchmark, avg over 10 runs:

CPU w/o opt
CPU opt
GPU w/o opt
GPU opt

original
0.922506
0.273149
0.284497
0.234114

yours
0.323355
-
0.279174
-

@WindQAQ I'm getting some errors when building from source with bazel:
ERROR: /home/lukasz/kaggle/kaggle-recursion/tfrec/addons/build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/cc_toolchain_config.bzl:1491:17: name 'CcToolchainConfigInfo' is not defined ERROR: error loading package 'build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0': Extension 'build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/cc_toolchain_config.bzl' has errors INFO: Elapsed time: 0.142s INFO: 0 processes. FAILED: Build did NOT complete successfully (0 packages loaded)     currently loading: build_deps/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0

Thanks for a quick response, any idea about my bazel compilation error? I have cuda-10.0 installed

Another error, and a quick google search gives no answers :-(
ERROR: /home/lukasz/kaggle/kaggle-recursion/tfrec/addons/tensorflow_addons/custom_ops/activations/BUILD:8:1: C++ compilation of rule '//tensorflow_addons/custom_ops/activations:gelu_op_gpu' failed (Exit 1)
gcc: error: unrecognized command line option '-nvcc_options=relaxed-constexpr'
gcc: error: unrecognized command line option '-nvcc_options=ftz=true'
Target //:build_pip_pkg failed to build
Use --verbose_failures to see the command lines of failed build steps.

Hi, @lukaszgrad could you take a try within the docker image tensorflow/tensorflow:custom-op-gpu-ubuntu16?

Closing this as we packages are now built with optimizations. Please feel free to re-open if this problem is persisting.

authors: WindQAQ facaiy lukaszgrad seanpmorgan

repo: tensorflow/addons


text:
tighten the tests for the tutorial

Some improvements to the codebase mean we should expect better results now.
See the comments in this recently merged PR that improved performance: https://github.com/tensorflow/cleverhans/pull/249

All the tensorflow tests failed. It's a little bit strange, because this passes on my desktop, even when I prevent the use of GPU. It's hard to tell how bad the failure was because the tests use assertTrue rather than assertGreater so they didn't print the failing values. I'm pushing a commit to use assertGreater so we can get more info on how it's failing on Travis.

LGTM given comments made in #249

authors: goodfeli npapernot

repo: tensorflow/cleverhans


text:
music_vae checkpoints download

Hi there!
When I'm playing with music_vae, I found that the checkpoints files are only on the Colab environment, not like other model with mag file download links. But I want to retrain the model based on existed checkpoints files, I'm just wondering if there's any chance to have a checkpoint file download link of music_vae?

Hi Liu!
The checkpoint downloads are linked from the top of the Colab. Let me know
if you have any issues!
-Adam
On Mon, Jun 4, 2018 at 9:50 AM Liu Yihong notifications@github.com wrote:

Hi there!
When I'm playing with music_vae
https://github.com/tensorflow/magenta/tree/master/magenta/models/music_vae,
I found that the checkpoints files are only on the Colab
https://g.co/magenta/musicvae-colabenvironment, not like other model
with mag file download links. But I want to retrain the model based on
existed checkpoints files, I'm just wondering if there's any chance to have
a checkpoint file download link of music_vae?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/magenta/issues/1194, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABCa6IxRqW1kZYRAdcTzAGmTy1wOtxxcks5t5WVrgaJpZM4UZb3i
.

authors: RoyLiuYihong adarob

repo: tensorflow/magenta


text:
AttributeError: 'AdafactorOptimizer' object has no attribute 'get_gradients'

Hi,
When I am trying to reproduce the adafactor experiments on en-de translation task, I encountered the following issue.
AttributeError: 'AdafactorOptimizer' object has no attribute 'get_gradients'
Could any one tell me how to use adafactor optimizer? Below is my running code:
  --data_dir=t2t_data \
  --problem=translate_ende_wmt32k \
  --model=transformer \
  --hparams_set=transformer_base \
  --hparams="batch_size=1024,optimizer=adafactor" \
  --schedule=continuous_train_and_eval \
  --output_dir=translate_ende_wmt32k_adafactor \
  --train_steps=300000 \
  --worker_gpu=10 \
  --eval_steps=100

Thanks!

fixed by commenting out line 241 and use line 245
https://github.com/tensorflow/tensor2tensor/blob/ef1fccebe8d2c0cf482f41f9d940e2938c816c78/tensor2tensor/utils/optimize.py#L241


text:
Feature request : warning for feeding unused values

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): custom
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04.4 LTS
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.4.1
Python version:  3.6.3
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce: N/A

Describe the problem
(It's not a bug. Just a feature discussion)
Probably it would be reasonable to add some kind of warnings for situations when the value you feed, is never used within computation?
Example:
a = tf.abs(2)
b = 2 * a
#a = tf.identity(a, name='a')

with tf.Session() as sess:
    print(sess.run([a, b], feed_dict={a: 3}))

In this everything works fine.  However if 3d line will be uncommented the feeded value will be useless. It can lead to hard debugging.
Probably it would we useful to add some warnings?
Thanx

Nagging Assignee @shivaniag: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 45 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 60 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 61 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 76 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 91 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 106 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 121 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 138 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 154 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 169 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 184 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 199 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 214 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 229 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @shivaniag: It has been 244 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@yurijvolkov ,
We see that you are using tf version 1.4.1, 1.x is not actively supported, please update to 2.x and let us know if you are facing same issue.Thanks!

authors: tensorflowbutler tilakrayal yurijvolkov

repo: tensorflow/tensorflow


text:
Cannot convert custom .pb file to .tflite file

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):  macOS Mojave, version 10.14.2
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.12.0
Python version: 3.6.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce:

Describe the problem
I have a custom model pre-trained in Pytorch framework. I have converted it to ONNX model, and then to TensorFlow model, so that it can be further converted to TensorFlow Lite model for deployment on mobile devices.
I used the tflite_convert command for the conversion,
tflite_convert
--graph_def_file=model.pb
--output_file=model.lite
--input_format=TENSORFLOW_GRAPH
--output_format=TFLITE
--input_arrays=0
--output_arrays=add_51 \
Yet I got the following error message:
Traceback (most recent call last):
  File "/miniconda3/bin/tflite_convert", line 11, in
    sys.exit(main())
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 412, in main app.run(main=run_main, argv=sys.argv[:1])
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 408, in run_main _convert_model(tflite_flags)
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 162, in _convert_model output_data = converter.convert()
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/lite.py", line 453, in convert **converter_kwargs)
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/convert.py", line 342, in toco_convert_impl input_data.SerializeToString())
  File "/miniconda3/lib/python3.6/site-packages/tensorflow/contrib/lite/python/convert.py", line 135, in toco_convert_protos(stdout, stderr))
RuntimeError: TOCO failed see console for info.
b"2018-12-20 17:44:31.127219: F tensorflow/contrib/lite/toco/import_tensorflow.cc:2137] Check failed: status.ok() Unexpected value for attribute 'data_format'. Expected 'NHWC'\n"
None
The custom pre-trained model is adapted from a Pytorch ResNet model. Therefore I also evaluated converting the ResNet model from the torchvision package to TensorFlow model through the ONNX framework, and further to TensorFlow Lite model. And I got the following error message:
Converting unsupported operation: PyFunc\n2018-12-20 18:09:03.357810: F tensorflow/contrib/lite/toco/import_tensorflow.cc:112] Check failed: attr.value_case() == AttrValue::kType (1 vs. 6)\n'
None
I wonder if the errors above were due to some unsupported operators from the model, probably caused from converting Pytorch model to TensorFlow model through the ONNX framework. As I can convert the pre-trained Mobilenet model (https://www.tensorflow.org/lite/models) to tflite model without error on my platform. Thanks.

How did you do this? I created a new conda environment and installed with pip the tf-nightly CPU, but the error is the same: "Check failed: status.ok() Unexpected value for attribute 'data_format'. Expected 'NHWC'
Fatal Python error: Aborted". Did you call tflite_convert in some other way?

I am getting the same error even when switching from Tensorflow GPU to CPU.

I am getting the same error even when switching from Tensorflow GPU to CPU.

Can you fix it?

Can you fix it?

One fix for dimension mismatch is to have a model like this:
class Model(nn.Module):     def __init__(self, model):         super().__init__()         self.model = model     def forward(self,x):         return self.model(x.permute(0,3,1,2))
This time I am still having :
ConverterError: TOCO failed. See console for info. 2019-04-03 14:06:45.559722: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: PyFunc 2019-04-03 14:06:45.571306: F tensorflow/lite/toco/import_tensorflow.cc:114] Check failed: attr.value_case() == AttrValue::kType (1 vs. 6)

Check failed: status.ok() Unexpected value for attribute 'data_format'. Expected 'NHWC'\n"
i am using tensorflow 1.12.0 ( ive tried nightly )
i am using cuda* as device ( which is cpu)
and still get this i cant seem to find out how to solve this where do i state that i want nhwc

I;ve created a tensorflow mode of my own with custom operator and when I'm trying to convert it to lite I get something similar like it doesn't find my custom operation (MyAdd)
I've tried both command line utilities tflite_convert & toco (with same error):
tflite_convert --output_file lite_version_model.tflite --saved_model_dir ./SavedModel --output_format TFLITE --allow_custom_ops
toco --output_file lite_model_version.tflite --saved_model_dir ./SavedModel --output_format=TFLITE

I am trying to convert .pb to .tflite using ssd mobilenet v1 pets config.
Create .pb file using this command
sudo python export_inference_graph.py --input_type image_tensor --pipeline_config_path /home/ubuntu/training/data/ssd_mobilenet_v1_pets.config --trained_checkpoint_prefix /home/ubuntu/training/data/model.ckpt-78386 --output_directory /home/ubuntu/training/ --input_shape 1,300,300,3
then summarize the frozen_inference_graph.pb
its return this output
Found 1 possible inputs: (name=image_tensor, type=uint8(4), shape=[1,300,300,3])
No variables spotted.
Found 4 possible outputs: (name=detection_boxes, op=Identity) (name=detection_scores, op=Identity) (name=detection_classes, op=Identity) (name=num_detections, op=Identity)
Found 6130346 (6.13M) const parameters, 0 (0) variable parameters, and 187 control_edges
Op types used: 1781 Const, 255 GatherV2, 230 Identity, 224 Reshape, 207 Minimum, 164 Maximum, 116 Slice, 113 Cast, 103 Mul, 99 Sub, 95 ConcatV2, 88 Greater, 82 Where, 82 Split, 72 Add, 63 Pack, 63 StridedSlice, 50 Shape, 50 Unpack, 45 ExpandDims, 43 Squeeze, 41 ZerosLike, 41 NonMaxSuppressionV3, 39 Fill, 37 Tile, 35 Relu6, 35 FusedBatchNorm, 34 Conv2D, 33 RealDiv, 21 Switch, 16 Range, 13 DepthwiseConv2dNative, 12 BiasAdd, 6 Merge, 6 Sqrt, 3 Assert, 3 Equal, 3 Transpose, 2 Exp, 1 All, 1 TopKV2, 1 Size, 1 Sigmoid, 1 Placeholder
To use with tensorflow/tools/benchmark:benchmark_model try these arguments:
bazel run tensorflow/tools/benchmark:benchmark_model -- --graph=/home/jayasri/Downloads/ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18/original/frozen_inference_graph.pb --show_flops --input_layer=image_tensor --input_layer_type=uint8 --input_layer_shape=1,300,300,3 --output_layer=detection_boxes,detection_scores,detection_classes,num_detections
Then trying to convert tflite using bazel-bin/tensorflow/lite/toco/toco
But I got this error
tensorflow/lite/toco/import_tensorflow.cc:1336] Converting unsupported operation: Size
2019-05-29 15:46:02.589145: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 2780 operators, 4997 arrays (0 quantized)
2019-05-29 15:46:02.794908: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After Removing unused ops pass 1: 2741 operators, 4915 arrays (0 quantized)
2019-05-29 15:46:03.070306: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 2741 operators, 4915 arrays (0 quantized)
2019-05-29 15:46:03.162066: F tensorflow/lite/toco/graph_transformations/resolve_constant_slice.cc:59] Check failed: dim_size >= 1 (0 vs. 1)
Still i can't find the solution

I also meet this problem: F tensorflow/lite/toco/import_tensorflow.cc:1994] Check failed: status.ok() Unexpected value for attribute 'data_format'. Expected 'NHWC'. I have tried tf-cpu, tf-gpu, tf-nightly-gpu, but I failed. Is there a solution? Thanks.

sadly this forum is closed so expect no answers. ive had to put the small project i was working on on hold.

I have a question Can't The people who are working on tensorflow see these errors.
I have never seen any tensorflow team member commenting on these issues.
Obviously they have better knowledge about these issues

How did you do this? I created a new conda environment and installed with pip the tf-nightly CPU, but the error is the same: "Check failed: status.ok() Unexpected value for attribute 'data_format'. Expected 'NHWC'
Fatal Python error: Aborted". Did you call tflite_convert in some other way?

Excuse me,Do you solve it?

Hi,
I am sorry no i didnt. i was just working on this a few months ago as a
personal project. So there wasnt any reason to actually get it working.
Good luck & Best wishes.
On Mon, Dec 21, 2020 at 1:38 AM YIQIHYM notifications@github.com wrote:

How did you do this? I created a new conda environment and installed with
pip the tf-nightly CPU, but the error is the same: "Check failed:
status.ok() Unexpected value for attribute 'data_format'. Expected 'NHWC'
Fatal Python error: Aborted". Did you call tflite_convert in some other
way?
Excuse me,Do you solve it?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/24491#issuecomment-748770065,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEVTVVNWFS7NZUNNW2NH7RTSV3NOXANCNFSM4GLVBRMQ
.

I also meet the same problem,and i try all the ways in this page,but it's useless.
finally,i try this way and then solved the problem.

The main solution is use tf.transpose change the NCHW to NHWC.

I also meet the same problem,and i try all the ways in this page,but it's useless.
finally,i try this way and then solved the problem.

The main solution is use tf.transpose change the NCHW to NHWC.

The link is not working.

I also meet the same problem,and i try all the ways in this page,but it's useless.
finally,i try this way and then solved the problem.
https://github.com/paulbauriegel/tensorflow-tools/blob/master/convert-model-to-NHWC.py
The main solution is use tf.transpose change the NCHW to NHWC.

The link is not working.

I also meet the same problem,and i try all the ways in this page,but it's useless.
finally,i try this way and then solved the problem.
https://github.com/paulbauriegel/tensorflow-tools/blob/master/convert-model-to-NHWC.py
The main solution is use tf.transpose change the NCHW to NHWC.


text:
Feature Request: Support for YARN cluster manager for Distributed TensorFlow

The Distributed TensorFlow documentation said to open an issue to request support for a specific cluster manager, support for YARN would be beneficial.
Thanks

Yes, we'd like to do that. Nobody is working on it right now though.

@j9ac9k In the meantime if you are interested there is an initial work on Mesos that already can use GPU resources at https://github.com/tensorflow/tensorflow/issues/1996

Is anyone working on this now?

We've started looking at this internally. @jhseu will be leading up this effort.

So are you supporting internally yarn and on contribution mesos?

We'll likely do both.

Great. Any timeline on this?

There are some prerequisites that need to be done first for it to be useful (e.g. HDFS support). Definitely soonish, but I'd prefer not commit externally to a timeline :)

Also, YARN has some complications that don't exist in Mesos and Kubernetes, so we're not completely sure we'll add support for it.

How tensorflow will support hdfs ? Any progress? I'm going to contribute on it.

We're working on HDFS, it's close.

@martinwicke hi guy, how does the HDFS support going on? very looking forward to this feature.

@martinwicke Looking forward to this feature. Let me know if you need any help on this to get it out quicker

Hi @jhseu and @martinwicke , I am from YARN community. Just noticed some comments above to say extra complications on YARN over Mesos/Kubernetes. Can you shed more light on this? I can see how can I help here, especially from YARN side. Thx!

Any update ??

No update. We'll publish Mesos/Kubernetes examples fairly soon and then consider YARN next.
@JunpingDu I'll follow up with you through e-mail.

is there any HDFS support release plan recently?
Thanks.

HDFS support is available in 0.11 (and at head). See this howto.

You can ignore those instructions and just start using hdfs paths if you use this Docker image:
https://github.com/tensorflow/ecosystem/blob/master/docker/Dockerfile.hdfs

I am working on this and will use Slider to support tensorflow on YARN. I have created a JIRA in Slider Project to make a introduction. https://issues.apache.org/jira/browse/SLIDER-1174

Nice! When you have a working configuration, would you mind sending a pull request to github.com/tensorflow/ecosystem?

Sure, I will send a pull request to ecosystem when it could work

@JunpingDu @jhseu , this is zhankun from intel focused on Hadoop. We are glad to help on YARN too. How can I join your thread? :)

@wangyang0918 A great job. Any progresses?

Thanks for your attention. @formath
I have already done most of the INCLUDED features in SLIDER-1174 and will attach a alpha patch next week.

Hi! Yarn support would be great! Could you share your progress please? Thank you for your efforts!

@wangyang0918 I didn't find a PR there yet. Hope you're progressing.

@drpngx Note there is also YARN-6043 TensorFlow on YARN in the Hadoop community that doesn't require docker. You may check out the details in the design document.

I have a high level question regarding this feature request: what are the key benefits of a native approach (directly running TF on YARN) compared with a Spark based approach, most notably https://github.com/yahoo/TensorFlowOnSpark?

In Hadoop 3.0, YARN native services can support running Tensorflow services on YARN without adding any dependencies or implement a new YARN application master.
Please see our blogpost: https://hortonworks.com/blog/distributed-tensorflow-assembly-hadoop-yarn/ and let me know if you have any questions. Thanks!

Nice

In the meantime, while waiting for Hadoop 3.0, this could be an alternative, an YARN native implementation:
https://github.com/hopshadoop/hops-tensorflow

Not sure there's any update for this effort?

Just to echo @tobiajo  - there is a version of distributed YARN-native tensorflow available here:
https://github.com/hopshadoop/hops-tensorflow
This also includes support for GPUs as a resource in YARN:
https://hopshadoop.atlassian.net/browse/HOPS-11
GUI support for running distributed TF jobs on YARN will be provided in Hopsworks by the end of the month (including Jupyter to write/run jobs):
https://www.youtube.com/watch?v=rop6bhuBHT0

@jimdowling Thanks for the pointers. One quick question - does this YARN-native Tensorflow work with regular Hadoop (i.e. Apache Hadoop, rather than Hops)?

@xiaoyaozhuzi  - no. Hops Hadoop is a fork of Apache Hadoop. The YARN code is identical to the Apache code (version 2.7.3) with only a few additions: cluster state is recovered from MySQL Cluster and support for GPUs. If somebody wanted they could take our code and introduce it to YARN. But the YARN folks would prob reject the PR as they want to redesign YARN's resource model before accepting support for GPUs. That will take time.

@xiaoyaozhuzi no since we added GPU support as a resource Apache Hadoop is not supported. One could, with just a little effort, fork the repo, strip the GPU stuff and just change the dependencies in the pom.xml to Apache Hadoop.

It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly.

I'm closing this -- a feature request for YARN support and any discussion should probably happen at  tensorflow/ecosystem instead.

authors: JunpingDu bhack butterluo c-val drpngx dzhwinter formath j9ac9k jhseu jimdowling leftnoteasy manuzhang martinwicke mrry nchivukula niuchenglei tangzhankun tensorflowbutler tobiajo wangyang0918 xiaoyongzhu zhe-thoughts zjujimmy


text:
in nmt_with_attention,the gru layer in decoder confuse me

nmt_with_attention file link
In the Decoder class, the gru layer is defined, and it is used in call() function, the code is:
output, state = self.gru(x)

My question is, the gru cell state may not be updated? In the same batch, every timestamp use the same cell state? Is my understand correct?
output, state = self.gru(x,initial_state=context_vector)

I then use this code and retrained the model, there is no obvious difference. Maybe someone can help me...
Forgive my lame English.

The loss when I use context_vector as the gru's initial_state
···
Epoch 5 Batch 0 Loss 0.4456
···

@yashk2810 Can you PTAL? Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

same problem here, in decoder:
output, state = self.gru(x)
decoder call is called by time step, this will make decoder GRU stateless, there is no previous state passed to this time step.
and after I change it to stateful:
output, state = self.gru(x, initial_state = hidden)
the model's loss is increased compare to the stateless model
can anyone explain?

authors: hihell sailist ymodak