text:
fix(latex): install style/ etc in auctex recipe

In order for AUCTeX to work properly, various ingredients (such as the style subdirectory) must be present in the package's build directory. For reasons I don't understand, after a "fresh" doom sync (i.e. after nuking .local/straight), the auctex build directory contains All The Things from the repo directory (including autotools junk), so that AUCTeX should initially work fine. However, the next time doom sync is run, the auctex package gets rebuilt, and we end up (per Straight's defaults) with only the top-level .el files symlinked into the build directory, breaking AUCTeX.
This PR updates the auctex package recipe to explicitly list the "extra" stuff that AUCTeX needs. I've confirmed that the package's build directory now has the expected structure and that AUCTeX works.  The "fresh" doom sync no longer installs autoconf junk, and the package is no longer rebuilt on the next doom sync.
At https://github.com/raxod502/straight.el/issues/240 there's further discussion on the intricacies of installing AUCTeX with Straight. I don't understand all of the nuances nor the solutions suggested there. The simple fix in this PR seems to be sufficient.

I should note that this fix still doesn't provide a "complete" AUCTeX installation. To generate some of the docs, make must currently be manually run inside the repo.  Buried somewhere in raxod502/straight.el#240 is probably a viable way to automate the process. But that's an issue for another day. For now, at least all of AUCTeX's functionality is working.

To generate some of the docs, make must currently be manually run inside the repo.

Does that make command work in a shallow clone of the project? If so, adding :post-build ("make") to the recipe should do it. If not, then I'll worry about that when I have the time; a jury rigged solution will be necessary.

OK, so Straight is actually installing AUCTeX from https://github.com/emacs-straight/auctex rather than upstream. This "fork" already contains almost all of the make-generated stuff (as listed in GNUmakefile); the only exceptions are doc/preview-dtxdoc.texi and doc/version.texi. So, a few options from here:

Use this fix as-is, and declare that the lack of preview-dtxdoc.texi and version.texi is not the end of the world.
Raise an issue to see if Radon can get emacs-straight/auctex to include these two missing files. There's no dedicated issue tracker for that repo, but I guess I could just add a comment to https://github.com/raxod502/straight.el/issues/240. Not sure if it's worth bothering him.
Override the built-in recipe to clone from upstream (GNU Savannah) rather than emacs-straight/auctex. I tried doing the following:

(package! auctex
  :recipe (:host nil
           :repo "git://git.savannah.gnu.org/auctex.git"
           :files ("*.el" "*.info" "dir"
                   "doc" "etc" "images" "latex" "style")
           :pre-build (("./autogen.sh") ("./configure") ("make")))
  :pin "3b0a080ae596c26c17b15ba9c71fc5542eae238b")

But then I got an error during a (fresh) doom sync:
      > Cloning project...
      - Checked out project: 401e4aa68e5f46fe72ca6f9a1cb07cd84f5ee981
      > Building project...
      > Building project > Cloning xref...
      > Building project > Building xref...
      x There was an unexpected error
        Message: error
        Error: (error ":pre-build command error in \"xref\" recipe in command \"(\\\"./autogen.sh\\\")\"")
        Backtrace:
          (signal error (":pre-build command error in \"xref\" recipe in command \"(...
          (error ":pre-build command error in %S recipe in command %S" "xref" "(\"./...
          (condition-case err (if (cl-every #'stringp command) (if (apply #'straight...
          (let ((command (car --dolist-tail--))) (condition-case err (if (cl-every #...
          (while --dolist-tail-- (let ((command (car --dolist-tail--))) (condition-c...
          (let ((--dolist-tail-- (if (cl-every #'listp commands) commands (list comm...
          (let ((default-directory repo) (commanderror nil)) (let ((--dolist-tail-- ...
          (if repo (let ((default-directory repo) (commanderror nil)) (let ((--dolis...
          (let* ((commands (and t (if post post-build pre-build))) (repo (and comman...
          (let* ((--cl-rest-- recipe) (pre-build (car (cdr (plist-member --cl-rest--...
      ! Extended backtrace logged to ~/apps/doom-emacs/.local/doom.error.log

So it looks like the :pre-build options are "leaking" from the auctex recipe into those for other packages (in this case, xref). Not sure if I'm doing it wrong or if this is a bug in Straight or core-packages.
Worth noting: Doing a full build from Savannah would probably break the recipe for Windows users who don't have a GNU toolchain installed. Simply omitting the :pre-build on Windows wouldn't work, because if we use the upstream repo, we have to do the :pre-build.
tl;dr The easiest and most conservative option seems to be sticking with emacs-straight/auctex and the amended :files. Interestingly, based on https://github.com/raxod502/straight.el/blob/b712a2f31f9bc452a2fa55aebc6753d946bfbb34/straight.el#L3223, it looks like we should not need to do this trick with the :files. No idea why Straight isn't doing what the code says it should be doing.

This was a messier issue than I had expected. While incurring Auctex's build process would be the most correct, it would also be the least cross-compatible, so I think this :files solution is our best bet for now. I'll revisit this after the CLI rewrite to see if I can get straight to behave. Anyhow, thanks for looking into this, and for your help!

authors: hlissner mjkramer


text:
feat: implement company template

Moves some logic from the category template into the shared code so that the same format can be reused in multiple templates.
The page itself isn't very exciting right now since we don't have any company data yet, but all the functionality is there.
Fix #86
Test URLs:

Before: https://main--channelco-crn-com--hlxsites.hlx.page/news/computing/
Before: https://main--channelco-crn-com--hlxsites.hlx.page/news/amazon-web-services-dwp
After: https://issue-86--channelco-crn-com--hlxsites.hlx.page/news/computing/
After: https://issue-86--channelco-crn-com--hlxsites.hlx.page/news/amazon-web-services-dwp

@mfrisbey we dont have enough data from company pages yet right that's why the pages are blank right?
What I am also seeing is when the page loads, i see a quick flickr with all the links on initial load and then that gets decorated into the UI we want. Expected ?

@kunwarsaluja we dont have enough data from company pages yet right that's why the pages are blank right?

yes. I tested the template locally with placeholder article data, and everything should be in place.

i see a quick flickr with all the links on initial load

Interesting, I haven't seen that before but I was able to reproduce. That's probably what's causing the CLS, so I'll take a look and fix. Good catch!

@mfrisbey  You may want to look at it once again as I dont see it rendering the data anymore. @varunmitra also finalized the changes to the company template to showcase the correct metadata and have migrated all company landing pages (not the underlying articles)

Closing this in favor of a future PR that will incorporate the CLS and speed optimizations that I've been working through.

source: gharchive/pull-request

authors: kunwarsaluja mfrisbey

repo: hlxsites/channelco-crn-com

license_type: permissive

license_source: github-api


text:
Product support cards

Product support cards and header are overlapping at the top in mobile.

Duplicate of #277

Closing in favor of #277

source: gharchive/issue

authors: Acidham krizel4 meryllblanchet

repo: hlxsites/mammotome


text:
Tags Block change

For tag home page, tags block should fetch each and every tag from the tags sheet corresponding to the language selected. Don't need to add metadata in this case on the page.
At other places we fetch details (tag key) from the metadata and get name from the tags sheet.

URL creation logic will be in the code
tag key --> immune-system
url --> /healthy-thinking/tag?feed-tags=immune-system
tag key --> immune-system-jp
url --> /jp/healthy-thinking/tag?feed-tags=immune-system-jp
Changes will be done after the discussion between @sdmcraft and @ehrro

Few suggestions:

Introduce an all variant of tags block whicvh fetches all the tags.
Keeps a separate tags.xlsx which has all the tags with the following columns, id ,  type, url
Keep the tag names as usual in placeholders.xlsx
Introduce a fetchTags method in scripts.js to fetch tags. It can take type , id, locale as optional parameters.
The above method should return tuples containing id, type, url, name.
All word docs which have tags in metadata should be re-imported to have the id of the tag in this metadata.
All word docs that need to show all the tags, should remove tags metadata and instead use the tags(all) block to show all the tags.

authors: jindaliiita sdmcraft

repo: hlxsites/sunstar


text:
APIM API Setup

JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/PUB-1144
https://tools.hmcts.net/jira/browse/PUB-1147
https://tools.hmcts.net/jira/browse/PUB-1148
https://tools.hmcts.net/jira/browse/PUB-1149
https://tools.hmcts.net/jira/browse/PUB-1193
Change description

Add new API to APIM
Add secrets
Add swagger validation

Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[ X ] No

moved back

repo: hmcts/pip-data-management

license: MIT


text:
[Swift 6] use isolated(any) for factory closures

The 2.4 update added Sendable conformance to the factory closures which breaks using GAIT types in Container and elsewhere.  For example:
@DaemonActor
public final class XPCDaemon {
    public static let shared = XPCDaemon()

    public init() {}
}

@MainActor
public final class State {
    public init() {}
}

extension Container {
    @DaemonActor var daemon: Factory<XPCDaemon> {
        Factory(self) { XPCDaemon.shared } // Global actor 'DaemonActor'-isolated class property 'shared' can not be referenced from a Sendable closure
    }

    // also a problem for `MainActor` types
    @MainActor var state: Factory<State> {
        self { .init() } // Converting function value of type '@MainActor @Sendable () -> State' to '@Sendable () -> State' loses global actor 'MainActor'
    }
}

By adding isolated(any) to the factory closures we can explicitly mark which global actor the closure should be isolated on. The above can now be written without any errors or intermediate wrapper types:
extension Container {
    @DaemonActor var daemon: Factory<XPCDaemon> {
        Factory(self) { @DaemonActor in XPCDaemon.shared }
    }

    @MainActor var state: Factory<State> {
        self { @MainActor in .init() }
    }
}

Some important details to consider. From the 431 evolution proposal it mentions that functions that use isolated(any) are assumed to cross an isolation boundary and are thus implicitly async and must be awaited. It seems the compiler allows these functions to be assigned to properties where isolated(any) is not used and then called without using await. Not sure if this is intended or possibly a compiler bug. The proposal also mentions that when the compiler can detect that an isolation region is not crossed then it seems the function wouldn't need to be awaited. All of those conditions would be satisfied if the getter in Container is using the GAIT (e.g. @MainActor) and the factory closure is also using the GAIT. Then if no isolation region is crossed no await is needed otherwise an await will be required by the compiler.
Currently it's possible to write something like this:
extension Container {
    // missing `@MainActor` attribute
    var state: Factory<State> {
        self { @MainActor in .init() }
    }
}

and then use it like this:
    @MainActor func doSomething() {
        DispatchQueue.global().async {
            _ = Container.shared.state() // crash here without compiler error because `state()` is `nonisolated`
        }
    }

Which causes a quick quit easter egg. The only way I can think of avoiding this situation is if the calls to the factory closure were always awaited on which sort of defeats the purpose of having everything assigned to the same global actor. I'm open to ideas if anyone has any clues of how to force the compiler to recognize the isolation context. It would be nice if this worked:
extension Container {
    @MainActor var state: Factory<State> {
        // `isolated(any)` picks up `@MainActor` from property definition, which would require using the `@MainActor` attribute for the property definition
        self { .init() }
    }
}

From the 431 proposal:

A function value with this type dynamically carries the isolation of the function that was used to initialize it.

Perhaps there's something I'm not understanding there or the feature isn't working the way I think that it says it's supposed to.

Would love to see this merged

It's not this PR but check out the concurrency branch for 2.4.2.

Hello @hmlongco, will it be merged soon?

@Arideno +1 👀

Concurrency was merged

This PR broke the package for Swift 5: https://github.com/hmlongco/Factory/issues/250

authors: Arideno balazserdeszlogmein hmlongco mlink mooshee tareksabry1337

repo: hmlongco/Factory


text:
🛑 celebration is down

In d26155a, celebration (https://celebration.minung.dev) was down:

HTTP code: 0
Response time: 0 ms

Resolved: celebration is back up in ea7a9b4.


text:
Support for rgb color

It would be really nice to have some kind of translation from rgb to hue values.
That would count for both ways
your app could publish rgb values next to the hue values.
It could also listen for rgb values and send hue values to the bridge.
(Was in my old version, with the same name)

PR welcome :-)

authors: hobbyquaker svrooij


text:
allow GP solves to return near_dual_feas solutions

Closes #1048
@whoburg, ready for review

thanks @bqpd! this seems to have increased our solution reliability

@1ozturkbe run your code on this branch and it should work

also, I think the D8 repo will start reliably passing tests once this is merged to master

@bqpd i added a warning message to this

@whoburg ready for rereview

@whoburg hold off going to add a unit test

@whoburg this is ready for review the unit test can't be written until relaxed constants solution heuristic is added to gpkit. once this is merged i will made an issue to add the requisite unit test so we don't forget

also...I think that D8 will pass tests once this is merged

@mayork that other branch is merged, this is ready for a test!

now that I think about this, I think it's better without a test4. For one, I've only been able to generate near optimal in large signomial programs (>1,500 variables) and I don't think we want our tests to run slow because of this one units test. Further, near_optimal is returned non-deterministically so I'm not sure a good test can be formulated regardless of speed.

interesting point. I just saw it yesterday with one of Ali's robust GPs, maybe we should use that as a test instead?

Yeah that seems like a better test. if you post a commit hash I can build a test.

On Apr 20, 2017, at 14:53, bqpd notifications@github.com wrote:
interesting point. I just saw it yesterday with one of Ali's robust GPs, maybe we should use that as a test instead?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

test models please

Looks good to me. I added one commit. This looks ready to merge once it passes tests.

@bqpd can this be merged? It'd get the full D8 suite running back on master

i added a few more aircraft models to the D8 tests....it will pass once this is merged to master

Merging, will create issue for test.

authors: bqpd mayork whoburg

repo: hoburg/gpkit

license: mit

license_source: bigquery


text:
Non-jcenter dependecies are not resolved

Hi,
I'm trying to run this script
#!/usr/bin/env kscript

@file:MavenRepository("maven-central","http://central.maven.org/maven2/")
@file:DependsOnMaven("javax.jms:jms:1.1")
@file:DependsOnMaven("org.apache.activemq:activemq-all:5.15.4")

import javax.jms.Session

import org.apache.activemq.ActiveMQConnectionFactory

And here is what I have as a result
[kscript] Resolving dependencies...Failed to lookup dependencies. Maven reported the following error:
[ERROR] Failed to execute goal on project maven_template: Could not resolve dependencies for project kscript:maven_template:jar:1.0: Failed to collect dependencies at org.apache.activemq:activemq-all:jar:5.15.4: Failed to read artifact descriptor for org.apache.activemq:activemq-all:jar:5.15.4: Could not transfer artifact org.apache.activemq:activemq-all:pom:5.15.4 from/to jcenter (http://jcenter.bintray.com/): /home/ignis/.m2/repository/org/apache/activemq/activemq-all/5.15.4/activemq-all-5.15.4.pom.part.lock (No such file or directory) -> [Help 1]
If I remove a file:MavenRepository line, the result is exactly the same.
Kscript v2.4.4, Kotlin 1.2.50, both installed with sdkman.

Your example works/runs correctly for me.
What is you version of Maven? Mine is
# mvn --version
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T21:39:06+02:00)
Maven home: /Users/brandl/.sdkman/candidates/maven/current
Java version: 1.8.0_162, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.4", arch: "x86_64", family: "mac"

Are you sitting behind some kind of firewall (i.e. requiring a proxy)?

Thank you for replying.
mvn --version
Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T20:49:05+01:00)
Maven home: /home/ignis/.sdkman/candidates/maven/current
Java version: 1.8.0_161, vendor: Oracle Corporation
Java home: /home/ignis/.sdkman/candidates/java/8u161-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.14.48-2-manjaro", arch: "amd64", family: "unix"

Strangely enough, I've just wiped out my .m2 directory and now, after fetching everything from scratch, it somehow just works. I should probably blame poisoned maven cache.

I think it was docker creating files in .m2 as a root user (I was mounting ~/.m2 into a container). I had to use 'sudo' to delete .m2 directory's contents.

authors: holgerbrandl vivus-ignis

repo: holgerbrandl/kscript


text:
hvplot is not compatible with latest release of bokeh

Thanks for contacting us! Please read and follow these instructions carefully, then delete this introductory text to keep your issue easy to read. Note that the issue tracker is NOT the place for usage questions and technical assistance; post those at Discourse instead. Issues without the required information below may be closed immediately.
ALL software version info
(this library, plus any other relevant software, e.g. bokeh, python, notebook, OS, browser, etc)
bokeh 2.0.0, python 3.7.6, hvplot 0.5.2
Description of expected behavior and the observed behavior
hvplot 0.5.2 is not compatible with bokeh 2.0.0, giving the following error message:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yy939/mgh/miniconda3/envs/pegasus-github/lib/python3.7/site-packages/hvplot/__init__.py", line 12, in <module>
    from .converter import HoloViewsConverter
  File "/Users/yy939/mgh/miniconda3/envs/pegasus-github/lib/python3.7/site-packages/hvplot/converter.py", line 25, in <module>
    from holoviews.plotting.bokeh import OverlayPlot
  File "/Users/yy939/mgh/miniconda3/envs/pegasus-github/lib/python3.7/site-packages/holoviews/plotting/bokeh/__init__.py", line 30, in <module>
    from .annotation import (TextPlot, LineAnnotationPlot, SplinePlot,
  File "/Users/yy939/mgh/miniconda3/envs/pegasus-github/lib/python3.7/site-packages/holoviews/plotting/bokeh/annotation.py", line 23, in <module>
    from .element import AnnotationPlot, ElementPlot, CompositeElementPlot, ColorbarPlot
  File "/Users/yy939/mgh/miniconda3/envs/pegasus-github/lib/python3.7/site-packages/holoviews/plotting/bokeh/element.py", line 23, in <module>
    from bokeh.plotting.helpers import _known_tools as known_tools
ModuleNotFoundError: No module named 'bokeh.plotting.helpers'

But it's compatible with bokeh 1.4.0.
Complete, minimal, self-contained example code that reproduces the issue
pip install --upgrade hvplot
pip install --upgrade bokeh
python
>>> import hvplot

Stack traceback and/or browser JavaScript console output
Screenshots or screencasts of the bug in action

The import issue
from bokeh.plotting.helpers import _known_tools as known_tools
ModuleNotFoundError: No module named 'bokeh.plotting.helpers'

could be replaced with:
from bokeh.plotting._tools import TOOLS_MAP as known_tools

Which gives the next API change issue:
import hvplot.pandas

...python
from bokeh.embed.notebook import encode_utf8, notebook_content
ImportError: cannot import name 'encode_utf8' from 'bokeh.embed.notebook'

From the Bokeh 2.0.0 release list of changes I see:

bokeh.util.string Functions encode_utf8, decode_utf8, and escape removed

This has already been fixed in HoloViews and will be addressed once HoloViews 1.13.0 is out (see https://github.com/holoviz/holoviews/issues/4293).

Thank you so much for your clarification!

I'm confused, this issue has been closed, and I have Holoviews 1.13.0, so this should be working? I thought @philippjfr 's comment only meant that this WILL BE addressed in hvplot once Holoviews 1.13 is out?

This should definitely be fixed as of the HoloViews 1.13 release, i.e. it should be working now.

Yes. My case works now.

can't make it work on linux, works on mac in JLab though, despite not having pyviz/jupyterlab_pyviz, but docs say it's needed, confusing. Will make a new issue.

Strange, let us know what you find. jupyterlab_pyviz is only needed for comms and therefore interactive features relying on the jupyter kernel to work.

Looks like it's a Bokeh problem, will report there.

Looks like the extension wasnt loaded so probably a HoloViews issue if anything.

gosh, i wish i could tell you yet if it's Holoviews but it's amazingly hard to find a simple data example code cell that doesn't depend on the assets folder. holoviews' very first example notebook depends on it, but it doesn't say in that notebook what needs to be done to have that assets folder available. :(

ok, found it, i was looking for an "installation" section, but because my screen height was just optimally cutting off the main page below the example graphics, the installation section on that page was "invisible" to me. :) Maybe would be good to add a general installation pointer in a TOC somewhere..
Now trying the examples...

ok, now i'm stumped. no errors with bokeh itself, and no errors with Holoviews, so why does hvplot create above errors?
of course, retrying the hvplot example now, it works. Is there such thing as bokeh needs to be run once before it works maybe? Sorry for the noise...

authors: epifanio michaelaye philippjfr yihming

repo: holoviz/hvplot


text:
hvplot ignores default legend_position

ALL software version info
bokeh                     2.3.0
holoviews                 1.14.2
hvplot                    0.7.1
Description of expected behavior and the observed behavior
Hello,
when setting default values for the legend position in holoviews, I expect it to be also respected by hvplot, if not specified otherwise in the hvplot call
df.hvplot()

but the code above plots the legend at the standard hvplot position on the right and not on the specified default position.
Complete, minimal, self-contained example code that reproduces the issue
import holoviews as hv
import hvplot.pandas
import pandas as pd
from holoviews import opts
import pandas.util.testing as tm

opts.defaults(opts.NdOverlay(show_legend=True, legend_position='top_left', legend_cols=False))

df = tm.makeDataFrame()
plt = df.hvplot()
hv.save(plt,"test.html")

Sadly most HoloViews defaults are currently ignored. Whether to use HoloViews defaults or have a separate defaults management system is also up for debate.

I would vote against adding some separate defaults managements system.

The problem with that is that hvPlot defaults are already very different from HoloViews defaults, so if we want to unify the defaults management we'd probably have to apply the hvPlot defaults to HoloViews on import.

Having HoloViews change all of its defaults when hvPlot is imported would be very surprising to me, but it might possibly be useful, because someone who gets e.g. an hvPlot scatterplot then builds a HoloViews scatterplot by hand might well want the  styling of the HoloViews plot to match their default hvPlot outputs. Even so, I think I'd be uncomfortable applying those defaults to HoloViews automatically on hvPlot import.  Instead it would be nice for a user to be able to say explicitly "use hvPlot defaults for native HoloViews plots", and to have that mentioned as a potentially useful thing to do in an advanced section of hvPlot docs. But of course making that work would require extracting hvPlot's defaults into something declarative, when I assume they are currently embedded into the hvPlot code.
With that in mind, it seems to me like hvPlot and HoloViews could share the same defaults management system without actually sharing the same defaults. E.g. hv.config and hvplot.config might be the same code but with different sets of defaults, ideally with hvplot.opts.defaults falling back to hv.opts.default values for anything not overridden in hvplot.opts.defaults. To me it seems like that would make things work the most naturally, with users able to set any default they want and make them match if they need to, but it does sound like a good bit of work.

I suppose apart from the huge amount of effort I just don't see how that would work for a rather large number of reasons:

If they don't share the same defaults then they by definition cannot be set in the same way, there would have to be two different APIs the set them (which kind of defeats the point).
hvPlot users shouldn't have to know about HoloViews element names, e.g. to set defaults for an hvPlot line plot a user shouldn't have to set hv.Curve defaults.
hvPlot does not neatly have to map to one particular HoloViews element, e.g. there might be composite objects returned by hvPlot which require very different defaults from the elements that make up the composite object.
There is no way for hvPlot to maintain a separate options tree like hv.opts.defaults because, once the object is returned by hvPlot, there is no way to HoloViews that it should load a different set of defaults.

So overall I see a shared system not only technically challenging but also resulting in a seriously sub-optimal user experience.

I suppose apart from the huge amount of effort I just don't see how that would work for a rather large number of reasons:

Definitely, the amount of work involved is probably prohibitive. Still, it's worth having even quite difficult roadmap items if they are desirable, because at the moment there's no straightforward way for a user to configure hvPlot defaults at all, which doesn't seem sustainable in the long term. Plus, as we want to add a Matplotlib backend for hvPlot, I it seems like something of this nature would be needed anyway. Plus making and maintaining a separate defaults management system is also a huge amount of effort, in the long run. So let's try to discuss what would be desirable separately from the difficulty of doing it.

If they don't share the same defaults then they by definition cannot be set in the same way, there would have to be two different APIs the set them (which kind of defeats the point).

I don't think that's true; can't they be set the same way with hvplot.opts.defaults() to set the ones specifically for hvPlot, and hv.opts.defaults() setting ones that apply either to both libraries (for options not already overridden in hvplot.opts.defaults either by hvPlot itself or by a user's changes) or specifically to HoloViews.  That's the same API, but acting on different objects.

hvPlot users shouldn't have to know about HoloViews element names, e.g. to set defaults for an hvPlot line plot a user shouldn't have to set hv.Curve defaults.

I agree that's annoying and confusing, but it seems like you'd have to have not just a new defaults management system but also a new options management system to avoid that. Advanced hvPlot users already have to know about HoloViews elements and .opts if they want to change any of the options on a created object, and that seems like something they would typically learn first, before ever messing with hv.opts.defaults at all. Only once they've applied .opts a lot would they ever be interested in changing the global defaults, and by that point, they've already made such a mental mapping. We could add some aliases to the options system to help smooth over such differences (accept opts.Line or opts.line for opts.Curve, etc.), but I don't have a strong opinion whether that's a good idea. In any case I agree that it's a shame there will be this impedance mismatch between hvPlot's method names and HoloView's ELement names, but I don't see a practical way to avoid that gap existing somewhere.

hvPlot does not neatly have to map to one particular HoloViews element, e.g. there might be composite objects returned by hvPlot which require very different defaults from the elements that make up the composite object.

Sure. In that case we'd presumably want to make a dummy hvPlot-specific element to hold such default values, even if it has no plotting implementation. We could even consider whether the implementation for such objects should be in a HoloViews element in general, so that HoloViews users could access that functionality in a natural way. In any case we don't necessarily have to solve such cases; they currently aren't configurable in a defaults system anyway, and so they would be no worse in my proposed unified defaults system.

The hvPlot options deliberately do not map directly to HoloViews option names, so to set defaults via a shared system a user would have to learn what the HoloViews options are called, which is different from what they would use in an hvPlot call.

Again, unless you are proposing not just a new defaults system but a new options system, I don't see how that's relevant. Yes, advanced or picky hvPlot users do have to figure out that hvPlot returns HoloViews objects and that such objects have different option names etc., but that's true already and is not affected by a defaults management system. And I think there are far more options that don't exist at all in hvPlot calls than that do, and so a default system that only addresses the hvPlot syntax will be severely limited. People would still need to reach for HoloViews options at that point, so again it seems to me that a separate defaults system is just going to make things worse.
In any case, despite strongly pushing hvPlot as our primary point of entry for users, I still would vote against adding another separate options management system, for the same reasons as voting against a separate defaults management system. I think users are better served by having solid, well documented functionality in hvPlot, along with a clearly demarcated boundary where they fall back to the underlying HoloViews way of doing things. Adding yet more complexity trying to hide the underlying system seems unworkable to me.

5.There is no way for hvPlot to maintain a separate options tree like hv.opts.defaults because, once the object is returned by hvPlot, there is no way to tell HoloViews that it should query a different set of defaults.

There are surely lots of ways! hvPlot could make a call before it returns the objects, looking up the hvPlot-specific options from hvplot.opts.defaults and applying them to the object before returning it. The result would be a pure HoloViews object with hvPlot's options applied. Or, HoloViews objects could have a pointer to HoloView's root options tree, while hvPlot would return objects pointing to a different root, so that they work exactly the same but with different option values. There are probably other approaches I haven't thought of, and definitely some way to make that work!
Anyway, my main concern here is that we need systems that we can maintain and defend and that users can understand. Adding some partial, always limited new defaults management system for hvPlot will make some things easier for some class of hvPlot users and some tasks. But we'd still either have continual pressure to make it cover more cases (inventing new hvPlot versions of all the HoloViews options that don't currently show up as hvPlot method arguments), or else users would have to drop down to HoloViews option and defaults management anyway. My vote is for having a clean break: hvPlot lets you build plots the hvPlot way, using function arguments. Anything else is done the HoloViews way, both for options and for defaults, even if we might do some aliasing in such cases to help hvPlot users make the mapping from hvPlot to HoloViews.

Hello,
I also think an additional options system is more confusing, as already stated an advanced user already knows about the connection to holoviews. The confusion for me comes more from the fact, that some default options are ignored. I basicly use hvplot to easily create plots from dataframes and then compose them further or add interactive functionality using the returned holoviews objects.
The defaults from hvplot are basicly set implicitly by the handling of the input arguments in the hvplot call.
For me the most intuitive way would be if an argument is provided to the hvplot call it is used and if its not, hvplot would check if there are user defaults set for that parameter and use that instead of the hvplot default.
This could either be achieved when handling the parameters in hvplot or afterwards with applying the options.
For setting defaults directly from hvplot,  I could think of an api using the hvplot names, that applies the options to the corresponding holoviews options tree. e.g hvplot.opts.default.line
Parameters like legend_position, that apply to line but not to Curve can than be applied NdOverlay for example.

authors: HugoMichi jbednar philippjfr


text:
Param documentation

Param's website param.pyviz.org is appalling.  It has a brief intro, but no real documentation, with no clue about any of the more powerful features of Param (certainly not any of the more recently added ones like depends, watch, and output) and outdated pointers to ParamNB and ParamTk and no mention of Panel.  Some previous issues mention bits of docs that do exist that need to be moved into the website (https://github.com/pyviz/param/issues/157, https://github.com/pyviz/param/issues/94), but really it needs some dedicated effort to provide actual, up to date content, because it's such a big part of how to use Panel.

Just to start a grab bag of items that need documentation:

Controlling order (definition order on py3.6+, sorted alphanumerically py2.7-3.5, within each precedence value; see https://github.com/pyviz/param/issues/185).

Hi, having these items documented would be really helpful. Do you think the documentation will be updated in the near future? Or, are there places I can read more about them -- esp. item 3,4,5,6?
Thanks!

Extends #47 and #157 (have closed those).

@cocoaaa Bit late for your question, but I would look at panel's docs and examples for now (e.g. see panel.pyviz.org).

@jbednar is there any documentation anywhere for some of those things? I am trying to figure out what watch does by turning it off and on, and can't tell. The controlling order thing is also something I've been trying to find and it sounds like it is possible.

The docs at panel.pyviz.org are your best bet for now; search those for "param" and "watch" and you should see examples of watch in heavy usage. Watch doesn't need Panel, of course, but that's where it's currently documented. I apologize for that; my only defense is that no one has ever paid us to document Param, and we're paid to work on a whole lot of other things that don't leave any time to work on Param docs.  I can't even guess how long "document Param" has been #7 on my priority list, and only the first few ever get addressed. :-/

No problem, I totally understand and appreciate the work you do!

FYI: We now have paid-for time to develop proper Param documentation, so it will happen sometime in 2020!  Sooner rather than later, I hope, but we have a large set of tasks in the current plan of work, and this one isn't the first in the list.

Param has now a proper documentation website, closing :)

authors: ceball cocoaaa jbednar kgullikson88 maximlt

repo: holoviz/param


text:
Add more tests

This PR doesn't touch any file in param or numbergen, it only adds more tests. Adding tests was motivated by the Sentinel PR, that in its current implementation is dynamically computing the default value of Parameter attributes (e.g. instantiate), opposed to know where most of them are just statically declared in the signature definitions. The tests added captures the current default values of most of the Parameters. This PR also adds some rather simple tests to previously untested Parameters such as Path.

Going to self-merge as this just includes changes to the test files.


text:
light.mqtt_json reference causing configuration errors

Running HA 0.84.3 on Centos 7, I'm seeing logged issues and reports of configuration errors regarding light.mqtt_json.
Note:  I do not have any such components configured in my system.
The associated log entry is:
2018-12-18 09:43:51 ERROR (Thread-10) [homeassistant.loader] Error loading homeassistant.components.light.mqtt_json. Make sure all dependencies are installed
Traceback (most recent call last):
File "/usr/lib/python3.5/site-packages/homeassistant/loader.py", line 92, in get_component
module = importlib.import_module(path)
File "/usr/lib64/python3.5/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 985, in _gcd_import
File "", line 968, in _find_and_load
File "", line 957, in _find_and_load_unlocked
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/usr/lib/python3.5/site-packages/homeassistant/components/light/mqtt_json.py", line 20, in
from homeassistant.components.light.mqtt import CONF_BRIGHTNESS_SCALE

Please read the breaking changes: https://www.home-assistant.io/blog/2018/12/12/release-84/#breaking-changes

And also make sure you're not using any custom components (fill out the issue template please, that was included there)

Reopen.
I am not using any MQTT light components as noted in the bug report.
This is a residual issue with your codebase
On 12/18/18, Otto Winter notifications@github.com wrote:

And also make sure you're not using any custom components (fill out the
issue template please, that was included there)
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/home-assistant/home-assistant/issues/19434#issuecomment-448323499

--
If you want your mail flagged by the government
just use this phrase often:  purchase plastic explosives

Blech.  Confirmed... found one squirreled away that wasn't even active.

authors: LouPaloma OttoWinter

repo: home-assistant/home-assistant


text:
Waze still throws unhandled exception (but not on startup)

Home Assistant release with the issue:
0.87.1
Last working Home Assistant release (if known):
N/a, this error was supposed to be fixed with #20316 but it fixed it only for startup
Operating environment (Hass.io/Docker/Windows/etc.):
Hassbian on a Pi3B
Component/platform:
Waze Traveltime sensor
Description of problem:
Waze still throws unhandled exceptions, just not at startup anymore. I have 3 sensors set up, and I have had 2 of these exceptions in the 24 hrs that I have 0.87.1 installed. Either exception happend at different sensors - I just copied one example here. Timing seems to be random. This particular sensor uses a device tracker and a zone, but the other is a zone-to-zone (work-home) setup (so no changing coordinates or so).  It is not the same error as #20953 (even though I do see that error too at startup). This is a different error at different point in time.
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
### sensors.yaml
- platform: waze_travel_time
  name: Reistijd Bob naar huis
  origin: device_tracker.bob_s9
  destination: zone.home
  region: 'EU'

Traceback (if applicable):
2019-02-13 18:55:01 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.reistijd_bob_naar_huis fails
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 221, in async_update_ha_state
    await self.async_device_update()
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 349, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/util/__init__.py", line 315, in wrapper
    result = method(*args, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/sensor/waze_travel_time.py", line 207, in update
    routes = params.calc_all_routes_info(real_time=self._realtime)
  File "/srv/homeassistant/lib/python3.6/site-packages/WazeRouteCalculator/WazeRouteCalculator.py", line 145, in calc_all_routes_info
    routes = self.get_route(npaths, time_delta)
  File "/srv/homeassistant/lib/python3.6/site-packages/WazeRouteCalculator/WazeRouteCalculator.py", line 95, in get_route
    response_json = response.json()
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Additional information:
Sensor does update itself after this error, when the new update_interval trigger is activated (no breaking-breaking error).

I am also still receiving this error, but not on startup.  I do have a thought.  I noticed that user: hmmbob also is using a device tracker as the starting point (as am I), wonder if the error is tied to the device tracker being unavailable, or reporting bad data at the time?  I dont have any waze sensors at the time that just use zones, or static coordinates.  but I'll try that out and see if the errors go away
2019-02-13 19:37:19 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.pauls_time_to_home fails
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 221, in async_update_ha_state
    await self.async_device_update()
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 349, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/util/__init__.py", line 315, in wrapper
    result = method(*args, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/sensor/waze_travel_time.py", line 207, in update
    routes = params.calc_all_routes_info(real_time=self._realtime)
  File "/srv/homeassistant/lib/python3.6/site-packages/WazeRouteCalculator/WazeRouteCalculator.py", line 145, in calc_all_routes_info
    routes = self.get_route(npaths, time_delta)
  File "/srv/homeassistant/lib/python3.6/site-packages/WazeRouteCalculator/WazeRouteCalculator.py", line 95, in get_route
    response_json = response.json()
  File "/srv/homeassistant/lib/python3.6/site-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/simplejson/__init__.py", line 518, in loads
    return _default_decoder.decode(s)
  File "/srv/homeassistant/lib/python3.6/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/srv/homeassistant/lib/python3.6/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Nope, if you read my post carefully you will see that I reported this happens to both my device_tracker-zone sensor AND my zone-zone sensors. I just copy-pasted one of them, als the error does not differ besides the time stamp.
My guesstimate is that the Waze server is returning an empty response (for whatever reason) that is not handled correctly.

this is was happens because the waze component is starting too early, there is a fix for this problem in the way  @ https://github.com/home-assistant/home-assistant/pull/21087
Sorry about that :)

@VirtualL Sorry, but no. Please read back what I reported in the first post. This is NOT the same issue.

I think this error needs to be fixed in the waze library that home assistant uses.

OK, I checked the upstream library (https://github.com/kovacsbalu/WazeRouteCalculator), and it is at version 0.7.2 (instead of 0.6 which HA uses). It is supposed to have a few fixes, especially for the error handling.
However, pulling the sensor into a custom sensor, updating the required version of the package to 0.7.2 led me to not being able to get any route, as the routing did not accept any GPS coordinates anymore. There is a PR for the upstream package to get it to accept GPS coordinates again, but that hasn't been merged.
Reverted all my changes, back to the currently shipped version that comes with HA.

authors: MartinHjelmare VirtualL hmmbob ptdalen


text:
504 errors when running behind apache proxy

Home Assistant release (hass --version): 0.41.0
Python release (python3 --version): 3.4.3
Component/platform: Web Interface / aiohttp
Description of problem:
When  running homeassistant behind an Apache proxy server, you will seen 504 errors (Gateway timeout)
in  the apache logs. Issue has been reported both for Apache as NGinx as proxy server; has been closed every time as not being a HA issue.
For example: https://github.com/home-assistant/home-assistant/issues/6676
Expected:
No 504 errors should normally occur
My info on this issue:
I have seen this issue as well; currently running HA 0.41.0
I have seen them on camera-url's on gpslogger requests, even on static content, etc.
I traced the network traffic between Apache and HA to find out who is sending the 504 and why.
I ran tcpdump on my Ubuntu server to catch the network traffic from Apache to HA (running on the same server, over the lo-adapter):

sudo tcpdump -i lo -w /tmp/tcpdump.pcap

What I see in Wireshark is the following:

Apache sends all requests to HA, event the ones we see the 504 error for in Apache log files
Apache is using a connection pool for the proxy; requests will go from apache->HA using tcp ports (for example 42635->8123)
Since Apache uses connection pooling, this port will get reused later on
in my case I have seen a port used (secs are offset to trace start):

Request 1:
0.06 sec: GET /api/camera_proxy/camera.schuur?token=966...40b&time=1490557176003
0.32 sec: 200 OK
Request 2:
30.0 sec: GET /api/camera_proxy/camera.voordeur?token=47c...5d7&time=1490557206005
30.26 sec: 200 OK
105.073 sec: 504 Gateway timeout (NOTE: no request for this response! see below for the content of this 504 sent by HA!)
Request 3:
110.05 sec: GET /api/camera_proxy/camera.schuur?token=cb2...cf0&time=1490557286013
110.05 sec: tcp port is reset by HA (8123 → 42635 [RST] Seq=64315 Win=0 Len=0)

SO: it looks like the 504-errors are related to Apache connection pooling

this is something you would never see without a proxy in front
Big question is

why is HA/aiohttp sending 504's
and why is it being sent without a request?

If I turn TCP port reuse off in apache proxy settings; i no longer get 504 errors (at least for now):

ProxyPass /api/websocket ws://localhost:8123/api/websocket disablereuse=On
ProxyPassReverse /api/websocket ws://localhost:8123/api/websocket
ProxyPass / http://localhost:8123/ disablereuse=On
ProxyPassReverse / http://localhost:8123/

This is a workaround to prevent the errors from occuring
It seems to work, for now...  But what will be the fix for this?
504 returned by HA/aiohttp without a request....

HTTP/1.1 504 Gateway Timeout\r\n
Content-Type: text/html; charset=utf-8\r\n
Content-Length: 182\r\n
Connection: close\r\n
Date: Sun, 26 Mar 2017 19:41:21 GMT\r\n
Server: Python/3.4 aiohttp/1.3.5\r\n

That's great news! I'll update my config and see if this works for me too!
Sebastian

let me know it it works for you as well. For me, I still am not seeing any 504 errors in Apache logs since yesterday....

After a short test, it's looking good so far... I'll keep you updated in case this changes.
Sebastian

Glad I saw this - I was getting the issue with my mjpeg camera, hadn't noticed it was only when I was accessing remotely via an Apache proxy. This workaround fixed it for me - the logs look clean today.

It's still looking good for me too. Thanks a lot @mjj4791 for finding that workaround!
I already created a PR to update the documentation.
Sebastian

Thanks @mjj4791 it works for me too!

Does it works with basic-auth set on Apache?
My config is working on nginx - any ideas how to copy that config there?

I can only answer your first question: yes, it works with basic auth.

This will be fixed by the aiohttp 2 upgrade

@balloob, so it was a HA (/component) issue after all?

I discovered that we could end up sending a 504 when losing connection after a connection was already used. Only happens when streaming camera's.
We will now only raise a 504 when the initial connection times out:
https://github.com/home-assistant/home-assistant/blob/dev/homeassistant/helpers/aiohttp_client.py#L82
So yes, it was an HA issue but I am still a bit confused how it can be triggered 🤔  Closing this as aiohttp 2 has been merged into develop: #6835

Just to complete the issue, for anyone searching for this error.
I upgraded to HA to 0.42.3 and removed the apache workaround I suggested above
(i.e. i removed the "disablereuse=On" from my apache config for HA)
This new config has been running for some hours now; have not yet seen any 504 error in the apache logs.

I can confirm this. I've been running 0.42.x since last Sunday and also removed the disablereuse-workaround when I update - haven't had a single 504 error since.
Sebastian

authors: alexmogavero andriej balloob mjj4791 seanb-uk sebk-666


text:
Home assistant (docker installation) upon RaspberryPi 3 is failing and the error message in the raspberrypi3 is 'Application Architecture incompatible with RaspberryPi 3 : exec format error' . We selected the Docker installation instead of manual installation , to use home-assistant with resin.io

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
Python release (python3 --version):
Component/platform:
Description of problem:
Expected:
Problem-relevant configuration.yaml entries and steps to reproduce:

Traceback (if applicable):

Additional info:

you need an ARM Home Assistant Docker Image. However, since you're using Resin.io, you are probably interested in: https://github.com/home-assistant/hassio

Thanks Thomas , I used lroguet/rpi-home-assistant with resin.io . Everything runs smooth

I have also written a Docker I will test it out and share the docker registry with you

authors: Srinidhi-SH tbrasser


text:
Fail when POSTing new automation in the editor

Home Assistant release (hass --version):
0.47.1
Python release (python3 --version):
Python 3.4.2
Component/platform:
Automations Editor
Description of problem:
Chrome fails @ POST saving a new automation
Expected:
Should save to automations.yaml
Traceback (if applicable): From chrome
POST http://10.0.0.10:8123/api/config/automation/config/1498756277666 404 (Not Found)
Object {error: "Response error: 404", status_code: 404, body: "404: Not Found"}

This is a duplicate of  #7823 and can be fixed by following the suggestion in https://github.com/home-assistant/home-assistant/issues/7823#issuecomment-314694547
Cheers,
Whytey

authors: Whytey Xstasy


text:
[RFC] Filter helper decorator

Description:
When dealing with physical sensors (those who read real life data) it is common to get errors and noise (due to the quality of the sensors or undesirable events). Good devices implement internally filters that remove outliers and smooth data. But many other devices do not (especially DIY devices/hardware and cheap devices).
This Pull request introduces a helper decorator Filter that can be added to the state() method or to any other @property method.
Two files are provided:

helpers/filter.py : implements the decorator and filter algorithms
components/sensor/filter.py : implements a meta sensor that uses the fore mentioned decorator and can be used regardless of components being updated with the Filter decorator

Filters have been implemented as decorators since I believe they are of broader use then to this single meta filter component (I personally use it for two different custom_components that handle sensor information coming from DIY devices). Furthermore this implementation makes it trivial to add filtering to existing and future components.
Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#<home-assistant.github.io PR number goes here>
Example entry for configuration.yaml (if applicable):
sensor:
  - platform: filter
    entity_id: sensor.relative_humidity
    name: lowpass
    options:
      time_constant: 10

Checklist:

[x] The code change is tested and works locally.
[ ] Documentation added/updated in home-assistant.github.io
[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Yep, perfect chain (can actually have multiple Filter decorators) to the same @property
Have been working and discussing this contribution with @robmarkcole who has expressed interest in providing some of the said filters. This PR is for now a proof of concept to get core team feedback before proceeding.

Tried addressing all comments. The filter stats do nonetheless require the collaboration of the component author (injecting the attribute into device_state_attributes() )
Planning on moving into WIP and start writing tests and doc.
I believe the naive filters should be enough for an initial PR, and further PR can each add it's own filter.

I don't like the name "filter" for the helper. It's too generic.
Also, shouldn't there already be a package out there that has all these filters and if not, should that be created and used inside Home Assistant? That way data scientists all over the world can use it.

Scipy signal processing is recommended for filters and could be incorporated to this component

That pulls in numpy. I don't want to add such a big dependency to the core.

@balloob appears a few components use numpy already? I'm also just about to submit another component that requires numpy, and generally most scientific packages require numpy so not having it would be a large handicap.

@robmarkcole numpy (+ scipy) are huge dependencies, especially concerning how often I believe this platform would be used. Also: I assure you lots of people will have problem with compiling the C extensions. Next, the signal module that was linked to has lots of complicated filters, but not really the ones we would like here (or I'm just not seeing them). Additionally, AFAIK scipy is better for fixed-size arrays and not so much for a constant stream of data

@robmarkcole there is a difference between core and component dependencies. Core dependencies are always installed. Component dependencies are installed on demand.

We have very few core dependencies which is good. Means that we have very little overhead of just loading the base system, lower memory footprint.

@balloob OK didn't appreciate the difference between core and component dependencies. Therefore if we stick the numpy requirement in the components only we are OK - or still advised against numpy as per @OttoWinter comment?

I had already expressed the same concern about adding large libraries, unfortunately that does leave out the libraries every data-scientist is using (numpy, scipy).
Since filters are bound to become common, they should be as simple as possible and without any dependencies beyond standard python libraries such as math and statistics in order to assure performance and minimal requirements.
As for the name "filters", I'm open for suggestions, but this is way to obvious IMHO.
Will do the recommend changes later today.

I'm not sure if we need it as helper. With the "filter" platform, you can cover all available sensor platforms and for custom component you can use a own library. I don't like that inside core to maintain explicit custom components.

What I was trying to avoid was yet another explosion in entities in the state machine.
Currently half of my entities are templates, with filter component at least one third would become filter components (in addition to the existing templates).
I think helpers is the way to go to limit the amount of entities.

@dgomes the problem is that we have to update every sensor platform ever to apply a filter. And that doesn't make much sense either as we need to be able to differentiate between 1st order (raw data from devices) and 2nd order (data derived from raw data).
So yes, @pvizeli has a point that this doesn't need to be in a helper.

Could the filter just append an attribute to a sensor?

No. That's a hack.

Don't think we need/should update every sensor. Since each sensor has a different model it isn't even viable for someone external to the sensor developer to go and add filters. It must be a task accomplished by each developers in its own interest (many devices already implement filters in hardware, or their libraries do all the filtering).
This is why I think it should be a helper:

Common functionality
Easy access for developers to integrate

@pvizeli don't understand your comment "inside core to maintain explicit custom components." - helpers have no dependence on the custom component (they could have been split in 2 PR)

We still consider the data "Raw" when the developer has run it through a filter before serving it to us via an API. We should not add a filter config option to every sensor.

I'm not proposing to add such option, my reasoning is that the developer configures a filter in it's own code and serves as it is to the end user.
Think about a lousy weather station (such as my own), I error is so high that I get 10ºC drops every now or then. I would use the outlier filter helper in my component code to filter those errors off.
Or think about an energy sensor monitoring Vrms (looks like a saw with several spikes) but what we actually wan't is something smoother, again as a developer of such sensor I would apply a Filter helper.
In all these use cases, the filter arguments would NOT be exposed to the end user.
The filter component is a different beast... that would expose everything, but it will also become an experts only sensor as parameters need to be tuned.

developer configures a filter in it's own code

That code would not live inside Home Assistant, as we deal with raw data.

Adding a filter sensor: ✅
Adding a filter helper: ❌

I also think that we should call it data_filter to indicate what we're filtering. (it's not for images or water)

@robmarkcole and I have discussed the idea of adding image filters to this component.
The only issue is detecting the type of sensor being fed to the filter, but can see a new PR coming in the near future in that direction.

@dgomes you example config in the description needs to be updated   - platform: data_filter

@robmarkcole guess your comment belongs in #4725 ;)
will update

@dgomes your docs are correct but the config at top of this page requires platform: data_filter
Re graphs, a picture tells a thousand words :)

I have a suggestion for the config schema:
Why not do something like:
sensor:
  - platform: data_filter
    name: lowpass_relative_humidity
    filters:
      - filter: lowpass
        time_constant: 10
      - filter: mean  # doesn't exist yet
        last_values: 42

For me, I think that would have several benefits:

Readability: Now the filter and it's options sit "closer" to one another in the config, thus increasing readability. Having a separate options: key seems like a bit of a hack to me.
Easy chaining of filters: If I understand correctly, with your current config you'd have to create multiple filter platforms to apply multiple data_filters to a raw data stream. What if I want to have my humidity sensor do lowpass detection & do some weighed history average? With filters: this would be a lot nicer. Also: this would put less pressure on the state machine.

Thanks for the feedback @OttoWinter !
Let me start by saying how much I'm enjoying working in this project, and the all interesting discussions within the developers community.
I think the data_filter platform has become a Frankenstein after all this discussion...
The platform was supposed to be a demonstrator of the Filter decorators, the platform was a way to demo the decorators without interfering with any other platform, it was not an end in itself (heck! it could have never existed and we could still have filters!). The choice of very simple filter (BASIC) was to not draw attention to the filters but to the helper. Yet, everyone focused on the platform....
My vision was that developers would have incorporated the decorators with options tuned for each sensor (no need for end-user to change any filter option value, the sensor platform developer would tune according to his use case/hardware and would expose a high level option at most!). Filtering would have been enabled/disabled per sensor platform through a configuration option in each sensor platform (not in the data_filter platform or helper).
It was never supposed to be done through the data_filter component like is currently the case.
I accepted the core team decision not to have Filters as helpers, but as a platform. I've moved the decorators away into the data_filter platform (yes... very hackish, but was my way of still being able to import the decorators into other components while keeping the filters in the official repository and exposing the possibility to other developers).
This brings us to today:
The current PR pushes filters towards the usage pattern of the template_sensor which I hate very dearly as it pollutes the state machine with extra entities, but from which I cannot escape...
I'm no longer proud of this PR... :(
This component does not address my goals and the changes requested, although very appropriate have created / will create something very different from what was my initial goal. With that the code has become very weird. The current data_filter doesn't support chaining as suggested by @OttoWinter, adding it in the current code would make things even more weird...
I could have ended this very long and sorrow comment with a "goodbye and thanks for the fish", but I like this community too much.
If people here agree that the template_sensor pattern is the way to go, I will start working a new platform from scratch (lets call it filter_sensor), that doesn't need to have idempotent filter algorithms (which currently adds complexity but was need to implement the filter decorator pattern).
Final bitching remark: I'm keeping the decorators in my custom_components dir :P

authors: OttoWinter balloob dgomes pvizeli robmarkcole


text:
Add support for Kwikset 914 Convert - lock.zwave id0446 (#19710)

Description:
Related issue (if applicable): fixes #19709
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#<home-assistant.io
dc562e4
Example entry for configuration.yaml (if applicable):

Checklist:

[X] The code change is tested and works locally.
[X ] Local tests pass with tox. Your PR cannot be merged unless tests pass
[X ] There is no commented out code in this PR.

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New or updated dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

Hi @cdkonecny,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!

authors: cdkonecny homeassistant


text:
Add support for SET_AWAY_MODE and TURN_ON/OFF on Daikin Climate

Description:
Adds support for SET_AWAY_MODE and TURN_ON/OFF on Daikin Climate
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#9370
Example entry for configuration.yaml (if applicable):

Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.

If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly (example).
[ ] New dependencies have been added to requirements in the manifest (example).
[ ] New or updated dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

This looks good -- but please the docs need to be updated and I couldn't find a PR in https://github.com/home-assistant/home-assistant.io.

@andrewsayre thanks! There are some issues with the next branch for Daikin in https://github.com/home-assistant/home-assistant.io, with many changes old changes that doesn't show up in current (which are correct).
I'm not sure how to proceed with the documentation changes to the next branch, but I did the changes that would merge nicely to the current branch: https://github.com/home-assistant/home-assistant.io/compare/current...fredrike:patch-8

@andrewsayre The documentation is updated now.

authors: andrewsayre fredrike


text:
kodi platform: following jsonrpc-request version bump (0.3), let kodi…

Description:
Hand over the collection item, the url or the file to play directly to kodi's file abstraction layer
Related issue (if applicable): fixes #
fixes kodi platform's implementation of media_player.play_media()
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#
Example entry for configuration.yaml (if applicable):

Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If code communicates with devices:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Looks good! :dolphin:

authors: balloob patkap


text:
Refactor Netatmo integration

Breaking Change:
A lot

for authentication only client id and secret are required, no need to pass your username and password anymore
public weather sensors are removed and will be added back in a separate PR
Service set_schedule now requires the name of the home
webhooks are removed and will be added back in a separate PR
Services to toggle the light/set it to auto have been removed until implemented in pyatmo

Description:

Implements OAuth2 account linking
Fixed set_schedule, now requires the name of the home
Presence camera flood light is exposed as an actual light (currently only its state)

Related issue (if applicable): fixes #
Pull request with documentation for home-assistant.io (if applicable): home-assistant/home-assistant.io#<home-assistant.io PR number goes here>
Example entry for configuration.yaml (if applicable):
netatmo:
  client_id: your_client_id
  client_secret: your_client_secret

Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[ ] There is no commented out code in this PR.
[ ] I have followed the development checklist

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly. Update and include derived files by running python3 -m script.hassfest.
[x] New or updated dependencies have been added to requirements_all.txt by running python3 -m script.gen_requirements_all.
[ ] Untested files have been added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

Waiting for upstream merge of https://github.com/jabesq/netatmo-api-python/pull/96.

This should be it. Now I'll just have to wait for a new pyatmo release so that the requirements are matched.

Looks good to me. Giving Martin final call 👍

Copy and pasting your suggestions wasn't that hard. 😉

I missed a docs PR. Please link it in the description.

That is true, I’ll fix that ASAP.

authors: MartinHjelmare balloob cgtobi


text:
[FR] Error message improvements

New error message improvements 🎉
We renamed area to pièce instead of zone.
zone is already used for zone (An external concept like your kid's school or your work place)
We felt the wording Aucun appareil ou entité appelé ... was a bit heavy for a voice interface so we simplified it to je ne connais pas l'appareil ...
The biggest change is that we now translate domains and device_classes for some errors.
Here are a few examples of interactions you will get from this PR:

Ouvre les rideaux du jardin

Before:  Jardin ne continent pas de curtain
Now ✨: Désolé, je n'ai pas trouvé de rideaux dans jardin

Allume les lumières du jardin

Before:  Jardin ne continent pas de light
Now ✨: Désolé, je n'ai pas trouvé de lumières dans jardin

What's "not perfect"
{area} cannot be prefaced with the correct article.
We cannot say:

Dans le jardin
Dans la cuisine
Dans l'entrée

... because we simply do not know
For Cover_HassTurnOn and Cover_HassTurnOff, we have a hybrid mapping of a single word: "Store" matching 3 device classes:
lists:
  ...
  cover_classes:
    values:
      - in: "store[s]"
        out:
          - "shade"
          - "blind"
          - "awning"
      - ....

When we say Ouvre les stores dans {area} ... If area does not contain any of these device classes, then unfortunately, the response will only return the first one alphabetically (awning) ....
As such

Note: I tried to move the order so that shade would be the first because shade is translated as "Store" .. but I think HASIL is re-ordering them alphabetically...
Note 2: Our hybrid mapping does work in case you are wondering, we are really targeting all 3 device classes as shown here (And also on the tests of this repo)
intent:
  name: HassTurnOn
slots:
  device_class:
    - shade
    - blind
    - awning
  area: jardin
  domain: cover
details:
  device_class:
    name: device_class
    value:
      - shade
      - blind
      - awning
    text: stores
  area:
    name: area
    value: jardin
    text: jardin
  domain:
    name: domain
    value: cover
    text: ''
targets: {}
match: true
sentence_template: <ouvre> [<tous>] [<le>]{cover_classes:device_class} [<dans>] [<le>]{area}
unmatched_slots: {}

This only issue is how the response is constructed, and this is not something we can control here unfortunately
EDIT
After talking to @piitaya , we are simplifying our cover device classes mapping
      - in: "store[s] [vénitien[s]]"
        out:
          - "blind"
          - "shade"

"Store" targets both type of "Store" and both device classes are now translated as "store" (and not "Store vénitien")
JLo

@piitaya
Question for you regarding the limitation on the cover device classes.
We can also simply remove the mapping for "awning", "blind" and "shade"
It would work for the other but not for our weird "Hybrid mapping"
Example:

Ouvre les rideaux du jardin

Désolé, je n'ai pas trouvé de rideaux dans jardin

Ouvre les volets du jardin

Désolé, je n'ai pas trouvé de volets dans jardin

Ouvre les stores du jardin

Désolé, je n'ai rien trouvé de correspondant  dans jardin

(We fallback to the generic one)

Here's a thought from a poor-French-speaking-intruder :D In Romanian, I named all areas "spaces", both in the core/frontend translations and in the intents. Does "espace" make more sense in French than "pièce", which AFAIK literally means room?

@tetele I think overall "Espace" would be better as it fits open floor.
But unfortunately, this ship has sailed for the Frenchies a long time ago when we decided to name area "Pièce" everywhere.
I guess we could, one day, make an effort to change all translations in one shot to keep consistency 🙈

authors: jlpouffier tetele

repo: home-assistant/intents